2024-04-20T21:00:10.224 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T21:00:10.224 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} --name rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --owner scheduled_rishabh@teuthology --verbose -- /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405/orig.config.yaml 2024-04-20T21:00:10.282 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T21:00:10.371 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405 branch: wip-rishabh-testing-20240416.193735 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7665405' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: a8c84f5997950fd7d008204839253291d52e6eda subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7954 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAXRcctz8qWnJkfkKzMfepnmRHdg99/wZeDzBv2PXjD1R27OlTS8c9NJ5ZyJ/wD1rLapEjN6LAnt4wyA1i8gbPo= smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBII995n8+j7s9P+oKxzAF7j1Qrg2cvouQaBQxRHEw4S/nTOqFycPqHN9eaS5EdEjDbTwZRdvRl0tOA59InC24zU= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHh2kZNUTGlXQpaIf/Re/UpcoG6gxbpABInNp+oGhB2UtHQIcVHuDq6Y/TXERCv+GxePAkH0S56PO/9VIe4+3eE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_20:50:37 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-20T21:00:10.371 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-20T21:00:10.372 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-20T21:00:10.372 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T21:00:10.373 INFO:teuthology.task.internal:Checking packages... 2024-04-20T21:00:10.395 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-20T21:00:10.395 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T21:00:10.395 INFO:teuthology.packaging:ref: None 2024-04-20T21:00:10.395 INFO:teuthology.packaging:tag: None 2024-04-20T21:00:10.395 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-20T21:00:10.395 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:00:10.396 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-20T21:00:10.535 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-20T21:00:10.536 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T21:00:10.543 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T21:00:10.544 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T21:00:10.673 INFO:teuthology.task.internal:Saving configuration 2024-04-20T21:00:10.697 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T21:00:10.717 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T21:00:10.743 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:43.907839', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAXRcctz8qWnJkfkKzMfepnmRHdg99/wZeDzBv2PXjD1R27OlTS8c9NJ5ZyJ/wD1rLapEjN6LAnt4wyA1i8gbPo='} 2024-04-20T21:00:10.770 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:43.911319', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBII995n8+j7s9P+oKxzAF7j1Qrg2cvouQaBQxRHEw4S/nTOqFycPqHN9eaS5EdEjDbTwZRdvRl0tOA59InC24zU='} 2024-04-20T21:00:10.793 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:43.909793', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHh2kZNUTGlXQpaIf/Re/UpcoG6gxbpABInNp+oGhB2UtHQIcVHuDq6Y/TXERCv+GxePAkH0S56PO/9VIe4+3eE='} 2024-04-20T21:00:10.793 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T21:00:10.799 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-20T21:00:10.799 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-20T21:00:10.799 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-20T21:00:10.799 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T21:00:10.945 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f640800d160>, signals=[15]) 2024-04-20T21:00:10.945 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T21:00:10.951 INFO:teuthology.task.internal:Opening connections... 2024-04-20T21:00:10.952 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2024-04-20T21:00:10.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:00:11.035 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-04-20T21:00:11.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:00:11.110 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-04-20T21:00:11.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:00:11.183 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T21:00:11.190 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-04-20T21:00:11.208 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-04-20T21:00:11.208 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-04-20T21:00:11.265 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2024-04-20T21:00:11.265 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2024-04-20T21:00:11.265 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2024-04-20T21:00:11.265 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2024-04-20T21:00:11.265 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2024-04-20T21:00:11.265 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2024-04-20T21:00:11.265 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T21:00:11.265 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2024-04-20T21:00:11.266 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2024-04-20T21:00:11.266 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T21:00:11.266 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2024-04-20T21:00:11.266 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T21:00:11.266 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T21:00:11.266 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T21:00:11.266 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-04-20T21:00:11.290 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-04-20T21:00:11.308 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-04-20T21:00:11.308 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-04-20T21:00:11.366 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-04-20T21:00:11.366 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-04-20T21:00:11.366 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-04-20T21:00:11.366 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-04-20T21:00:11.366 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-04-20T21:00:11.366 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-04-20T21:00:11.366 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T21:00:11.366 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-04-20T21:00:11.367 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-04-20T21:00:11.367 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T21:00:11.367 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-04-20T21:00:11.367 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T21:00:11.367 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T21:00:11.367 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T21:00:11.367 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-04-20T21:00:11.389 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-04-20T21:00:11.407 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-04-20T21:00:11.408 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T21:00:11.465 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T21:00:11.466 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-04-20T21:00:11.491 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T21:00:11.501 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T21:00:11.559 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T21:00:11.560 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2024-04-20T21:00:11.562 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-04-20T21:00:11.564 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-04-20T21:00:11.580 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T21:00:11.621 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T21:00:11.621 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2024-04-20T21:00:11.623 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-04-20T21:00:11.626 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-04-20T21:00:11.654 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T21:00:11.735 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T21:00:11.749 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-20T21:00:11.749 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-20T21:00:11.749 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-20T21:00:11.750 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T21:00:11.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:00:11.767 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-20T21:00:12.022 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T21:00:12.022 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T21:00:12.023 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-04-20T21:00:12.040 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-437.el9.x86_64 2024-04-20T21:00:12.040 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T21:00:12.041 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T21:00:12.041 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum upgrade -y linux-firmware 2024-04-20T21:00:14.769 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 7.5 MB/s | 8.0 MB 00:01 2024-04-20T21:00:16.936 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-20T21:00:22.438 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.2 MB 00:01 2024-04-20T21:00:24.820 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-20T21:00:26.405 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-20T21:00:31.597 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-04-20T21:00:33.730 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-04-20T21:00:33.731 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-04-20T21:00:33.731 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-04-20T21:00:33.855 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-20T21:00:33.856 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T21:00:34.081 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T21:00:34.082 DEBUG:teuthology.orchestra.run.smithi022:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T21:00:34.085 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi022.front.sepia.ceph.com to complete... 2024-04-20T21:08:49.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20 21:08:49 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T21:08:49.969 INFO:teuthology.orchestra.run.smithi022.stderr:FINISHED --2024-04-20 21:08:49-- 2024-04-20T21:08:49.969 INFO:teuthology.orchestra.run.smithi022.stderr:Total wall clock time: 8m 16s 2024-04-20T21:08:49.969 INFO:teuthology.orchestra.run.smithi022.stderr:Downloaded: 1 files, 341M in 8m 15s (705 KB/s) 2024-04-20T21:08:49.972 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-20T21:08:49.972 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T21:08:52.156 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying... ######################################## 2024-04-20T21:08:52.280 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing... ######################################## 2024-04-20T21:08:52.409 INFO:teuthology.orchestra.run.smithi022.stdout:Updating / installing... 2024-04-20T21:09:00.692 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T21:09:38.518 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T21:09:38.518 DEBUG:teuthology.orchestra.run.smithi022:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T21:09:38.611 INFO:teuthology.orchestra.run.smithi022.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:38.611 INFO:teuthology.orchestra.run.smithi022.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:38.611 INFO:teuthology.orchestra.run.smithi022.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:38.611 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T21:09:38.611 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T21:09:38.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T21:09:38.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T21:09:38.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T21:09:38.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T21:09:38.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T21:09:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T21:09:38.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T21:09:38.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T21:09:38.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T21:09:38.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T21:09:38.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T21:09:38.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T21:09:38.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T21:09:38.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T21:09:38.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T21:09:38.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T21:09:38.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T21:09:38.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T21:09:38.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T21:09:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T21:09:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T21:09:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T21:09:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T21:09:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T21:09:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T21:09:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T21:09:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T21:09:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T21:09:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T21:09:38.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T21:09:38.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T21:09:38.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T21:09:38.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T21:09:38.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T21:09:38.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T21:09:38.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T21:09:38.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T21:09:38.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T21:09:38.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T21:09:38.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T21:09:38.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T21:09:38.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T21:09:38.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T21:09:38.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T21:09:38.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T21:09:38.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T21:09:38.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T21:09:38.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T21:09:38.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T21:09:38.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T21:09:38.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T21:09:38.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T21:09:38.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T21:09:38.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T21:09:38.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T21:09:38.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T21:09:38.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T21:09:38.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T21:09:38.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T21:09:38.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T21:09:38.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T21:09:38.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T21:09:38.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T21:09:38.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T21:09:38.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T21:09:38.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T21:09:38.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T21:09:38.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T21:09:38.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T21:09:38.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T21:09:38.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T21:09:38.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T21:09:38.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T21:09:38.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T21:09:38.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T21:09:38.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T21:09:38.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T21:09:38.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T21:09:38.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T21:09:38.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T21:09:38.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T21:09:38.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T21:09:38.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T21:09:38.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T21:09:38.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T21:09:38.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T21:09:38.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T21:09:38.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T21:09:38.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T21:09:38.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T21:09:38.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T21:09:38.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T21:09:38.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T21:09:38.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T21:09:38.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T21:09:38.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T21:09:38.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T21:09:38.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T21:09:38.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T21:09:38.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T21:09:38.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T21:09:38.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T21:09:38.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T21:09:38.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T21:09:38.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T21:09:38.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T21:09:38.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T21:09:38.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T21:09:38.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T21:09:38.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T21:09:38.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T21:09:38.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T21:09:38.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T21:09:38.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T21:09:38.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T21:09:38.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T21:09:38.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T21:09:38.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T21:09:38.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T21:09:38.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T21:09:38.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T21:09:38.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T21:09:38.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T21:09:38.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T21:09:38.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T21:09:38.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T21:09:38.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T21:09:38.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T21:09:38.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T21:09:38.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T21:09:38.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T21:09:38.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T21:09:38.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T21:09:38.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T21:09:38.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T21:09:38.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T21:09:38.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T21:09:38.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T21:09:38.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T21:09:38.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T21:09:38.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T21:09:38.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T21:09:38.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T21:09:38.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T21:09:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T21:09:38.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T21:09:38.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T21:09:38.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T21:09:38.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T21:09:38.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T21:09:38.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T21:09:38.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T21:09:38.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T21:09:38.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T21:09:38.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T21:09:38.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T21:09:38.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T21:09:38.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T21:09:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T21:09:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T21:09:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T21:09:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T21:09:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T21:09:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T21:09:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T21:09:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T21:09:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T21:09:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T21:09:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T21:09:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T21:09:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T21:09:38.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T21:09:38.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T21:09:38.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T21:09:38.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T21:09:38.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T21:09:38.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T21:09:38.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T21:09:38.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T21:09:38.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T21:09:38.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T21:09:38.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T21:09:38.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T21:09:38.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T21:09:38.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T21:09:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T21:09:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T21:09:38.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T21:09:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T21:09:38.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T21:09:38.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T21:09:38.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T21:09:38.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T21:09:38.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T21:09:38.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T21:09:38.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T21:09:38.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T21:09:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T21:09:38.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T21:09:38.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T21:09:38.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T21:09:38.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T21:09:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T21:09:38.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T21:09:38.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T21:09:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T21:09:38.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T21:09:38.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T21:09:38.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T21:09:38.680 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T21:09:38.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T21:09:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T21:09:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T21:09:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T21:09:38.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T21:09:38.686 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T21:09:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T21:09:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T21:09:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T21:09:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T21:09:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T21:09:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T21:09:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T21:09:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T21:09:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T21:09:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T21:09:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T21:09:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T21:09:38.688 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T21:09:38.689 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T21:09:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T21:09:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T21:09:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T21:09:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T21:09:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T21:09:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T21:09:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T21:09:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T21:09:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T21:09:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T21:09:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T21:09:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T21:09:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T21:09:38.693 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T21:09:38.693 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T21:09:38.693 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T21:09:38.693 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T21:09:38.693 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T21:09:38.693 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T21:09:38.694 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T21:09:38.694 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T21:09:38.694 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T21:09:38.694 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T21:09:38.694 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T21:09:38.694 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T21:09:38.694 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T21:09:38.694 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T21:09:38.694 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T21:09:38.694 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T21:09:38.695 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T21:09:38.695 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T21:09:38.695 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T21:09:38.695 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T21:09:38.695 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T21:09:38.695 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T21:09:38.695 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T21:09:38.695 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T21:09:38.695 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T21:09:38.695 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T21:09:38.695 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T21:09:38.696 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T21:09:38.697 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T21:09:38.698 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T21:09:38.701 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T21:09:38.701 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T21:09:38.702 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T21:09:38.703 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T21:09:38.704 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T21:09:38.705 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T21:09:38.706 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T21:09:38.706 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T21:09:38.706 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T21:09:38.706 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T21:09:38.706 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T21:09:38.706 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T21:09:38.706 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T21:09:38.706 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T21:09:38.706 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T21:09:38.707 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T21:09:38.707 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T21:09:38.707 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T21:09:38.707 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T21:09:38.707 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T21:09:38.707 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T21:09:38.707 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T21:09:38.708 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T21:09:38.708 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T21:09:38.708 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T21:09:38.708 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T21:09:38.708 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T21:09:38.708 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T21:09:38.708 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T21:09:38.709 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T21:09:38.709 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T21:09:38.709 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T21:09:38.709 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T21:09:38.709 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T21:09:38.709 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T21:09:38.709 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T21:09:38.709 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T21:09:38.709 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T21:09:38.709 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T21:09:38.709 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T21:09:38.709 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T21:09:38.710 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T21:09:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T21:09:38.712 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T21:09:38.712 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T21:09:38.712 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T21:09:38.712 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T21:09:38.712 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T21:09:38.712 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T21:09:38.712 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T21:09:38.712 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T21:09:38.712 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T21:09:38.712 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T21:09:38.712 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T21:09:38.712 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T21:09:38.713 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T21:09:38.713 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T21:09:38.713 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T21:09:38.713 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T21:09:38.713 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T21:09:38.713 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T21:09:38.713 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T21:09:38.714 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T21:09:38.714 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T21:09:38.714 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T21:09:38.714 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T21:09:38.714 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T21:09:38.714 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T21:09:38.714 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T21:09:38.714 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T21:09:38.714 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T21:09:38.715 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T21:09:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T21:09:38.717 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T21:09:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T21:09:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T21:09:38.720 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T21:09:38.721 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T21:09:38.722 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T21:09:38.722 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T21:09:38.722 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T21:09:38.722 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T21:09:38.722 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T21:09:38.722 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T21:09:38.722 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T21:09:38.722 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T21:09:38.722 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T21:09:38.722 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T21:09:38.723 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T21:09:38.724 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T21:09:38.725 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T21:09:38.726 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T21:09:38.726 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T21:09:38.726 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T21:09:38.726 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T21:09:38.726 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T21:09:38.726 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T21:09:38.726 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T21:09:38.726 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T21:09:38.726 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T21:09:38.726 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T21:09:38.726 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T21:09:38.727 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T21:09:38.727 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T21:09:38.727 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T21:09:38.727 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T21:09:38.727 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T21:09:38.727 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T21:09:38.727 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T21:09:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T21:09:38.729 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T21:09:38.730 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T21:09:38.731 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T21:09:38.732 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T21:09:38.733 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T21:09:38.734 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T21:09:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T21:09:38.736 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T21:09:38.737 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T21:09:38.738 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T21:09:38.738 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T21:09:38.738 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T21:09:38.738 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T21:09:38.738 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T21:09:38.738 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T21:09:38.738 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T21:09:38.738 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T21:09:38.738 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T21:09:38.738 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T21:09:38.739 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T21:09:38.740 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T21:09:38.741 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T21:09:38.742 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T21:09:38.743 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T21:09:38.744 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T21:09:38.745 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T21:09:38.746 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T21:09:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T21:09:38.748 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T21:09:38.748 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T21:09:38.748 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T21:09:38.748 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T21:09:38.748 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T21:09:38.748 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T21:09:38.748 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T21:09:38.748 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T21:09:38.748 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T21:09:38.748 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T21:09:38.748 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T21:09:38.749 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T21:09:38.749 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T21:09:38.749 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T21:09:38.749 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T21:09:38.749 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T21:09:38.749 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T21:09:38.749 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T21:09:38.749 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T21:09:38.749 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T21:09:38.749 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T21:09:38.749 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T21:09:38.749 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T21:09:38.750 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T21:09:38.751 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T21:09:38.752 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T21:09:38.753 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T21:09:38.753 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T21:09:38.753 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T21:09:38.753 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T21:09:38.753 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T21:09:38.753 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T21:09:38.753 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T21:09:38.753 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T21:09:38.753 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T21:09:38.753 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T21:09:38.753 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T21:09:38.753 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T21:09:38.754 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T21:09:38.755 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T21:09:38.756 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T21:09:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T21:09:38.758 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T21:09:38.759 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T21:09:38.759 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T21:09:38.759 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T21:09:38.759 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T21:09:38.759 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T21:09:38.759 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T21:09:38.763 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:38.763 DEBUG:teuthology.orchestra.run.smithi022:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T21:09:38.787 INFO:teuthology.orchestra.run.smithi022.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T21:09:38.787 INFO:teuthology.orchestra.run.smithi022.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T21:09:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T21:09:38.787 INFO:teuthology.orchestra.run.smithi022.stdout:fi 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout:for file in vmlinuz System.map config; do 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: fi 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout:done 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout:fi 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:38.788 INFO:teuthology.orchestra.run.smithi022.stdout:fi 2024-04-20T21:09:38.789 INFO:teuthology.orchestra.run.smithi022.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T21:09:38.789 INFO:teuthology.orchestra.run.smithi022.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T21:09:38.789 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:38.789 INFO:teuthology.orchestra.run.smithi022.stdout:fi 2024-04-20T21:09:38.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -qi grub2-tools 2024-04-20T21:09:38.860 INFO:teuthology.orchestra.run.smithi022.stdout:Name : grub2-tools 2024-04-20T21:09:38.860 INFO:teuthology.orchestra.run.smithi022.stdout:Epoch : 1 2024-04-20T21:09:38.860 INFO:teuthology.orchestra.run.smithi022.stdout:Version : 2.06 2024-04-20T21:09:38.860 INFO:teuthology.orchestra.run.smithi022.stdout:Release : 68.el9 2024-04-20T21:09:38.860 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: x86_64 2024-04-20T21:09:38.860 INFO:teuthology.orchestra.run.smithi022.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T21:09:38.860 INFO:teuthology.orchestra.run.smithi022.stdout:Group : Unspecified 2024-04-20T21:09:38.861 INFO:teuthology.orchestra.run.smithi022.stdout:Size : 8264254 2024-04-20T21:09:38.861 INFO:teuthology.orchestra.run.smithi022.stdout:License : GPLv3+ 2024-04-20T21:09:38.861 INFO:teuthology.orchestra.run.smithi022.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T21:09:38.861 INFO:teuthology.orchestra.run.smithi022.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T21:09:38.861 INFO:teuthology.orchestra.run.smithi022.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T21:09:38.861 INFO:teuthology.orchestra.run.smithi022.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T21:09:38.861 INFO:teuthology.orchestra.run.smithi022.stdout:Packager : builder@centos.org 2024-04-20T21:09:38.861 INFO:teuthology.orchestra.run.smithi022.stdout:Vendor : CentOS 2024-04-20T21:09:38.861 INFO:teuthology.orchestra.run.smithi022.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T21:09:38.861 INFO:teuthology.orchestra.run.smithi022.stdout:Summary : Support tools for GRUB. 2024-04-20T21:09:38.861 INFO:teuthology.orchestra.run.smithi022.stdout:Description : 2024-04-20T21:09:38.861 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:09:38.862 INFO:teuthology.orchestra.run.smithi022.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T21:09:38.862 INFO:teuthology.orchestra.run.smithi022.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T21:09:38.862 INFO:teuthology.orchestra.run.smithi022.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T21:09:38.862 INFO:teuthology.orchestra.run.smithi022.stdout:hardware devices. 2024-04-20T21:09:38.862 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:09:38.862 INFO:teuthology.orchestra.run.smithi022.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T21:09:38.863 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T21:09:38.863 INFO:teuthology.task.kernel:Updating grub on smithi022 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:38.863 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T21:09:39.501 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2024-04-20T21:09:40.898 INFO:teuthology.orchestra.run.smithi022.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T21:09:40.926 INFO:teuthology.orchestra.run.smithi022.stderr:done 2024-04-20T21:09:40.928 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-04-20T21:09:40.944 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.Pq67Fhl0PO 2024-04-20T21:09:40.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Pq67Fhl0PO 2024-04-20T21:09:41.011 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.Pq67Fhl0PO 2024-04-20T21:09:41.165 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.Pq67Fhl0PO is 6KB 2024-04-20T21:09:41.186 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.Pq67Fhl0PO 2024-04-20T21:09:41.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T21:09:41.263 INFO:teuthology.orchestra.run.smithi022.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T21:09:41.263 INFO:teuthology.orchestra.run.smithi022.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T21:09:41.263 INFO:teuthology.orchestra.run.smithi022.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T21:09:41.264 INFO:teuthology.orchestra.run.smithi022.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T21:09:41.265 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:41.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2024-04-20T21:10:11.405 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-20T21:10:11.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-20T21:10:11.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:10:29.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-04-20T21:10:38.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-20T21:10:38.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:11:38.923 DEBUG:teuthology.orchestra.remote:timed out 2024-04-20T21:11:50.924 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-20T21:11:50.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:11:51.361 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-04-20T21:11:52.040 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-20T21:11:52.041 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T21:11:52.041 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-04-20T21:11:52.096 INFO:teuthology.orchestra.run.smithi022.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T21:11:52.096 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T21:11:52.096 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T21:11:52.096 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T21:11:53.097 DEBUG:teuthology.parallel:result is None 2024-04-20T21:11:53.098 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T21:11:53.105 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T21:11:53.105 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T21:11:53.113 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T21:11:53.116 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T21:11:53.164 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T21:11:53.171 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T21:11:53.177 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T21:11:53.178 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T21:11:53.183 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T21:11:53.185 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T21:11:53.254 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T21:11:53.261 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T21:11:53.261 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T21:11:53.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:11:53.285 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T21:11:53.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:11:53.302 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T21:11:53.309 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T21:11:53.329 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T21:11:53.369 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:11:53.384 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:11:53.403 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:11:53.421 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:11:53.429 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:11:53.483 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:11:53.488 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T21:11:53.499 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T21:11:53.499 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T21:11:53.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T21:11:53.508 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T21:11:53.591 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T21:11:53.600 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T21:11:53.600 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T21:11:53.607 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T21:11:53.610 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T21:11:53.657 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:11:53.742 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:11:53.838 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:11:53.922 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:11:54.014 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:11:54.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T21:11:54.105 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:11:54.149 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:11:54.200 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:11:54.286 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:11:54.331 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-20T21:11:54.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T21:11:54.398 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:11:54.443 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:11:54.498 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:11:54.583 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:11:54.626 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-20T21:11:54.627 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T21:11:54.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2024-04-20T21:11:54.701 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-04-20T21:11:54.704 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-04-20T21:11:54.742 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:11:54.768 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:11:54.814 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:11:55.235 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T21:11:55.243 INFO:teuthology.task.internal:Starting timer... 2024-04-20T21:11:55.243 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T21:11:55.253 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T21:11:55.262 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-20T21:11:55.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2024-04-20T21:11:55.389 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2024-04-20T21:11:55.394 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-04-20T21:11:55.445 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-04-20T21:11:55.447 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-04-20T21:11:55.526 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-04-20T21:11:55.528 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T21:11:55.529 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2024-04-20T21:11:55.602 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2024-04-20T21:11:55.602 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-04-20T21:11:55.626 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-04-20T21:11:55.627 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-04-20T21:11:55.652 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-04-20T21:11:55.653 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi122.front.sepia.ceph.com': 'permissive'} 2024-04-20T21:11:55.653 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:11:55.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:11:55.739 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:11:55.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:11:55.766 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:11:55.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:11:55.793 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T21:11:55.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2024-04-20T21:11:55.863 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-04-20T21:11:55.888 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-04-20T21:11:55.913 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T21:11:55.922 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-20T21:11:55.927 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-20T21:11:56.056 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T21:11:56.074 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T21:11:56.104 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi033.front.sepia.ceph.com,smithi122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T21:14:19.798 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com')] 2024-04-20T21:14:19.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-20T21:14:19.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:14:19.930 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-04-20T21:14:20.089 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-04-20T21:14:20.089 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-20T21:14:20.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:14:20.187 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-04-20T21:14:20.260 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-20T21:14:20.260 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-20T21:14:20.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:14:20.347 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-04-20T21:14:20.424 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-20T21:14:20.424 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T21:14:20.447 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T21:14:20.447 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T21:14:20.448 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:14:20.457 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T21:14:20.457 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:14:20.460 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T21:14:20.460 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:14:20.493 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T21:14:20.496 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T21:14:20.509 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T21:14:20.511 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T21:14:20.536 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-04-20T21:14:20.536 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-04-20T21:14:20.546 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-04-20T21:14:20.547 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-04-20T21:14:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T21:14:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T21:14:20.564 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T21:14:20.572 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T21:14:20.576 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T21:14:20.596 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T21:14:20.627 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:14:20.632 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:14:20.632 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-04-20T21:14:20.632 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:20.632 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:20.632 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:20.632 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:14:20.641 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:14:20.641 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-04-20T21:14:20.641 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:20.641 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:20.641 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:20.641 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:20.681 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2024-04-20T21:14:20.716 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2024-04-20T21:14:20.760 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T21:14:20.805 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T21:14:20.944 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:14:20.952 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:14:20.953 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-04-20T21:14:20.953 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:20.953 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:20.953 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:20.953 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:20.954 INFO:teuthology.run_tasks:Running task install... 2024-04-20T21:14:20.962 DEBUG:teuthology.task.install:project ceph 2024-04-20T21:14:20.962 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-20T21:14:20.962 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-20T21:14:20.963 INFO:teuthology.task.install:Using flavor: default 2024-04-20T21:14:20.970 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T21:14:20.970 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-20T21:14:20.970 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T21:14:20.971 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:14:20.973 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T21:14:20.973 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:14:20.974 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T21:14:20.974 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:14:21.205 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T21:14:21.205 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T21:14:21.220 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T21:14:21.220 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T21:14:21.232 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T21:14:21.233 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T21:14:21.495 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T21:14:21.495 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:14:21.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T21:14:21.583 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T21:14:21.583 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-20T21:14:21.584 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T21:14:21.596 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T21:14:21.597 DEBUG:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T21:14:21.620 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T21:14:21.620 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T21:14:21.687 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T21:14:21.733 DEBUG:teuthology.orchestra.run.smithi022:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T21:14:21.762 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T21:14:21.826 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-04-20T21:14:21.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-04-20T21:14:21.855 DEBUG:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T21:14:21.943 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2024-04-20T21:14:21.950 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum clean all 2024-04-20T21:14:22.061 INFO:teuthology.orchestra.run.smithi033.stdout:48 files removed 2024-04-20T21:14:22.089 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T21:14:22.281 INFO:teuthology.orchestra.run.smithi022.stdout:48 files removed 2024-04-20T21:14:22.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T21:14:22.413 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T21:14:22.414 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-20T21:14:22.414 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T21:14:22.442 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T21:14:22.442 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T21:14:22.510 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T21:14:22.586 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T21:14:22.650 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-04-20T21:14:22.652 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-04-20T21:14:22.880 INFO:teuthology.orchestra.run.smithi122.stdout:48 files removed 2024-04-20T21:14:22.906 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T21:14:23.350 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 85 kB/s | 84 kB 00:00 2024-04-20T21:14:24.069 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 93 kB/s | 84 kB 00:00 2024-04-20T21:14:24.226 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 14 kB/s | 12 kB 00:00 2024-04-20T21:14:24.356 INFO:teuthology.orchestra.run.smithi022.stdout:ceph packages for x86_64 53 kB/s | 84 kB 00:01 2024-04-20T21:14:24.871 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 15 kB/s | 12 kB 00:00 2024-04-20T21:14:25.118 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 2.2 kB/s | 1.9 kB 00:00 2024-04-20T21:14:25.332 INFO:teuthology.orchestra.run.smithi022.stdout:ceph noarch packages 13 kB/s | 12 kB 00:00 2024-04-20T21:14:25.737 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 2.3 kB/s | 1.9 kB 00:00 2024-04-20T21:14:26.315 INFO:teuthology.orchestra.run.smithi022.stdout:ceph source packages 2.1 kB/s | 1.9 kB 00:00 2024-04-20T21:14:27.557 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.0 MB 00:01 2024-04-20T21:14:29.598 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-20T21:14:29.737 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.0 MB 00:03 2024-04-20T21:14:34.109 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 920 kB/s | 8.0 MB 00:08 2024-04-20T21:14:34.232 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.2 MB 00:00 2024-04-20T21:14:36.158 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-04-20T21:14:36.326 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-20T21:14:40.810 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.2 MB 00:00 2024-04-20T21:14:42.470 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-20T21:14:43.840 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-20T21:14:45.853 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:15 2024-04-20T21:14:48.979 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-20T21:14:50.229 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 6.2 MB 00:00 2024-04-20T21:14:51.256 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:14:51.256 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:14:51.268 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T21:14:51.268 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T21:14:51.269 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T21:14:51.270 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T21:14:51.272 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T21:14:51.274 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T21:14:51.275 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T21:14:51.338 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-20T21:14:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-04-20T21:14:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-20T21:14:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-04-20T21:14:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-20T21:14:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T21:14:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T21:14:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:14:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T21:14:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T21:14:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T21:14:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T21:14:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T21:14:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T21:14:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T21:14:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T21:14:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T21:14:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T21:14:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T21:14:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T21:14:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:Install 148 Packages 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 2 Packages 2024-04-20T21:14:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:14:51.368 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 218 M 2024-04-20T21:14:51.368 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-20T21:14:52.564 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-20T21:14:52.672 INFO:teuthology.orchestra.run.smithi033.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-20T21:14:54.230 INFO:teuthology.orchestra.run.smithi033.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 573 kB/s | 893 kB 00:01 2024-04-20T21:14:54.447 INFO:teuthology.orchestra.run.smithi033.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 669 kB/s | 145 kB 00:00 2024-04-20T21:14:54.507 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-20T21:14:55.296 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:18 2024-04-20T21:14:56.189 INFO:teuthology.orchestra.run.smithi033.stdout:(4/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 1.4 MB/s | 2.4 MB 00:01 2024-04-20T21:14:57.230 INFO:teuthology.orchestra.run.smithi033.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 1.4 MB/s | 1.5 MB 00:01 2024-04-20T21:14:58.472 INFO:teuthology.orchestra.run.smithi033.stdout:(6/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 877 kB/s | 5.4 MB 00:06 2024-04-20T21:14:59.540 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-20T21:15:00.747 INFO:teuthology.orchestra.run.smithi033.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 1.3 MB/s | 4.7 MB 00:03 2024-04-20T21:15:01.440 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-20T21:15:01.716 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:15:01.717 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:15:01.728 INFO:teuthology.orchestra.run.smithi122.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T21:15:01.729 INFO:teuthology.orchestra.run.smithi122.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T21:15:01.730 INFO:teuthology.orchestra.run.smithi122.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T21:15:01.732 INFO:teuthology.orchestra.run.smithi122.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T21:15:01.732 INFO:teuthology.orchestra.run.smithi122.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T21:15:01.734 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T21:15:01.735 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T21:15:01.795 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-04-20T21:15:01.812 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-04-20T21:15:01.812 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T21:15:01.813 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T21:15:01.814 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T21:15:01.815 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T21:15:01.815 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T21:15:01.815 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T21:15:01.815 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T21:15:01.815 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T21:15:01.815 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T21:15:01.815 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T21:15:01.816 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T21:15:01.816 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T21:15:01.816 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:15:01.816 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T21:15:01.816 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T21:15:01.816 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-04-20T21:15:01.816 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T21:15:01.816 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:15:01.816 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-04-20T21:15:01.817 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T21:15:01.817 INFO:teuthology.orchestra.run.smithi122.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T21:15:01.817 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T21:15:01.817 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T21:15:01.817 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T21:15:01.817 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T21:15:01.817 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T21:15:01.817 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T21:15:01.817 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T21:15:01.817 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T21:15:01.818 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T21:15:01.818 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T21:15:01.818 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T21:15:01.818 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T21:15:01.818 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T21:15:01.818 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T21:15:01.818 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T21:15:01.818 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T21:15:01.818 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T21:15:01.818 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T21:15:01.818 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T21:15:01.819 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T21:15:01.819 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T21:15:01.819 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T21:15:01.819 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T21:15:01.819 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T21:15:01.819 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T21:15:01.819 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:15:01.819 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T21:15:01.819 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T21:15:01.819 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T21:15:01.820 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T21:15:01.821 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T21:15:01.822 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T21:15:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T21:15:01.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:15:01.825 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-04-20T21:15:01.826 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-04-20T21:15:01.826 INFO:teuthology.orchestra.run.smithi122.stdout:Install 148 Packages 2024-04-20T21:15:01.826 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 2 Packages 2024-04-20T21:15:01.826 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:15:01.826 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 218 M 2024-04-20T21:15:01.827 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-04-20T21:15:02.934 INFO:teuthology.orchestra.run.smithi122.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-20T21:15:04.012 INFO:teuthology.orchestra.run.smithi022.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:15:04.014 INFO:teuthology.orchestra.run.smithi022.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:15:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T21:15:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T21:15:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T21:15:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T21:15:04.041 INFO:teuthology.orchestra.run.smithi022.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T21:15:04.044 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T21:15:04.045 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T21:15:04.120 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-04-20T21:15:04.136 INFO:teuthology.orchestra.run.smithi022.stdout:======================================================================================= 2024-04-20T21:15:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-04-20T21:15:04.136 INFO:teuthology.orchestra.run.smithi022.stdout:======================================================================================= 2024-04-20T21:15:04.136 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-04-20T21:15:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T21:15:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T21:15:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:15:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T21:15:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T21:15:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T21:15:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T21:15:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T21:15:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T21:15:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T21:15:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T21:15:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T21:15:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T21:15:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T21:15:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T21:15:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T21:15:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T21:15:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T21:15:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T21:15:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T21:15:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T21:15:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T21:15:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T21:15:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T21:15:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T21:15:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T21:15:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T21:15:04.147 INFO:teuthology.orchestra.run.smithi022.stdout:Installing weak dependencies: 2024-04-20T21:15:04.148 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T21:15:04.148 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:15:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-04-20T21:15:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:======================================================================================= 2024-04-20T21:15:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:Install 148 Packages 2024-04-20T21:15:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 2 Packages 2024-04-20T21:15:04.148 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:15:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 218 M 2024-04-20T21:15:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-04-20T21:15:04.726 INFO:teuthology.orchestra.run.smithi122.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 498 kB/s | 893 kB 00:01 2024-04-20T21:15:04.976 INFO:teuthology.orchestra.run.smithi122.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 580 kB/s | 145 kB 00:00 2024-04-20T21:15:05.508 INFO:teuthology.orchestra.run.smithi022.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-04-20T21:15:07.467 INFO:teuthology.orchestra.run.smithi122.stdout:(4/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 970 kB/s | 2.4 MB 00:02 2024-04-20T21:15:07.855 INFO:teuthology.orchestra.run.smithi033.stdout:(8/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 1.4 MB/s | 21 MB 00:15 2024-04-20T21:15:07.947 INFO:teuthology.orchestra.run.smithi033.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 275 kB/s | 25 kB 00:00 2024-04-20T21:15:08.083 INFO:teuthology.orchestra.run.smithi022.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 347 kB/s | 893 kB 00:02 2024-04-20T21:15:08.602 INFO:teuthology.orchestra.run.smithi022.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 280 kB/s | 145 kB 00:00 2024-04-20T21:15:08.876 INFO:teuthology.orchestra.run.smithi122.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 1.1 MB/s | 1.5 MB 00:01 2024-04-20T21:15:11.786 INFO:teuthology.orchestra.run.smithi022.stdout:(4/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 803 kB/s | 5.4 MB 00:06 2024-04-20T21:15:13.043 INFO:teuthology.orchestra.run.smithi122.stdout:(6/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 1.1 MB/s | 4.7 MB 00:04 2024-04-20T21:15:13.236 INFO:teuthology.orchestra.run.smithi022.stdout:(5/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 522 kB/s | 2.4 MB 00:04 2024-04-20T21:15:13.486 INFO:teuthology.orchestra.run.smithi022.stdout:(6/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 903 kB/s | 1.5 MB 00:01 2024-04-20T21:15:15.246 INFO:teuthology.orchestra.run.smithi033.stdout:(10/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 748 kB/s | 11 MB 00:14 2024-04-20T21:15:15.738 INFO:teuthology.orchestra.run.smithi033.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 457 kB/s | 225 kB 00:00 2024-04-20T21:15:15.888 INFO:teuthology.orchestra.run.smithi033.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 180 kB/s | 27 kB 00:00 2024-04-20T21:15:16.030 INFO:teuthology.orchestra.run.smithi033.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 224 kB/s | 32 kB 00:00 2024-04-20T21:15:16.093 INFO:teuthology.orchestra.run.smithi122.stdout:(7/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 406 kB/s | 5.4 MB 00:13 2024-04-20T21:15:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 389 kB/s | 748 kB 00:01 2024-04-20T21:15:18.347 INFO:teuthology.orchestra.run.smithi033.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 417 kB/s | 163 kB 00:00 2024-04-20T21:15:18.639 INFO:teuthology.orchestra.run.smithi033.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 434 kB/s | 126 kB 00:00 2024-04-20T21:15:19.122 INFO:teuthology.orchestra.run.smithi033.stdout:(17/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 835 kB/s | 17 MB 00:20 2024-04-20T21:15:19.531 INFO:teuthology.orchestra.run.smithi033.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 561 kB/s | 501 kB 00:00 2024-04-20T21:15:19.648 INFO:teuthology.orchestra.run.smithi033.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 388 kB/s | 45 kB 00:00 2024-04-20T21:15:19.948 INFO:teuthology.orchestra.run.smithi033.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 449 kB/s | 135 kB 00:00 2024-04-20T21:15:20.748 INFO:teuthology.orchestra.run.smithi033.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 203 kB/s | 162 kB 00:00 2024-04-20T21:15:21.507 INFO:teuthology.orchestra.run.smithi033.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 423 kB/s | 321 kB 00:00 2024-04-20T21:15:22.215 INFO:teuthology.orchestra.run.smithi033.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 422 kB/s | 299 kB 00:00 2024-04-20T21:15:22.377 INFO:teuthology.orchestra.run.smithi022.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 526 kB/s | 4.7 MB 00:09 2024-04-20T21:15:22.857 INFO:teuthology.orchestra.run.smithi033.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 155 kB/s | 99 kB 00:00 2024-04-20T21:15:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 284 kB/s | 85 kB 00:00 2024-04-20T21:15:29.924 INFO:teuthology.orchestra.run.smithi033.stdout:(26/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 509 kB/s | 5.4 MB 00:10 2024-04-20T21:15:30.258 INFO:teuthology.orchestra.run.smithi033.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 511 kB/s | 170 kB 00:00 2024-04-20T21:15:30.358 INFO:teuthology.orchestra.run.smithi033.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 342 kB/s | 34 kB 00:00 2024-04-20T21:15:30.608 INFO:teuthology.orchestra.run.smithi033.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 569 kB/s | 142 kB 00:00 2024-04-20T21:15:32.333 INFO:teuthology.orchestra.run.smithi033.stdout:(30/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 345 kB/s | 3.1 MB 00:09 2024-04-20T21:15:33.942 INFO:teuthology.orchestra.run.smithi122.stdout:(8/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x86 825 kB/s | 17 MB 00:20 2024-04-20T21:15:34.067 INFO:teuthology.orchestra.run.smithi122.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 202 kB/s | 25 kB 00:00 2024-04-20T21:15:34.693 INFO:teuthology.orchestra.run.smithi122.stdout:(10/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 583 kB/s | 11 MB 00:18 2024-04-20T21:15:36.218 INFO:teuthology.orchestra.run.smithi122.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 147 kB/s | 225 kB 00:01 2024-04-20T21:15:36.426 INFO:teuthology.orchestra.run.smithi122.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 130 kB/s | 27 kB 00:00 2024-04-20T21:15:36.627 INFO:teuthology.orchestra.run.smithi122.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 159 kB/s | 32 kB 00:00 2024-04-20T21:15:38.008 INFO:teuthology.orchestra.run.smithi033.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 490 kB/s | 3.5 MB 00:07 2024-04-20T21:15:38.833 INFO:teuthology.orchestra.run.smithi033.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 302 kB/s | 249 kB 00:00 2024-04-20T21:15:39.016 INFO:teuthology.orchestra.run.smithi033.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 269 kB/s | 49 kB 00:00 2024-04-20T21:15:39.360 INFO:teuthology.orchestra.run.smithi122.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 274 kB/s | 748 kB 00:02 2024-04-20T21:15:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 47 kB/s | 16 kB 00:00 2024-04-20T21:15:39.886 INFO:teuthology.orchestra.run.smithi022.stdout:(8/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x86 653 kB/s | 17 MB 00:26 2024-04-20T21:15:39.960 INFO:teuthology.orchestra.run.smithi122.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 272 kB/s | 163 kB 00:00 2024-04-20T21:15:40.067 INFO:teuthology.orchestra.run.smithi022.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 138 kB/s | 25 kB 00:00 2024-04-20T21:15:40.367 INFO:teuthology.orchestra.run.smithi033.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 273 kB/s | 273 kB 00:00 2024-04-20T21:15:40.504 INFO:teuthology.orchestra.run.smithi122.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 234 kB/s | 126 kB 00:00 2024-04-20T21:15:42.327 INFO:teuthology.orchestra.run.smithi122.stdout:(17/150): libradosstriper1-19.0.0-3133.ga8c84f5 274 kB/s | 501 kB 00:01 2024-04-20T21:15:42.991 INFO:teuthology.orchestra.run.smithi033.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 291 kB/s | 763 kB 00:02 2024-04-20T21:15:43.550 INFO:teuthology.orchestra.run.smithi033.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 33 kB/s | 18 kB 00:00 2024-04-20T21:15:43.667 INFO:teuthology.orchestra.run.smithi033.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 689 kB/s | 80 kB 00:00 2024-04-20T21:15:43.776 INFO:teuthology.orchestra.run.smithi033.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2024-04-20T21:15:43.818 INFO:teuthology.orchestra.run.smithi033.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-20T21:15:43.859 INFO:teuthology.orchestra.run.smithi033.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-20T21:15:43.935 INFO:teuthology.orchestra.run.smithi033.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-20T21:15:43.969 INFO:teuthology.orchestra.run.smithi033.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-20T21:15:44.010 INFO:teuthology.orchestra.run.smithi033.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-20T21:15:44.052 INFO:teuthology.orchestra.run.smithi033.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-20T21:15:44.094 INFO:teuthology.orchestra.run.smithi033.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-20T21:15:44.169 INFO:teuthology.orchestra.run.smithi122.stdout:(18/150): ceph-common-19.0.0-3133.ga8c84f59.el9 520 kB/s | 21 MB 00:41 2024-04-20T21:15:44.370 INFO:teuthology.orchestra.run.smithi122.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 226 kB/s | 45 kB 00:00 2024-04-20T21:15:44.478 INFO:teuthology.orchestra.run.smithi033.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-04-20T21:15:44.870 INFO:teuthology.orchestra.run.smithi122.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 269 kB/s | 135 kB 00:00 2024-04-20T21:15:44.986 INFO:teuthology.orchestra.run.smithi033.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 943 kB 00:00 2024-04-20T21:15:45.270 INFO:teuthology.orchestra.run.smithi033.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 378 kB/s | 107 kB 00:00 2024-04-20T21:15:45.445 INFO:teuthology.orchestra.run.smithi122.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 282 kB/s | 162 kB 00:00 2024-04-20T21:15:45.587 INFO:teuthology.orchestra.run.smithi033.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 68 kB/s | 21 kB 00:00 2024-04-20T21:15:45.974 INFO:teuthology.orchestra.run.smithi022.stdout:(10/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 460 kB/s | 11 MB 00:23 2024-04-20T21:15:46.070 INFO:teuthology.orchestra.run.smithi033.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 649 kB/s | 313 kB 00:00 2024-04-20T21:15:46.321 INFO:teuthology.orchestra.run.smithi033.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 131 kB/s | 33 kB 00:00 2024-04-20T21:15:46.741 INFO:teuthology.orchestra.run.smithi022.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 293 kB/s | 225 kB 00:00 2024-04-20T21:15:46.855 INFO:teuthology.orchestra.run.smithi033.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.6 MB/s | 3.0 MB 00:00 2024-04-20T21:15:46.945 INFO:teuthology.orchestra.run.smithi122.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 214 kB/s | 321 kB 00:01 2024-04-20T21:15:46.959 INFO:teuthology.orchestra.run.smithi022.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 125 kB/s | 27 kB 00:00 2024-04-20T21:15:47.063 INFO:teuthology.orchestra.run.smithi033.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 87 kB/s | 18 kB 00:00 2024-04-20T21:15:47.322 INFO:teuthology.orchestra.run.smithi033.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 43 kB/s | 11 kB 00:00 2024-04-20T21:15:47.402 INFO:teuthology.orchestra.run.smithi022.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 72 kB/s | 32 kB 00:00 2024-04-20T21:15:47.655 INFO:teuthology.orchestra.run.smithi033.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 481 kB/s | 160 kB 00:00 2024-04-20T21:15:47.931 INFO:teuthology.orchestra.run.smithi033.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 165 kB/s | 45 kB 00:00 2024-04-20T21:15:48.312 INFO:teuthology.orchestra.run.smithi122.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 218 kB/s | 299 kB 00:01 2024-04-20T21:15:48.347 INFO:teuthology.orchestra.run.smithi033.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 1.6 MB/s | 662 kB 00:00 2024-04-20T21:15:48.746 INFO:teuthology.orchestra.run.smithi122.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 230 kB/s | 99 kB 00:00 2024-04-20T21:15:48.931 INFO:teuthology.orchestra.run.smithi033.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 998 kB/s | 582 kB 00:00 2024-04-20T21:15:49.096 INFO:teuthology.orchestra.run.smithi122.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 243 kB/s | 85 kB 00:00 2024-04-20T21:15:49.198 INFO:teuthology.orchestra.run.smithi033.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 95 kB/s | 25 kB 00:00 2024-04-20T21:15:49.548 INFO:teuthology.orchestra.run.smithi033.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 136 kB/s | 47 kB 00:00 2024-04-20T21:15:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 798 kB/s | 292 kB 00:00 2024-04-20T21:15:50.232 INFO:teuthology.orchestra.run.smithi033.stdout:(63/150): ceph-mgr-diskprediction-local-19.0.0- 424 kB/s | 7.4 MB 00:17 2024-04-20T21:15:50.324 INFO:teuthology.orchestra.run.smithi033.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 462 kB/s | 188 kB 00:00 2024-04-20T21:15:50.350 INFO:teuthology.orchestra.run.smithi022.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 254 kB/s | 748 kB 00:02 2024-04-20T21:15:50.466 INFO:teuthology.orchestra.run.smithi033.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 151 kB/s | 35 kB 00:00 2024-04-20T21:15:51.008 INFO:teuthology.orchestra.run.smithi033.stdout:(66/150): openblas-openmp-0.3.21-2.el9.x86_64.r 6.9 MB/s | 4.7 MB 00:00 2024-04-20T21:15:51.084 INFO:teuthology.orchestra.run.smithi022.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 223 kB/s | 163 kB 00:00 2024-04-20T21:15:51.516 INFO:teuthology.orchestra.run.smithi033.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 3.1 MB/s | 1.6 MB 00:00 2024-04-20T21:15:51.583 INFO:teuthology.orchestra.run.smithi033.stdout:(68/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 3.7 MB/s | 4.1 MB 00:01 2024-04-20T21:15:51.684 INFO:teuthology.orchestra.run.smithi022.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 211 kB/s | 126 kB 00:00 2024-04-20T21:15:51.933 INFO:teuthology.orchestra.run.smithi033.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 412 kB/s | 133 kB 00:00 2024-04-20T21:15:52.034 INFO:teuthology.orchestra.run.smithi033.stdout:(70/150): postgresql-contrib-13.14-1.el9.x86_64 1.7 MB/s | 911 kB 00:00 2024-04-20T21:15:52.559 INFO:teuthology.orchestra.run.smithi033.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 1.9 MB/s | 1.0 MB 00:00 2024-04-20T21:15:52.901 INFO:teuthology.orchestra.run.smithi033.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 256 kB/s | 87 kB 00:00 2024-04-20T21:15:53.043 INFO:teuthology.orchestra.run.smithi033.stdout:(73/150): postgresql-server-13.14-1.el9.x86_64. 5.2 MB/s | 5.8 MB 00:01 2024-04-20T21:15:53.292 INFO:teuthology.orchestra.run.smithi022.stdout:(17/150): libradosstriper1-19.0.0-3133.ga8c84f5 311 kB/s | 501 kB 00:01 2024-04-20T21:15:53.376 INFO:teuthology.orchestra.run.smithi033.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 736 kB/s | 244 kB 00:00 2024-04-20T21:15:53.677 INFO:teuthology.orchestra.run.smithi033.stdout:(75/150): python3-babel-2.9.1-2.el9.noarch.rpm 7.7 MB/s | 6.0 MB 00:00 2024-04-20T21:15:53.827 INFO:teuthology.orchestra.run.smithi033.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 551 kB/s | 248 kB 00:00 2024-04-20T21:15:53.919 INFO:teuthology.orchestra.run.smithi033.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 106 kB/s | 26 kB 00:00 2024-04-20T21:15:54.036 INFO:teuthology.orchestra.run.smithi033.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 92 kB/s | 19 kB 00:00 2024-04-20T21:15:54.219 INFO:teuthology.orchestra.run.smithi033.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 574 kB/s | 172 kB 00:00 2024-04-20T21:15:54.278 INFO:teuthology.orchestra.run.smithi033.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 144 kB/s | 35 kB 00:00 2024-04-20T21:15:54.620 INFO:teuthology.orchestra.run.smithi033.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 792 kB/s | 250 kB 00:00 2024-04-20T21:15:54.945 INFO:teuthology.orchestra.run.smithi033.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 684 kB/s | 222 kB 00:00 2024-04-20T21:15:55.012 INFO:teuthology.orchestra.run.smithi033.stdout:(83/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.4 MB/s | 5.1 MB 00:00 2024-04-20T21:15:55.246 INFO:teuthology.orchestra.run.smithi033.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 257 kB/s | 77 kB 00:00 2024-04-20T21:15:55.304 INFO:teuthology.orchestra.run.smithi033.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 159 kB/s | 42 kB 00:00 2024-04-20T21:15:55.601 INFO:teuthology.orchestra.run.smithi022.stdout:(18/150): ceph-common-19.0.0-3133.ga8c84f59.el9 427 kB/s | 21 MB 00:50 2024-04-20T21:15:55.654 INFO:teuthology.orchestra.run.smithi033.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 659 kB/s | 269 kB 00:00 2024-04-20T21:15:55.680 INFO:teuthology.orchestra.run.smithi033.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 454 kB/s | 159 kB 00:00 2024-04-20T21:15:55.834 INFO:teuthology.orchestra.run.smithi022.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 194 kB/s | 45 kB 00:00 2024-04-20T21:15:55.955 INFO:teuthology.orchestra.run.smithi033.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 186 kB/s | 51 kB 00:00 2024-04-20T21:15:56.055 INFO:teuthology.orchestra.run.smithi033.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 696 kB/s | 279 kB 00:00 2024-04-20T21:15:56.239 INFO:teuthology.orchestra.run.smithi033.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 189 kB/s | 54 kB 00:00 2024-04-20T21:15:56.484 INFO:teuthology.orchestra.run.smithi022.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 207 kB/s | 135 kB 00:00 2024-04-20T21:15:56.548 INFO:teuthology.orchestra.run.smithi033.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 135 kB/s | 42 kB 00:00 2024-04-20T21:15:56.646 INFO:teuthology.orchestra.run.smithi122.stdout:(26/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 384 kB/s | 5.4 MB 00:14 2024-04-20T21:15:56.865 INFO:teuthology.orchestra.run.smithi033.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 139 kB/s | 44 kB 00:00 2024-04-20T21:15:56.968 INFO:teuthology.orchestra.run.smithi022.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 336 kB/s | 162 kB 00:00 2024-04-20T21:15:57.054 INFO:teuthology.orchestra.run.smithi122.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 417 kB/s | 170 kB 00:00 2024-04-20T21:15:57.189 INFO:teuthology.orchestra.run.smithi033.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 680 kB/s | 221 kB 00:00 2024-04-20T21:15:57.398 INFO:teuthology.orchestra.run.smithi033.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 172 kB/s | 36 kB 00:00 2024-04-20T21:15:57.405 INFO:teuthology.orchestra.run.smithi122.stdout:(28/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 381 kB/s | 3.1 MB 00:08 2024-04-20T21:15:57.649 INFO:teuthology.orchestra.run.smithi033.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 190 kB/s | 47 kB 00:00 2024-04-20T21:15:57.663 INFO:teuthology.orchestra.run.smithi122.stdout:(29/150): ceph-grafana-dashboards-19.0.0-3133.g 56 kB/s | 34 kB 00:00 2024-04-20T21:15:57.797 INFO:teuthology.orchestra.run.smithi122.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 363 kB/s | 142 kB 00:00 2024-04-20T21:15:57.815 INFO:teuthology.orchestra.run.smithi033.stdout:(96/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.2 MB/s | 16 MB 00:01 2024-04-20T21:15:57.835 INFO:teuthology.orchestra.run.smithi022.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 371 kB/s | 321 kB 00:00 2024-04-20T21:15:58.141 INFO:teuthology.orchestra.run.smithi033.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 194 kB/s | 58 kB 00:00 2024-04-20T21:15:58.166 INFO:teuthology.orchestra.run.smithi033.stdout:(98/150): socat-1.7.4.1-5.el9.x86_64.rpm 590 kB/s | 305 kB 00:00 2024-04-20T21:15:58.441 INFO:teuthology.orchestra.run.smithi033.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 232 kB/s | 64 kB 00:00 2024-04-20T21:15:58.466 INFO:teuthology.orchestra.run.smithi033.stdout:(100/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 288 kB/s | 94 kB 00:00 2024-04-20T21:15:58.585 INFO:teuthology.orchestra.run.smithi022.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 399 kB/s | 299 kB 00:00 2024-04-20T21:15:58.591 INFO:teuthology.orchestra.run.smithi033.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 179 kB/s | 22 kB 00:00 2024-04-20T21:15:58.758 INFO:teuthology.orchestra.run.smithi033.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 141 kB/s | 45 kB 00:00 2024-04-20T21:15:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-04-20T21:15:58.944 INFO:teuthology.orchestra.run.smithi022.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 278 kB/s | 99 kB 00:00 2024-04-20T21:15:59.184 INFO:teuthology.orchestra.run.smithi033.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 806 kB/s | 309 kB 00:00 2024-04-20T21:15:59.218 INFO:teuthology.orchestra.run.smithi033.stdout:(105/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-20T21:15:59.276 INFO:teuthology.orchestra.run.smithi033.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-04-20T21:15:59.360 INFO:teuthology.orchestra.run.smithi033.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-04-20T21:15:59.418 INFO:teuthology.orchestra.run.smithi033.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 834 kB/s | 49 kB 00:00 2024-04-20T21:15:59.485 INFO:teuthology.orchestra.run.smithi033.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-20T21:15:59.561 INFO:teuthology.orchestra.run.smithi033.stdout:(110/150): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-20T21:15:59.586 INFO:teuthology.orchestra.run.smithi033.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-20T21:15:59.645 INFO:teuthology.orchestra.run.smithi033.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-20T21:15:59.695 INFO:teuthology.orchestra.run.smithi033.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 869 kB/s | 43 kB 00:00 2024-04-20T21:15:59.737 INFO:teuthology.orchestra.run.smithi033.stdout:(114/150): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-04-20T21:15:59.762 INFO:teuthology.orchestra.run.smithi033.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-04-20T21:15:59.796 INFO:teuthology.orchestra.run.smithi033.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-20T21:15:59.810 INFO:teuthology.orchestra.run.smithi022.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 98 kB/s | 85 kB 00:00 2024-04-20T21:15:59.821 INFO:teuthology.orchestra.run.smithi033.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-20T21:15:59.880 INFO:teuthology.orchestra.run.smithi033.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-20T21:15:59.905 INFO:teuthology.orchestra.run.smithi033.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-20T21:15:59.955 INFO:teuthology.orchestra.run.smithi033.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 470 kB/s | 35 kB 00:00 2024-04-20T21:15:59.981 INFO:teuthology.orchestra.run.smithi033.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-04-20T21:16:00.039 INFO:teuthology.orchestra.run.smithi033.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-20T21:16:00.098 INFO:teuthology.orchestra.run.smithi033.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-04-20T21:16:00.156 INFO:teuthology.orchestra.run.smithi033.stdout:(124/150): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-04-20T21:16:00.182 INFO:teuthology.orchestra.run.smithi033.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-04-20T21:16:00.215 INFO:teuthology.orchestra.run.smithi033.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-04-20T21:16:00.240 INFO:teuthology.orchestra.run.smithi033.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-04-20T21:16:00.274 INFO:teuthology.orchestra.run.smithi033.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-04-20T21:16:00.341 INFO:teuthology.orchestra.run.smithi033.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-20T21:16:00.366 INFO:teuthology.orchestra.run.smithi033.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 504 kB/s | 46 kB 00:00 2024-04-20T21:16:00.408 INFO:teuthology.orchestra.run.smithi033.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-20T21:16:00.433 INFO:teuthology.orchestra.run.smithi033.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 865 kB/s | 58 kB 00:00 2024-04-20T21:16:00.484 INFO:teuthology.orchestra.run.smithi033.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-20T21:16:00.509 INFO:teuthology.orchestra.run.smithi033.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-20T21:16:00.543 INFO:teuthology.orchestra.run.smithi033.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-20T21:16:00.568 INFO:teuthology.orchestra.run.smithi033.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 483 kB/s | 28 kB 00:00 2024-04-20T21:16:00.601 INFO:teuthology.orchestra.run.smithi033.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 527 kB/s | 31 kB 00:00 2024-04-20T21:16:00.627 INFO:teuthology.orchestra.run.smithi033.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-04-20T21:16:00.660 INFO:teuthology.orchestra.run.smithi033.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-20T21:16:00.686 INFO:teuthology.orchestra.run.smithi033.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-04-20T21:16:00.719 INFO:teuthology.orchestra.run.smithi033.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 870 kB/s | 51 kB 00:00 2024-04-20T21:16:00.745 INFO:teuthology.orchestra.run.smithi033.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-20T21:16:00.778 INFO:teuthology.orchestra.run.smithi033.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-20T21:16:00.812 INFO:teuthology.orchestra.run.smithi033.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-04-20T21:16:00.837 INFO:teuthology.orchestra.run.smithi033.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-04-20T21:16:00.879 INFO:teuthology.orchestra.run.smithi033.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-20T21:16:00.938 INFO:teuthology.orchestra.run.smithi033.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-20T21:16:07.455 INFO:teuthology.orchestra.run.smithi122.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 370 kB/s | 3.5 MB 00:09 2024-04-20T21:16:07.947 INFO:teuthology.orchestra.run.smithi122.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 506 kB/s | 249 kB 00:00 2024-04-20T21:16:08.064 INFO:teuthology.orchestra.run.smithi122.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 422 kB/s | 49 kB 00:00 2024-04-20T21:16:08.189 INFO:teuthology.orchestra.run.smithi122.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 132 kB/s | 16 kB 00:00 2024-04-20T21:16:08.714 INFO:teuthology.orchestra.run.smithi122.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 520 kB/s | 273 kB 00:00 2024-04-20T21:16:08.734 INFO:teuthology.orchestra.run.smithi022.stdout:(26/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 356 kB/s | 5.4 MB 00:15 2024-04-20T21:16:09.276 INFO:teuthology.orchestra.run.smithi022.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 315 kB/s | 170 kB 00:00 2024-04-20T21:16:09.402 INFO:teuthology.orchestra.run.smithi022.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 275 kB/s | 34 kB 00:00 2024-04-20T21:16:10.093 INFO:teuthology.orchestra.run.smithi022.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 206 kB/s | 142 kB 00:00 2024-04-20T21:16:10.323 INFO:teuthology.orchestra.run.smithi122.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 474 kB/s | 763 kB 00:01 2024-04-20T21:16:10.448 INFO:teuthology.orchestra.run.smithi122.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 146 kB/s | 18 kB 00:00 2024-04-20T21:16:10.690 INFO:teuthology.orchestra.run.smithi122.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 332 kB/s | 80 kB 00:00 2024-04-20T21:16:10.832 INFO:teuthology.orchestra.run.smithi122.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 5.5 MB/s | 803 kB 00:00 2024-04-20T21:16:10.899 INFO:teuthology.orchestra.run.smithi122.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-20T21:16:10.975 INFO:teuthology.orchestra.run.smithi122.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-20T21:16:11.108 INFO:teuthology.orchestra.run.smithi122.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-04-20T21:16:11.167 INFO:teuthology.orchestra.run.smithi122.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-20T21:16:11.234 INFO:teuthology.orchestra.run.smithi122.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-20T21:16:11.293 INFO:teuthology.orchestra.run.smithi122.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-20T21:16:11.393 INFO:teuthology.orchestra.run.smithi122.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-20T21:16:11.643 INFO:teuthology.orchestra.run.smithi122.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2024-04-20T21:16:11.769 INFO:teuthology.orchestra.run.smithi122.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 7.3 MB/s | 943 kB 00:00 2024-04-20T21:16:11.861 INFO:teuthology.orchestra.run.smithi122.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-20T21:16:11.920 INFO:teuthology.orchestra.run.smithi122.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 370 kB/s | 21 kB 00:00 2024-04-20T21:16:11.978 INFO:teuthology.orchestra.run.smithi122.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-20T21:16:12.037 INFO:teuthology.orchestra.run.smithi122.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 561 kB/s | 33 kB 00:00 2024-04-20T21:16:12.196 INFO:teuthology.orchestra.run.smithi122.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-20T21:16:12.254 INFO:teuthology.orchestra.run.smithi122.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-04-20T21:16:12.314 INFO:teuthology.orchestra.run.smithi122.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-04-20T21:16:12.371 INFO:teuthology.orchestra.run.smithi122.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-20T21:16:12.430 INFO:teuthology.orchestra.run.smithi122.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-04-20T21:16:12.530 INFO:teuthology.orchestra.run.smithi122.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-04-20T21:16:12.631 INFO:teuthology.orchestra.run.smithi122.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-04-20T21:16:12.706 INFO:teuthology.orchestra.run.smithi122.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 339 kB/s | 25 kB 00:00 2024-04-20T21:16:12.773 INFO:teuthology.orchestra.run.smithi122.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-04-20T21:16:12.832 INFO:teuthology.orchestra.run.smithi122.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-20T21:16:12.891 INFO:teuthology.orchestra.run.smithi122.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-20T21:16:12.941 INFO:teuthology.orchestra.run.smithi122.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-04-20T21:16:12.961 INFO:teuthology.orchestra.run.smithi033.stdout:(148/150): librados2-19.0.0-3133.ga8c84f59.el9. 288 kB/s | 3.4 MB 00:12 2024-04-20T21:16:13.085 INFO:teuthology.orchestra.run.smithi022.stdout:(30/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 238 kB/s | 3.1 MB 00:13 2024-04-20T21:16:13.103 INFO:teuthology.orchestra.run.smithi033.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 264 kB/s | 3.1 MB 00:12 2024-04-20T21:16:13.174 INFO:teuthology.orchestra.run.smithi122.stdout:(65/150): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-04-20T21:16:13.358 INFO:teuthology.orchestra.run.smithi122.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 23 MB/s | 4.1 MB 00:00 2024-04-20T21:16:13.467 INFO:teuthology.orchestra.run.smithi122.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-20T21:16:13.543 INFO:teuthology.orchestra.run.smithi122.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-20T21:16:13.601 INFO:teuthology.orchestra.run.smithi122.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-20T21:16:13.818 INFO:teuthology.orchestra.run.smithi122.stdout:(70/150): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-04-20T21:16:13.893 INFO:teuthology.orchestra.run.smithi122.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-20T21:16:13.953 INFO:teuthology.orchestra.run.smithi122.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-20T21:16:14.203 INFO:teuthology.orchestra.run.smithi122.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2024-04-20T21:16:14.262 INFO:teuthology.orchestra.run.smithi122.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-20T21:16:14.320 INFO:teuthology.orchestra.run.smithi122.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-20T21:16:14.371 INFO:teuthology.orchestra.run.smithi122.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-04-20T21:16:14.429 INFO:teuthology.orchestra.run.smithi122.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-04-20T21:16:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-20T21:16:14.547 INFO:teuthology.orchestra.run.smithi122.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 596 kB/s | 35 kB 00:00 2024-04-20T21:16:14.755 INFO:teuthology.orchestra.run.smithi122.stdout:(80/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-04-20T21:16:14.814 INFO:teuthology.orchestra.run.smithi122.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-20T21:16:14.873 INFO:teuthology.orchestra.run.smithi122.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-20T21:16:14.932 INFO:teuthology.orchestra.run.smithi122.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-20T21:16:14.990 INFO:teuthology.orchestra.run.smithi122.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-04-20T21:16:15.048 INFO:teuthology.orchestra.run.smithi122.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-20T21:16:15.107 INFO:teuthology.orchestra.run.smithi122.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-20T21:16:15.166 INFO:teuthology.orchestra.run.smithi122.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-20T21:16:15.233 INFO:teuthology.orchestra.run.smithi122.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 765 kB/s | 51 kB 00:00 2024-04-20T21:16:15.317 INFO:teuthology.orchestra.run.smithi122.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 644 kB/s | 54 kB 00:00 2024-04-20T21:16:15.975 INFO:teuthology.orchestra.run.smithi122.stdout:(90/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-04-20T21:16:16.026 INFO:teuthology.orchestra.run.smithi122.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 837 kB/s | 42 kB 00:00 2024-04-20T21:16:16.076 INFO:teuthology.orchestra.run.smithi122.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-04-20T21:16:16.134 INFO:teuthology.orchestra.run.smithi122.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-20T21:16:16.185 INFO:teuthology.orchestra.run.smithi122.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-04-20T21:16:16.244 INFO:teuthology.orchestra.run.smithi122.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-04-20T21:16:16.302 INFO:teuthology.orchestra.run.smithi122.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-20T21:16:16.361 INFO:teuthology.orchestra.run.smithi122.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 992 kB/s | 58 kB 00:00 2024-04-20T21:16:16.420 INFO:teuthology.orchestra.run.smithi122.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-20T21:16:16.479 INFO:teuthology.orchestra.run.smithi122.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-20T21:16:16.537 INFO:teuthology.orchestra.run.smithi122.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 768 kB/s | 45 kB 00:00 2024-04-20T21:16:16.621 INFO:teuthology.orchestra.run.smithi122.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-04-20T21:16:16.821 INFO:teuthology.orchestra.run.smithi122.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-04-20T21:16:17.430 INFO:teuthology.orchestra.run.smithi122.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 901 kB/s | 548 kB 00:00 2024-04-20T21:16:17.564 INFO:teuthology.orchestra.run.smithi122.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 MB/s | 309 kB 00:00 2024-04-20T21:16:17.689 INFO:teuthology.orchestra.run.smithi122.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-04-20T21:16:18.064 INFO:teuthology.orchestra.run.smithi122.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-20T21:16:18.164 INFO:teuthology.orchestra.run.smithi122.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-20T21:16:18.273 INFO:teuthology.orchestra.run.smithi122.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2024-04-20T21:16:18.373 INFO:teuthology.orchestra.run.smithi122.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 675 kB/s | 67 kB 00:00 2024-04-20T21:16:18.474 INFO:teuthology.orchestra.run.smithi122.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-20T21:16:18.607 INFO:teuthology.orchestra.run.smithi122.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 838 kB 00:00 2024-04-20T21:16:18.725 INFO:teuthology.orchestra.run.smithi122.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-04-20T21:16:18.833 INFO:teuthology.orchestra.run.smithi122.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 402 kB/s | 43 kB 00:00 2024-04-20T21:16:18.933 INFO:teuthology.orchestra.run.smithi122.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-04-20T21:16:19.042 INFO:teuthology.orchestra.run.smithi122.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-20T21:16:19.134 INFO:teuthology.orchestra.run.smithi122.stdout:(116/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-20T21:16:19.243 INFO:teuthology.orchestra.run.smithi122.stdout:(117/150): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-20T21:16:19.351 INFO:teuthology.orchestra.run.smithi122.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-20T21:16:19.461 INFO:teuthology.orchestra.run.smithi122.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-04-20T21:16:19.569 INFO:teuthology.orchestra.run.smithi122.stdout:(120/150): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-04-20T21:16:19.711 INFO:teuthology.orchestra.run.smithi122.stdout:(121/150): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-20T21:16:19.820 INFO:teuthology.orchestra.run.smithi122.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-20T21:16:19.920 INFO:teuthology.orchestra.run.smithi122.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-20T21:16:20.012 INFO:teuthology.orchestra.run.smithi122.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 193 kB/s | 18 kB 00:00 2024-04-20T21:16:20.104 INFO:teuthology.orchestra.run.smithi122.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 253 kB/s | 23 kB 00:00 2024-04-20T21:16:20.288 INFO:teuthology.orchestra.run.smithi122.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 106 kB/s | 19 kB 00:00 2024-04-20T21:16:20.380 INFO:teuthology.orchestra.run.smithi122.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-20T21:16:20.522 INFO:teuthology.orchestra.run.smithi122.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-04-20T21:16:20.622 INFO:teuthology.orchestra.run.smithi122.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-04-20T21:16:20.748 INFO:teuthology.orchestra.run.smithi122.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 631 kB/s | 79 kB 00:00 2024-04-20T21:16:20.873 INFO:teuthology.orchestra.run.smithi122.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 461 kB/s | 58 kB 00:00 2024-04-20T21:16:21.024 INFO:teuthology.orchestra.run.smithi122.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-04-20T21:16:21.140 INFO:teuthology.orchestra.run.smithi122.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-04-20T21:16:21.249 INFO:teuthology.orchestra.run.smithi122.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 830 kB/s | 90 kB 00:00 2024-04-20T21:16:21.358 INFO:teuthology.orchestra.run.smithi122.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-20T21:16:21.466 INFO:teuthology.orchestra.run.smithi122.stdout:(136/150): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2024-04-20T21:16:21.575 INFO:teuthology.orchestra.run.smithi122.stdout:(137/150): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-20T21:16:21.692 INFO:teuthology.orchestra.run.smithi122.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 508 kB/s | 59 kB 00:00 2024-04-20T21:16:21.792 INFO:teuthology.orchestra.run.smithi122.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 357 kB/s | 36 kB 00:00 2024-04-20T21:16:21.893 INFO:teuthology.orchestra.run.smithi122.stdout:(140/150): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-04-20T21:16:22.001 INFO:teuthology.orchestra.run.smithi122.stdout:(141/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-20T21:16:22.119 INFO:teuthology.orchestra.run.smithi122.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 767 kB/s | 90 kB 00:00 2024-04-20T21:16:22.227 INFO:teuthology.orchestra.run.smithi122.stdout:(143/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-20T21:16:22.319 INFO:teuthology.orchestra.run.smithi122.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-04-20T21:16:22.428 INFO:teuthology.orchestra.run.smithi122.stdout:(145/150): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-20T21:16:22.587 INFO:teuthology.orchestra.run.smithi122.stdout:(146/150): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-20T21:16:22.670 INFO:teuthology.orchestra.run.smithi122.stdout:(147/150): ceph-mgr-diskprediction-local-19.0.0 305 kB/s | 7.4 MB 00:24 2024-04-20T21:16:23.326 INFO:teuthology.orchestra.run.smithi022.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 274 kB/s | 3.5 MB 00:13 2024-04-20T21:16:24.434 INFO:teuthology.orchestra.run.smithi022.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 225 kB/s | 249 kB 00:01 2024-04-20T21:16:24.651 INFO:teuthology.orchestra.run.smithi022.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 228 kB/s | 49 kB 00:00 2024-04-20T21:16:25.459 INFO:teuthology.orchestra.run.smithi022.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 20 kB/s | 16 kB 00:00 2024-04-20T21:16:26.618 INFO:teuthology.orchestra.run.smithi022.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 236 kB/s | 273 kB 00:01 2024-04-20T21:16:29.793 INFO:teuthology.orchestra.run.smithi033.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 615 kB/s | 49 MB 01:21 2024-04-20T21:16:29.796 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-20T21:16:29.796 INFO:teuthology.orchestra.run.smithi033.stdout:Total 2.2 MB/s | 218 MB 01:38 2024-04-20T21:16:29.834 INFO:teuthology.orchestra.run.smithi022.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 237 kB/s | 763 kB 00:03 2024-04-20T21:16:29.960 INFO:teuthology.orchestra.run.smithi022.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 147 kB/s | 18 kB 00:00 2024-04-20T21:16:30.076 INFO:teuthology.orchestra.run.smithi022.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 699 kB/s | 80 kB 00:00 2024-04-20T21:16:30.194 INFO:teuthology.orchestra.run.smithi022.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 6.8 MB/s | 803 kB 00:00 2024-04-20T21:16:30.244 INFO:teuthology.orchestra.run.smithi022.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-20T21:16:30.302 INFO:teuthology.orchestra.run.smithi022.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-04-20T21:16:30.594 INFO:teuthology.orchestra.run.smithi022.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-04-20T21:16:30.670 INFO:teuthology.orchestra.run.smithi022.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-20T21:16:30.745 INFO:teuthology.orchestra.run.smithi022.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-20T21:16:30.795 INFO:teuthology.orchestra.run.smithi022.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-20T21:16:30.818 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-20T21:16:30.846 INFO:teuthology.orchestra.run.smithi022.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-20T21:16:30.955 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-20T21:16:30.955 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-20T21:16:31.079 INFO:teuthology.orchestra.run.smithi022.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-04-20T21:16:31.246 INFO:teuthology.orchestra.run.smithi022.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 5.6 MB/s | 943 kB 00:00 2024-04-20T21:16:31.437 INFO:teuthology.orchestra.run.smithi122.stdout:(148/150): librados2-19.0.0-3133.ga8c84f59.el9. 393 kB/s | 3.4 MB 00:08 2024-04-20T21:16:31.447 INFO:teuthology.orchestra.run.smithi022.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 535 kB/s | 107 kB 00:00 2024-04-20T21:16:31.497 INFO:teuthology.orchestra.run.smithi022.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 437 kB/s | 21 kB 00:00 2024-04-20T21:16:31.597 INFO:teuthology.orchestra.run.smithi022.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-04-20T21:16:31.656 INFO:teuthology.orchestra.run.smithi022.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 568 kB/s | 33 kB 00:00 2024-04-20T21:16:32.098 INFO:teuthology.orchestra.run.smithi022.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.8 MB/s | 3.0 MB 00:00 2024-04-20T21:16:32.156 INFO:teuthology.orchestra.run.smithi022.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 318 kB/s | 18 kB 00:00 2024-04-20T21:16:32.215 INFO:teuthology.orchestra.run.smithi022.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 192 kB/s | 11 kB 00:00 2024-04-20T21:16:32.273 INFO:teuthology.orchestra.run.smithi022.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-20T21:16:32.357 INFO:teuthology.orchestra.run.smithi022.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 548 kB/s | 45 kB 00:00 2024-04-20T21:16:32.483 INFO:teuthology.orchestra.run.smithi022.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-04-20T21:16:32.599 INFO:teuthology.orchestra.run.smithi022.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-04-20T21:16:32.650 INFO:teuthology.orchestra.run.smithi022.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 515 kB/s | 25 kB 00:00 2024-04-20T21:16:32.700 INFO:teuthology.orchestra.run.smithi022.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 964 kB/s | 47 kB 00:00 2024-04-20T21:16:32.781 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-20T21:16:32.781 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-20T21:16:32.792 INFO:teuthology.orchestra.run.smithi022.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-04-20T21:16:32.851 INFO:teuthology.orchestra.run.smithi022.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-20T21:16:32.934 INFO:teuthology.orchestra.run.smithi022.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 427 kB/s | 35 kB 00:00 2024-04-20T21:16:33.576 INFO:teuthology.orchestra.run.smithi022.stdout:(65/150): openblas-openmp-0.3.21-2.el9.x86_64.r 7.4 MB/s | 4.7 MB 00:00 2024-04-20T21:16:34.110 INFO:teuthology.orchestra.run.smithi022.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 7.8 MB/s | 4.1 MB 00:00 2024-04-20T21:16:34.327 INFO:teuthology.orchestra.run.smithi022.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 7.4 MB/s | 1.6 MB 00:00 2024-04-20T21:16:34.452 INFO:teuthology.orchestra.run.smithi022.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 7.2 MB/s | 911 kB 00:00 2024-04-20T21:16:34.511 INFO:teuthology.orchestra.run.smithi022.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-20T21:16:34.836 INFO:teuthology.orchestra.run.smithi122.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 264 kB/s | 3.1 MB 00:12 2024-04-20T21:16:35.111 INFO:teuthology.orchestra.run.smithi022.stdout:(70/150): postgresql-server-13.14-1.el9.x86_64. 9.6 MB/s | 5.8 MB 00:00 2024-04-20T21:16:35.245 INFO:teuthology.orchestra.run.smithi022.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-04-20T21:16:35.303 INFO:teuthology.orchestra.run.smithi022.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-20T21:16:35.870 INFO:teuthology.orchestra.run.smithi022.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-20T21:16:35.929 INFO:teuthology.orchestra.run.smithi022.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 4.2 MB/s | 244 kB 00:00 2024-04-20T21:16:35.988 INFO:teuthology.orchestra.run.smithi022.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-20T21:16:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 521 kB/s | 26 kB 00:00 2024-04-20T21:16:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 332 kB/s | 19 kB 00:00 2024-04-20T21:16:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-20T21:16:36.214 INFO:teuthology.orchestra.run.smithi022.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 600 kB/s | 35 kB 00:00 2024-04-20T21:16:36.631 INFO:teuthology.orchestra.run.smithi022.stdout:(80/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-20T21:16:36.689 INFO:teuthology.orchestra.run.smithi022.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-04-20T21:16:36.748 INFO:teuthology.orchestra.run.smithi022.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-04-20T21:16:36.806 INFO:teuthology.orchestra.run.smithi022.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-20T21:16:36.865 INFO:teuthology.orchestra.run.smithi022.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 733 kB/s | 42 kB 00:00 2024-04-20T21:16:36.924 INFO:teuthology.orchestra.run.smithi022.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-04-20T21:16:36.982 INFO:teuthology.orchestra.run.smithi022.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-20T21:16:37.041 INFO:teuthology.orchestra.run.smithi022.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-20T21:16:37.091 INFO:teuthology.orchestra.run.smithi022.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-20T21:16:37.142 INFO:teuthology.orchestra.run.smithi022.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-20T21:16:38.122 INFO:teuthology.orchestra.run.smithi022.stdout:(90/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-20T21:16:38.200 INFO:teuthology.orchestra.run.smithi022.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 538 kB/s | 42 kB 00:00 2024-04-20T21:16:38.276 INFO:teuthology.orchestra.run.smithi022.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 595 kB/s | 44 kB 00:00 2024-04-20T21:16:38.368 INFO:teuthology.orchestra.run.smithi022.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-04-20T21:16:38.443 INFO:teuthology.orchestra.run.smithi022.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 483 kB/s | 36 kB 00:00 2024-04-20T21:16:38.519 INFO:teuthology.orchestra.run.smithi022.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 638 kB/s | 47 kB 00:00 2024-04-20T21:16:38.577 INFO:teuthology.orchestra.run.smithi022.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-20T21:16:38.652 INFO:teuthology.orchestra.run.smithi022.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 776 kB/s | 58 kB 00:00 2024-04-20T21:16:38.711 INFO:teuthology.orchestra.run.smithi022.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-20T21:16:38.788 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-20T21:16:38.794 INFO:teuthology.orchestra.run.smithi022.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 769 kB/s | 64 kB 00:00 2024-04-20T21:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T21:16:38.870 INFO:teuthology.orchestra.run.smithi022.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-04-20T21:16:38.896 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T21:16:38.940 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T21:16:39.154 INFO:teuthology.orchestra.run.smithi022.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-04-20T21:16:39.294 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T21:16:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:16:39.531 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:16:39.557 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:16:39.650 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:16:39.692 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:16:39.720 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T21:16:39.721 INFO:teuthology.orchestra.run.smithi022.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.5 MB/s | 863 kB 00:00 2024-04-20T21:16:39.746 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T21:16:39.908 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T21:16:39.968 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T21:16:39.996 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:16:40.012 INFO:teuthology.orchestra.run.smithi022.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-04-20T21:16:40.088 INFO:teuthology.orchestra.run.smithi022.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-04-20T21:16:40.101 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:16:40.130 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:16:40.163 INFO:teuthology.orchestra.run.smithi022.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 260 kB/s | 19 kB 00:00 2024-04-20T21:16:40.176 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:16:40.198 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:16:40.216 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:16:40.426 INFO:teuthology.orchestra.run.smithi022.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-20T21:16:40.472 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:16:40.505 INFO:teuthology.orchestra.run.smithi022.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 332 kB/s | 26 kB 00:00 2024-04-20T21:16:40.506 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:16:40.581 INFO:teuthology.orchestra.run.smithi022.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 652 kB/s | 49 kB 00:00 2024-04-20T21:16:40.618 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T21:16:40.639 INFO:teuthology.orchestra.run.smithi022.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-20T21:16:40.698 INFO:teuthology.orchestra.run.smithi022.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-04-20T21:16:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T21:16:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T21:16:40.782 INFO:teuthology.orchestra.run.smithi022.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.9 MB/s | 838 kB 00:00 2024-04-20T21:16:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T21:16:40.857 INFO:teuthology.orchestra.run.smithi022.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 7.2 MB/s | 548 kB 00:00 2024-04-20T21:16:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T21:16:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T21:16:40.932 INFO:teuthology.orchestra.run.smithi022.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 585 kB/s | 43 kB 00:00 2024-04-20T21:16:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T21:16:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T21:16:41.008 INFO:teuthology.orchestra.run.smithi022.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 432 kB/s | 32 kB 00:00 2024-04-20T21:16:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T21:16:41.083 INFO:teuthology.orchestra.run.smithi022.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 202 kB/s | 15 kB 00:00 2024-04-20T21:16:41.099 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T21:16:41.141 INFO:teuthology.orchestra.run.smithi022.stdout:(116/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-20T21:16:41.151 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T21:16:41.179 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T21:16:41.208 INFO:teuthology.orchestra.run.smithi022.stdout:(117/150): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-04-20T21:16:41.223 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T21:16:41.275 INFO:teuthology.orchestra.run.smithi022.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-04-20T21:16:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T21:16:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T21:16:41.350 INFO:teuthology.orchestra.run.smithi022.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 474 kB/s | 35 kB 00:00 2024-04-20T21:16:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T21:16:41.418 INFO:teuthology.orchestra.run.smithi022.stdout:(120/150): python3-google-auth-2.28.2-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-04-20T21:16:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T21:16:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T21:16:41.543 INFO:teuthology.orchestra.run.smithi022.stdout:(121/150): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-04-20T21:16:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T21:16:41.610 INFO:teuthology.orchestra.run.smithi022.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-20T21:16:41.635 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T21:16:41.694 INFO:teuthology.orchestra.run.smithi022.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2024-04-20T21:16:41.777 INFO:teuthology.orchestra.run.smithi022.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 214 kB/s | 18 kB 00:00 2024-04-20T21:16:41.781 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T21:16:41.833 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T21:16:41.861 INFO:teuthology.orchestra.run.smithi022.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 280 kB/s | 23 kB 00:00 2024-04-20T21:16:41.945 INFO:teuthology.orchestra.run.smithi022.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 235 kB/s | 19 kB 00:00 2024-04-20T21:16:42.029 INFO:teuthology.orchestra.run.smithi022.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 238 kB/s | 20 kB 00:00 2024-04-20T21:16:42.137 INFO:teuthology.orchestra.run.smithi022.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 9.5 MB/s | 1.0 MB 00:00 2024-04-20T21:16:42.212 INFO:teuthology.orchestra.run.smithi022.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 623 kB/s | 46 kB 00:00 2024-04-20T21:16:42.271 INFO:teuthology.orchestra.run.smithi022.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-20T21:16:42.346 INFO:teuthology.orchestra.run.smithi022.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 775 kB/s | 58 kB 00:00 2024-04-20T21:16:42.399 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T21:16:42.413 INFO:teuthology.orchestra.run.smithi022.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-20T21:16:42.489 INFO:teuthology.orchestra.run.smithi022.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 220 kB/s | 16 kB 00:00 2024-04-20T21:16:42.515 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T21:16:42.547 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T21:16:42.548 INFO:teuthology.orchestra.run.smithi022.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-20T21:16:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T21:16:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T21:16:42.622 INFO:teuthology.orchestra.run.smithi022.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 380 kB/s | 28 kB 00:00 2024-04-20T21:16:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T21:16:42.698 INFO:teuthology.orchestra.run.smithi022.stdout:(136/150): python3-repoze-lru-0.7-16.el9.noarch 413 kB/s | 31 kB 00:00 2024-04-20T21:16:42.710 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T21:16:42.765 INFO:teuthology.orchestra.run.smithi022.stdout:(137/150): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-04-20T21:16:42.784 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T21:16:42.820 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T21:16:42.840 INFO:teuthology.orchestra.run.smithi022.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 794 kB/s | 59 kB 00:00 2024-04-20T21:16:42.857 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T21:16:42.906 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T21:16:42.915 INFO:teuthology.orchestra.run.smithi022.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 483 kB/s | 36 kB 00:00 2024-04-20T21:16:42.940 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T21:16:42.984 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T21:16:42.991 INFO:teuthology.orchestra.run.smithi022.stdout:(140/150): python3-typing-extensions-4.4.0-2.el 686 kB/s | 51 kB 00:00 2024-04-20T21:16:43.027 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T21:16:43.049 INFO:teuthology.orchestra.run.smithi022.stdout:(141/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-04-20T21:16:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-20T21:16:43.173 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T21:16:43.223 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T21:16:43.225 INFO:teuthology.orchestra.run.smithi022.stdout:(143/150): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-20T21:16:43.252 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T21:16:43.295 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T21:16:43.300 INFO:teuthology.orchestra.run.smithi022.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 269 kB/s | 20 kB 00:00 2024-04-20T21:16:43.367 INFO:teuthology.orchestra.run.smithi022.stdout:(145/150): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-20T21:16:43.394 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T21:16:43.435 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T21:16:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T21:16:43.476 INFO:teuthology.orchestra.run.smithi022.stdout:(146/150): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-20T21:16:43.509 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T21:16:43.533 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T21:16:43.613 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T21:16:43.753 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T21:16:43.791 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T21:16:43.819 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T21:16:43.854 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T21:16:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T21:16:43.936 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T21:16:43.976 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T21:16:44.004 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T21:16:44.039 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T21:16:44.073 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T21:16:44.115 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T21:16:44.149 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T21:16:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T21:16:44.289 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T21:16:44.319 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T21:16:44.367 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T21:16:44.481 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T21:16:44.538 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T21:16:44.859 INFO:teuthology.orchestra.run.smithi022.stdout:(147/150): ceph-mgr-diskprediction-local-19.0.0 239 kB/s | 7.4 MB 00:31 2024-04-20T21:16:45.334 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T21:16:45.384 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T21:16:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T21:16:45.522 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T21:16:45.576 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T21:16:45.668 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T21:16:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T21:16:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T21:16:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T21:16:46.006 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T21:16:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T21:16:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T21:16:48.182 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T21:16:48.262 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T21:16:48.290 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T21:16:48.323 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T21:16:48.652 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T21:16:48.686 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:16:48.777 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:16:48.807 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T21:16:48.844 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T21:16:49.350 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T21:16:49.388 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:16:49.443 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:16:49.473 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T21:16:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:16:51.878 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:16:52.449 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:16:52.489 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:16:52.581 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:16:52.581 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T21:16:52.581 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T21:16:52.581 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:16:52.616 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:16:54.350 INFO:teuthology.orchestra.run.smithi022.stdout:(148/150): librados2-19.0.0-3133.ga8c84f59.el9. 320 kB/s | 3.4 MB 00:10 2024-04-20T21:16:54.475 INFO:teuthology.orchestra.run.smithi022.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 335 kB/s | 3.1 MB 00:09 2024-04-20T21:17:01.927 INFO:teuthology.orchestra.run.smithi122.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 573 kB/s | 49 MB 01:27 2024-04-20T21:17:01.929 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-04-20T21:17:01.929 INFO:teuthology.orchestra.run.smithi122.stdout:Total 1.8 MB/s | 218 MB 02:00 2024-04-20T21:17:02.943 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-04-20T21:17:03.068 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-04-20T21:17:03.068 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-04-20T21:17:04.438 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:17:04.438 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-04-20T21:17:04.438 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-04-20T21:17:04.438 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-04-20T21:17:04.438 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-04-20T21:17:04.438 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-04-20T21:17:04.438 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /root 2024-04-20T21:17:04.438 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-04-20T21:17:04.438 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:17:04.618 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:17:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:17:04.634 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:04.634 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T21:17:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:17:04.773 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-04-20T21:17:04.773 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-04-20T21:17:06.212 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:17:06.239 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:17:06.239 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:06.239 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T21:17:06.239 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:17:06.239 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:17:06.239 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:17:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:17:06.495 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:17:06.495 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:06.495 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T21:17:06.495 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:17:06.495 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:17:06.495 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:17:06.955 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:17:06.978 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:17:06.979 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:06.979 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T21:17:06.979 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:17:06.979 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:17:06.979 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:17:07.087 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T21:17:07.145 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T21:17:07.209 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T21:17:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T21:17:07.465 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T21:17:07.546 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T21:17:07.635 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T21:17:07.787 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T21:17:07.840 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T21:17:07.872 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T21:17:07.899 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T21:17:08.327 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:17:08.356 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:17:08.389 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T21:17:08.452 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T21:17:08.479 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:17:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:17:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:17:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-04-20T21:17:10.631 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:17:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:17:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T21:17:10.721 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T21:17:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:17:10.763 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T21:17:10.935 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T21:17:10.966 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:17:10.990 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:17:10.990 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:10.990 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T21:17:10.990 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:17:10.991 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:17:10.991 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:17:11.051 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:17:11.062 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:17:11.102 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T21:17:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T21:17:11.207 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:17:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T21:17:11.311 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:17:11.342 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:17:11.432 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:17:11.475 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:17:11.505 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T21:17:11.530 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T21:17:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T21:17:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T21:17:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:17:11.885 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:17:11.908 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:17:11.970 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:17:11.992 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:17:12.017 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:17:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T21:17:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:17:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:17:12.252 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:12.252 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T21:17:12.252 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:17:12.252 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:17:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:17:12.279 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:17:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:17:12.317 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:17:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:17:12.327 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:12.327 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T21:17:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:17:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:17:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:17:12.394 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:12.395 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T21:17:12.395 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:17:12.395 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:17:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:17:12.423 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T21:17:12.521 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T21:17:12.568 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T21:17:12.609 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T21:17:12.675 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T21:17:12.717 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T21:17:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:17:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:17:12.745 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:12.745 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T21:17:12.745 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:17:12.745 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:17:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:17:12.769 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T21:17:12.802 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T21:17:12.830 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T21:17:12.898 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T21:17:12.952 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T21:17:12.991 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T21:17:13.026 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T21:17:13.102 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T21:17:13.138 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T21:17:13.181 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T21:17:13.210 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T21:17:13.281 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T21:17:13.325 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T21:17:13.400 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T21:17:13.541 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T21:17:13.602 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T21:17:14.126 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T21:17:14.215 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T21:17:14.253 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T21:17:14.296 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T21:17:14.334 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T21:17:14.367 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T21:17:14.424 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T21:17:14.497 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T21:17:14.535 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T21:17:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T21:17:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T21:17:14.655 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T21:17:14.707 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T21:17:14.756 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T21:17:14.895 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T21:17:14.946 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T21:17:14.976 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T21:17:15.018 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T21:17:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T21:17:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T21:17:15.192 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T21:17:15.217 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T21:17:15.241 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T21:17:15.316 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T21:17:15.453 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T21:17:15.500 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T21:17:15.527 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T21:17:15.562 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T21:17:15.607 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T21:17:15.644 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T21:17:15.701 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T21:17:15.731 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T21:17:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T21:17:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T21:17:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T21:17:15.875 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T21:17:15.971 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T21:17:16.023 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T21:17:16.054 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T21:17:16.101 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T21:17:16.201 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T21:17:16.263 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T21:17:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T21:17:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T21:17:17.078 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T21:17:17.179 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T21:17:17.237 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T21:17:17.325 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T21:17:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T21:17:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T21:17:17.622 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T21:17:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T21:17:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T21:17:17.706 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T21:17:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T21:17:17.740 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T21:17:17.767 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T21:17:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T21:17:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T21:17:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T21:17:18.358 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T21:17:18.480 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T21:17:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:17:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:17:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:17:19.568 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T21:17:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T21:17:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T21:17:19.716 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T21:17:19.788 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T21:17:19.802 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T21:17:19.823 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T21:17:19.823 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:17:19.830 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T21:17:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T21:17:19.907 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:17:19.907 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:17:20.174 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T21:17:20.226 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:17:20.300 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:17:20.331 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T21:17:20.375 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T21:17:20.853 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T21:17:20.888 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:17:20.959 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:17:20.981 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T21:17:23.271 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:17:23.312 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:17:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:17:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:17:23.980 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:17:23.980 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T21:17:23.980 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T21:17:23.980 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:24.008 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:17:25.880 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:17:25.880 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T21:17:25.881 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T21:17:25.882 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T21:17:25.893 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T21:17:25.893 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T21:17:25.893 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T21:17:25.893 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T21:17:25.893 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T21:17:25.893 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T21:17:25.893 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T21:17:25.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T21:17:25.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T21:17:25.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T21:17:25.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T21:17:25.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T21:17:25.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T21:17:25.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T21:17:25.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T21:17:25.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T21:17:25.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T21:17:25.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T21:17:25.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T21:17:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:17:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:17:28.037 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-04-20T21:17:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.037 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-20T21:17:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T21:17:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T21:17:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T21:17:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T21:17:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T21:17:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T21:17:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T21:17:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T21:17:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T21:17:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T21:17:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T21:17:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T21:17:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T21:17:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T21:17:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T21:17:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T21:17:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T21:17:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T21:17:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T21:17:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T21:17:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T21:17:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T21:17:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T21:17:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T21:17:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T21:17:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T21:17:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T21:17:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T21:17:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T21:17:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T21:17:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T21:17:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T21:17:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T21:17:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T21:17:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T21:17:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T21:17:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T21:17:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T21:17:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T21:17:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T21:17:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T21:17:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T21:17:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T21:17:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T21:17:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T21:17:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-20T21:17:28.049 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-20T21:17:28.395 DEBUG:teuthology.parallel:result is None 2024-04-20T21:17:30.304 INFO:teuthology.orchestra.run.smithi022.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 456 kB/s | 49 MB 01:50 2024-04-20T21:17:30.309 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-04-20T21:17:30.309 INFO:teuthology.orchestra.run.smithi022.stdout:Total 1.5 MB/s | 218 MB 02:26 2024-04-20T21:17:33.113 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-04-20T21:17:33.348 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-04-20T21:17:33.348 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-04-20T21:17:35.484 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:17:35.484 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-04-20T21:17:35.485 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-04-20T21:17:35.485 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-04-20T21:17:35.485 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-04-20T21:17:35.485 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-04-20T21:17:35.485 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-04-20T21:17:35.485 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-04-20T21:17:35.485 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:17:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:17:35.673 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:35.673 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T21:17:35.674 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-04-20T21:17:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-04-20T21:17:37.137 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:17:37.162 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:17:37.162 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:37.162 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T21:17:37.163 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:17:37.163 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:17:37.163 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:17:37.410 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:17:37.410 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:37.410 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T21:17:37.410 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:17:37.410 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:17:37.410 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:17:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:17:37.877 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:37.878 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T21:17:37.878 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:17:37.878 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:17:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:37.985 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T21:17:38.033 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T21:17:38.089 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T21:17:38.281 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T21:17:38.332 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T21:17:38.409 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T21:17:38.497 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T21:17:38.636 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T21:17:38.674 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T21:17:38.706 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T21:17:38.734 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T21:17:39.171 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:17:39.199 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:17:39.232 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T21:17:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T21:17:39.323 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:17:40.350 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:17:40.383 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:17:41.347 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:17:41.375 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:17:41.475 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:17:41.639 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T21:17:41.678 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:17:41.700 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:17:41.700 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:41.700 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T21:17:41.700 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:17:41.701 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:17:41.701 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:17:41.764 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:17:41.864 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T21:17:41.918 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T21:17:42.360 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-04-20T21:17:42.447 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T21:17:42.483 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T21:17:42.601 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T21:17:42.844 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T21:17:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:17:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:17:42.896 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:42.896 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T21:17:42.896 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:17:42.896 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:17:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:17:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:17:42.963 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:42.963 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T21:17:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:17:43.030 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:17:43.030 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:43.030 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T21:17:43.030 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:17:43.030 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:17:43.030 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:43.085 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T21:17:43.223 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:17:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:17:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:17:43.357 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:17:43.357 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T21:17:43.357 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:17:43.357 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:17:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:43.415 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:17:43.449 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:17:43.567 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:17:43.633 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:17:43.665 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T21:17:43.703 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T21:17:44.134 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T21:17:44.200 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T21:17:44.230 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:17:44.400 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:17:44.431 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:17:44.526 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:17:44.556 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:17:44.633 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:17:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:17:45.234 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:17:45.436 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T21:17:45.564 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T21:17:45.630 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T21:17:45.851 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T21:17:45.980 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T21:17:46.032 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T21:17:46.131 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T21:17:46.197 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T21:17:46.242 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T21:17:46.373 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T21:17:46.445 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T21:17:46.483 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T21:17:46.544 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T21:17:46.665 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T21:17:46.720 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T21:17:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T21:17:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T21:17:46.940 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T21:17:47.005 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T21:17:47.119 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T21:17:47.301 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T21:17:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T21:17:47.963 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T21:17:48.013 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T21:17:48.062 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T21:17:48.103 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T21:17:48.107 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T21:17:48.130 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T21:17:48.218 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T21:17:48.257 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T21:17:48.304 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T21:17:48.309 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T21:17:48.352 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T21:17:48.354 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T21:17:48.382 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T21:17:48.430 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T21:17:48.532 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T21:17:48.641 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T21:17:48.683 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T21:17:48.842 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T21:17:48.930 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T21:17:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:17:48.976 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T21:17:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:17:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:17:49.051 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T21:17:49.122 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T21:17:49.339 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T21:17:49.395 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T21:17:49.432 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T21:17:49.490 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T21:17:49.662 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T21:17:49.706 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T21:17:49.762 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T21:17:49.781 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T21:17:49.797 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T21:17:49.829 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T21:17:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T21:17:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T21:17:49.982 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T21:17:50.024 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T21:17:50.024 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:17:50.060 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T21:17:50.117 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:17:50.117 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:17:50.245 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T21:17:50.286 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T21:17:50.318 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T21:17:50.364 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T21:17:50.437 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T21:17:50.497 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T21:17:50.548 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T21:17:50.590 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T21:17:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T21:17:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T21:17:50.798 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T21:17:50.847 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T21:17:50.999 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T21:17:51.062 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T21:17:51.119 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T21:17:51.193 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T21:17:51.357 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T21:17:51.424 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T21:17:52.441 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T21:17:52.506 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T21:17:52.539 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T21:17:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T21:17:52.735 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T21:17:52.846 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T21:17:52.891 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T21:17:53.132 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T21:17:53.170 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T21:17:53.282 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T21:17:54.203 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T21:17:54.394 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T21:17:56.125 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T21:17:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T21:17:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T21:17:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T21:17:56.436 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:17:56.436 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T21:17:56.436 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T21:17:56.436 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T21:17:56.436 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T21:17:56.436 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T21:17:56.437 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T21:17:56.438 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T21:17:56.438 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T21:17:56.438 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T21:17:56.438 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T21:17:56.438 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T21:17:56.438 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T21:17:56.438 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T21:17:56.438 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T21:17:56.438 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T21:17:56.438 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T21:17:56.438 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T21:17:56.438 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T21:17:56.439 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T21:17:56.440 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T21:17:56.441 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T21:17:56.442 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T21:17:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T21:17:56.444 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T21:17:56.445 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T21:17:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T21:17:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T21:17:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T21:17:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T21:17:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T21:17:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T21:17:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T21:17:56.447 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T21:17:56.675 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T21:17:56.714 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:17:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:17:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T21:17:56.929 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T21:17:57.484 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T21:17:57.517 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:17:57.635 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:17:57.660 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T21:17:58.501 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T21:17:58.508 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi122.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi122.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T21:17:58.513 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T21:17:58.513 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:17:58.513 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-04-20T21:17:58.896 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:00.787 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:18:00.832 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:18:01.510 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:18:01.560 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:18:01.711 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:18:01.711 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T21:18:01.712 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T21:18:01.712 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:01.761 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:18:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:18:15.242 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /sys 2024-04-20T21:18:15.242 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /proc 2024-04-20T21:18:15.242 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /mnt 2024-04-20T21:18:15.242 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /var/tmp 2024-04-20T21:18:15.242 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /home 2024-04-20T21:18:15.242 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /root 2024-04-20T21:18:15.242 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /tmp 2024-04-20T21:18:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:15.485 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:18:15.537 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:18:15.537 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:15.537 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T21:18:15.537 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:17.236 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:18:17.317 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:18:17.317 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:17.317 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T21:18:17.317 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:18:17.318 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:18:17.318 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:17.578 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:18:17.649 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:18:17.649 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:17.649 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T21:18:17.649 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:18:17.649 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:18:17.649 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:18.170 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:18:18.241 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:18:18.241 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:18.241 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T21:18:18.241 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:18:18.241 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:18:18.241 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:18.380 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T21:18:18.443 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T21:18:18.523 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T21:18:18.785 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T21:18:18.861 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T21:18:18.960 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T21:18:19.080 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T21:18:19.258 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T21:18:19.328 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T21:18:19.367 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T21:18:19.398 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T21:18:19.992 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:18:20.028 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:18:20.086 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T21:18:20.198 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T21:18:20.228 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:18:21.594 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:18:21.647 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:18:22.816 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:18:22.847 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:18:23.002 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:18:23.203 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T21:18:23.233 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:18:23.322 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:18:23.322 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:23.322 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T21:18:23.322 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:18:23.322 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:18:23.323 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:23.392 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:18:23.431 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:18:23.550 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T21:18:23.622 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T21:18:24.710 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T21:18:24.745 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:18:24.817 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:18:24.817 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:24.817 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T21:18:24.817 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:18:24.817 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:18:24.817 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:24.870 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:18:24.943 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:18:24.943 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:24.943 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T21:18:24.943 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:18:25.088 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:18:25.088 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:25.088 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T21:18:25.088 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:18:25.088 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:18:25.088 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:25.471 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:18:25.545 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:18:25.546 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:25.546 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T21:18:25.546 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:18:25.546 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:18:25.546 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:30.861 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T21:18:30.915 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T21:18:30.999 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T21:18:31.048 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T21:18:31.099 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T21:18:31.430 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T21:18:31.491 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T21:18:31.519 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T21:18:32.451 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:18:32.492 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:18:32.547 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:18:33.899 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T21:18:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T21:18:34.095 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T21:18:34.269 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T21:18:34.338 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T21:18:34.339 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:18:34.453 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:18:34.453 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:18:34.973 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:18:34.973 INFO:teuthology.orchestra.run.smithi022.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T21:18:34.973 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:40.864 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T21:18:40.865 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T21:18:40.866 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T21:18:40.866 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T21:18:40.866 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T21:18:40.866 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T21:18:40.866 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T21:18:40.866 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T21:18:40.866 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T21:18:40.866 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T21:18:40.866 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T21:18:40.866 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T21:18:40.866 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T21:18:40.866 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T21:18:40.867 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T21:18:40.868 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T21:18:40.869 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T21:18:40.870 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T21:18:40.871 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T21:18:40.872 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T21:18:40.873 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T21:18:40.874 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T21:18:40.875 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T21:18:40.875 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T21:18:40.875 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T21:18:40.875 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T21:18:40.875 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T21:18:40.875 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T21:18:40.875 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T21:18:43.477 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:18:43.477 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:43.478 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2024-04-20T21:18:43.478 INFO:teuthology.orchestra.run.smithi022.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.478 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.478 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-04-20T21:18:43.478 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T21:18:43.478 INFO:teuthology.orchestra.run.smithi022.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T21:18:43.478 INFO:teuthology.orchestra.run.smithi022.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T21:18:43.478 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T21:18:43.478 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.478 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.478 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.478 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:43.479 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T21:18:43.480 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.481 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T21:18:43.482 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T21:18:43.483 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.484 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.484 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.484 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T21:18:43.484 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T21:18:43.484 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T21:18:43.484 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T21:18:43.484 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T21:18:43.484 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T21:18:43.484 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T21:18:43.484 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T21:18:43.484 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T21:18:43.484 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T21:18:43.485 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T21:18:43.486 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T21:18:43.486 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T21:18:43.486 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T21:18:43.486 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T21:18:43.486 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T21:18:43.486 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T21:18:43.486 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T21:18:43.486 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T21:18:43.486 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T21:18:43.486 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T21:18:43.486 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T21:18:43.486 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T21:18:43.487 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T21:18:43.488 INFO:teuthology.orchestra.run.smithi022.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T21:18:43.489 INFO:teuthology.orchestra.run.smithi022.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T21:18:43.489 INFO:teuthology.orchestra.run.smithi022.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T21:18:43.489 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T21:18:43.489 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T21:18:43.489 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T21:18:43.489 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:18:43.489 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-04-20T21:18:44.064 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:44.065 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:18:44.455 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T21:18:44.511 INFO:teuthology.orchestra.run.smithi022.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T21:18:44.512 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T21:18:44.512 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T21:18:44.514 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:18:44.843 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T21:18:44.871 INFO:teuthology.orchestra.run.smithi033.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T21:18:44.871 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T21:18:44.871 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T21:18:44.873 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:18:45.158 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T21:18:45.183 INFO:teuthology.orchestra.run.smithi122.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T21:18:45.183 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T21:18:45.183 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T21:18:45.185 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T21:18:45.185 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:18:45.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T21:18:45.253 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-20T21:18:45.253 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T21:18:45.285 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-20T21:18:45.286 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T21:18:45.315 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T21:18:45.316 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:18:45.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T21:18:45.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T21:18:45.481 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-20T21:18:45.481 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T21:18:45.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T21:18:45.581 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-20T21:18:45.581 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T21:18:45.610 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T21:18:45.676 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T21:18:45.676 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:18:45.676 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T21:18:45.735 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T21:18:45.839 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-20T21:18:45.839 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T21:18:45.865 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T21:18:45.929 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-20T21:18:45.930 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T21:18:45.954 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T21:18:46.017 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T21:18:46.017 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:18:46.017 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T21:18:46.080 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T21:18:46.182 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-20T21:18:46.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T21:18:46.207 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T21:18:46.271 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-20T21:18:46.272 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T21:18:46.295 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T21:18:46.357 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-20T21:18:46.452 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'subvols': {'create': 2}} 2024-04-20T21:18:46.452 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:18:46.453 INFO:tasks.cephadm:Cluster fsid is 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:18:46.453 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-20T21:18:46.453 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.22', 'mon.b': '172.21.15.33', 'mon.c': '172.21.15.122'} 2024-04-20T21:18:46.453 INFO:tasks.cephadm:First mon is mon.a on smithi022 2024-04-20T21:18:46.453 INFO:tasks.cephadm:First mgr is x 2024-04-20T21:18:46.453 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-20T21:18:46.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname $(hostname -s) 2024-04-20T21:18:46.521 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname $(hostname -s) 2024-04-20T21:18:46.546 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname $(hostname -s) 2024-04-20T21:18:46.570 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-20T21:18:46.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:18:46.716 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78637/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3133-ga8c84f59', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3133.ga8c84f59'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-16 21:41:56.138585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240416.193735', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-20T21:18:46.858 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-rishabh-testing-20240416.193735, sha1 a8c84f5997950fd7d008204839253291d52e6eda from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:18:46.859 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm 2024-04-20T21:18:46.860 DEBUG:teuthology.orchestra.run.smithi022:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:48.401 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 20 21:18 /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:48.401 DEBUG:teuthology.orchestra.run.smithi033:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:49.544 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 20 21:18 /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:49.544 DEBUG:teuthology.orchestra.run.smithi122:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:50.947 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 20 21:18 /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:50.947 DEBUG:teuthology.orchestra.run.smithi022:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:51.010 DEBUG:teuthology.orchestra.run.smithi033:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:51.030 DEBUG:teuthology.orchestra.run.smithi122:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:51.064 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda on all hosts... 2024-04-20T21:18:51.064 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-20T21:18:51.072 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-20T21:18:51.074 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-20T21:18:52.577 INFO:teuthology.orchestra.run.smithi033.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T21:18:52.744 INFO:teuthology.orchestra.run.smithi122.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T21:18:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T21:20:00.746 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-20T21:20:00.747 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:20:00.747 INFO:teuthology.orchestra.run.smithi122.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-20T21:20:00.747 INFO:teuthology.orchestra.run.smithi122.stdout: "repo_digests": [ 2024-04-20T21:20:00.747 INFO:teuthology.orchestra.run.smithi122.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-20T21:20:00.747 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-20T21:20:00.747 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-20T21:21:29.356 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-20T21:21:29.357 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:21:29.357 INFO:teuthology.orchestra.run.smithi033.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-20T21:21:29.357 INFO:teuthology.orchestra.run.smithi033.stdout: "repo_digests": [ 2024-04-20T21:21:29.357 INFO:teuthology.orchestra.run.smithi033.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-20T21:21:29.357 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-20T21:21:29.357 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-20T21:22:02.100 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:22:02.100 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:22:02.100 INFO:teuthology.orchestra.run.smithi022.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-20T21:22:02.100 INFO:teuthology.orchestra.run.smithi022.stdout: "repo_digests": [ 2024-04-20T21:22:02.100 INFO:teuthology.orchestra.run.smithi022.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-20T21:22:02.100 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-20T21:22:02.100 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:22:02.139 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph 2024-04-20T21:22:02.229 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph 2024-04-20T21:22:02.262 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph 2024-04-20T21:22:02.294 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /etc/ceph 2024-04-20T21:22:02.384 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /etc/ceph 2024-04-20T21:22:02.413 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /etc/ceph 2024-04-20T21:22:02.440 INFO:tasks.cephadm:Writing seed config... 2024-04-20T21:22:02.441 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-20T21:22:02.441 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-20T21:22:02.441 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-20T21:22:02.441 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-20T21:22:02.441 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-20T21:22:02.441 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-20T21:22:02.441 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-20T21:22:02.441 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-20T21:22:02.441 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-20T21:22:02.441 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-20T21:22:02.441 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-20T21:22:02.442 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-20T21:22:02.442 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-20T21:22:02.442 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-20T21:22:02.442 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-20T21:22:02.442 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-20T21:22:02.442 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-20T21:22:02.442 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-20T21:22:02.442 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-20T21:22:02.442 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-20T21:22:02.442 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-20T21:22:02.442 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-20T21:22:02.442 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-20T21:22:02.443 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-20T21:22:02.444 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-20T21:22:02.444 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-20T21:22:02.444 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-20T21:22:02.444 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-20T21:22:02.444 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-20T21:22:02.444 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-20T21:22:02.444 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-20T21:22:02.444 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-20T21:22:02.444 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-20T21:22:02.445 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:22:02.445 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-20T21:22:02.504 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 92c49c5c-ff5b-11ee-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-20T21:22:02.504 DEBUG:teuthology.orchestra.run.smithi022:mon.a> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.a.service 2024-04-20T21:22:02.552 DEBUG:teuthology.orchestra.run.smithi022:mgr.x> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.x.service 2024-04-20T21:22:02.598 INFO:tasks.cephadm:Bootstrapping... 2024-04-20T21:22:02.598 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda -v bootstrap --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.22 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:22:02.881 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-04-20T21:22:02.882 INFO:teuthology.orchestra.run.smithi022.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda', '-v', 'bootstrap', '--fsid', '92c49c5c-ff5b-11ee-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.22', '--skip-admin-label'] 2024-04-20T21:22:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-20T21:22:02.882 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying podman|docker is present... 2024-04-20T21:22:02.950 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-20T21:22:02.951 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying lvm2 is present... 2024-04-20T21:22:02.951 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying time synchronization is in place... 2024-04-20T21:22:02.973 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-20T21:22:02.973 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-20T21:22:02.992 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-20T21:22:02.992 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-04-20T21:22:03.012 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-04-20T21:22:03.032 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-04-20T21:22:03.032 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-04-20T21:22:03.032 INFO:teuthology.orchestra.run.smithi022.stdout:Repeating the final host check... 2024-04-20T21:22:03.095 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-20T21:22:03.095 INFO:teuthology.orchestra.run.smithi022.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-20T21:22:03.095 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl is present 2024-04-20T21:22:03.095 INFO:teuthology.orchestra.run.smithi022.stdout:lvcreate is present 2024-04-20T21:22:03.110 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-20T21:22:03.110 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-20T21:22:03.125 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-20T21:22:03.125 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-04-20T21:22:03.143 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-04-20T21:22:03.159 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-04-20T21:22:03.159 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-04-20T21:22:03.159 INFO:teuthology.orchestra.run.smithi022.stdout:Host looks OK 2024-04-20T21:22:03.159 INFO:teuthology.orchestra.run.smithi022.stdout:Cluster fsid: 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:03.159 INFO:teuthology.orchestra.run.smithi022.stdout:Acquiring lock 140087010835616 on /run/cephadm/92c49c5c-ff5b-11ee-bc93-c7b262605968.lock 2024-04-20T21:22:03.160 INFO:teuthology.orchestra.run.smithi022.stdout:Lock 140087010835616 acquired on /run/cephadm/92c49c5c-ff5b-11ee-bc93-c7b262605968.lock 2024-04-20T21:22:03.160 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 3300 ... 2024-04-20T21:22:03.160 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 6789 ... 2024-04-20T21:22:03.161 INFO:teuthology.orchestra.run.smithi022.stdout:Base mon IP(s) is [172.21.15.22:3300, 172.21.15.22:6789], mon addrv is [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-04-20T21:22:03.169 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.22 metric 100 2024-04-20T21:22:03.169 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.22 metric 100 2024-04-20T21:22:03.176 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-20T21:22:03.176 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-20T21:22:03.176 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-20T21:22:03.183 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe59:5de4/64 scope link noprefixroute 2024-04-20T21:22:03.183 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-20T21:22:03.183 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-04-20T21:22:03.183 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-04-20T21:22:03.183 INFO:teuthology.orchestra.run.smithi022.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi022.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-20T21:22:03.185 INFO:teuthology.orchestra.run.smithi022.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T21:22:03.815 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-20T21:22:03.815 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T21:22:03.815 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Getting image source signatures 2024-04-20T21:22:03.815 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:9473d3fd79377e8e2904ab0f292b04da81cb2f2bfccde15ec927b481a725f6f4 2024-04-20T21:22:03.816 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-20T21:22:03.816 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying config sha256:76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-20T21:22:03.816 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-20T21:22:05.003 INFO:teuthology.orchestra.run.smithi022.stdout:ceph: stdout ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:22:05.003 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph version: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:22:05.003 INFO:teuthology.orchestra.run.smithi022.stdout:Extracting ceph user uid/gid from container image... 2024-04-20T21:22:06.104 INFO:teuthology.orchestra.run.smithi022.stdout:stat: stdout 167 167 2024-04-20T21:22:06.105 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial keys... 2024-04-20T21:22:07.184 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQD+MSRmFaJgIhAAwpMarh26b89SJzSo6UzIKw== 2024-04-20T21:22:08.333 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQD/MSRmEdWhJBAAJx5iTNdHHLKmh3l+gVfUew== 2024-04-20T21:22:09.458 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQAAMiRmHv9xLxAAOHvBQisjL/W7YKCA2Y8l9w== 2024-04-20T21:22:09.461 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial monmap... 2024-04-20T21:22:10.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-20T21:22:10.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-20T21:22:10.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:10.534 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-20T21:22:10.534 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool for a [v2:172.21.15.22:3300,v1:172.21.15.22:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-20T21:22:10.535 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = quincy 2024-04-20T21:22:10.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: set fsid to 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:10.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-20T21:22:10.535 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:22:10.535 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mon... 2024-04-20T21:22:11.601 INFO:teuthology.orchestra.run.smithi022.stdout:create mon.a on 2024-04-20T21:22:12.044 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-20T21:22:12.298 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-20T21:22:12.587 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968.target. 2024-04-20T21:22:12.587 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968.target. 2024-04-20T21:22:12.979 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.a 2024-04-20T21:22:12.979 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.a.service: Unit ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.a.service not loaded. 2024-04-20T21:22:13.266 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968.target.wants/ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@.service. 2024-04-20T21:22:13.978 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:13 smithi022 podman[16459]: 2024-04-20 21:22:13.685931743 +0000 UTC m=+0.136222046 container create 0bebbf11976aea0a9b606d837305a50be5c461980f37667f74a0a1a009ab762f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2) 2024-04-20T21:22:13.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:13 smithi022 podman[16459]: 2024-04-20 21:22:13.978693465 +0000 UTC m=+0.428983731 container init 0bebbf11976aea0a9b606d837305a50be5c461980f37667f74a0a1a009ab762f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:22:14.366 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:13 smithi022 podman[16459]: 2024-04-20 21:22:13.990412684 +0000 UTC m=+0.440702973 container start 0bebbf11976aea0a9b606d837305a50be5c461980f37667f74a0a1a009ab762f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-20T21:22:14.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:14 smithi022 ceph-mon[16478]: mkfs 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:14.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:14 smithi022 ceph-mon[16478]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-20T21:22:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:15 smithi022 ceph-mon[16478]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-20T21:22:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:15 smithi022 ceph-mon[16478]: monmap epoch 1 2024-04-20T21:22:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:15 smithi022 ceph-mon[16478]: fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:15 smithi022 ceph-mon[16478]: last_changed 2024-04-20T21:22:09.923834+0000 2024-04-20T21:22:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:15 smithi022 ceph-mon[16478]: created 2024-04-20T21:22:09.923834+0000 2024-04-20T21:22:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:15 smithi022 ceph-mon[16478]: min_mon_release 19 (squid) 2024-04-20T21:22:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:15 smithi022 ceph-mon[16478]: election_strategy: 1 2024-04-20T21:22:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:15 smithi022 ceph-mon[16478]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-04-20T21:22:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:15 smithi022 ceph-mon[16478]: fsmap 2024-04-20T21:22:15.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:15 smithi022 ceph-mon[16478]: osdmap e1: 0 total, 0 up, 0 in 2024-04-20T21:22:15.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:15 smithi022 ceph-mon[16478]: mgrmap e1: no daemons active 2024-04-20T21:22:19.136 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T21:22:19.137 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-04-20T21:22:19.158 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T21:22:19.158 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-04-20T21:22:19.159 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-04-20T21:22:19.159 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-04-20T21:22:19.160 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon to start... 2024-04-20T21:22:19.160 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon... 2024-04-20T21:22:19.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:19 smithi022 bash[16459]: 0bebbf11976aea0a9b606d837305a50be5c461980f37667f74a0a1a009ab762f 2024-04-20T21:22:19.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:19 smithi022 systemd[1]: Started Ceph mon.a for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:22:22.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:22 smithi022 ceph-mon[16478]: from='client.? 172.21.15.22:0/1320728046' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-20T21:22:23.027 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout cluster: 2024-04-20T21:22:23.031 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout id: 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout services: 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 8s) 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout data: 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pgs: 2024-04-20T21:22:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.292+0000 7fe420800700 1 Processor -- start 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.292+0000 7fe420800700 1 -- start start 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.292+0000 7fe420800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 0x7fe41810f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.292+0000 7fe420800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe41810fa40 con 0x7fe41810f050 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.293+0000 7fe41de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 0x7fe41810f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.294+0000 7fe41de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 0x7fe41810f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38192/0 (socket says 172.21.15.22:38192) 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.294+0000 7fe41de00700 1 -- 172.21.15.22:0/2498865715 learned_addr learned my addr 172.21.15.22:0/2498865715 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.294+0000 7fe41de00700 1 -- 172.21.15.22:0/2498865715 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41810fd70 con 0x7fe41810f050 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.295+0000 7fe41de00700 1 --2- 172.21.15.22:0/2498865715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 0x7fe41810f470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0098a0 tx=0x7fe40c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e21616dcaa88d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.295+0000 7fe41ca00700 1 -- 172.21.15.22:0/2498865715 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe40c004160 con 0x7fe41810f050 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.295+0000 7fe41ca00700 1 -- 172.21.15.22:0/2498865715 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40c004300 con 0x7fe41810f050 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.295+0000 7fe41ca00700 1 -- 172.21.15.22:0/2498865715 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe40c0036a0 con 0x7fe41810f050 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.296+0000 7fe420800700 1 -- 172.21.15.22:0/2498865715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 msgr2=0x7fe41810f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.033 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.296+0000 7fe420800700 1 --2- 172.21.15.22:0/2498865715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 0x7fe41810f470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0098a0 tx=0x7fe40c009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.034 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.296+0000 7fe420800700 1 -- 172.21.15.22:0/2498865715 shutdown_connections 2024-04-20T21:22:23.034 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.296+0000 7fe420800700 1 --2- 172.21.15.22:0/2498865715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 0x7fe41810f470 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.034 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.296+0000 7fe420800700 1 -- 172.21.15.22:0/2498865715 >> 172.21.15.22:0/2498865715 conn(0x7fe41807ddd0 msgr2=0x7fe4180801f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:23.034 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.296+0000 7fe420800700 1 -- 172.21.15.22:0/2498865715 shutdown_connections 2024-04-20T21:22:23.034 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.296+0000 7fe420800700 1 -- 172.21.15.22:0/2498865715 wait complete. 2024-04-20T21:22:23.034 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.297+0000 7fe420800700 1 Processor -- start 2024-04-20T21:22:23.034 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.297+0000 7fe420800700 1 -- start start 2024-04-20T21:22:23.034 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.298+0000 7fe420800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 0x7fe4181a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.034 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.298+0000 7fe420800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe4181102a0 con 0x7fe41810f050 2024-04-20T21:22:23.034 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.298+0000 7fe41de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 0x7fe4181a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.036 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.298+0000 7fe41de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 0x7fe4181a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38206/0 (socket says 172.21.15.22:38206) 2024-04-20T21:22:23.036 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.298+0000 7fe41de00700 1 -- 172.21.15.22:0/1320728046 learned_addr learned my addr 172.21.15.22:0/1320728046 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.298+0000 7fe41de00700 1 -- 172.21.15.22:0/1320728046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4181a2760 con 0x7fe41810f050 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.299+0000 7fe41de00700 1 --2- 172.21.15.22:0/1320728046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 0x7fe4181a2220 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009fb0 tx=0x7fe40c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.299+0000 7fe412a00700 1 -- 172.21.15.22:0/1320728046 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe40c003de0 con 0x7fe41810f050 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.299+0000 7fe412a00700 1 -- 172.21.15.22:0/1320728046 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40c011630 con 0x7fe41810f050 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.299+0000 7fe412a00700 1 -- 172.21.15.22:0/1320728046 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe40c011930 con 0x7fe41810f050 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.299+0000 7fe420800700 1 -- 172.21.15.22:0/1320728046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4181a29f0 con 0x7fe41810f050 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.299+0000 7fe420800700 1 -- 172.21.15.22:0/1320728046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4181a56e0 con 0x7fe41810f050 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.299+0000 7fe412a00700 1 -- 172.21.15.22:0/1320728046 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe40c011ad0 con 0x7fe41810f050 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.299+0000 7fe412a00700 1 -- 172.21.15.22:0/1320728046 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe40c01b450 con 0x7fe41810f050 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.300+0000 7fe420800700 1 -- 172.21.15.22:0/1320728046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4180640b0 con 0x7fe41810f050 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.304+0000 7fe412a00700 1 -- 172.21.15.22:0/1320728046 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fe40c004260 con 0x7fe41810f050 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.386+0000 7fe420800700 1 -- 172.21.15.22:0/1320728046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe4181a5c50 con 0x7fe41810f050 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.387+0000 7fe412a00700 1 -- 172.21.15.22:0/1320728046 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+310 (secure 0 0 0) 0x7fe40c025480 con 0x7fe41810f050 2024-04-20T21:22:23.037 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.388+0000 7fe420800700 1 -- 172.21.15.22:0/1320728046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 msgr2=0x7fe4181a2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.038 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.388+0000 7fe420800700 1 --2- 172.21.15.22:0/1320728046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 0x7fe4181a2220 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009fb0 tx=0x7fe40c004750 comp rx=0 tx=0).stop 2024-04-20T21:22:23.038 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.388+0000 7fe420800700 1 -- 172.21.15.22:0/1320728046 shutdown_connections 2024-04-20T21:22:23.038 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.388+0000 7fe420800700 1 --2- 172.21.15.22:0/1320728046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe41810f050 0x7fe4181a2220 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.038 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.388+0000 7fe420800700 1 -- 172.21.15.22:0/1320728046 >> 172.21.15.22:0/1320728046 conn(0x7fe41807ddd0 msgr2=0x7fe418196590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:23.038 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.389+0000 7fe420800700 1 -- 172.21.15.22:0/1320728046 shutdown_connections 2024-04-20T21:22:23.038 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.389+0000 7fe420800700 1 -- 172.21.15.22:0/1320728046 wait complete. 2024-04-20T21:22:23.038 INFO:teuthology.orchestra.run.smithi022.stdout:mon is available 2024-04-20T21:22:23.038 INFO:teuthology.orchestra.run.smithi022.stdout:Assimilating anything we can from ceph.conf... 2024-04-20T21:22:24.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:23 smithi022 ceph-mon[16478]: from='client.? 172.21.15.22:0/4189509715' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-20T21:22:24.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:23 smithi022 ceph-mon[16478]: from='client.? 172.21.15.22:0/4189509715' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-20T21:22:24.579 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-20T21:22:24.580 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.806+0000 7f80b9800700 1 Processor -- start 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.806+0000 7f80b9800700 1 -- start start 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.806+0000 7f80b9800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 0x7f80b410f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.807+0000 7f80b9800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80b410fa40 con 0x7f80b410f050 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.807+0000 7f80b2a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 0x7f80b410f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.807+0000 7f80b2a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 0x7f80b410f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35240/0 (socket says 172.21.15.22:35240) 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.807+0000 7f80b2a00700 1 -- 172.21.15.22:0/2844259121 learned_addr learned my addr 172.21.15.22:0/2844259121 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.808+0000 7f80b2a00700 1 -- 172.21.15.22:0/2844259121 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b410fd70 con 0x7f80b410f050 2024-04-20T21:22:24.581 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.808+0000 7f80b2a00700 1 --2- 172.21.15.22:0/2844259121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 0x7f80b410f470 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f80a40098a0 tx=0x7f80a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb905ee4f8899dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:24.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.809+0000 7f80b1600700 1 -- 172.21.15.22:0/2844259121 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80a4004160 con 0x7f80b410f050 2024-04-20T21:22:24.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.809+0000 7f80b1600700 1 -- 172.21.15.22:0/2844259121 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a4004300 con 0x7f80b410f050 2024-04-20T21:22:24.582 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.809+0000 7f80b1600700 1 -- 172.21.15.22:0/2844259121 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80a40036a0 con 0x7f80b410f050 2024-04-20T21:22:24.585 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.809+0000 7f80b9800700 1 -- 172.21.15.22:0/2844259121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 msgr2=0x7f80b410f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.585 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.809+0000 7f80b9800700 1 --2- 172.21.15.22:0/2844259121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 0x7f80b410f470 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f80a40098a0 tx=0x7f80a4009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.586 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.809+0000 7f80b9800700 1 -- 172.21.15.22:0/2844259121 shutdown_connections 2024-04-20T21:22:24.586 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.809+0000 7f80b9800700 1 --2- 172.21.15.22:0/2844259121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 0x7f80b410f470 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.586 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.809+0000 7f80b9800700 1 -- 172.21.15.22:0/2844259121 >> 172.21.15.22:0/2844259121 conn(0x7f80b407ddd0 msgr2=0x7f80b40801f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:24.586 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.810+0000 7f80b9800700 1 -- 172.21.15.22:0/2844259121 shutdown_connections 2024-04-20T21:22:24.586 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.810+0000 7f80b9800700 1 -- 172.21.15.22:0/2844259121 wait complete. 2024-04-20T21:22:24.586 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.811+0000 7f80b9800700 1 Processor -- start 2024-04-20T21:22:24.586 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.811+0000 7f80b9800700 1 -- start start 2024-04-20T21:22:24.586 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.811+0000 7f80b9800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 0x7f80b41a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.586 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.811+0000 7f80b9800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41102a0 con 0x7f80b410f050 2024-04-20T21:22:24.586 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.812+0000 7f80b2a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 0x7f80b41a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:24.586 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.812+0000 7f80b2a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 0x7f80b41a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35250/0 (socket says 172.21.15.22:35250) 2024-04-20T21:22:24.586 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.812+0000 7f80b2a00700 1 -- 172.21.15.22:0/4189509715 learned_addr learned my addr 172.21.15.22:0/4189509715 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.812+0000 7f80b2a00700 1 -- 172.21.15.22:0/4189509715 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b41a2760 con 0x7f80b410f050 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.812+0000 7f80b2a00700 1 --2- 172.21.15.22:0/4189509715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 0x7f80b41a2220 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f80a4000c00 tx=0x7f80a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.813+0000 7f80ab400700 1 -- 172.21.15.22:0/4189509715 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80a4003db0 con 0x7f80b410f050 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.813+0000 7f80ab400700 1 -- 172.21.15.22:0/4189509715 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a4011630 con 0x7f80b410f050 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.813+0000 7f80ab400700 1 -- 172.21.15.22:0/4189509715 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80a4011950 con 0x7f80b410f050 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.813+0000 7f80b9800700 1 -- 172.21.15.22:0/4189509715 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b41a29f0 con 0x7f80b410f050 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.813+0000 7f80b9800700 1 -- 172.21.15.22:0/4189509715 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b41a56e0 con 0x7f80b410f050 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.813+0000 7f80ab400700 1 -- 172.21.15.22:0/4189509715 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f80a4011af0 con 0x7f80b410f050 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.813+0000 7f80ab400700 1 -- 172.21.15.22:0/4189509715 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f80a401c790 con 0x7f80b410f050 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.814+0000 7f80b9800700 1 -- 172.21.15.22:0/4189509715 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b40640b0 con 0x7f80b410f050 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.817+0000 7f80ab400700 1 -- 172.21.15.22:0/4189509715 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f80a4032020 con 0x7f80b410f050 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.895+0000 7f80b9800700 1 -- 172.21.15.22:0/4189509715 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f80b41a58b0 con 0x7f80b410f050 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.902+0000 7f80ab400700 1 -- 172.21.15.22:0/4189509715 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+456 (secure 0 0 0) 0x7f80a4004260 con 0x7f80b410f050 2024-04-20T21:22:24.587 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.904+0000 7f80ab400700 1 -- 172.21.15.22:0/4189509715 <== mon.0 v2:172.21.15.22:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80a401ce50 con 0x7f80b410f050 2024-04-20T21:22:24.588 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.904+0000 7f80b9800700 1 -- 172.21.15.22:0/4189509715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 msgr2=0x7f80b41a2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.588 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.904+0000 7f80b9800700 1 --2- 172.21.15.22:0/4189509715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 0x7f80b41a2220 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f80a4000c00 tx=0x7f80a4004750 comp rx=0 tx=0).stop 2024-04-20T21:22:24.588 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.904+0000 7f80b9800700 1 -- 172.21.15.22:0/4189509715 shutdown_connections 2024-04-20T21:22:24.588 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.904+0000 7f80b9800700 1 --2- 172.21.15.22:0/4189509715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b410f050 0x7f80b41a2220 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.588 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.904+0000 7f80b9800700 1 -- 172.21.15.22:0/4189509715 >> 172.21.15.22:0/4189509715 conn(0x7f80b407ddd0 msgr2=0x7f80b4196590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:24.588 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.904+0000 7f80b9800700 1 -- 172.21.15.22:0/4189509715 shutdown_connections 2024-04-20T21:22:24.588 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.904+0000 7f80b9800700 1 -- 172.21.15.22:0/4189509715 wait complete. 2024-04-20T21:22:24.588 INFO:teuthology.orchestra.run.smithi022.stdout:Generating new minimal ceph.conf... 2024-04-20T21:22:25.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:25 smithi022 ceph-mon[16478]: from='client.? 172.21.15.22:0/483900590' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:22:26.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.289+0000 7ff0c4c00700 1 Processor -- start 2024-04-20T21:22:26.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.290+0000 7ff0c4c00700 1 -- start start 2024-04-20T21:22:26.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.290+0000 7ff0c4c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 0x7ff0c010f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.290+0000 7ff0c4c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c010fa40 con 0x7ff0c010f050 2024-04-20T21:22:26.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.290+0000 7ff0bde00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 0x7ff0c010f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.290+0000 7ff0bde00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 0x7ff0c010f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35264/0 (socket says 172.21.15.22:35264) 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.290+0000 7ff0bde00700 1 -- 172.21.15.22:0/2319891491 learned_addr learned my addr 172.21.15.22:0/2319891491 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.291+0000 7ff0bde00700 1 -- 172.21.15.22:0/2319891491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c010fd70 con 0x7ff0c010f050 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.291+0000 7ff0bde00700 1 --2- 172.21.15.22:0/2319891491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 0x7ff0c010f470 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00098a0 tx=0x7ff0b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d8fd99d280877b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.292+0000 7ff0bca00700 1 -- 172.21.15.22:0/2319891491 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0b0004160 con 0x7ff0c010f050 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.292+0000 7ff0bca00700 1 -- 172.21.15.22:0/2319891491 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff0b0004300 con 0x7ff0c010f050 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.292+0000 7ff0bca00700 1 -- 172.21.15.22:0/2319891491 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0b0011b50 con 0x7ff0c010f050 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.292+0000 7ff0c4c00700 1 -- 172.21.15.22:0/2319891491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 msgr2=0x7ff0c010f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.292+0000 7ff0c4c00700 1 --2- 172.21.15.22:0/2319891491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 0x7ff0c010f470 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00098a0 tx=0x7ff0b0009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.293+0000 7ff0c4c00700 1 -- 172.21.15.22:0/2319891491 shutdown_connections 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.293+0000 7ff0c4c00700 1 --2- 172.21.15.22:0/2319891491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 0x7ff0c010f470 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.293+0000 7ff0c4c00700 1 -- 172.21.15.22:0/2319891491 >> 172.21.15.22:0/2319891491 conn(0x7ff0c007ddd0 msgr2=0x7ff0c00801f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.293+0000 7ff0c4c00700 1 -- 172.21.15.22:0/2319891491 shutdown_connections 2024-04-20T21:22:26.080 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.293+0000 7ff0c4c00700 1 -- 172.21.15.22:0/2319891491 wait complete. 2024-04-20T21:22:26.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.294+0000 7ff0c4c00700 1 Processor -- start 2024-04-20T21:22:26.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.294+0000 7ff0c4c00700 1 -- start start 2024-04-20T21:22:26.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.294+0000 7ff0c4c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 0x7ff0c01a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.294+0000 7ff0c4c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c0113120 con 0x7ff0c010f050 2024-04-20T21:22:26.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.295+0000 7ff0bde00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 0x7ff0c01a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.295+0000 7ff0bde00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 0x7ff0c01a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35266/0 (socket says 172.21.15.22:35266) 2024-04-20T21:22:26.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.295+0000 7ff0bde00700 1 -- 172.21.15.22:0/483900590 learned_addr learned my addr 172.21.15.22:0/483900590 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:26.081 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.296+0000 7ff0bde00700 1 -- 172.21.15.22:0/483900590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c01a3000 con 0x7ff0c010f050 2024-04-20T21:22:26.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.296+0000 7ff0bde00700 1 --2- 172.21.15.22:0/483900590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 0x7ff0c01a2ac0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0011b60 tx=0x7ff0b0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:26.083 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.296+0000 7ff0b6a00700 1 -- 172.21.15.22:0/483900590 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0b0003b20 con 0x7ff0c010f050 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.296+0000 7ff0b6a00700 1 -- 172.21.15.22:0/483900590 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff0b0003cc0 con 0x7ff0c010f050 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.296+0000 7ff0b6a00700 1 -- 172.21.15.22:0/483900590 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0b0024840 con 0x7ff0c010f050 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.296+0000 7ff0c4c00700 1 -- 172.21.15.22:0/483900590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c01a3290 con 0x7ff0c010f050 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.296+0000 7ff0c4c00700 1 -- 172.21.15.22:0/483900590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c01a5f80 con 0x7ff0c010f050 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.297+0000 7ff0b6a00700 1 -- 172.21.15.22:0/483900590 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff0b0022020 con 0x7ff0c010f050 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.297+0000 7ff0b6a00700 1 -- 172.21.15.22:0/483900590 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff0b0011e00 con 0x7ff0c010f050 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.298+0000 7ff0c4c00700 1 -- 172.21.15.22:0/483900590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c00640b0 con 0x7ff0c010f050 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.300+0000 7ff0b6a00700 1 -- 172.21.15.22:0/483900590 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7ff0b0043e50 con 0x7ff0c010f050 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.379+0000 7ff0c4c00700 1 -- 172.21.15.22:0/483900590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff0c01a6660 con 0x7ff0c010f050 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.380+0000 7ff0b6a00700 1 -- 172.21.15.22:0/483900590 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff0b0032020 con 0x7ff0c010f050 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.382+0000 7ff0c4c00700 1 -- 172.21.15.22:0/483900590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 msgr2=0x7ff0c01a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.382+0000 7ff0c4c00700 1 --2- 172.21.15.22:0/483900590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 0x7ff0c01a2ac0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0011b60 tx=0x7ff0b0011c40 comp rx=0 tx=0).stop 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.382+0000 7ff0c4c00700 1 -- 172.21.15.22:0/483900590 shutdown_connections 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.382+0000 7ff0c4c00700 1 --2- 172.21.15.22:0/483900590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c010f050 0x7ff0c01a2ac0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.382+0000 7ff0c4c00700 1 -- 172.21.15.22:0/483900590 >> 172.21.15.22:0/483900590 conn(0x7ff0c007ddd0 msgr2=0x7ff0c007f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:26.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.382+0000 7ff0c4c00700 1 -- 172.21.15.22:0/483900590 shutdown_connections 2024-04-20T21:22:26.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:25.382+0000 7ff0c4c00700 1 -- 172.21.15.22:0/483900590 wait complete. 2024-04-20T21:22:26.085 INFO:teuthology.orchestra.run.smithi022.stdout:Restarting the monitor... 2024-04-20T21:22:26.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:26 smithi022 systemd[1]: Stopping Ceph mon.a for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:22:26.864 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:26 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a[16474]: 2024-04-20T21:22:26.607+0000 7f512a200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:22:26.864 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:26 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a[16474]: 2024-04-20T21:22:26.607+0000 7f512a200700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-20T21:22:27.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:26 smithi022 podman[16720]: 2024-04-20 21:22:26.865428226 +0000 UTC m=+0.381509908 container died 0bebbf11976aea0a9b606d837305a50be5c461980f37667f74a0a1a009ab762f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-20T21:22:27.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:27 smithi022 podman[16720]: 2024-04-20 21:22:27.450813288 +0000 UTC m=+0.966894979 container cleanup 0bebbf11976aea0a9b606d837305a50be5c461980f37667f74a0a1a009ab762f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-20T21:22:27.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:27 smithi022 bash[16720]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a 2024-04-20T21:22:28.026 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:27 smithi022 podman[16734]: 2024-04-20 21:22:27.76061099 +0000 UTC m=+0.882244724 container remove 0bebbf11976aea0a9b606d837305a50be5c461980f37667f74a0a1a009ab762f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_CLEAN=True) 2024-04-20T21:22:28.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 systemd[1]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-20T21:22:28.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 systemd[1]: Stopped Ceph mon.a for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:22:28.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 systemd[1]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.a.service: Consumed 2.029s CPU time. 2024-04-20T21:22:28.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 systemd[1]: Starting Ceph mon.a for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:22:28.739 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 podman[16824]: 2024-04-20 21:22:28.378852838 +0000 UTC m=+0.034838827 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:22:28.739 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 podman[16824]: 2024-04-20 21:22:28.503139384 +0000 UTC m=+0.159125410 container create 4f99bfab2067b8b8eb61f95fc827b57a5dc4e2949e812d4b08305a36cd942d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:22:28.739 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 podman[16824]: 2024-04-20 21:22:28.739980063 +0000 UTC m=+0.395966095 container init 4f99bfab2067b8b8eb61f95fc827b57a5dc4e2949e812d4b08305a36cd942d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:22:28.825 INFO:teuthology.orchestra.run.smithi022.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-20T21:22:29.111 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 podman[16824]: 2024-04-20 21:22:28.751122549 +0000 UTC m=+0.407108553 container start 4f99bfab2067b8b8eb61f95fc827b57a5dc4e2949e812d4b08305a36cd942d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:22:29.111 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: set uid:gid to 167:167 (ceph:ceph) 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: pidfile_write: ignore empty --pid-file 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: load: jerasure load: lrc 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: RocksDB version: 7.9.2 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Git sha 0 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: DB SUMMARY 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: DB Session ID: ZG80QB9I43JLUW4ZSRHZ 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: CURRENT file: CURRENT 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: IDENTITY file: IDENTITY 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105731 ; 2024-04-20T21:22:29.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.error_if_exists: 0 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.create_if_missing: 0 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.paranoid_checks: 1 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.env: 0x557019617cc0 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.fs: PosixFileSystem 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.info_log: 0x5570552cfc60 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_file_opening_threads: 16 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.statistics: (nil) 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.use_fsync: 0 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_log_file_size: 0 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-20T21:22:29.113 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.keep_log_file_num: 1000 2024-04-20T21:22:29.114 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.recycle_log_file_num: 0 2024-04-20T21:22:29.114 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.allow_fallocate: 1 2024-04-20T21:22:29.114 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.allow_mmap_reads: 0 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.allow_mmap_writes: 0 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.use_direct_reads: 0 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.create_missing_column_families: 0 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.db_log_dir: 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.wal_dir: 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.advise_random_on_open: 1 2024-04-20T21:22:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.db_write_buffer_size: 0 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.write_buffer_manager: 0x55705529e1e0 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.rate_limiter: (nil) 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.wal_recovery_mode: 2 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.enable_thread_tracking: 0 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.enable_pipelined_write: 0 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.unordered_write: 0 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-20T21:22:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.row_cache: None 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.wal_filter: None 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.allow_ingest_behind: 0 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.two_write_queues: 0 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.manual_wal_flush: 0 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.wal_compression: 0 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.atomic_flush: 0 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.log_readahead_size: 0 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.best_efforts_recovery: 0 2024-04-20T21:22:29.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.allow_data_in_errors: 0 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.db_host_id: __hostname__ 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_background_jobs: 2 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_background_compactions: -1 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_subcompactions: 1 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_total_wal_size: 0 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-20T21:22:29.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_open_files: -1 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bytes_per_sync: 0 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_readahead_size: 0 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_background_flushes: -1 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Compression algorithms supported: 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: kZSTD supported: 0 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: kXpressCompression supported: 0 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: kLZ4HCCompression supported: 1 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: kZlibCompression supported: 1 2024-04-20T21:22:29.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: kSnappyCompression supported: 1 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: kLZ4Compression supported: 1 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: kBZip2Compression supported: 0 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.merge_operator: 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_filter: None 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_filter_factory: None 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.sst_partitioner_factory: None 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-20T21:22:29.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5570552cfe20) 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks: 1 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_top_level_index_and_filter: 1 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_type: 0 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_index_type: 0 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_shortening: 1 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: checksum: 4 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: no_block_cache: 0 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache: 0x5570552d1090 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_name: BinnedLRUCache 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_options: 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: capacity : 536870912 2024-04-20T21:22:29.121 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_shard_bits : 4 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: strict_capacity_limit : 0 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: high_pri_pool_ratio: 0.000 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_compressed: (nil) 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: persistent_cache: (nil) 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size: 4096 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size_deviation: 10 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_restart_interval: 16 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_block_restart_interval: 1 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: metadata_block_size: 4096 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: partition_filters: 0 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: use_delta_encoding: 1 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: filter_policy: bloomfilter 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: whole_key_filtering: 1 2024-04-20T21:22:29.122 INFO:journalctl@ceph.mon.a.smithi022.stdout: verify_compression: 0 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout: read_amp_bytes_per_bit: 0 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout: format_version: 5 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_index_compression: 1 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_align: 0 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout: max_auto_readahead_size: 262144 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout: prepopulate_block_cache: 0 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout: initial_auto_readahead_size: 8192 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_file_reads_for_auto_readahead: 2 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.write_buffer_size: 33554432 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_write_buffer_number: 2 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compression: NoCompression 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bottommost_compression: Disabled 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.prefix_extractor: nullptr 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-20T21:22:29.123 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.num_levels: 7 2024-04-20T21:22:29.124 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-20T21:22:29.124 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 bash[16824]: 4f99bfab2067b8b8eb61f95fc827b57a5dc4e2949e812d4b08305a36cd942d4c 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compression_opts.level: 32767 2024-04-20T21:22:29.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compression_opts.strategy: 0 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compression_opts.enabled: false 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.target_file_size_base: 67108864 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-20T21:22:29.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.arena_block_size: 1048576 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-20T21:22:29.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.disable_auto_compactions: 0 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.inplace_update_support: 0 2024-04-20T21:22:29.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.bloom_locality: 0 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.max_successive_merges: 0 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.paranoid_file_checks: 0 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.force_consistency_checks: 1 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.report_bg_io_stats: 0 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.ttl: 2592000 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-20T21:22:29.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.enable_blob_files: false 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.min_blob_size: 0 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.blob_file_size: 268435456 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.blob_file_starting_level: 0 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2d8d9d08-311d-4c4c-b5aa-a8fa59cc10ae 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713648148811392, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-20T21:22:29.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713648148813267, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100664, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 98549, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14298, "raw_average_key_size": 49, "raw_value_size": 90804, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713648148, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2d8d9d08-311d-4c4c-b5aa-a8fa59cc10ae", "db_session_id": "ZG80QB9I43JLUW4ZSRHZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713648148813425, "job": 1, "event": "recovery_finished"} 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5570553be000 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: DB pointer 0x5570553a8000 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** DB Stats ** 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T21:22:29.131 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: L0 2/0 100.17 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: Sum 2/0 100.17 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-20T21:22:29.132 INFO:journalctl@ceph.mon.a.smithi022.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 77.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative compaction: 0.00 GB write, 11.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval compaction: 0.00 GB write, 11.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: Block cache BinnedLRUCache@0x5570552d1090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-20T21:22:29.133 INFO:journalctl@ceph.mon.a.smithi022.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: starting mon.a rank 0 at public addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] at bind addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: mon.a@-1(???) e1 preinit fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: mon.a@-1(???).mds e1 new map 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: mon.a@-1(???).mds e1 print_map 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout: e1 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout: legacy client fscid: -1 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout: No filesystems configured 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-20T21:22:29.134 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 systemd[1]: Started Ceph mon.a for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: monmap epoch 1 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: last_changed 2024-04-20T21:22:09.923834+0000 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: created 2024-04-20T21:22:09.923834+0000 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: min_mon_release 19 (squid) 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: election_strategy: 1 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: fsmap 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: osdmap e1: 0 total, 0 up, 0 in 2024-04-20T21:22:29.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:28 smithi022 ceph-mon[16843]: mgrmap e1: no daemons active 2024-04-20T21:22:30.221 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.466+0000 7f8e78200700 1 Processor -- start 2024-04-20T21:22:30.221 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.466+0000 7f8e78200700 1 -- start start 2024-04-20T21:22:30.222 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.467+0000 7f8e78200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 0x7f8e70108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.222 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.467+0000 7f8e78200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70108f50 con 0x7f8e70108560 2024-04-20T21:22:30.222 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.468+0000 7f8e75800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 0x7f8e70108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.222 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.468+0000 7f8e75800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 0x7f8e70108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35282/0 (socket says 172.21.15.22:35282) 2024-04-20T21:22:30.223 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.468+0000 7f8e75800700 1 -- 172.21.15.22:0/559376835 learned_addr learned my addr 172.21.15.22:0/559376835 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:30.223 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.468+0000 7f8e75800700 1 -- 172.21.15.22:0/559376835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e70109710 con 0x7f8e70108560 2024-04-20T21:22:30.223 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.469+0000 7f8e75800700 1 --2- 172.21.15.22:0/559376835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 0x7f8e70108980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009e80 tx=0x7f8e6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c5132563c473cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:30.223 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.469+0000 7f8e6fe00700 1 -- 172.21.15.22:0/559376835 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e64017070 con 0x7f8e70108560 2024-04-20T21:22:30.223 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.469+0000 7f8e6fe00700 1 -- 172.21.15.22:0/559376835 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8e6400b8c0 con 0x7f8e70108560 2024-04-20T21:22:30.224 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.469+0000 7f8e6fe00700 1 -- 172.21.15.22:0/559376835 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e640045c0 con 0x7f8e70108560 2024-04-20T21:22:30.224 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.470+0000 7f8e78200700 1 -- 172.21.15.22:0/559376835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 msgr2=0x7f8e70108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.224 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.470+0000 7f8e78200700 1 --2- 172.21.15.22:0/559376835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 0x7f8e70108980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009e80 tx=0x7f8e6400b330 comp rx=0 tx=0).stop 2024-04-20T21:22:30.224 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.470+0000 7f8e78200700 1 -- 172.21.15.22:0/559376835 shutdown_connections 2024-04-20T21:22:30.225 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.470+0000 7f8e78200700 1 --2- 172.21.15.22:0/559376835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 0x7f8e70108980 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.225 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.470+0000 7f8e78200700 1 -- 172.21.15.22:0/559376835 >> 172.21.15.22:0/559376835 conn(0x7f8e70103b80 msgr2=0x7f8e70105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:30.225 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.470+0000 7f8e78200700 1 -- 172.21.15.22:0/559376835 shutdown_connections 2024-04-20T21:22:30.225 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.470+0000 7f8e78200700 1 -- 172.21.15.22:0/559376835 wait complete. 2024-04-20T21:22:30.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.471+0000 7f8e78200700 1 Processor -- start 2024-04-20T21:22:30.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.472+0000 7f8e78200700 1 -- start start 2024-04-20T21:22:30.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.472+0000 7f8e78200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 0x7f8e7019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.472+0000 7f8e78200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010cac0 con 0x7f8e70108560 2024-04-20T21:22:30.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.472+0000 7f8e75800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 0x7f8e7019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.472+0000 7f8e75800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 0x7f8e7019e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35288/0 (socket says 172.21.15.22:35288) 2024-04-20T21:22:30.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.472+0000 7f8e75800700 1 -- 172.21.15.22:0/1799945718 learned_addr learned my addr 172.21.15.22:0/1799945718 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:30.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.473+0000 7f8e75800700 1 -- 172.21.15.22:0/1799945718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7019ec20 con 0x7f8e70108560 2024-04-20T21:22:30.228 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.473+0000 7f8e75800700 1 --2- 172.21.15.22:0/1799945718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 0x7f8e7019e6e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8e640072a0 tx=0x7f8e6400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:30.228 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.473+0000 7f8e6e000700 1 -- 172.21.15.22:0/1799945718 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e64020020 con 0x7f8e70108560 2024-04-20T21:22:30.228 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.473+0000 7f8e6e000700 1 -- 172.21.15.22:0/1799945718 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8e64003b50 con 0x7f8e70108560 2024-04-20T21:22:30.228 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.473+0000 7f8e78200700 1 -- 172.21.15.22:0/1799945718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7019eeb0 con 0x7f8e70108560 2024-04-20T21:22:30.229 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.473+0000 7f8e6e000700 1 -- 172.21.15.22:0/1799945718 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e64017040 con 0x7f8e70108560 2024-04-20T21:22:30.229 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.474+0000 7f8e78200700 1 -- 172.21.15.22:0/1799945718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e701a1ba0 con 0x7f8e70108560 2024-04-20T21:22:30.229 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.474+0000 7f8e6e000700 1 -- 172.21.15.22:0/1799945718 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8e64023020 con 0x7f8e70108560 2024-04-20T21:22:30.229 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.474+0000 7f8e6e000700 1 -- 172.21.15.22:0/1799945718 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8e6400f050 con 0x7f8e70108560 2024-04-20T21:22:30.229 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.475+0000 7f8e78200700 1 -- 172.21.15.22:0/1799945718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e700640b0 con 0x7f8e70108560 2024-04-20T21:22:30.230 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.478+0000 7f8e6e000700 1 -- 172.21.15.22:0/1799945718 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f8e6402a020 con 0x7f8e70108560 2024-04-20T21:22:30.230 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.555+0000 7f8e78200700 1 -- 172.21.15.22:0/1799945718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f8e701a1d70 con 0x7f8e70108560 2024-04-20T21:22:30.230 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.559+0000 7f8e6e000700 1 -- 172.21.15.22:0/1799945718 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f8e701a1d70 con 0x7f8e70108560 2024-04-20T21:22:30.230 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.560+0000 7f8e78200700 1 -- 172.21.15.22:0/1799945718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 msgr2=0x7f8e7019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.230 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.560+0000 7f8e78200700 1 --2- 172.21.15.22:0/1799945718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 0x7f8e7019e6e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8e640072a0 tx=0x7f8e6400bd50 comp rx=0 tx=0).stop 2024-04-20T21:22:30.231 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.560+0000 7f8e78200700 1 -- 172.21.15.22:0/1799945718 shutdown_connections 2024-04-20T21:22:30.231 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.560+0000 7f8e78200700 1 --2- 172.21.15.22:0/1799945718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e70108560 0x7f8e7019e6e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.231 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.560+0000 7f8e78200700 1 -- 172.21.15.22:0/1799945718 >> 172.21.15.22:0/1799945718 conn(0x7f8e70103b80 msgr2=0x7f8e70104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:30.231 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.561+0000 7f8e78200700 1 -- 172.21.15.22:0/1799945718 shutdown_connections 2024-04-20T21:22:30.231 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:29.561+0000 7f8e78200700 1 -- 172.21.15.22:0/1799945718 wait complete. 2024-04-20T21:22:30.232 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-20T21:22:30.261 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:22:30.261 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mgr... 2024-04-20T21:22:30.263 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-20T21:22:30.264 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-20T21:22:30.730 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.x 2024-04-20T21:22:30.730 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.x.service: Unit ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-20T21:22:30.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:30 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1799945718' entity='client.admin' 2024-04-20T21:22:30.996 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968.target.wants/ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@.service. 2024-04-20T21:22:31.400 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:31 smithi022 systemd[1]: Starting Ceph mgr.x for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:22:31.658 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:31 smithi022 podman[17057]: 2024-04-20 21:22:31.307667513 +0000 UTC m=+0.038331102 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:22:31.658 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:31 smithi022 podman[17057]: 2024-04-20 21:22:31.406963379 +0000 UTC m=+0.137627003 container create 3f9d41b7577e9669ba391993b737e7ff6eacccbd34d3538e76d306dea63e8437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-20T21:22:31.763 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T21:22:31.763 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-04-20T21:22:31.779 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T21:22:31.779 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-04-20T21:22:31.779 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-04-20T21:22:31.779 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-04-20T21:22:31.798 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T21:22:31.798 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-04-20T21:22:31.814 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T21:22:31.814 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-04-20T21:22:31.814 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-04-20T21:22:31.814 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-20T21:22:31.814 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr to start... 2024-04-20T21:22:31.814 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr... 2024-04-20T21:22:31.920 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:31 smithi022 podman[17057]: 2024-04-20 21:22:31.66064272 +0000 UTC m=+0.391306347 container init 3f9d41b7577e9669ba391993b737e7ff6eacccbd34d3538e76d306dea63e8437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-20T21:22:31.920 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:31 smithi022 podman[17057]: 2024-04-20 21:22:31.672647898 +0000 UTC m=+0.403311491 container start 3f9d41b7577e9669ba391993b737e7ff6eacccbd34d3538e76d306dea63e8437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:22:31.920 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:31 smithi022 bash[17057]: 3f9d41b7577e9669ba391993b737e7ff6eacccbd34d3538e76d306dea63e8437 2024-04-20T21:22:31.921 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:31 smithi022 systemd[1]: Started Ceph mgr.x for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:22:32.185 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:31 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:31.919+0000 7f9a5477f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:22:32.436 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:32.196+0000 7f9a5477f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:22:32.436 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:32.338+0000 7f9a5477f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:22:32.790 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:32 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1116599417' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T21:22:33.156 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:32.790+0000 7f9a5477f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:22:33.156 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:32.916+0000 7f9a5477f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:22:33.230 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:33.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:22:33.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "92c49c5c-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:22:33.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T21:22:33.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T21:22:33.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T21:22:33.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:33.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T21:22:33.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:33.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T21:22:33.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T21:22:33.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T21:22:33.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T21:22:33.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T21:22:33.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T21:22:33.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:33.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T21:22:33.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T21:22:33.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T21:22:33.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T21:22:14.364851+0000", 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.433+0000 7f2a3c000700 1 Processor -- start 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.433+0000 7f2a3c000700 1 -- start start 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.434+0000 7f2a3c000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 0x7f2a34109210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.434+0000 7f2a3c000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3407bea0 con 0x7f2a34106de0 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.434+0000 7f2a39600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 0x7f2a34109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.435+0000 7f2a39600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 0x7f2a34109210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34348/0 (socket says 172.21.15.22:34348) 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.435+0000 7f2a39600700 1 -- 172.21.15.22:0/3504925528 learned_addr learned my addr 172.21.15.22:0/3504925528 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.435+0000 7f2a39600700 1 -- 172.21.15.22:0/3504925528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a34109750 con 0x7f2a34106de0 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.436+0000 7f2a39600700 1 --2- 172.21.15.22:0/3504925528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 0x7f2a34109210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2a280098a0 tx=0x7f2a28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd436d94f6945f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.436+0000 7f2a33e00700 1 -- 172.21.15.22:0/3504925528 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a28004160 con 0x7f2a34106de0 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.436+0000 7f2a33e00700 1 -- 172.21.15.22:0/3504925528 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2a28004300 con 0x7f2a34106de0 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.436+0000 7f2a33e00700 1 -- 172.21.15.22:0/3504925528 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a28011b50 con 0x7f2a34106de0 2024-04-20T21:22:33.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.437+0000 7f2a3c000700 1 -- 172.21.15.22:0/3504925528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 msgr2=0x7f2a34109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.437+0000 7f2a3c000700 1 --2- 172.21.15.22:0/3504925528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 0x7f2a34109210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2a280098a0 tx=0x7f2a28009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.437+0000 7f2a3c000700 1 -- 172.21.15.22:0/3504925528 shutdown_connections 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.437+0000 7f2a3c000700 1 --2- 172.21.15.22:0/3504925528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 0x7f2a34109210 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.437+0000 7f2a3c000700 1 -- 172.21.15.22:0/3504925528 >> 172.21.15.22:0/3504925528 conn(0x7f2a34102a20 msgr2=0x7f2a34104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.438+0000 7f2a3c000700 1 -- 172.21.15.22:0/3504925528 shutdown_connections 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.438+0000 7f2a3c000700 1 -- 172.21.15.22:0/3504925528 wait complete. 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.439+0000 7f2a3c000700 1 Processor -- start 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.439+0000 7f2a3c000700 1 -- start start 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.440+0000 7f2a3c000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 0x7f2a341a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.440+0000 7f2a3c000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3410a7e0 con 0x7f2a34106de0 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.440+0000 7f2a39600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 0x7f2a341a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.440+0000 7f2a39600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 0x7f2a341a2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34364/0 (socket says 172.21.15.22:34364) 2024-04-20T21:22:33.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.440+0000 7f2a39600700 1 -- 172.21.15.22:0/1116599417 learned_addr learned my addr 172.21.15.22:0/1116599417 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.441+0000 7f2a39600700 1 -- 172.21.15.22:0/1116599417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a341a30e0 con 0x7f2a34106de0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.441+0000 7f2a39600700 1 --2- 172.21.15.22:0/1116599417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 0x7f2a341a2ba0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2a28011b60 tx=0x7f2a28011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.441+0000 7f2a32000700 1 -- 172.21.15.22:0/1116599417 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a28003970 con 0x7f2a34106de0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.441+0000 7f2a32000700 1 -- 172.21.15.22:0/1116599417 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2a28003b10 con 0x7f2a34106de0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.441+0000 7f2a3c000700 1 -- 172.21.15.22:0/1116599417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a341a3370 con 0x7f2a34106de0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.441+0000 7f2a32000700 1 -- 172.21.15.22:0/1116599417 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a28024610 con 0x7f2a34106de0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.441+0000 7f2a3c000700 1 -- 172.21.15.22:0/1116599417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a341a6060 con 0x7f2a34106de0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.442+0000 7f2a32000700 1 -- 172.21.15.22:0/1116599417 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2a28024c50 con 0x7f2a34106de0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.442+0000 7f2a32000700 1 -- 172.21.15.22:0/1116599417 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2a28011e00 con 0x7f2a34106de0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.443+0000 7f2a3c000700 1 -- 172.21.15.22:0/1116599417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a340640b0 con 0x7f2a34106de0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.445+0000 7f2a32000700 1 -- 172.21.15.22:0/1116599417 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f2a2801a770 con 0x7f2a34106de0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.543+0000 7f2a3c000700 1 -- 172.21.15.22:0/1116599417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2a3419c100 con 0x7f2a34106de0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.543+0000 7f2a32000700 1 -- 172.21.15.22:0/1116599417 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2a3419c100 con 0x7f2a34106de0 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.544+0000 7f2a3c000700 1 -- 172.21.15.22:0/1116599417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 msgr2=0x7f2a341a2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:33.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.544+0000 7f2a3c000700 1 --2- 172.21.15.22:0/1116599417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 0x7f2a341a2ba0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2a28011b60 tx=0x7f2a28011c40 comp rx=0 tx=0).stop 2024-04-20T21:22:33.248 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.544+0000 7f2a3c000700 1 -- 172.21.15.22:0/1116599417 shutdown_connections 2024-04-20T21:22:33.248 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.545+0000 7f2a3c000700 1 --2- 172.21.15.22:0/1116599417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a34106de0 0x7f2a341a2ba0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:33.248 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.545+0000 7f2a3c000700 1 -- 172.21.15.22:0/1116599417 >> 172.21.15.22:0/1116599417 conn(0x7f2a34102a20 msgr2=0x7f2a34104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:33.248 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.545+0000 7f2a3c000700 1 -- 172.21.15.22:0/1116599417 shutdown_connections 2024-04-20T21:22:33.248 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:32.545+0000 7f2a3c000700 1 -- 172.21.15.22:0/1116599417 wait complete. 2024-04-20T21:22:33.248 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (1/15)... 2024-04-20T21:22:34.906 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:34 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:34.617+0000 7f9a5477f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:22:34.907 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:34 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:34.854+0000 7f9a5477f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:22:35.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:35 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:35.072+0000 7f9a5477f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:22:35.699 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:35 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:35.481+0000 7f9a5477f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:22:35.699 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:35 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:35.611+0000 7f9a5477f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:22:35.996 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:35 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:35.882+0000 7f9a5477f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:22:36.386 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:36 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2680992578' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T21:22:36.386 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:36 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:36.198+0000 7f9a5477f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:22:36.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:36.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:22:36.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "92c49c5c-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:22:36.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-20T21:22:36.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T21:22:36.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:36.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T21:22:36.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T21:22:36.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:36.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T21:22:36.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:36.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T21:22:36.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T21:22:36.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T21:22:36.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T21:22:36.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T21:22:36.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T21:22:36.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:36.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T21:22:36.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T21:22:36.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T21:22:36.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T21:22:36.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T21:22:36.724 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T21:22:36.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T21:22:36.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T21:22:36.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T21:22:36.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:36.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T21:22:36.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:36.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T21:22:36.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T21:22:36.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:36.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T21:22:36.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T21:22:36.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T21:22:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T21:22:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T21:22:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T21:22:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T21:22:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:22:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T21:22:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T21:22:14.364851+0000", 2024-04-20T21:22:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:22:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.861+0000 7f5f2aa00700 1 Processor -- start 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.861+0000 7f5f2aa00700 1 -- start start 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.862+0000 7f5f2aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 0x7f5f2410b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.862+0000 7f5f2aa00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2407be20 con 0x7f5f24109070 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.862+0000 7f5f23e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 0x7f5f2410b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.862+0000 7f5f23e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 0x7f5f2410b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34370/0 (socket says 172.21.15.22:34370) 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.862+0000 7f5f23e00700 1 -- 172.21.15.22:0/3445463912 learned_addr learned my addr 172.21.15.22:0/3445463912 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.863+0000 7f5f23e00700 1 -- 172.21.15.22:0/3445463912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2410b9a0 con 0x7f5f24109070 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.863+0000 7f5f23e00700 1 --2- 172.21.15.22:0/3445463912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 0x7f5f2410b460 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5f140098a0 tx=0x7f5f14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec343e5f030868e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.864+0000 7f5f22a00700 1 -- 172.21.15.22:0/3445463912 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f14004160 con 0x7f5f24109070 2024-04-20T21:22:36.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.864+0000 7f5f22a00700 1 -- 172.21.15.22:0/3445463912 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5f14004300 con 0x7f5f24109070 2024-04-20T21:22:36.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.864+0000 7f5f22a00700 1 -- 172.21.15.22:0/3445463912 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f14011b50 con 0x7f5f24109070 2024-04-20T21:22:36.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.864+0000 7f5f2aa00700 1 -- 172.21.15.22:0/3445463912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 msgr2=0x7f5f2410b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.864+0000 7f5f2aa00700 1 --2- 172.21.15.22:0/3445463912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 0x7f5f2410b460 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5f140098a0 tx=0x7f5f14009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.864+0000 7f5f2aa00700 1 -- 172.21.15.22:0/3445463912 shutdown_connections 2024-04-20T21:22:36.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.864+0000 7f5f2aa00700 1 --2- 172.21.15.22:0/3445463912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 0x7f5f2410b460 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.864+0000 7f5f2aa00700 1 -- 172.21.15.22:0/3445463912 >> 172.21.15.22:0/3445463912 conn(0x7f5f24102a20 msgr2=0x7f5f24104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:36.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.865+0000 7f5f2aa00700 1 -- 172.21.15.22:0/3445463912 shutdown_connections 2024-04-20T21:22:36.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.865+0000 7f5f2aa00700 1 -- 172.21.15.22:0/3445463912 wait complete. 2024-04-20T21:22:36.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.866+0000 7f5f2aa00700 1 Processor -- start 2024-04-20T21:22:36.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.866+0000 7f5f2aa00700 1 -- start start 2024-04-20T21:22:36.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.866+0000 7f5f2aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 0x7f5f2419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.728 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.866+0000 7f5f2aa00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2410c960 con 0x7f5f24109070 2024-04-20T21:22:36.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.866+0000 7f5f23e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 0x7f5f2419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.867+0000 7f5f23e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 0x7f5f2419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34372/0 (socket says 172.21.15.22:34372) 2024-04-20T21:22:36.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.867+0000 7f5f23e00700 1 -- 172.21.15.22:0/2680992578 learned_addr learned my addr 172.21.15.22:0/2680992578 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:36.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.867+0000 7f5f23e00700 1 -- 172.21.15.22:0/2680992578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2419ed10 con 0x7f5f24109070 2024-04-20T21:22:36.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.867+0000 7f5f23e00700 1 --2- 172.21.15.22:0/2680992578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 0x7f5f2419e7d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5f14011b60 tx=0x7f5f14011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.868+0000 7f5f20c00700 1 -- 172.21.15.22:0/2680992578 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f14003970 con 0x7f5f24109070 2024-04-20T21:22:36.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.868+0000 7f5f20c00700 1 -- 172.21.15.22:0/2680992578 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5f14003b10 con 0x7f5f24109070 2024-04-20T21:22:36.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.868+0000 7f5f20c00700 1 -- 172.21.15.22:0/2680992578 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f14024610 con 0x7f5f24109070 2024-04-20T21:22:36.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.868+0000 7f5f2aa00700 1 -- 172.21.15.22:0/2680992578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2419efa0 con 0x7f5f24109070 2024-04-20T21:22:36.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.868+0000 7f5f2aa00700 1 -- 172.21.15.22:0/2680992578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f241a1c90 con 0x7f5f24109070 2024-04-20T21:22:36.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.868+0000 7f5f20c00700 1 -- 172.21.15.22:0/2680992578 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5f14024ca0 con 0x7f5f24109070 2024-04-20T21:22:36.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.868+0000 7f5f20c00700 1 -- 172.21.15.22:0/2680992578 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5f14011e00 con 0x7f5f24109070 2024-04-20T21:22:36.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.869+0000 7f5f2aa00700 1 -- 172.21.15.22:0/2680992578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f240640b0 con 0x7f5f24109070 2024-04-20T21:22:36.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.874+0000 7f5f20c00700 1 -- 172.21.15.22:0/2680992578 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f5f1401a9f0 con 0x7f5f24109070 2024-04-20T21:22:36.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.996+0000 7f5f2aa00700 1 -- 172.21.15.22:0/2680992578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5f241a1e60 con 0x7f5f24109070 2024-04-20T21:22:36.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.997+0000 7f5f20c00700 1 -- 172.21.15.22:0/2680992578 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5f1401a430 con 0x7f5f24109070 2024-04-20T21:22:36.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.999+0000 7f5f2aa00700 1 -- 172.21.15.22:0/2680992578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 msgr2=0x7f5f2419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.999+0000 7f5f2aa00700 1 --2- 172.21.15.22:0/2680992578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 0x7f5f2419e7d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5f14011b60 tx=0x7f5f14011c40 comp rx=0 tx=0).stop 2024-04-20T21:22:36.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.999+0000 7f5f2aa00700 1 -- 172.21.15.22:0/2680992578 shutdown_connections 2024-04-20T21:22:36.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.999+0000 7f5f2aa00700 1 --2- 172.21.15.22:0/2680992578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f24109070 0x7f5f2419e7d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.999+0000 7f5f2aa00700 1 -- 172.21.15.22:0/2680992578 >> 172.21.15.22:0/2680992578 conn(0x7f5f24102a20 msgr2=0x7f5f24103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:36.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.999+0000 7f5f2aa00700 1 -- 172.21.15.22:0/2680992578 shutdown_connections 2024-04-20T21:22:36.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.999+0000 7f5f2aa00700 1 -- 172.21.15.22:0/2680992578 wait complete. 2024-04-20T21:22:36.731 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (2/15)... 2024-04-20T21:22:37.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:37 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:37.021+0000 7f9a5477f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:22:37.906 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:37 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:37.597+0000 7f9a5477f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:22:37.907 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:37 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:37.728+0000 7f9a5477f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:22:38.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:38 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:38.002+0000 7f9a5477f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:22:39.021 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:38 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:38.674+0000 7f9a5477f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:22:39.021 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:38 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:38.811+0000 7f9a5477f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:22:39.288 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:39.160+0000 7f9a5477f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:22:39.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:39 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3847965944' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T21:22:39.656 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:39.354+0000 7f9a5477f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:22:39.657 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:39.502+0000 7f9a5477f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:22:40.072 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:39.663+0000 7f9a5477f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:22:40.088 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:40.089 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:22:40.089 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "92c49c5c-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:22:40.089 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T21:22:40.089 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T21:22:40.092 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T21:22:40.092 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T21:22:40.092 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:40.092 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T21:22:40.092 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T21:22:40.092 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-04-20T21:22:40.092 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:40.092 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T21:22:40.092 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T21:22:40.092 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:40.092 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-20T21:22:40.092 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T21:22:40.093 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:40.093 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T21:22:40.093 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T21:22:40.093 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:40.093 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T21:22:40.094 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:40.094 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T21:22:40.094 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T21:22:40.094 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T21:22:40.094 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T21:22:40.094 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T21:22:40.094 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T21:22:40.094 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:40.095 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T21:22:40.095 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T21:22:40.099 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T21:22:40.099 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T21:22:40.099 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T21:22:40.099 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T21:22:40.099 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T21:22:40.099 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T21:22:40.099 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T21:22:40.099 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:40.099 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T21:22:40.099 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:40.099 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T21:22:40.100 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T21:22:14.364851+0000", 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.288+0000 7fbb71c00700 1 Processor -- start 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.288+0000 7fbb71c00700 1 -- start start 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.288+0000 7fbb71c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 0x7fbb6c1089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.288+0000 7fbb71c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c108f80 con 0x7fbb6c108590 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.289+0000 7fbb6aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 0x7fbb6c1089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.289+0000 7fbb6aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 0x7fbb6c1089b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34380/0 (socket says 172.21.15.22:34380) 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.289+0000 7fbb6aa00700 1 -- 172.21.15.22:0/2146108624 learned_addr learned my addr 172.21.15.22:0/2146108624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.289+0000 7fbb6aa00700 1 -- 172.21.15.22:0/2146108624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c109740 con 0x7fbb6c108590 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.289+0000 7fbb6aa00700 1 --2- 172.21.15.22:0/2146108624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 0x7fbb6c1089b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0098a0 tx=0x7fbb5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d934de86d74ebe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.290+0000 7fbb69600700 1 -- 172.21.15.22:0/2146108624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb5c004160 con 0x7fbb6c108590 2024-04-20T21:22:40.101 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.290+0000 7fbb69600700 1 -- 172.21.15.22:0/2146108624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb5c004300 con 0x7fbb6c108590 2024-04-20T21:22:40.102 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.290+0000 7fbb69600700 1 -- 172.21.15.22:0/2146108624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb5c011b50 con 0x7fbb6c108590 2024-04-20T21:22:40.102 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.291+0000 7fbb71c00700 1 -- 172.21.15.22:0/2146108624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 msgr2=0x7fbb6c1089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:40.105 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.291+0000 7fbb71c00700 1 --2- 172.21.15.22:0/2146108624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 0x7fbb6c1089b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0098a0 tx=0x7fbb5c009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:40.105 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.291+0000 7fbb71c00700 1 -- 172.21.15.22:0/2146108624 shutdown_connections 2024-04-20T21:22:40.105 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.291+0000 7fbb71c00700 1 --2- 172.21.15.22:0/2146108624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 0x7fbb6c1089b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:40.105 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.291+0000 7fbb71c00700 1 -- 172.21.15.22:0/2146108624 >> 172.21.15.22:0/2146108624 conn(0x7fbb6c103bb0 msgr2=0x7fbb6c105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:40.105 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.291+0000 7fbb71c00700 1 -- 172.21.15.22:0/2146108624 shutdown_connections 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.291+0000 7fbb71c00700 1 -- 172.21.15.22:0/2146108624 wait complete. 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.292+0000 7fbb71c00700 1 Processor -- start 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.292+0000 7fbb71c00700 1 -- start start 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.293+0000 7fbb71c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 0x7fbb6c07db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.293+0000 7fbb71c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c10caf0 con 0x7fbb6c108590 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.293+0000 7fbb6aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 0x7fbb6c07db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.293+0000 7fbb6aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 0x7fbb6c07db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34390/0 (socket says 172.21.15.22:34390) 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.293+0000 7fbb6aa00700 1 -- 172.21.15.22:0/3847965944 learned_addr learned my addr 172.21.15.22:0/3847965944 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.293+0000 7fbb6aa00700 1 -- 172.21.15.22:0/3847965944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c07e0a0 con 0x7fbb6c108590 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.294+0000 7fbb6aa00700 1 --2- 172.21.15.22:0/3847965944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 0x7fbb6c07db60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c011670 tx=0x7fbb5c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.294+0000 7fbb63400700 1 -- 172.21.15.22:0/3847965944 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb5c0039c0 con 0x7fbb6c108590 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.294+0000 7fbb63400700 1 -- 172.21.15.22:0/3847965944 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb5c011e50 con 0x7fbb6c108590 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.294+0000 7fbb71c00700 1 -- 172.21.15.22:0/3847965944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6c07c290 con 0x7fbb6c108590 2024-04-20T21:22:40.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.294+0000 7fbb63400700 1 -- 172.21.15.22:0/3847965944 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb5c01a5c0 con 0x7fbb6c108590 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.294+0000 7fbb71c00700 1 -- 172.21.15.22:0/3847965944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6c07c7d0 con 0x7fbb6c108590 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.295+0000 7fbb63400700 1 -- 172.21.15.22:0/3847965944 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbb5c01ac00 con 0x7fbb6c108590 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.295+0000 7fbb63400700 1 -- 172.21.15.22:0/3847965944 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbb5c019700 con 0x7fbb6c108590 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.296+0000 7fbb71c00700 1 -- 172.21.15.22:0/3847965944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb6c0640b0 con 0x7fbb6c108590 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.298+0000 7fbb63400700 1 -- 172.21.15.22:0/3847965944 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fbb5c003b60 con 0x7fbb6c108590 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.376+0000 7fbb71c00700 1 -- 172.21.15.22:0/3847965944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbb6c07cab0 con 0x7fbb6c108590 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.377+0000 7fbb63400700 1 -- 172.21.15.22:0/3847965944 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fbb5c0199a0 con 0x7fbb6c108590 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.378+0000 7fbb71c00700 1 -- 172.21.15.22:0/3847965944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 msgr2=0x7fbb6c07db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.378+0000 7fbb71c00700 1 --2- 172.21.15.22:0/3847965944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 0x7fbb6c07db60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c011670 tx=0x7fbb5c011b60 comp rx=0 tx=0).stop 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.378+0000 7fbb71c00700 1 -- 172.21.15.22:0/3847965944 shutdown_connections 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.378+0000 7fbb71c00700 1 --2- 172.21.15.22:0/3847965944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb6c108590 0x7fbb6c07db60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.378+0000 7fbb71c00700 1 -- 172.21.15.22:0/3847965944 >> 172.21.15.22:0/3847965944 conn(0x7fbb6c103bb0 msgr2=0x7fbb6c1047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.378+0000 7fbb71c00700 1 -- 172.21.15.22:0/3847965944 shutdown_connections 2024-04-20T21:22:40.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:39.378+0000 7fbb71c00700 1 -- 172.21.15.22:0/3847965944 wait complete. 2024-04-20T21:22:40.108 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (3/15)... 2024-04-20T21:22:40.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:40 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:40.072+0000 7f9a5477f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:22:41.156 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:40 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:40.869+0000 7f9a5477f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:22:41.656 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:41.165+0000 7f9a5477f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:22:41.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-mon[16843]: Activating manager daemon x 2024-04-20T21:22:41.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-mon[16843]: mgrmap e2: x(active, starting, since 0.0101819s) 2024-04-20T21:22:41.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-mon[16843]: from='mgr.14100 172.21.15.22:0/3139287386' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T21:22:41.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-mon[16843]: from='mgr.14100 172.21.15.22:0/3139287386' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T21:22:41.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-mon[16843]: from='mgr.14100 172.21.15.22:0/3139287386' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T21:22:41.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-mon[16843]: from='mgr.14100 172.21.15.22:0/3139287386' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:22:41.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-mon[16843]: from='mgr.14100 172.21.15.22:0/3139287386' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T21:22:41.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-mon[16843]: Manager daemon x is now available 2024-04-20T21:22:41.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-mon[16843]: from='mgr.14100 172.21.15.22:0/3139287386' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:22:41.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-mon[16843]: from='mgr.14100 172.21.15.22:0/3139287386' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:22:41.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-mon[16843]: from='mgr.14100 172.21.15.22:0/3139287386' entity='mgr.x' 2024-04-20T21:22:41.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:41 smithi022 ceph-mon[16843]: from='mgr.14100 172.21.15.22:0/3139287386' entity='mgr.x' 2024-04-20T21:22:42.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:42 smithi022 ceph-mon[16843]: from='mgr.14100 172.21.15.22:0/3139287386' entity='mgr.x' 2024-04-20T21:22:42.523 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:42 smithi022 ceph-mon[16843]: mgrmap e3: x(active, since 1.01273s) 2024-04-20T21:22:43.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:43 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3031417409' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T21:22:43.699 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "92c49c5c-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T21:22:43.700 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T21:22:43.701 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:43.701 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-20T21:22:43.701 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T21:22:43.701 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:43.701 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T21:22:43.701 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T21:22:43.701 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:43.702 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T21:22:43.702 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:43.702 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T21:22:43.703 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T21:22:43.703 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T21:22:43.703 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T21:22:43.703 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T21:22:43.703 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T21:22:43.703 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:43.703 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T21:22:43.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T21:22:43.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T21:22:43.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T21:22:43.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T21:22:43.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T21:22:43.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T21:22:43.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T21:22:43.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T21:22:43.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:43.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:43.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T21:22:14.364851+0000", 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.685+0000 7fdbf7200700 1 Processor -- start 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.685+0000 7fdbf7200700 1 -- start start 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.686+0000 7fdbf7200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf810a5c0 0x7fdbf810a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.686+0000 7fdbf7200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810afb0 con 0x7fdbf810a5c0 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.686+0000 7fdbf5e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf810a5c0 0x7fdbf810a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.686+0000 7fdbf5e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf810a5c0 0x7fdbf810a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45928/0 (socket says 172.21.15.22:45928) 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.686+0000 7fdbf5e00700 1 -- 172.21.15.22:0/591585601 learned_addr learned my addr 172.21.15.22:0/591585601 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:43.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.687+0000 7fdbf5e00700 1 -- 172.21.15.22:0/591585601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf810b770 con 0x7fdbf810a5c0 2024-04-20T21:22:43.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.687+0000 7fdbf5e00700 1 --2- 172.21.15.22:0/591585601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf810a5c0 0x7fdbf810a9e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80098a0 tx=0x7fdbe8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f91bbfab3522997 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:43.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.687+0000 7fdbf4a00700 1 -- 172.21.15.22:0/591585601 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbe8004160 con 0x7fdbf810a5c0 2024-04-20T21:22:43.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.687+0000 7fdbf4a00700 1 -- 172.21.15.22:0/591585601 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbe8004300 con 0x7fdbf810a5c0 2024-04-20T21:22:43.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.687+0000 7fdbf4a00700 1 -- 172.21.15.22:0/591585601 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbe8011b50 con 0x7fdbf810a5c0 2024-04-20T21:22:43.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.688+0000 7fdbf7200700 1 -- 172.21.15.22:0/591585601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf810a5c0 msgr2=0x7fdbf810a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:43.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.688+0000 7fdbf7200700 1 --2- 172.21.15.22:0/591585601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf810a5c0 0x7fdbf810a9e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80098a0 tx=0x7fdbe8009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:43.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.688+0000 7fdbf7200700 1 -- 172.21.15.22:0/591585601 shutdown_connections 2024-04-20T21:22:43.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.688+0000 7fdbf7200700 1 --2- 172.21.15.22:0/591585601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf810a5c0 0x7fdbf810a9e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:43.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.688+0000 7fdbf7200700 1 -- 172.21.15.22:0/591585601 >> 172.21.15.22:0/591585601 conn(0x7fdbf807dbb0 msgr2=0x7fdbf81084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:43.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.688+0000 7fdbf7200700 1 -- 172.21.15.22:0/591585601 shutdown_connections 2024-04-20T21:22:43.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.688+0000 7fdbf7200700 1 -- 172.21.15.22:0/591585601 wait complete. 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.689+0000 7fdbf7200700 1 Processor -- start 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.690+0000 7fdbf7200700 1 -- start start 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.690+0000 7fdbf7200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf819e7b0 0x7fdbf819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.690+0000 7fdbf7200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810eb20 con 0x7fdbf819e7b0 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.690+0000 7fdbf5e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf819e7b0 0x7fdbf819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.690+0000 7fdbf5e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf819e7b0 0x7fdbf819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45944/0 (socket says 172.21.15.22:45944) 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.690+0000 7fdbf5e00700 1 -- 172.21.15.22:0/3031417409 learned_addr learned my addr 172.21.15.22:0/3031417409 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.690+0000 7fdbf5e00700 1 -- 172.21.15.22:0/3031417409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf819f110 con 0x7fdbf819e7b0 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.691+0000 7fdbf5e00700 1 --2- 172.21.15.22:0/3031417409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf819e7b0 0x7fdbf819ebd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8011850 tx=0x7fdbe8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.691+0000 7fdbeea00700 1 -- 172.21.15.22:0/3031417409 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbe80036a0 con 0x7fdbf819e7b0 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.691+0000 7fdbeea00700 1 -- 172.21.15.22:0/3031417409 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbe801a460 con 0x7fdbf819e7b0 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.691+0000 7fdbf7200700 1 -- 172.21.15.22:0/3031417409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf819f3a0 con 0x7fdbf819e7b0 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.691+0000 7fdbeea00700 1 -- 172.21.15.22:0/3031417409 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbe801b440 con 0x7fdbf819e7b0 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.691+0000 7fdbf7200700 1 -- 172.21.15.22:0/3031417409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf81a1e80 con 0x7fdbf819e7b0 2024-04-20T21:22:43.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.692+0000 7fdbeea00700 1 -- 172.21.15.22:0/3031417409 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7fdbe8003840 con 0x7fdbf819e7b0 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.692+0000 7fdbf7200700 1 -- 172.21.15.22:0/3031417409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf80640b0 con 0x7fdbf819e7b0 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.693+0000 7fdbeea00700 1 --2- 172.21.15.22:0/3031417409 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7fdbdc039450 0x7fdbdc03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.693+0000 7fdbeea00700 1 -- 172.21.15.22:0/3031417409 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdbe804d9f0 con 0x7fdbf819e7b0 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.693+0000 7fdbf5400700 1 --2- 172.21.15.22:0/3031417409 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7fdbdc039450 0x7fdbdc03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.694+0000 7fdbf5400700 1 --2- 172.21.15.22:0/3031417409 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7fdbdc039450 0x7fdbdc03b910 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4006fd0 tx=0x7fdbe4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.699+0000 7fdbeea00700 1 -- 172.21.15.22:0/3031417409 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdbe800fe70 con 0x7fdbf819e7b0 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.002+0000 7fdbf7200700 1 -- 172.21.15.22:0/3031417409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdbf81a2710 con 0x7fdbf819e7b0 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.002+0000 7fdbeea00700 1 -- 172.21.15.22:0/3031417409 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdbe8004230 con 0x7fdbf819e7b0 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.005+0000 7fdbf7200700 1 -- 172.21.15.22:0/3031417409 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7fdbdc039450 msgr2=0x7fdbdc03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.005+0000 7fdbf7200700 1 --2- 172.21.15.22:0/3031417409 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7fdbdc039450 0x7fdbdc03b910 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4006fd0 tx=0x7fdbe4006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.005+0000 7fdbf7200700 1 -- 172.21.15.22:0/3031417409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf819e7b0 msgr2=0x7fdbf819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.005+0000 7fdbf7200700 1 --2- 172.21.15.22:0/3031417409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf819e7b0 0x7fdbf819ebd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8011850 tx=0x7fdbe8011c10 comp rx=0 tx=0).stop 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.005+0000 7fdbf7200700 1 -- 172.21.15.22:0/3031417409 shutdown_connections 2024-04-20T21:22:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.005+0000 7fdbf7200700 1 --2- 172.21.15.22:0/3031417409 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7fdbdc039450 0x7fdbdc03b910 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:43.718 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.005+0000 7fdbf7200700 1 --2- 172.21.15.22:0/3031417409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbf819e7b0 0x7fdbf819ebd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:43.718 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.005+0000 7fdbf7200700 1 -- 172.21.15.22:0/3031417409 >> 172.21.15.22:0/3031417409 conn(0x7fdbf807dbb0 msgr2=0x7fdbf8107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:43.718 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.005+0000 7fdbf7200700 1 -- 172.21.15.22:0/3031417409 shutdown_connections 2024-04-20T21:22:43.718 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.005+0000 7fdbf7200700 1 -- 172.21.15.22:0/3031417409 wait complete. 2024-04-20T21:22:43.718 INFO:teuthology.orchestra.run.smithi022.stdout:mgr is available 2024-04-20T21:22:44.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:44 smithi022 ceph-mon[16843]: mgrmap e4: x(active, since 2s) 2024-04-20T21:22:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-04-20T21:22:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-20T21:22:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-20T21:22:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-20T21:22:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-20T21:22:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-20T21:22:45.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-20T21:22:45.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:45.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-04-20T21:22:45.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-20T21:22:45.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-20T21:22:45.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-20T21:22:45.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-20T21:22:45.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-20T21:22:45.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-20T21:22:45.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.319+0000 7f2a6de00700 1 Processor -- start 2024-04-20T21:22:45.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.319+0000 7f2a6de00700 1 -- start start 2024-04-20T21:22:45.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.320+0000 7f2a6de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 0x7f2a68108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:45.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.320+0000 7f2a6de00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68108f50 con 0x7f2a68108560 2024-04-20T21:22:45.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.320+0000 7f2a66a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 0x7f2a68108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:45.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.321+0000 7f2a66a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 0x7f2a68108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45946/0 (socket says 172.21.15.22:45946) 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.321+0000 7f2a66a00700 1 -- 172.21.15.22:0/604195903 learned_addr learned my addr 172.21.15.22:0/604195903 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.321+0000 7f2a66a00700 1 -- 172.21.15.22:0/604195903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a68109710 con 0x7f2a68108560 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.321+0000 7f2a66a00700 1 --2- 172.21.15.22:0/604195903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 0x7f2a68108980 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2a580098a0 tx=0x7f2a58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2a074f59e8989 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.322+0000 7f2a65600700 1 -- 172.21.15.22:0/604195903 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a58004160 con 0x7f2a68108560 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.322+0000 7f2a65600700 1 -- 172.21.15.22:0/604195903 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2a58004300 con 0x7f2a68108560 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.322+0000 7f2a65600700 1 -- 172.21.15.22:0/604195903 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a58011b50 con 0x7f2a68108560 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.322+0000 7f2a6de00700 1 -- 172.21.15.22:0/604195903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 msgr2=0x7f2a68108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.322+0000 7f2a6de00700 1 --2- 172.21.15.22:0/604195903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 0x7f2a68108980 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2a580098a0 tx=0x7f2a58009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.322+0000 7f2a6de00700 1 -- 172.21.15.22:0/604195903 shutdown_connections 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.322+0000 7f2a6de00700 1 --2- 172.21.15.22:0/604195903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 0x7f2a68108980 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.322+0000 7f2a6de00700 1 -- 172.21.15.22:0/604195903 >> 172.21.15.22:0/604195903 conn(0x7f2a68103b80 msgr2=0x7f2a68105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.323+0000 7f2a6de00700 1 -- 172.21.15.22:0/604195903 shutdown_connections 2024-04-20T21:22:45.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.323+0000 7f2a6de00700 1 -- 172.21.15.22:0/604195903 wait complete. 2024-04-20T21:22:45.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.324+0000 7f2a6de00700 1 Processor -- start 2024-04-20T21:22:45.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.324+0000 7f2a6de00700 1 -- start start 2024-04-20T21:22:45.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.324+0000 7f2a6de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 0x7f2a6819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:45.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.324+0000 7f2a6de00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6810cac0 con 0x7f2a68108560 2024-04-20T21:22:45.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.324+0000 7f2a66a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 0x7f2a6819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:45.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.324+0000 7f2a66a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 0x7f2a6819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45958/0 (socket says 172.21.15.22:45958) 2024-04-20T21:22:45.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.324+0000 7f2a66a00700 1 -- 172.21.15.22:0/380550068 learned_addr learned my addr 172.21.15.22:0/380550068 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:45.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.325+0000 7f2a66a00700 1 -- 172.21.15.22:0/380550068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6819ec30 con 0x7f2a68108560 2024-04-20T21:22:45.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.325+0000 7f2a66a00700 1 --2- 172.21.15.22:0/380550068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 0x7f2a6819e6f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2a58011670 tx=0x7f2a58011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:45.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.325+0000 7f2a6ca00700 1 -- 172.21.15.22:0/380550068 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a580039c0 con 0x7f2a68108560 2024-04-20T21:22:45.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.325+0000 7f2a6ca00700 1 -- 172.21.15.22:0/380550068 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2a58011e50 con 0x7f2a68108560 2024-04-20T21:22:45.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.325+0000 7f2a6de00700 1 -- 172.21.15.22:0/380550068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6819eec0 con 0x7f2a68108560 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.325+0000 7f2a6ca00700 1 -- 172.21.15.22:0/380550068 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a5801a5c0 con 0x7f2a68108560 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.326+0000 7f2a6de00700 1 -- 172.21.15.22:0/380550068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a681a1bb0 con 0x7f2a68108560 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.327+0000 7f2a6ca00700 1 -- 172.21.15.22:0/380550068 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f2a58024430 con 0x7f2a68108560 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.327+0000 7f2a6de00700 1 -- 172.21.15.22:0/380550068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a680640b0 con 0x7f2a68108560 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.327+0000 7f2a6ca00700 1 --2- 172.21.15.22:0/380550068 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f2a4c039510 0x7f2a4c03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.327+0000 7f2a6ca00700 1 -- 172.21.15.22:0/380550068 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2a5804da80 con 0x7f2a68108560 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.328+0000 7f2a66000700 1 --2- 172.21.15.22:0/380550068 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f2a4c039510 0x7f2a4c03b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.328+0000 7f2a66000700 1 --2- 172.21.15.22:0/380550068 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f2a4c039510 0x7f2a4c03b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a54006fd0 tx=0x7f2a54006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.333+0000 7f2a6ca00700 1 -- 172.21.15.22:0/380550068 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2a5804fe20 con 0x7f2a68108560 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.559+0000 7f2a6de00700 1 -- 172.21.15.22:0/380550068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f2a681a1d80 con 0x7f2a68108560 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.562+0000 7f2a6ca00700 1 -- 172.21.15.22:0/380550068 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+400 (secure 0 0 0) 0x7f2a681a1d80 con 0x7f2a68108560 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.565+0000 7f2a6de00700 1 -- 172.21.15.22:0/380550068 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f2a4c039510 msgr2=0x7f2a4c03b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.565+0000 7f2a6de00700 1 --2- 172.21.15.22:0/380550068 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f2a4c039510 0x7f2a4c03b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a54006fd0 tx=0x7f2a54006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:45.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.565+0000 7f2a6de00700 1 -- 172.21.15.22:0/380550068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 msgr2=0x7f2a6819e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:45.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.565+0000 7f2a6de00700 1 --2- 172.21.15.22:0/380550068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 0x7f2a6819e6f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2a58011670 tx=0x7f2a58011b60 comp rx=0 tx=0).stop 2024-04-20T21:22:45.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.566+0000 7f2a6de00700 1 -- 172.21.15.22:0/380550068 shutdown_connections 2024-04-20T21:22:45.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.566+0000 7f2a6de00700 1 --2- 172.21.15.22:0/380550068 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f2a4c039510 0x7f2a4c03b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.566+0000 7f2a6de00700 1 --2- 172.21.15.22:0/380550068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a68108560 0x7f2a6819e6f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.566+0000 7f2a6de00700 1 -- 172.21.15.22:0/380550068 >> 172.21.15.22:0/380550068 conn(0x7f2a68103b80 msgr2=0x7f2a68104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:45.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.566+0000 7f2a6de00700 1 -- 172.21.15.22:0/380550068 shutdown_connections 2024-04-20T21:22:45.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.566+0000 7f2a6de00700 1 -- 172.21.15.22:0/380550068 wait complete. 2024-04-20T21:22:45.245 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling cephadm module... 2024-04-20T21:22:45.562 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:45 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/380550068' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-20T21:22:46.537 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:46 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/206015449' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-20T21:22:46.537 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:46 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: ignoring --setuser ceph since I am not root 2024-04-20T21:22:46.537 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:46 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: ignoring --setgroup ceph since I am not root 2024-04-20T21:22:46.537 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:46 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:46.536+0000 7fe03edbd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:22:46.906 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:46 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:46.786+0000 7fe03edbd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:22:46.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.866+0000 7fc46d000700 1 Processor -- start 2024-04-20T21:22:46.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.866+0000 7fc46d000700 1 -- start start 2024-04-20T21:22:46.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.867+0000 7fc46d000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 0x7fc46810f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.867+0000 7fc46d000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810fa40 con 0x7fc46810f050 2024-04-20T21:22:46.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.867+0000 7fc465e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 0x7fc46810f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.867+0000 7fc465e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 0x7fc46810f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45972/0 (socket says 172.21.15.22:45972) 2024-04-20T21:22:46.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.867+0000 7fc465e00700 1 -- 172.21.15.22:0/4073919329 learned_addr learned my addr 172.21.15.22:0/4073919329 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:46.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.868+0000 7fc465e00700 1 -- 172.21.15.22:0/4073919329 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46810fd70 con 0x7fc46810f050 2024-04-20T21:22:46.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.868+0000 7fc465e00700 1 --2- 172.21.15.22:0/4073919329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 0x7fc46810f470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc4580098a0 tx=0x7fc458009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775af67a8c6db195 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:46.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.869+0000 7fc465400700 1 -- 172.21.15.22:0/4073919329 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc458004160 con 0x7fc46810f050 2024-04-20T21:22:46.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.869+0000 7fc465400700 1 -- 172.21.15.22:0/4073919329 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc458004300 con 0x7fc46810f050 2024-04-20T21:22:46.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.869+0000 7fc465400700 1 -- 172.21.15.22:0/4073919329 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc458011b50 con 0x7fc46810f050 2024-04-20T21:22:46.948 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.869+0000 7fc46d000700 1 -- 172.21.15.22:0/4073919329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 msgr2=0x7fc46810f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.948 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.869+0000 7fc46d000700 1 --2- 172.21.15.22:0/4073919329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 0x7fc46810f470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc4580098a0 tx=0x7fc458009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.948 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.870+0000 7fc46d000700 1 -- 172.21.15.22:0/4073919329 shutdown_connections 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.870+0000 7fc46d000700 1 --2- 172.21.15.22:0/4073919329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 0x7fc46810f470 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.870+0000 7fc46d000700 1 -- 172.21.15.22:0/4073919329 >> 172.21.15.22:0/4073919329 conn(0x7fc46807ddd0 msgr2=0x7fc4680801f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.870+0000 7fc46d000700 1 -- 172.21.15.22:0/4073919329 shutdown_connections 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.870+0000 7fc46d000700 1 -- 172.21.15.22:0/4073919329 wait complete. 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.872+0000 7fc46d000700 1 Processor -- start 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.872+0000 7fc46d000700 1 -- start start 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.872+0000 7fc46d000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 0x7fc4681a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.872+0000 7fc46d000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc468113120 con 0x7fc46810f050 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.873+0000 7fc465e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 0x7fc4681a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.873+0000 7fc465e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 0x7fc4681a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45978/0 (socket says 172.21.15.22:45978) 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.873+0000 7fc465e00700 1 -- 172.21.15.22:0/206015449 learned_addr learned my addr 172.21.15.22:0/206015449 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.873+0000 7fc465e00700 1 -- 172.21.15.22:0/206015449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4681a3000 con 0x7fc46810f050 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.874+0000 7fc465e00700 1 --2- 172.21.15.22:0/206015449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 0x7fc4681a2ac0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc458011b60 tx=0x7fc458011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.874+0000 7fc45ea00700 1 -- 172.21.15.22:0/206015449 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc458003b20 con 0x7fc46810f050 2024-04-20T21:22:46.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.874+0000 7fc45ea00700 1 -- 172.21.15.22:0/206015449 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc458003cc0 con 0x7fc46810f050 2024-04-20T21:22:46.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.874+0000 7fc46d000700 1 -- 172.21.15.22:0/206015449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4681a3290 con 0x7fc46810f050 2024-04-20T21:22:46.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.874+0000 7fc45ea00700 1 -- 172.21.15.22:0/206015449 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc458024840 con 0x7fc46810f050 2024-04-20T21:22:46.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.874+0000 7fc46d000700 1 -- 172.21.15.22:0/206015449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4681a5f80 con 0x7fc46810f050 2024-04-20T21:22:46.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.875+0000 7fc45ea00700 1 -- 172.21.15.22:0/206015449 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fc458022020 con 0x7fc46810f050 2024-04-20T21:22:46.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.875+0000 7fc46d000700 1 -- 172.21.15.22:0/206015449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4680640b0 con 0x7fc46810f050 2024-04-20T21:22:46.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.876+0000 7fc45ea00700 1 --2- 172.21.15.22:0/206015449 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7fc44c0395b0 0x7fc44c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.876+0000 7fc45ea00700 1 -- 172.21.15.22:0/206015449 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc458011e00 con 0x7fc46810f050 2024-04-20T21:22:46.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.876+0000 7fc45fe00700 1 --2- 172.21.15.22:0/206015449 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7fc44c0395b0 0x7fc44c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.877+0000 7fc45fe00700 1 --2- 172.21.15.22:0/206015449 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7fc44c0395b0 0x7fc44c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc450006fd0 tx=0x7fc450006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:45.882+0000 7fc45ea00700 1 -- 172.21.15.22:0/206015449 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc45804d150 con 0x7fc46810f050 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.179+0000 7fc46d000700 1 -- 172.21.15.22:0/206015449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fc4681a6150 con 0x7fc46810f050 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.244+0000 7fc45ea00700 1 -- 172.21.15.22:0/206015449 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fc4681a6150 con 0x7fc46810f050 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.247+0000 7fc45ea00700 1 -- 172.21.15.22:0/206015449 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fc45804f0e0 con 0x7fc46810f050 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.248+0000 7fc46d000700 1 -- 172.21.15.22:0/206015449 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7fc44c0395b0 msgr2=0x7fc44c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.248+0000 7fc46d000700 1 --2- 172.21.15.22:0/206015449 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7fc44c0395b0 0x7fc44c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc450006fd0 tx=0x7fc450006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.248+0000 7fc46d000700 1 -- 172.21.15.22:0/206015449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 msgr2=0x7fc4681a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.248+0000 7fc46d000700 1 --2- 172.21.15.22:0/206015449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 0x7fc4681a2ac0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc458011b60 tx=0x7fc458011c40 comp rx=0 tx=0).stop 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.248+0000 7fc46d000700 1 -- 172.21.15.22:0/206015449 shutdown_connections 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.248+0000 7fc46d000700 1 --2- 172.21.15.22:0/206015449 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7fc44c0395b0 0x7fc44c03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.248+0000 7fc46d000700 1 --2- 172.21.15.22:0/206015449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc46810f050 0x7fc4681a2ac0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.248+0000 7fc46d000700 1 -- 172.21.15.22:0/206015449 >> 172.21.15.22:0/206015449 conn(0x7fc46807ddd0 msgr2=0x7fc46807f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.250+0000 7fc46d000700 1 -- 172.21.15.22:0/206015449 shutdown_connections 2024-04-20T21:22:46.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:46.250+0000 7fc46d000700 1 -- 172.21.15.22:0/206015449 wait complete. 2024-04-20T21:22:47.244 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:46 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:46.920+0000 7fe03edbd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:22:47.551 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:47 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/206015449' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-20T21:22:47.551 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:47 smithi022 ceph-mon[16843]: mgrmap e5: x(active, since 5s) 2024-04-20T21:22:47.551 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:47 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:47.351+0000 7fe03edbd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:22:47.552 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:47 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:47.469+0000 7fe03edbd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:22:48.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:48 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2186171948' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-20T21:22:48.769 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.551+0000 7f8916200700 1 Processor -- start 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.552+0000 7f8916200700 1 -- start start 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.552+0000 7f8916200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 0x7f89101078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.552+0000 7f8916200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8910107e70 con 0x7f8910107480 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.553+0000 7f890f400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 0x7f89101078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.553+0000 7f890f400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 0x7f89101078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45996/0 (socket says 172.21.15.22:45996) 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.553+0000 7f890f400700 1 -- 172.21.15.22:0/612598607 learned_addr learned my addr 172.21.15.22:0/612598607 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:48.770 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.553+0000 7f890f400700 1 -- 172.21.15.22:0/612598607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8910108680 con 0x7f8910107480 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.554+0000 7f890f400700 1 --2- 172.21.15.22:0/612598607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 0x7f89101078a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f89000098a0 tx=0x7f8900009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7666a9cec35f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.554+0000 7f890e000700 1 -- 172.21.15.22:0/612598607 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8900004160 con 0x7f8910107480 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.554+0000 7f890e000700 1 -- 172.21.15.22:0/612598607 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8900004300 con 0x7f8910107480 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.554+0000 7f890e000700 1 -- 172.21.15.22:0/612598607 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8900011b50 con 0x7f8910107480 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.555+0000 7f8916200700 1 -- 172.21.15.22:0/612598607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 msgr2=0x7f89101078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.555+0000 7f8916200700 1 --2- 172.21.15.22:0/612598607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 0x7f89101078a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f89000098a0 tx=0x7f8900009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.555+0000 7f8916200700 1 -- 172.21.15.22:0/612598607 shutdown_connections 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.555+0000 7f8916200700 1 --2- 172.21.15.22:0/612598607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 0x7f89101078a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.555+0000 7f8916200700 1 -- 172.21.15.22:0/612598607 >> 172.21.15.22:0/612598607 conn(0x7f8910102a20 msgr2=0x7f8910104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.555+0000 7f8916200700 1 -- 172.21.15.22:0/612598607 shutdown_connections 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.555+0000 7f8916200700 1 -- 172.21.15.22:0/612598607 wait complete. 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.557+0000 7f8916200700 1 Processor -- start 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.557+0000 7f8916200700 1 -- start start 2024-04-20T21:22:48.771 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.558+0000 7f8916200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 0x7f891019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.558+0000 7f8916200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f891007afc0 con 0x7f8910107480 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.558+0000 7f890f400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 0x7f891019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.558+0000 7f890f400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 0x7f891019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46002/0 (socket says 172.21.15.22:46002) 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.558+0000 7f890f400700 1 -- 172.21.15.22:0/2186171948 learned_addr learned my addr 172.21.15.22:0/2186171948 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.558+0000 7f890f400700 1 -- 172.21.15.22:0/2186171948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891019cb90 con 0x7f8910107480 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.559+0000 7f890f400700 1 --2- 172.21.15.22:0/2186171948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 0x7f891019c650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8900011850 tx=0x7f8900011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.559+0000 7f8907e00700 1 -- 172.21.15.22:0/2186171948 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8900003af0 con 0x7f8910107480 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.559+0000 7f8907e00700 1 -- 172.21.15.22:0/2186171948 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8900020070 con 0x7f8910107480 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.559+0000 7f8907e00700 1 -- 172.21.15.22:0/2186171948 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89000195a0 con 0x7f8910107480 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.559+0000 7f8916200700 1 -- 172.21.15.22:0/2186171948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891019ce20 con 0x7f8910107480 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.559+0000 7f8916200700 1 -- 172.21.15.22:0/2186171948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891019fac0 con 0x7f8910107480 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.560+0000 7f8907e00700 1 -- 172.21.15.22:0/2186171948 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f890001a430 con 0x7f8910107480 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.561+0000 7f8907e00700 1 --2- 172.21.15.22:0/2186171948 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f88f4039600 0x7f88f403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:48.772 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.561+0000 7f8916200700 1 -- 172.21.15.22:0/2186171948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89100640b0 con 0x7f8910107480 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.561+0000 7f8907e00700 1 -- 172.21.15.22:0/2186171948 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8900011e00 con 0x7f8910107480 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.561+0000 7f890ea00700 1 -- 172.21.15.22:0/2186171948 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f88f4039600 msgr2=0x7f88f403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3430200041 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.561+0000 7f890ea00700 1 --2- 172.21.15.22:0/2186171948 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f88f4039600 0x7f88f403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.566+0000 7f8907e00700 1 -- 172.21.15.22:0/2186171948 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8900013110 con 0x7f8910107480 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.762+0000 7f890ea00700 1 -- 172.21.15.22:0/2186171948 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f88f4039600 msgr2=0x7f88f403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3430200041 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.762+0000 7f890ea00700 1 --2- 172.21.15.22:0/2186171948 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f88f4039600 0x7f88f403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.895+0000 7f8916200700 1 -- 172.21.15.22:0/2186171948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f89100008d0 con 0x7f8910107480 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.896+0000 7f8907e00700 1 -- 172.21.15.22:0/2186171948 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f890000e260 con 0x7f8910107480 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.899+0000 7f8916200700 1 -- 172.21.15.22:0/2186171948 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f88f4039600 msgr2=0x7f88f403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.899+0000 7f8916200700 1 --2- 172.21.15.22:0/2186171948 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f88f4039600 0x7f88f403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.899+0000 7f8916200700 1 -- 172.21.15.22:0/2186171948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 msgr2=0x7f891019c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.899+0000 7f8916200700 1 --2- 172.21.15.22:0/2186171948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 0x7f891019c650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8900011850 tx=0x7f8900011c10 comp rx=0 tx=0).stop 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.899+0000 7f8916200700 1 -- 172.21.15.22:0/2186171948 shutdown_connections 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.899+0000 7f8916200700 1 --2- 172.21.15.22:0/2186171948 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f88f4039600 0x7f88f403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.773 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.899+0000 7f8916200700 1 --2- 172.21.15.22:0/2186171948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8910107480 0x7f891019c650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.899+0000 7f8916200700 1 -- 172.21.15.22:0/2186171948 >> 172.21.15.22:0/2186171948 conn(0x7f8910102a20 msgr2=0x7f8910103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:48.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.899+0000 7f8916200700 1 -- 172.21.15.22:0/2186171948 shutdown_connections 2024-04-20T21:22:48.774 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:47.900+0000 7f8916200700 1 -- 172.21.15.22:0/2186171948 wait complete. 2024-04-20T21:22:48.774 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-04-20T21:22:48.774 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 5... 2024-04-20T21:22:49.320 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:49.116+0000 7fe03edbd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:22:49.594 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:49.356+0000 7fe03edbd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:22:49.594 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:49.594+0000 7fe03edbd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:22:50.376 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:49.993+0000 7fe03edbd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:22:50.377 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:50 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:50.118+0000 7fe03edbd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:22:50.377 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:50 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:50.376+0000 7fe03edbd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:22:50.906 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:50 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:50.639+0000 7fe03edbd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:22:51.906 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:51 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:51.448+0000 7fe03edbd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:22:52.334 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:52 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:52.016+0000 7fe03edbd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:22:52.334 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:52 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:52.146+0000 7fe03edbd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:22:52.656 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:52 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:52.421+0000 7fe03edbd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:22:53.407 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:53 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:53.086+0000 7fe03edbd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:22:53.407 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:53 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:53.214+0000 7fe03edbd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:22:53.869 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:53 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:53.546+0000 7fe03edbd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:22:53.869 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:53 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:53.729+0000 7fe03edbd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:22:54.156 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:53 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:53.858+0000 7fe03edbd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:22:54.157 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:54 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:54.015+0000 7fe03edbd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:22:54.656 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:54 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:54.388+0000 7fe03edbd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:22:55.481 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:55.186+0000 7fe03edbd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:22:55.906 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:22:55.480+0000 7fe03edbd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:22:55.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-mon[16843]: Active manager daemon x restarted 2024-04-20T21:22:55.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-mon[16843]: Activating manager daemon x 2024-04-20T21:22:55.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-mon[16843]: osdmap e2: 0 total, 0 up, 0 in 2024-04-20T21:22:55.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-mon[16843]: mgrmap e6: x(active, starting, since 0.0090146s) 2024-04-20T21:22:55.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:22:55.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T21:22:55.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T21:22:55.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T21:22:55.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T21:22:55.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-mon[16843]: Manager daemon x is now available 2024-04-20T21:22:55.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:55 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:22:56.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:56 smithi022 ceph-mon[16843]: Found migration_current of "None". Setting to last migration. 2024-04-20T21:22:56.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:56 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:22:56.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:56 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:22:56.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:56 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:22:56.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:56 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:22:56.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:56 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:22:56.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:56 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:22:56.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:56 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:22:56.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:56 smithi022 ceph-mon[16843]: mgrmap e7: x(active, since 1.01651s) 2024-04-20T21:22:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:22:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-20T21:22:57.192 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.321+0000 7f86ef800700 1 Processor -- start 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.321+0000 7f86ef800700 1 -- start start 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.321+0000 7f86ef800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 0x7f86e8108a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.322+0000 7f86ef800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8108fe0 con 0x7f86e81085f0 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.322+0000 7f86ece00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 0x7f86e8108a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.322+0000 7f86ece00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 0x7f86e8108a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46018/0 (socket says 172.21.15.22:46018) 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.322+0000 7f86ece00700 1 -- 172.21.15.22:0/1439666891 learned_addr learned my addr 172.21.15.22:0/1439666891 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.323+0000 7f86ece00700 1 -- 172.21.15.22:0/1439666891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e81097a0 con 0x7f86e81085f0 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.323+0000 7f86ece00700 1 --2- 172.21.15.22:0/1439666891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 0x7f86e8108a10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0098a0 tx=0x7f86dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab80447c3621fcf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.323+0000 7f86e7400700 1 -- 172.21.15.22:0/1439666891 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86dc004160 con 0x7f86e81085f0 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.323+0000 7f86e7400700 1 -- 172.21.15.22:0/1439666891 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f86dc004300 con 0x7f86e81085f0 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.323+0000 7f86e7400700 1 -- 172.21.15.22:0/1439666891 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86dc011b50 con 0x7f86e81085f0 2024-04-20T21:22:57.193 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.324+0000 7f86ef800700 1 -- 172.21.15.22:0/1439666891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 msgr2=0x7f86e8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.200 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.324+0000 7f86ef800700 1 --2- 172.21.15.22:0/1439666891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 0x7f86e8108a10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0098a0 tx=0x7f86dc009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.200 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.324+0000 7f86ef800700 1 -- 172.21.15.22:0/1439666891 shutdown_connections 2024-04-20T21:22:57.200 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.324+0000 7f86ef800700 1 --2- 172.21.15.22:0/1439666891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 0x7f86e8108a10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.200 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.324+0000 7f86ef800700 1 -- 172.21.15.22:0/1439666891 >> 172.21.15.22:0/1439666891 conn(0x7f86e8103c10 msgr2=0x7f86e8106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:57.200 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.324+0000 7f86ef800700 1 -- 172.21.15.22:0/1439666891 shutdown_connections 2024-04-20T21:22:57.200 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.324+0000 7f86ef800700 1 -- 172.21.15.22:0/1439666891 wait complete. 2024-04-20T21:22:57.200 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.325+0000 7f86ef800700 1 Processor -- start 2024-04-20T21:22:57.200 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.326+0000 7f86ef800700 1 -- start start 2024-04-20T21:22:57.200 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.326+0000 7f86ef800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 0x7f86e807db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.200 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.326+0000 7f86ef800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86e810cb50 con 0x7f86e81085f0 2024-04-20T21:22:57.200 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.326+0000 7f86ece00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 0x7f86e807db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.200 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.326+0000 7f86ece00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 0x7f86e807db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46022/0 (socket says 172.21.15.22:46022) 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.326+0000 7f86ece00700 1 -- 172.21.15.22:0/487755561 learned_addr learned my addr 172.21.15.22:0/487755561 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.326+0000 7f86ece00700 1 -- 172.21.15.22:0/487755561 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e807e0a0 con 0x7f86e81085f0 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.327+0000 7f86ece00700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 0x7f86e807db60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f86dc011670 tx=0x7f86dc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.327+0000 7f86e5600700 1 -- 172.21.15.22:0/487755561 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86dc0039c0 con 0x7f86e81085f0 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.327+0000 7f86e5600700 1 -- 172.21.15.22:0/487755561 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f86dc011e50 con 0x7f86e81085f0 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.327+0000 7f86ef800700 1 -- 172.21.15.22:0/487755561 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e807c290 con 0x7f86e81085f0 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.327+0000 7f86e5600700 1 -- 172.21.15.22:0/487755561 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86dc01a5c0 con 0x7f86e81085f0 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.327+0000 7f86ef800700 1 -- 172.21.15.22:0/487755561 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e807c7d0 con 0x7f86e81085f0 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.329+0000 7f86e5600700 1 -- 172.21.15.22:0/487755561 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f86dc024430 con 0x7f86e81085f0 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.329+0000 7f86e5600700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 0x7f86d003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.329+0000 7f86e5600700 1 -- 172.21.15.22:0/487755561 --> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f86d003c130 con 0x7f86d0039560 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.329+0000 7f86e5600700 1 -- 172.21.15.22:0/487755561 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f86dc04db90 con 0x7f86e81085f0 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.329+0000 7f86e7e00700 1 -- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 msgr2=0x7f86d003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3430200041 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.329+0000 7f86e7e00700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 0x7f86d003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:22:57.201 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.530+0000 7f86e7e00700 1 -- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 msgr2=0x7f86d003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3430200041 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.530+0000 7f86e7e00700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 0x7f86d003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.931+0000 7f86e7e00700 1 -- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 msgr2=0x7f86d003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3430200041 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.931+0000 7f86e7e00700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 0x7f86d003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:50.732+0000 7f86e7e00700 1 -- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 msgr2=0x7f86d003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3430200041 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:50.732+0000 7f86e7e00700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 0x7f86d003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:52.334+0000 7f86e7e00700 1 -- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 msgr2=0x7f86d003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3430200041 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:52.334+0000 7f86e7e00700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 0x7f86d003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.489+0000 7f86e5600700 1 -- 172.21.15.22:0/487755561 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f86dc024770 con 0x7f86e81085f0 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.489+0000 7f86e5600700 1 -- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 msgr2=0x7f86d003ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.489+0000 7f86e5600700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 0x7f86d003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.494+0000 7f86e5600700 1 -- 172.21.15.22:0/487755561 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f86dc0188a0 con 0x7f86e81085f0 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.494+0000 7f86e5600700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f86d003ce50 0x7f86d003f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.202 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.494+0000 7f86e5600700 1 -- 172.21.15.22:0/487755561 --> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f86dc0068a0 con 0x7f86d003ce50 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.496+0000 7f86e7e00700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f86d003ce50 0x7f86d003f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.499+0000 7f86e7e00700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f86d003ce50 0x7f86d003f240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f86d4003a10 tx=0x7f86d4009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.501+0000 7f86e5600700 1 -- 172.21.15.22:0/487755561 <== mgr.14120 v2:172.21.15.22:6800/2098569388 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f86dc0068a0 con 0x7f86d003ce50 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.508+0000 7f86ef800700 1 -- 172.21.15.22:0/487755561 --> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f86e80640b0 con 0x7f86d003ce50 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.509+0000 7f86e5600700 1 -- 172.21.15.22:0/487755561 <== mgr.14120 v2:172.21.15.22:6800/2098569388 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f86e80640b0 con 0x7f86d003ce50 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.509+0000 7f86ef800700 1 -- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f86d003ce50 msgr2=0x7f86d003f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.509+0000 7f86ef800700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f86d003ce50 0x7f86d003f240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f86d4003a10 tx=0x7f86d4009290 comp rx=0 tx=0).stop 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.509+0000 7f86ef800700 1 -- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 msgr2=0x7f86e807db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.510+0000 7f86ef800700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 0x7f86e807db60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f86dc011670 tx=0x7f86dc011b60 comp rx=0 tx=0).stop 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.510+0000 7f86ef800700 1 -- 172.21.15.22:0/487755561 shutdown_connections 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.510+0000 7f86ef800700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f86d003ce50 0x7f86d003f240 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.510+0000 7f86ef800700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86e81085f0 0x7f86e807db60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.510+0000 7f86ef800700 1 --2- 172.21.15.22:0/487755561 >> [v2:172.21.15.22:6800/3430200041,v1:172.21.15.22:6801/3430200041] conn(0x7f86d0039560 0x7f86d003ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.510+0000 7f86ef800700 1 -- 172.21.15.22:0/487755561 >> 172.21.15.22:0/487755561 conn(0x7f86e8103c10 msgr2=0x7f86e8104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.510+0000 7f86ef800700 1 -- 172.21.15.22:0/487755561 shutdown_connections 2024-04-20T21:22:57.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:56.510+0000 7f86ef800700 1 -- 172.21.15.22:0/487755561 wait complete. 2024-04-20T21:22:57.204 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 5 is available 2024-04-20T21:22:57.204 INFO:teuthology.orchestra.run.smithi022.stdout:Setting orchestrator backend to cephadm... 2024-04-20T21:22:57.822 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:57 smithi022 ceph-mon[16843]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-20T21:22:57.822 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:57 smithi022 ceph-mon[16843]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-20T21:22:57.823 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:57 smithi022 ceph-mon[16843]: [20/Apr/2024:21:22:57] ENGINE Bus STARTING 2024-04-20T21:22:57.823 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:57 smithi022 ceph-mon[16843]: [20/Apr/2024:21:22:57] ENGINE Serving on https://172.21.15.22:7150 2024-04-20T21:22:57.823 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:57 smithi022 ceph-mon[16843]: [20/Apr/2024:21:22:57] ENGINE Serving on http://172.21.15.22:8765 2024-04-20T21:22:57.823 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:57 smithi022 ceph-mon[16843]: [20/Apr/2024:21:22:57] ENGINE Bus STARTED 2024-04-20T21:22:57.823 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:57 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:22:58.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:58 smithi022 ceph-mon[16843]: mgrmap e8: x(active, since 2s) 2024-04-20T21:22:58.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:58 smithi022 ceph-mon[16843]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:22:58.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:58 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:22:58.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:22:58 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:22:58.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.822+0000 7efc97400700 1 Processor -- start 2024-04-20T21:22:58.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.823+0000 7efc97400700 1 -- start start 2024-04-20T21:22:58.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.823+0000 7efc97400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 0x7efc90107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:58.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.823+0000 7efc97400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efc9007be40 con 0x7efc90106e40 2024-04-20T21:22:58.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.824+0000 7efc94a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 0x7efc90107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:58.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.824+0000 7efc94a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 0x7efc90107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52418/0 (socket says 172.21.15.22:52418) 2024-04-20T21:22:58.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.824+0000 7efc94a00700 1 -- 172.21.15.22:0/1193504780 learned_addr learned my addr 172.21.15.22:0/1193504780 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:58.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.824+0000 7efc94a00700 1 -- 172.21.15.22:0/1193504780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc901077a0 con 0x7efc90106e40 2024-04-20T21:22:58.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.825+0000 7efc94a00700 1 --2- 172.21.15.22:0/1193504780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 0x7efc90107260 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efc840098a0 tx=0x7efc84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c16ca512849931 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:58.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.825+0000 7efc8f400700 1 -- 172.21.15.22:0/1193504780 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc84004160 con 0x7efc90106e40 2024-04-20T21:22:58.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.825+0000 7efc8f400700 1 -- 172.21.15.22:0/1193504780 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc84004300 con 0x7efc90106e40 2024-04-20T21:22:58.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.825+0000 7efc8f400700 1 -- 172.21.15.22:0/1193504780 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc84011b50 con 0x7efc90106e40 2024-04-20T21:22:58.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.826+0000 7efc97400700 1 -- 172.21.15.22:0/1193504780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 msgr2=0x7efc90107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:58.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.826+0000 7efc97400700 1 --2- 172.21.15.22:0/1193504780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 0x7efc90107260 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efc840098a0 tx=0x7efc84009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.826+0000 7efc97400700 1 -- 172.21.15.22:0/1193504780 shutdown_connections 2024-04-20T21:22:58.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.826+0000 7efc97400700 1 --2- 172.21.15.22:0/1193504780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 0x7efc90107260 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.826+0000 7efc97400700 1 -- 172.21.15.22:0/1193504780 >> 172.21.15.22:0/1193504780 conn(0x7efc90102a20 msgr2=0x7efc90104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:58.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.826+0000 7efc97400700 1 -- 172.21.15.22:0/1193504780 shutdown_connections 2024-04-20T21:22:58.763 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.826+0000 7efc97400700 1 -- 172.21.15.22:0/1193504780 wait complete. 2024-04-20T21:22:58.763 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.827+0000 7efc97400700 1 Processor -- start 2024-04-20T21:22:58.764 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.827+0000 7efc97400700 1 -- start start 2024-04-20T21:22:58.764 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.828+0000 7efc97400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 0x7efc9019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.828+0000 7efc97400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efc90108830 con 0x7efc90106e40 2024-04-20T21:22:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.828+0000 7efc94a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 0x7efc9019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.828+0000 7efc94a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 0x7efc9019e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52422/0 (socket says 172.21.15.22:52422) 2024-04-20T21:22:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.828+0000 7efc94a00700 1 -- 172.21.15.22:0/118510269 learned_addr learned my addr 172.21.15.22:0/118510269 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:22:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.828+0000 7efc94a00700 1 -- 172.21.15.22:0/118510269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9019ece0 con 0x7efc90106e40 2024-04-20T21:22:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.829+0000 7efc94a00700 1 --2- 172.21.15.22:0/118510269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 0x7efc9019e7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efc84011b60 tx=0x7efc84011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.829+0000 7efc8d600700 1 -- 172.21.15.22:0/118510269 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc84003970 con 0x7efc90106e40 2024-04-20T21:22:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.829+0000 7efc8d600700 1 -- 172.21.15.22:0/118510269 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc84003b10 con 0x7efc90106e40 2024-04-20T21:22:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.829+0000 7efc8d600700 1 -- 172.21.15.22:0/118510269 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc84024610 con 0x7efc90106e40 2024-04-20T21:22:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.829+0000 7efc97400700 1 -- 172.21.15.22:0/118510269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9019ef70 con 0x7efc90106e40 2024-04-20T21:22:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.829+0000 7efc97400700 1 -- 172.21.15.22:0/118510269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc901a1c60 con 0x7efc90106e40 2024-04-20T21:22:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.830+0000 7efc8d600700 1 -- 172.21.15.22:0/118510269 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7efc8401a9f0 con 0x7efc90106e40 2024-04-20T21:22:58.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.830+0000 7efc97400700 1 -- 172.21.15.22:0/118510269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc900640b0 con 0x7efc90106e40 2024-04-20T21:22:58.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.831+0000 7efc8d600700 1 --2- 172.21.15.22:0/118510269 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7efc78039600 0x7efc7803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:58.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.831+0000 7efc8d600700 1 -- 172.21.15.22:0/118510269 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efc84011e00 con 0x7efc90106e40 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.831+0000 7efc8fe00700 1 --2- 172.21.15.22:0/118510269 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7efc78039600 0x7efc7803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.832+0000 7efc8fe00700 1 --2- 172.21.15.22:0/118510269 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7efc78039600 0x7efc7803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efc7c006fd0 tx=0x7efc7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.837+0000 7efc8d600700 1 -- 172.21.15.22:0/118510269 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efc8400b3e0 con 0x7efc90106e40 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:58.090+0000 7efc97400700 1 -- 172.21.15.22:0/118510269 --> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7efc9002e2d0 con 0x7efc78039600 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:58.103+0000 7efc8d600700 1 -- 172.21.15.22:0/118510269 <== mgr.14120 v2:172.21.15.22:6800/2098569388 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc9002e2d0 con 0x7efc78039600 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:58.106+0000 7efc97400700 1 -- 172.21.15.22:0/118510269 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7efc78039600 msgr2=0x7efc7803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:58.106+0000 7efc97400700 1 --2- 172.21.15.22:0/118510269 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7efc78039600 0x7efc7803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efc7c006fd0 tx=0x7efc7c006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:58.106+0000 7efc97400700 1 -- 172.21.15.22:0/118510269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 msgr2=0x7efc9019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:58.106+0000 7efc97400700 1 --2- 172.21.15.22:0/118510269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 0x7efc9019e7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efc84011b60 tx=0x7efc84011c40 comp rx=0 tx=0).stop 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:58.106+0000 7efc97400700 1 -- 172.21.15.22:0/118510269 shutdown_connections 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:58.106+0000 7efc97400700 1 --2- 172.21.15.22:0/118510269 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7efc78039600 0x7efc7803bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:58.106+0000 7efc97400700 1 --2- 172.21.15.22:0/118510269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc90106e40 0x7efc9019e7a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:58.106+0000 7efc97400700 1 -- 172.21.15.22:0/118510269 >> 172.21.15.22:0/118510269 conn(0x7efc90102a20 msgr2=0x7efc90195440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:58.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:58.107+0000 7efc97400700 1 -- 172.21.15.22:0/118510269 shutdown_connections 2024-04-20T21:22:58.769 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:58.107+0000 7efc97400700 1 -- 172.21.15.22:0/118510269 wait complete. 2024-04-20T21:23:00.294 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-20T21:23:00.294 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.420+0000 7f138a600700 1 Processor -- start 2024-04-20T21:23:00.294 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.420+0000 7f138a600700 1 -- start start 2024-04-20T21:23:00.294 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.420+0000 7f138a600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 0x7f13841078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.421+0000 7f138a600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1384107e70 con 0x7f1384107480 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.421+0000 7f1383400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 0x7f13841078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.421+0000 7f1383400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 0x7f13841078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52434/0 (socket says 172.21.15.22:52434) 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.421+0000 7f1383400700 1 -- 172.21.15.22:0/4155791124 learned_addr learned my addr 172.21.15.22:0/4155791124 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.422+0000 7f1383400700 1 -- 172.21.15.22:0/4155791124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1384108680 con 0x7f1384107480 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.422+0000 7f1383400700 1 --2- 172.21.15.22:0/4155791124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 0x7f13841078a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f13740098a0 tx=0x7f1374009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb74eef72618d995 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.422+0000 7f1382a00700 1 -- 172.21.15.22:0/4155791124 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1374004160 con 0x7f1384107480 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.422+0000 7f1382a00700 1 -- 172.21.15.22:0/4155791124 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1374004300 con 0x7f1384107480 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.422+0000 7f1382a00700 1 -- 172.21.15.22:0/4155791124 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1374011b50 con 0x7f1384107480 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.423+0000 7f138a600700 1 -- 172.21.15.22:0/4155791124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 msgr2=0x7f13841078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.423+0000 7f138a600700 1 --2- 172.21.15.22:0/4155791124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 0x7f13841078a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f13740098a0 tx=0x7f1374009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.423+0000 7f138a600700 1 -- 172.21.15.22:0/4155791124 shutdown_connections 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.423+0000 7f138a600700 1 --2- 172.21.15.22:0/4155791124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 0x7f13841078a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.295 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.423+0000 7f138a600700 1 -- 172.21.15.22:0/4155791124 >> 172.21.15.22:0/4155791124 conn(0x7f1384102a20 msgr2=0x7f1384104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.423+0000 7f138a600700 1 -- 172.21.15.22:0/4155791124 shutdown_connections 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.424+0000 7f138a600700 1 -- 172.21.15.22:0/4155791124 wait complete. 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.425+0000 7f138a600700 1 Processor -- start 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.425+0000 7f138a600700 1 -- start start 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.425+0000 7f138a600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 0x7f1384192630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.425+0000 7f138a600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f138410ba30 con 0x7f1384107480 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.426+0000 7f1383400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 0x7f1384192630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.426+0000 7f1383400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 0x7f1384192630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52444/0 (socket says 172.21.15.22:52444) 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.426+0000 7f1383400700 1 -- 172.21.15.22:0/2058284308 learned_addr learned my addr 172.21.15.22:0/2058284308 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.426+0000 7f1383400700 1 -- 172.21.15.22:0/2058284308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1384192b70 con 0x7f1384107480 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.426+0000 7f1383400700 1 --2- 172.21.15.22:0/2058284308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 0x7f1384192630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1374011850 tx=0x7f1374011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.427+0000 7f1380c00700 1 -- 172.21.15.22:0/2058284308 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1374003af0 con 0x7f1384107480 2024-04-20T21:23:00.296 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.427+0000 7f1380c00700 1 -- 172.21.15.22:0/2058284308 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1374020070 con 0x7f1384107480 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.427+0000 7f1380c00700 1 -- 172.21.15.22:0/2058284308 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13740195a0 con 0x7f1384107480 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.427+0000 7f138a600700 1 -- 172.21.15.22:0/2058284308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1384190d60 con 0x7f1384107480 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.427+0000 7f138a600700 1 -- 172.21.15.22:0/2058284308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13841912a0 con 0x7f1384107480 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.428+0000 7f1380c00700 1 -- 172.21.15.22:0/2058284308 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1374019850 con 0x7f1384107480 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.428+0000 7f138a600700 1 -- 172.21.15.22:0/2058284308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13840640b0 con 0x7f1384107480 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.429+0000 7f1380c00700 1 --2- 172.21.15.22:0/2058284308 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1368039600 0x7f136803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.429+0000 7f1380c00700 1 -- 172.21.15.22:0/2058284308 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1374011e00 con 0x7f1384107480 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.429+0000 7f137be00700 1 --2- 172.21.15.22:0/2058284308 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1368039600 0x7f136803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.430+0000 7f137be00700 1 --2- 172.21.15.22:0/2058284308 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1368039600 0x7f136803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f136c006fd0 tx=0x7f136c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.435+0000 7f1380c00700 1 -- 172.21.15.22:0/2058284308 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f137400e260 con 0x7f1384107480 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.669+0000 7f138a600700 1 -- 172.21.15.22:0/2058284308 --> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f1384196440 con 0x7f1368039600 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.671+0000 7f1380c00700 1 -- 172.21.15.22:0/2058284308 <== mgr.14120 v2:172.21.15.22:6800/2098569388 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f1384196440 con 0x7f1368039600 2024-04-20T21:23:00.297 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.674+0000 7f138a600700 1 -- 172.21.15.22:0/2058284308 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1368039600 msgr2=0x7f136803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.674+0000 7f138a600700 1 --2- 172.21.15.22:0/2058284308 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1368039600 0x7f136803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f136c006fd0 tx=0x7f136c006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:00.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.674+0000 7f138a600700 1 -- 172.21.15.22:0/2058284308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 msgr2=0x7f1384192630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.674+0000 7f138a600700 1 --2- 172.21.15.22:0/2058284308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 0x7f1384192630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1374011850 tx=0x7f1374011c10 comp rx=0 tx=0).stop 2024-04-20T21:23:00.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.674+0000 7f138a600700 1 -- 172.21.15.22:0/2058284308 shutdown_connections 2024-04-20T21:23:00.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.674+0000 7f138a600700 1 --2- 172.21.15.22:0/2058284308 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1368039600 0x7f136803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.674+0000 7f138a600700 1 --2- 172.21.15.22:0/2058284308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384107480 0x7f1384192630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.674+0000 7f138a600700 1 -- 172.21.15.22:0/2058284308 >> 172.21.15.22:0/2058284308 conn(0x7f1384102a20 msgr2=0x7f1384103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:00.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.674+0000 7f138a600700 1 -- 172.21.15.22:0/2058284308 shutdown_connections 2024-04-20T21:23:00.298 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.674+0000 7f138a600700 1 -- 172.21.15.22:0/2058284308 wait complete. 2024-04-20T21:23:00.298 INFO:teuthology.orchestra.run.smithi022.stdout:Generating ssh key... 2024-04-20T21:23:00.586 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:00 smithi022 ceph-mon[16843]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:01.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-mon[16843]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:01.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-mon[16843]: Generating ssh key... 2024-04-20T21:23:01.908 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: Generating public/private rsa key pair. 2024-04-20T21:23:01.908 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: Your identification has been saved in /tmp/tmpoovj6j7e/key. 2024-04-20T21:23:01.908 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: Your public key has been saved in /tmp/tmpoovj6j7e/key.pub. 2024-04-20T21:23:01.908 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: The key fingerprint is: 2024-04-20T21:23:01.908 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: SHA256:orCO4fDmwJafdUEQ7IK/glbxkH0zUsWcbNdLRP77E3M ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:01.908 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: The key's randomart image is: 2024-04-20T21:23:01.908 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: +---[RSA 3072]----+ 2024-04-20T21:23:01.908 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: | .o. =.. +o | 2024-04-20T21:23:01.908 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: | ... * ..o | 2024-04-20T21:23:01.909 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: | . + ... . ... | 2024-04-20T21:23:01.909 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: | . = +.+ .. | 2024-04-20T21:23:01.909 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: | o = +.S . | 2024-04-20T21:23:01.909 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: |. .= o .. oE| 2024-04-20T21:23:01.909 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: |=+o o. . .+| 2024-04-20T21:23:01.909 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: |=Oo.o . ..| 2024-04-20T21:23:01.909 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: |o+=o o| 2024-04-20T21:23:01.909 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: +----[SHA256]-----+ 2024-04-20T21:23:02.228 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.901+0000 7f410fe00700 1 Processor -- start 2024-04-20T21:23:02.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.901+0000 7f410fe00700 1 -- start start 2024-04-20T21:23:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.902+0000 7f410fe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 0x7f41101076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.902+0000 7f410fe00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4110107c90 con 0x7f41101072a0 2024-04-20T21:23:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.902+0000 7f410ea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 0x7f41101076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.903+0000 7f410ea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 0x7f41101076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52454/0 (socket says 172.21.15.22:52454) 2024-04-20T21:23:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.903+0000 7f410ea00700 1 -- 172.21.15.22:0/3182546594 learned_addr learned my addr 172.21.15.22:0/3182546594 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.903+0000 7f410ea00700 1 -- 172.21.15.22:0/3182546594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41101084a0 con 0x7f41101072a0 2024-04-20T21:23:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.904+0000 7f410ea00700 1 --2- 172.21.15.22:0/3182546594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 0x7f41101076c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f410000bb40 tx=0x7f410000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36020e32aa10383 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.904+0000 7f410e000700 1 -- 172.21.15.22:0/3182546594 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f410000dbe0 con 0x7f41101072a0 2024-04-20T21:23:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.904+0000 7f410e000700 1 -- 172.21.15.22:0/3182546594 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f41000044f0 con 0x7f41101072a0 2024-04-20T21:23:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.904+0000 7f410e000700 1 -- 172.21.15.22:0/3182546594 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4100010470 con 0x7f41101072a0 2024-04-20T21:23:02.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.905+0000 7f410fe00700 1 -- 172.21.15.22:0/3182546594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 msgr2=0x7f41101076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.905+0000 7f410fe00700 1 --2- 172.21.15.22:0/3182546594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 0x7f41101076c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f410000bb40 tx=0x7f410000be50 comp rx=0 tx=0).stop 2024-04-20T21:23:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.905+0000 7f410fe00700 1 -- 172.21.15.22:0/3182546594 shutdown_connections 2024-04-20T21:23:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.905+0000 7f410fe00700 1 --2- 172.21.15.22:0/3182546594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 0x7f41101076c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.905+0000 7f410fe00700 1 -- 172.21.15.22:0/3182546594 >> 172.21.15.22:0/3182546594 conn(0x7f4110102860 msgr2=0x7f4110104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.905+0000 7f410fe00700 1 -- 172.21.15.22:0/3182546594 shutdown_connections 2024-04-20T21:23:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.905+0000 7f410fe00700 1 -- 172.21.15.22:0/3182546594 wait complete. 2024-04-20T21:23:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.906+0000 7f410fe00700 1 Processor -- start 2024-04-20T21:23:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.906+0000 7f410fe00700 1 -- start start 2024-04-20T21:23:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.907+0000 7f410fe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 0x7f411019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.907+0000 7f410fe00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f411007afc0 con 0x7f41101072a0 2024-04-20T21:23:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.907+0000 7f410ea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 0x7f411019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.907+0000 7f410ea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 0x7f411019e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52470/0 (socket says 172.21.15.22:52470) 2024-04-20T21:23:02.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.907+0000 7f410ea00700 1 -- 172.21.15.22:0/2680955217 learned_addr learned my addr 172.21.15.22:0/2680955217 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:02.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.907+0000 7f410ea00700 1 -- 172.21.15.22:0/2680955217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411019eaf0 con 0x7f41101072a0 2024-04-20T21:23:02.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.908+0000 7f410ea00700 1 --2- 172.21.15.22:0/2680955217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 0x7f411019e5b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f410000b340 tx=0x7f410000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:02.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.908+0000 7f4107400700 1 -- 172.21.15.22:0/2680955217 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4100010680 con 0x7f41101072a0 2024-04-20T21:23:02.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.908+0000 7f4107400700 1 -- 172.21.15.22:0/2680955217 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f410001d070 con 0x7f41101072a0 2024-04-20T21:23:02.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.908+0000 7f4107400700 1 -- 172.21.15.22:0/2680955217 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4100021af0 con 0x7f41101072a0 2024-04-20T21:23:02.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.908+0000 7f410fe00700 1 -- 172.21.15.22:0/2680955217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411019ed80 con 0x7f41101072a0 2024-04-20T21:23:02.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.908+0000 7f410fe00700 1 -- 172.21.15.22:0/2680955217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41101a1a70 con 0x7f41101072a0 2024-04-20T21:23:02.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.909+0000 7f410fe00700 1 -- 172.21.15.22:0/2680955217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41100640b0 con 0x7f41101072a0 2024-04-20T21:23:02.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.909+0000 7f4107400700 1 -- 172.21.15.22:0/2680955217 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f410001f030 con 0x7f41101072a0 2024-04-20T21:23:02.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.910+0000 7f4107400700 1 --2- 172.21.15.22:0/2680955217 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f40f4039650 0x7f40f403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.910+0000 7f4107400700 1 -- 172.21.15.22:0/2680955217 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f410004d880 con 0x7f41101072a0 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.910+0000 7f4107e00700 1 --2- 172.21.15.22:0/2680955217 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f40f4039650 0x7f40f403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.911+0000 7f4107e00700 1 --2- 172.21.15.22:0/2680955217 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f40f4039650 0x7f40f403bb10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f40f8006fd0 tx=0x7f40f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:00.915+0000 7f4107400700 1 -- 172.21.15.22:0/2680955217 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4100015070 con 0x7f41101072a0 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:01.147+0000 7f410fe00700 1 -- 172.21.15.22:0/2680955217 --> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f4110062e90 con 0x7f40f4039650 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:01.616+0000 7f4107400700 1 -- 172.21.15.22:0/2680955217 <== mgr.14120 v2:172.21.15.22:6800/2098569388 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4110062e90 con 0x7f40f4039650 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:01.618+0000 7f410fe00700 1 -- 172.21.15.22:0/2680955217 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f40f4039650 msgr2=0x7f40f403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:01.618+0000 7f410fe00700 1 --2- 172.21.15.22:0/2680955217 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f40f4039650 0x7f40f403bb10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f40f8006fd0 tx=0x7f40f8006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:01.618+0000 7f410fe00700 1 -- 172.21.15.22:0/2680955217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 msgr2=0x7f411019e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:01.618+0000 7f410fe00700 1 --2- 172.21.15.22:0/2680955217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 0x7f411019e5b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f410000b340 tx=0x7f410000dd90 comp rx=0 tx=0).stop 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:01.619+0000 7f410fe00700 1 -- 172.21.15.22:0/2680955217 shutdown_connections 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:01.619+0000 7f410fe00700 1 --2- 172.21.15.22:0/2680955217 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f40f4039650 0x7f40f403bb10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:01.619+0000 7f410fe00700 1 --2- 172.21.15.22:0/2680955217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101072a0 0x7f411019e5b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:01.619+0000 7f410fe00700 1 -- 172.21.15.22:0/2680955217 >> 172.21.15.22:0/2680955217 conn(0x7f4110102860 msgr2=0x7f4110194bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:02.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:01.619+0000 7f410fe00700 1 -- 172.21.15.22:0/2680955217 shutdown_connections 2024-04-20T21:23:02.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:01.619+0000 7f410fe00700 1 -- 172.21.15.22:0/2680955217 wait complete. 2024-04-20T21:23:02.802 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:02 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:23:02.802 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:02 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:23:03.739 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:03 smithi022 ceph-mon[16843]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:03.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaAupTRE6xrMJ3Otr93e+WWUdSnuEhyzIx+TgCyj76BlkQyrXocGHEmuWElvD4l6YCSGPRQfd70Bejz3Z9ZgdptXZ/75ayPsOdg5g88L1w83p+d0pmPBX6GD2oW+6GjMQ+Dr0bClxQAXwIvuz87T+QxC0aUcO7jRRK3nkTKbTHN9ru/O/U7CWEPiJ6u2LYlDx/U3EDxbNI3/5rmZQZrN3pQEHFr06G36NqR20AuRspzSVtQbQMpBzfyrg12Vx2QK18zLmDSUpE7cqHtC3P6xzb7o5GJu95XB1KMDgujf2HBrcuSn/WRdUrJ6+Bv7HLKr17ihKScpJLkc+C96mSYmJl8zp1a05dnapCmhGOmVUNgrcaWFKZ9iadhRKIRiJO5ptBmvh73po+mNWjpXLlEDF+Uox9SIqtwb+wwR1oFvIM0cGcJvm6FRt+SqzswyyJQiski299vIhmil8nDTwd9KgUgXJ7FWkTKRZBCIv33rgh7WrOHL8d5Qr0vMOuVqYMSYk= ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:03.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.876+0000 7f357fc00700 1 Processor -- start 2024-04-20T21:23:03.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.877+0000 7f357fc00700 1 -- start start 2024-04-20T21:23:03.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.877+0000 7f357fc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 0x7f357807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.877+0000 7f357fc00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f357807c810 con 0x7f357807de20 2024-04-20T21:23:03.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.878+0000 7f357d200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 0x7f357807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.878+0000 7f357d200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 0x7f357807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55616/0 (socket says 172.21.15.22:55616) 2024-04-20T21:23:03.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.878+0000 7f357d200700 1 -- 172.21.15.22:0/1210415169 learned_addr learned my addr 172.21.15.22:0/1210415169 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:03.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.878+0000 7f357d200700 1 -- 172.21.15.22:0/1210415169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357807c990 con 0x7f357807de20 2024-04-20T21:23:03.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.878+0000 7f357d200700 1 --2- 172.21.15.22:0/1210415169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 0x7f357807c2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f356c0098a0 tx=0x7f356c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15231c9b93ca2ec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:03.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.879+0000 7f3577400700 1 -- 172.21.15.22:0/1210415169 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f356c004160 con 0x7f357807de20 2024-04-20T21:23:03.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.879+0000 7f3577400700 1 -- 172.21.15.22:0/1210415169 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f356c004300 con 0x7f357807de20 2024-04-20T21:23:03.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.879+0000 7f3577400700 1 -- 172.21.15.22:0/1210415169 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f356c011b50 con 0x7f357807de20 2024-04-20T21:23:03.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.879+0000 7f357fc00700 1 -- 172.21.15.22:0/1210415169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 msgr2=0x7f357807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.879+0000 7f357fc00700 1 --2- 172.21.15.22:0/1210415169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 0x7f357807c2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f356c0098a0 tx=0x7f356c009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.880+0000 7f357fc00700 1 -- 172.21.15.22:0/1210415169 shutdown_connections 2024-04-20T21:23:03.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.880+0000 7f357fc00700 1 --2- 172.21.15.22:0/1210415169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 0x7f357807c2d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.880+0000 7f357fc00700 1 -- 172.21.15.22:0/1210415169 >> 172.21.15.22:0/1210415169 conn(0x7f3578103b80 msgr2=0x7f3578105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:03.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.880+0000 7f357fc00700 1 -- 172.21.15.22:0/1210415169 shutdown_connections 2024-04-20T21:23:03.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.880+0000 7f357fc00700 1 -- 172.21.15.22:0/1210415169 wait complete. 2024-04-20T21:23:03.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.882+0000 7f357fc00700 1 Processor -- start 2024-04-20T21:23:03.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.882+0000 7f357fc00700 1 -- start start 2024-04-20T21:23:03.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.882+0000 7f357fc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 0x7f357819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.882+0000 7f357fc00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f357810eb30 con 0x7f357807de20 2024-04-20T21:23:03.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.882+0000 7f357d200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 0x7f357819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.883+0000 7f357d200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 0x7f357819e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55624/0 (socket says 172.21.15.22:55624) 2024-04-20T21:23:03.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.883+0000 7f357d200700 1 -- 172.21.15.22:0/2686050975 learned_addr learned my addr 172.21.15.22:0/2686050975 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:03.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.883+0000 7f357d200700 1 -- 172.21.15.22:0/2686050975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357819ec20 con 0x7f357807de20 2024-04-20T21:23:03.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.883+0000 7f357d200700 1 --2- 172.21.15.22:0/2686050975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 0x7f357819e6e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f356c011b60 tx=0x7f356c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:03.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.884+0000 7f3575600700 1 -- 172.21.15.22:0/2686050975 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f356c003b20 con 0x7f357807de20 2024-04-20T21:23:03.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.884+0000 7f3575600700 1 -- 172.21.15.22:0/2686050975 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f356c003cc0 con 0x7f357807de20 2024-04-20T21:23:03.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.884+0000 7f357fc00700 1 -- 172.21.15.22:0/2686050975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357819eeb0 con 0x7f357807de20 2024-04-20T21:23:03.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.884+0000 7f3575600700 1 -- 172.21.15.22:0/2686050975 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f356c024840 con 0x7f357807de20 2024-04-20T21:23:03.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.884+0000 7f357fc00700 1 -- 172.21.15.22:0/2686050975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35781a1ba0 con 0x7f357807de20 2024-04-20T21:23:03.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.885+0000 7f3575600700 1 -- 172.21.15.22:0/2686050975 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f356c022020 con 0x7f357807de20 2024-04-20T21:23:03.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.885+0000 7f357fc00700 1 -- 172.21.15.22:0/2686050975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35780640b0 con 0x7f357807de20 2024-04-20T21:23:03.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.886+0000 7f3575600700 1 --2- 172.21.15.22:0/2686050975 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f3560039600 0x7f356003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.886+0000 7f3575600700 1 -- 172.21.15.22:0/2686050975 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f356c011e00 con 0x7f357807de20 2024-04-20T21:23:03.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.886+0000 7f3577e00700 1 --2- 172.21.15.22:0/2686050975 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f3560039600 0x7f356003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.887+0000 7f3577e00700 1 --2- 172.21.15.22:0/2686050975 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f3560039600 0x7f356003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3568006fd0 tx=0x7f3568006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:03.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:02.891+0000 7f3575600700 1 -- 172.21.15.22:0/2686050975 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f356c00b3e0 con 0x7f357807de20 2024-04-20T21:23:03.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:03.118+0000 7f357fc00700 1 -- 172.21.15.22:0/2686050975 --> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f3578104e50 con 0x7f3560039600 2024-04-20T21:23:03.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:03.120+0000 7f3575600700 1 -- 172.21.15.22:0/2686050975 <== mgr.14120 v2:172.21.15.22:6800/2098569388 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f3578104e50 con 0x7f3560039600 2024-04-20T21:23:03.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:03.123+0000 7f357fc00700 1 -- 172.21.15.22:0/2686050975 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f3560039600 msgr2=0x7f356003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:03.123+0000 7f357fc00700 1 --2- 172.21.15.22:0/2686050975 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f3560039600 0x7f356003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3568006fd0 tx=0x7f3568006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:03.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:03.123+0000 7f357fc00700 1 -- 172.21.15.22:0/2686050975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 msgr2=0x7f357819e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:03.123+0000 7f357fc00700 1 --2- 172.21.15.22:0/2686050975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 0x7f357819e6e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f356c011b60 tx=0x7f356c011c40 comp rx=0 tx=0).stop 2024-04-20T21:23:03.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:03.123+0000 7f357fc00700 1 -- 172.21.15.22:0/2686050975 shutdown_connections 2024-04-20T21:23:03.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:03.123+0000 7f357fc00700 1 --2- 172.21.15.22:0/2686050975 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f3560039600 0x7f356003bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:03.123+0000 7f357fc00700 1 --2- 172.21.15.22:0/2686050975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f357807de20 0x7f357819e6e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.763 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:03.123+0000 7f357fc00700 1 -- 172.21.15.22:0/2686050975 >> 172.21.15.22:0/2686050975 conn(0x7f3578103b80 msgr2=0x7f3578104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:03.763 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:03.123+0000 7f357fc00700 1 -- 172.21.15.22:0/2686050975 shutdown_connections 2024-04-20T21:23:03.763 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:03.123+0000 7f357fc00700 1 -- 172.21.15.22:0/2686050975 wait complete. 2024-04-20T21:23:03.764 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-20T21:23:03.764 INFO:teuthology.orchestra.run.smithi022.stdout:Adding key to root@localhost authorized_keys... 2024-04-20T21:23:03.784 INFO:teuthology.orchestra.run.smithi022.stdout:Adding host smithi022... 2024-04-20T21:23:05.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:05 smithi022 ceph-mon[16843]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:06.602 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:06 smithi022 ceph-mon[16843]: Deploying cephadm binary to smithi022 2024-04-20T21:23:11.541 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Added host 'smithi022' with addr '172.21.15.22' 2024-04-20T21:23:11.541 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.361+0000 7fbfa8a00700 1 Processor -- start 2024-04-20T21:23:11.543 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.362+0000 7fbfa8a00700 1 -- start start 2024-04-20T21:23:11.543 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.362+0000 7fbfa8a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 0x7fbfa41089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.543 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.362+0000 7fbfa8a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4108fc0 con 0x7fbfa41085d0 2024-04-20T21:23:11.543 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.363+0000 7fbfa1e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 0x7fbfa41089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.543 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.363+0000 7fbfa1e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 0x7fbfa41089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55628/0 (socket says 172.21.15.22:55628) 2024-04-20T21:23:11.543 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.363+0000 7fbfa1e00700 1 -- 172.21.15.22:0/2324907891 learned_addr learned my addr 172.21.15.22:0/2324907891 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:11.543 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.364+0000 7fbfa1e00700 1 -- 172.21.15.22:0/2324907891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa4109730 con 0x7fbfa41085d0 2024-04-20T21:23:11.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.364+0000 7fbfa1e00700 1 --2- 172.21.15.22:0/2324907891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 0x7fbfa41089f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbf940098a0 tx=0x7fbf94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd245bff6799b945 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:11.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.364+0000 7fbfa0a00700 1 -- 172.21.15.22:0/2324907891 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf94004160 con 0x7fbfa41085d0 2024-04-20T21:23:11.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.364+0000 7fbfa0a00700 1 -- 172.21.15.22:0/2324907891 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbf94004300 con 0x7fbfa41085d0 2024-04-20T21:23:11.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.364+0000 7fbfa0a00700 1 -- 172.21.15.22:0/2324907891 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf94011b50 con 0x7fbfa41085d0 2024-04-20T21:23:11.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.365+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2324907891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 msgr2=0x7fbfa41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:11.544 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.365+0000 7fbfa8a00700 1 --2- 172.21.15.22:0/2324907891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 0x7fbfa41089f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbf940098a0 tx=0x7fbf94009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.365+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2324907891 shutdown_connections 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.365+0000 7fbfa8a00700 1 --2- 172.21.15.22:0/2324907891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 0x7fbfa41089f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.365+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2324907891 >> 172.21.15.22:0/2324907891 conn(0x7fbfa4103bb0 msgr2=0x7fbfa4105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.365+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2324907891 shutdown_connections 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.365+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2324907891 wait complete. 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.366+0000 7fbfa8a00700 1 Processor -- start 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.366+0000 7fbfa8a00700 1 -- start start 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.367+0000 7fbfa8a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 0x7fbfa419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.367+0000 7fbfa8a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa410cae0 con 0x7fbfa41085d0 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.367+0000 7fbfa1e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 0x7fbfa419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.367+0000 7fbfa1e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 0x7fbfa419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55630/0 (socket says 172.21.15.22:55630) 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.367+0000 7fbfa1e00700 1 -- 172.21.15.22:0/2706448241 learned_addr learned my addr 172.21.15.22:0/2706448241 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:11.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.367+0000 7fbfa1e00700 1 -- 172.21.15.22:0/2706448241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa419cac0 con 0x7fbfa41085d0 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.368+0000 7fbfa1e00700 1 --2- 172.21.15.22:0/2706448241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 0x7fbfa419c580 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbf94011850 tx=0x7fbf94011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.368+0000 7fbf9aa00700 1 -- 172.21.15.22:0/2706448241 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf94003af0 con 0x7fbfa41085d0 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.368+0000 7fbf9aa00700 1 -- 172.21.15.22:0/2706448241 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbf94020070 con 0x7fbfa41085d0 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.368+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2706448241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa419cd50 con 0x7fbfa41085d0 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.368+0000 7fbf9aa00700 1 -- 172.21.15.22:0/2706448241 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf940195a0 con 0x7fbfa41085d0 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.368+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2706448241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa419fa40 con 0x7fbfa41085d0 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.369+0000 7fbf9aa00700 1 -- 172.21.15.22:0/2706448241 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbf9401a430 con 0x7fbfa41085d0 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.369+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2706448241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa40640b0 con 0x7fbfa41085d0 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.370+0000 7fbf9aa00700 1 --2- 172.21.15.22:0/2706448241 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fbf88039600 0x7fbf8803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.370+0000 7fbf9aa00700 1 -- 172.21.15.22:0/2706448241 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbf94011e00 con 0x7fbfa41085d0 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.370+0000 7fbfa1400700 1 --2- 172.21.15.22:0/2706448241 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fbf88039600 0x7fbf8803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.371+0000 7fbfa1400700 1 --2- 172.21.15.22:0/2706448241 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fbf88039600 0x7fbf8803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbf90006fd0 tx=0x7fbf90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:11.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.376+0000 7fbf9aa00700 1 -- 172.21.15.22:0/2706448241 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbf9400e260 con 0x7fbfa41085d0 2024-04-20T21:23:11.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:04.610+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2706448241 --> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}) -- 0x7fbfa4104e90 con 0x7fbf88039600 2024-04-20T21:23:11.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:10.931+0000 7fbf9aa00700 1 -- 172.21.15.22:0/2706448241 <== mgr.14120 v2:172.21.15.22:6800/2098569388 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fbfa4104e90 con 0x7fbf88039600 2024-04-20T21:23:11.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:10.935+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2706448241 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fbf88039600 msgr2=0x7fbf8803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:11.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:10.935+0000 7fbfa8a00700 1 --2- 172.21.15.22:0/2706448241 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fbf88039600 0x7fbf8803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbf90006fd0 tx=0x7fbf90006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:11.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:10.935+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2706448241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 msgr2=0x7fbfa419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:11.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:10.935+0000 7fbfa8a00700 1 --2- 172.21.15.22:0/2706448241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 0x7fbfa419c580 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbf94011850 tx=0x7fbf94011c10 comp rx=0 tx=0).stop 2024-04-20T21:23:11.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:10.935+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2706448241 shutdown_connections 2024-04-20T21:23:11.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:10.935+0000 7fbfa8a00700 1 --2- 172.21.15.22:0/2706448241 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fbf88039600 0x7fbf8803bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.550 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:10.935+0000 7fbfa8a00700 1 --2- 172.21.15.22:0/2706448241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfa41085d0 0x7fbfa419c580 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:10.935+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2706448241 >> 172.21.15.22:0/2706448241 conn(0x7fbfa4103bb0 msgr2=0x7fbfa4104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:11.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:10.935+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2706448241 shutdown_connections 2024-04-20T21:23:11.551 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:10.936+0000 7fbfa8a00700 1 -- 172.21.15.22:0/2706448241 wait complete. 2024-04-20T21:23:11.551 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mon service... 2024-04-20T21:23:12.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:11 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:23:12.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:11 smithi022 ceph-mon[16843]: Added host smithi022 2024-04-20T21:23:12.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:11 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:13.051 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-20T21:23:13.051 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.125+0000 7f1ced600700 1 Processor -- start 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.125+0000 7f1ced600700 1 -- start start 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.125+0000 7f1ced600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 0x7f1ce810f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.125+0000 7f1ced600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce810fa40 con 0x7f1ce810f050 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.126+0000 7f1ce6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 0x7f1ce810f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.126+0000 7f1ce6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 0x7f1ce810f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55632/0 (socket says 172.21.15.22:55632) 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.126+0000 7f1ce6a00700 1 -- 172.21.15.22:0/3153211823 learned_addr learned my addr 172.21.15.22:0/3153211823 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.126+0000 7f1ce6a00700 1 -- 172.21.15.22:0/3153211823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce810fd70 con 0x7f1ce810f050 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.127+0000 7f1ce6a00700 1 --2- 172.21.15.22:0/3153211823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 0x7f1ce810f470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80098a0 tx=0x7f1cd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3263727b7455012f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.127+0000 7f1ce5600700 1 -- 172.21.15.22:0/3153211823 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cd8004160 con 0x7f1ce810f050 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.127+0000 7f1ce5600700 1 -- 172.21.15.22:0/3153211823 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1cd8004300 con 0x7f1ce810f050 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.127+0000 7f1ce5600700 1 -- 172.21.15.22:0/3153211823 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cd8011b50 con 0x7f1ce810f050 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.128+0000 7f1ced600700 1 -- 172.21.15.22:0/3153211823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 msgr2=0x7f1ce810f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:13.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.128+0000 7f1ced600700 1 --2- 172.21.15.22:0/3153211823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 0x7f1ce810f470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80098a0 tx=0x7f1cd8009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.128+0000 7f1ced600700 1 -- 172.21.15.22:0/3153211823 shutdown_connections 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.128+0000 7f1ced600700 1 --2- 172.21.15.22:0/3153211823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 0x7f1ce810f470 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.128+0000 7f1ced600700 1 -- 172.21.15.22:0/3153211823 >> 172.21.15.22:0/3153211823 conn(0x7f1ce807ddd0 msgr2=0x7f1ce80801f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.128+0000 7f1ced600700 1 -- 172.21.15.22:0/3153211823 shutdown_connections 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.128+0000 7f1ced600700 1 -- 172.21.15.22:0/3153211823 wait complete. 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.129+0000 7f1ced600700 1 Processor -- start 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.129+0000 7f1ced600700 1 -- start start 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.130+0000 7f1ced600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 0x7f1ce81a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.130+0000 7f1ced600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce8113120 con 0x7f1ce810f050 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.130+0000 7f1ce6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 0x7f1ce81a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.130+0000 7f1ce6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 0x7f1ce81a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55634/0 (socket says 172.21.15.22:55634) 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.130+0000 7f1ce6a00700 1 -- 172.21.15.22:0/2577596971 learned_addr learned my addr 172.21.15.22:0/2577596971 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.130+0000 7f1ce6a00700 1 -- 172.21.15.22:0/2577596971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce81a3000 con 0x7f1ce810f050 2024-04-20T21:23:13.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.131+0000 7f1ce6a00700 1 --2- 172.21.15.22:0/2577596971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 0x7f1ce81a2ac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8011b60 tx=0x7f1cd8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:13.054 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.131+0000 7f1cdf400700 1 -- 172.21.15.22:0/2577596971 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cd8003b20 con 0x7f1ce810f050 2024-04-20T21:23:13.054 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.131+0000 7f1cdf400700 1 -- 172.21.15.22:0/2577596971 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1cd8003cc0 con 0x7f1ce810f050 2024-04-20T21:23:13.054 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.131+0000 7f1cdf400700 1 -- 172.21.15.22:0/2577596971 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cd8024840 con 0x7f1ce810f050 2024-04-20T21:23:13.054 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.131+0000 7f1ced600700 1 -- 172.21.15.22:0/2577596971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce81a3290 con 0x7f1ce810f050 2024-04-20T21:23:13.054 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.131+0000 7f1ced600700 1 -- 172.21.15.22:0/2577596971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce81a5f80 con 0x7f1ce810f050 2024-04-20T21:23:13.054 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.132+0000 7f1cdf400700 1 -- 172.21.15.22:0/2577596971 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1cd8022020 con 0x7f1ce810f050 2024-04-20T21:23:13.054 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.133+0000 7f1ced600700 1 -- 172.21.15.22:0/2577596971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce80640b0 con 0x7f1ce810f050 2024-04-20T21:23:13.054 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.133+0000 7f1cdf400700 1 --2- 172.21.15.22:0/2577596971 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1ccc039600 0x7f1ccc03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:13.054 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.133+0000 7f1cdf400700 1 -- 172.21.15.22:0/2577596971 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1cd8011e00 con 0x7f1ce810f050 2024-04-20T21:23:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.133+0000 7f1ce6000700 1 --2- 172.21.15.22:0/2577596971 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1ccc039600 0x7f1ccc03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.134+0000 7f1ce6000700 1 --2- 172.21.15.22:0/2577596971 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1ccc039600 0x7f1ccc03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4006fd0 tx=0x7f1cd4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.139+0000 7f1cdf400700 1 -- 172.21.15.22:0/2577596971 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1cd804d1a0 con 0x7f1ce810f050 2024-04-20T21:23:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.387+0000 7f1ced600700 1 -- 172.21.15.22:0/2577596971 --> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1ce807ffd0 con 0x7f1ccc039600 2024-04-20T21:23:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.394+0000 7f1cdf400700 1 -- 172.21.15.22:0/2577596971 <== mgr.14120 v2:172.21.15.22:6800/2098569388 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1ce807ffd0 con 0x7f1ccc039600 2024-04-20T21:23:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.396+0000 7f1ced600700 1 -- 172.21.15.22:0/2577596971 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1ccc039600 msgr2=0x7f1ccc03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.397+0000 7f1ced600700 1 --2- 172.21.15.22:0/2577596971 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1ccc039600 0x7f1ccc03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4006fd0 tx=0x7f1cd4006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.397+0000 7f1ced600700 1 -- 172.21.15.22:0/2577596971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 msgr2=0x7f1ce81a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.397+0000 7f1ced600700 1 --2- 172.21.15.22:0/2577596971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 0x7f1ce81a2ac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8011b60 tx=0x7f1cd8011c40 comp rx=0 tx=0).stop 2024-04-20T21:23:13.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.397+0000 7f1ced600700 1 -- 172.21.15.22:0/2577596971 shutdown_connections 2024-04-20T21:23:13.057 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.397+0000 7f1ced600700 1 --2- 172.21.15.22:0/2577596971 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1ccc039600 0x7f1ccc03bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.057 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.397+0000 7f1ced600700 1 --2- 172.21.15.22:0/2577596971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce810f050 0x7f1ce81a2ac0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.057 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.397+0000 7f1ced600700 1 -- 172.21.15.22:0/2577596971 >> 172.21.15.22:0/2577596971 conn(0x7f1ce807ddd0 msgr2=0x7f1ce807f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:13.057 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.397+0000 7f1ced600700 1 -- 172.21.15.22:0/2577596971 shutdown_connections 2024-04-20T21:23:13.057 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:12.397+0000 7f1ced600700 1 -- 172.21.15.22:0/2577596971 wait complete. 2024-04-20T21:23:13.057 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mgr service... 2024-04-20T21:23:13.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:13 smithi022 ceph-mon[16843]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:13.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:13 smithi022 ceph-mon[16843]: Saving service mon spec with placement count:5 2024-04-20T21:23:13.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:13 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:23:15.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-20T21:23:15.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.195+0000 7f1343200700 1 Processor -- start 2024-04-20T21:23:15.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.196+0000 7f1343200700 1 -- start start 2024-04-20T21:23:15.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.196+0000 7f1343200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 0x7f133c110390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.196+0000 7f1343200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f133c110960 con 0x7f133c10ff70 2024-04-20T21:23:15.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.197+0000 7f1341e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 0x7f133c110390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.197+0000 7f1341e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 0x7f133c110390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40842/0 (socket says 172.21.15.22:40842) 2024-04-20T21:23:15.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.197+0000 7f1341e00700 1 -- 172.21.15.22:0/857872197 learned_addr learned my addr 172.21.15.22:0/857872197 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:15.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.197+0000 7f1341e00700 1 -- 172.21.15.22:0/857872197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c110ae0 con 0x7f133c10ff70 2024-04-20T21:23:15.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.197+0000 7f1341e00700 1 --2- 172.21.15.22:0/857872197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 0x7f133c110390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f13300098a0 tx=0x7f1330009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb9b309284f74ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:15.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.198+0000 7f1340a00700 1 -- 172.21.15.22:0/857872197 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1330004160 con 0x7f133c10ff70 2024-04-20T21:23:15.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.198+0000 7f1340a00700 1 -- 172.21.15.22:0/857872197 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1330004300 con 0x7f133c10ff70 2024-04-20T21:23:15.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.198+0000 7f1340a00700 1 -- 172.21.15.22:0/857872197 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1330011b50 con 0x7f133c10ff70 2024-04-20T21:23:15.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.198+0000 7f1343200700 1 -- 172.21.15.22:0/857872197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 msgr2=0x7f133c110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.198+0000 7f1343200700 1 --2- 172.21.15.22:0/857872197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 0x7f133c110390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f13300098a0 tx=0x7f1330009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.199+0000 7f1343200700 1 -- 172.21.15.22:0/857872197 shutdown_connections 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.199+0000 7f1343200700 1 --2- 172.21.15.22:0/857872197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 0x7f133c110390 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.199+0000 7f1343200700 1 -- 172.21.15.22:0/857872197 >> 172.21.15.22:0/857872197 conn(0x7f133c07dbb0 msgr2=0x7f133c07ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.199+0000 7f1343200700 1 -- 172.21.15.22:0/857872197 shutdown_connections 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.199+0000 7f1343200700 1 -- 172.21.15.22:0/857872197 wait complete. 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.200+0000 7f1343200700 1 Processor -- start 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.200+0000 7f1343200700 1 -- start start 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.200+0000 7f1343200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 0x7f133c1a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.200+0000 7f1343200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f133c112f00 con 0x7f133c10ff70 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.201+0000 7f1341e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 0x7f133c1a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.201+0000 7f1341e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 0x7f133c1a29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40852/0 (socket says 172.21.15.22:40852) 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.201+0000 7f1341e00700 1 -- 172.21.15.22:0/2181624596 learned_addr learned my addr 172.21.15.22:0/2181624596 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.201+0000 7f1341e00700 1 -- 172.21.15.22:0/2181624596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c1a2f20 con 0x7f133c10ff70 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.201+0000 7f1341e00700 1 --2- 172.21.15.22:0/2181624596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 0x7f133c1a29e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1330011850 tx=0x7f1330011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:15.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.201+0000 7f1336a00700 1 -- 172.21.15.22:0/2181624596 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1330003af0 con 0x7f133c10ff70 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.202+0000 7f1336a00700 1 -- 172.21.15.22:0/2181624596 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1330020070 con 0x7f133c10ff70 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.202+0000 7f1336a00700 1 -- 172.21.15.22:0/2181624596 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13300195a0 con 0x7f133c10ff70 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.202+0000 7f1343200700 1 -- 172.21.15.22:0/2181624596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c1a31b0 con 0x7f133c10ff70 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.202+0000 7f1343200700 1 -- 172.21.15.22:0/2181624596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c1a5ea0 con 0x7f133c10ff70 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.203+0000 7f1343200700 1 -- 172.21.15.22:0/2181624596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133c0640b0 con 0x7f133c10ff70 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.203+0000 7f1336a00700 1 -- 172.21.15.22:0/2181624596 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1330019850 con 0x7f133c10ff70 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.204+0000 7f1336a00700 1 --2- 172.21.15.22:0/2181624596 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1324039600 0x7f132403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.204+0000 7f1336a00700 1 -- 172.21.15.22:0/2181624596 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1330011e00 con 0x7f133c10ff70 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.204+0000 7f1341400700 1 --2- 172.21.15.22:0/2181624596 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1324039600 0x7f132403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.205+0000 7f1341400700 1 --2- 172.21.15.22:0/2181624596 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1324039600 0x7f132403bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f132c006fd0 tx=0x7f132c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.209+0000 7f1336a00700 1 -- 172.21.15.22:0/2181624596 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f133000e260 con 0x7f133c10ff70 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.459+0000 7f1343200700 1 -- 172.21.15.22:0/2181624596 --> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f133c07fe80 con 0x7f1324039600 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.469+0000 7f1336a00700 1 -- 172.21.15.22:0/2181624596 <== mgr.14120 v2:172.21.15.22:6800/2098569388 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f133c07fe80 con 0x7f1324039600 2024-04-20T21:23:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.472+0000 7f1343200700 1 -- 172.21.15.22:0/2181624596 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1324039600 msgr2=0x7f132403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.472+0000 7f1343200700 1 --2- 172.21.15.22:0/2181624596 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1324039600 0x7f132403bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f132c006fd0 tx=0x7f132c006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:15.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.472+0000 7f1343200700 1 -- 172.21.15.22:0/2181624596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 msgr2=0x7f133c1a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.472+0000 7f1343200700 1 --2- 172.21.15.22:0/2181624596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 0x7f133c1a29e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1330011850 tx=0x7f1330011c10 comp rx=0 tx=0).stop 2024-04-20T21:23:15.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.472+0000 7f1343200700 1 -- 172.21.15.22:0/2181624596 shutdown_connections 2024-04-20T21:23:15.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.472+0000 7f1343200700 1 --2- 172.21.15.22:0/2181624596 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f1324039600 0x7f132403bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.472+0000 7f1343200700 1 --2- 172.21.15.22:0/2181624596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f133c10ff70 0x7f133c1a29e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.472+0000 7f1343200700 1 -- 172.21.15.22:0/2181624596 >> 172.21.15.22:0/2181624596 conn(0x7f133c07dbb0 msgr2=0x7f133c07f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:15.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.473+0000 7f1343200700 1 -- 172.21.15.22:0/2181624596 shutdown_connections 2024-04-20T21:23:15.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:14.473+0000 7f1343200700 1 -- 172.21.15.22:0/2181624596 wait complete. 2024-04-20T21:23:15.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:15 smithi022 ceph-mon[16843]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:15.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:15 smithi022 ceph-mon[16843]: Saving service mgr spec with placement count:2 2024-04-20T21:23:15.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:15 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:23:15.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:15 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:23:16.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.798+0000 7f67b7c00700 1 Processor -- start 2024-04-20T21:23:16.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.798+0000 7f67b7c00700 1 -- start start 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.798+0000 7f67b7c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 0x7f67b007c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.798+0000 7f67b7c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67b007c810 con 0x7f67b007de20 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.799+0000 7f67b5200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 0x7f67b007c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.799+0000 7f67b5200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 0x7f67b007c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40864/0 (socket says 172.21.15.22:40864) 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.799+0000 7f67b5200700 1 -- 172.21.15.22:0/2945239732 learned_addr learned my addr 172.21.15.22:0/2945239732 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.799+0000 7f67b5200700 1 -- 172.21.15.22:0/2945239732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b007c990 con 0x7f67b007de20 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.800+0000 7f67b5200700 1 --2- 172.21.15.22:0/2945239732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 0x7f67b007c2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f67a40098a0 tx=0x7f67a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb312c4c2600627 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.800+0000 7f67af400700 1 -- 172.21.15.22:0/2945239732 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67a4004160 con 0x7f67b007de20 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.800+0000 7f67af400700 1 -- 172.21.15.22:0/2945239732 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a4004300 con 0x7f67b007de20 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.800+0000 7f67af400700 1 -- 172.21.15.22:0/2945239732 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67a4011b20 con 0x7f67b007de20 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.801+0000 7f67b7c00700 1 -- 172.21.15.22:0/2945239732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 msgr2=0x7f67b007c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.801+0000 7f67b7c00700 1 --2- 172.21.15.22:0/2945239732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 0x7f67b007c2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f67a40098a0 tx=0x7f67a4009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:16.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.801+0000 7f67b7c00700 1 -- 172.21.15.22:0/2945239732 shutdown_connections 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.801+0000 7f67b7c00700 1 --2- 172.21.15.22:0/2945239732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 0x7f67b007c2d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.801+0000 7f67b7c00700 1 -- 172.21.15.22:0/2945239732 >> 172.21.15.22:0/2945239732 conn(0x7f67b0103b80 msgr2=0x7f67b0105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.801+0000 7f67b7c00700 1 -- 172.21.15.22:0/2945239732 shutdown_connections 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.801+0000 7f67b7c00700 1 -- 172.21.15.22:0/2945239732 wait complete. 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.802+0000 7f67b7c00700 1 Processor -- start 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.802+0000 7f67b7c00700 1 -- start start 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.803+0000 7f67b7c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 0x7f67b019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.803+0000 7f67b7c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67b010eb20 con 0x7f67b007de20 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.803+0000 7f67b5200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 0x7f67b019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.803+0000 7f67b5200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 0x7f67b019e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40878/0 (socket says 172.21.15.22:40878) 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.803+0000 7f67b5200700 1 -- 172.21.15.22:0/703976871 learned_addr learned my addr 172.21.15.22:0/703976871 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.803+0000 7f67b5200700 1 -- 172.21.15.22:0/703976871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b019ec20 con 0x7f67b007de20 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.804+0000 7f67b5200700 1 --2- 172.21.15.22:0/703976871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 0x7f67b019e6e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67a40117e0 tx=0x7f67a4011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:16.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.804+0000 7f67ad600700 1 -- 172.21.15.22:0/703976871 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67a401a430 con 0x7f67b007de20 2024-04-20T21:23:16.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.804+0000 7f67ad600700 1 -- 172.21.15.22:0/703976871 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a401aab0 con 0x7f67b007de20 2024-04-20T21:23:16.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.804+0000 7f67b7c00700 1 -- 172.21.15.22:0/703976871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b019eeb0 con 0x7f67b007de20 2024-04-20T21:23:16.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.804+0000 7f67ad600700 1 -- 172.21.15.22:0/703976871 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67a4024a60 con 0x7f67b007de20 2024-04-20T21:23:16.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.804+0000 7f67b7c00700 1 -- 172.21.15.22:0/703976871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b019f310 con 0x7f67b007de20 2024-04-20T21:23:16.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.805+0000 7f67ad600700 1 -- 172.21.15.22:0/703976871 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f67a4022020 con 0x7f67b007de20 2024-04-20T21:23:16.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.805+0000 7f67b7c00700 1 -- 172.21.15.22:0/703976871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b00640b0 con 0x7f67b007de20 2024-04-20T21:23:16.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.806+0000 7f67ad600700 1 --2- 172.21.15.22:0/703976871 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f6798039600 0x7f679803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:16.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.806+0000 7f67ad600700 1 -- 172.21.15.22:0/703976871 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f67a404df00 con 0x7f67b007de20 2024-04-20T21:23:16.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.806+0000 7f67afe00700 1 --2- 172.21.15.22:0/703976871 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f6798039600 0x7f679803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:16.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.807+0000 7f67afe00700 1 --2- 172.21.15.22:0/703976871 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f6798039600 0x7f679803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f679c006fd0 tx=0x7f679c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:15.811+0000 7f67ad600700 1 -- 172.21.15.22:0/703976871 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f67a404d850 con 0x7f67b007de20 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:16.043+0000 7f67b7c00700 1 -- 172.21.15.22:0/703976871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f67b01a1ba0 con 0x7f67b007de20 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:16.048+0000 7f67ad600700 1 -- 172.21.15.22:0/703976871 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f67a402ad60 con 0x7f67b007de20 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:16.052+0000 7f67b7c00700 1 -- 172.21.15.22:0/703976871 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f6798039600 msgr2=0x7f679803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:16.052+0000 7f67b7c00700 1 --2- 172.21.15.22:0/703976871 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f6798039600 0x7f679803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f679c006fd0 tx=0x7f679c006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:16.052+0000 7f67b7c00700 1 -- 172.21.15.22:0/703976871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 msgr2=0x7f67b019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:16.052+0000 7f67b7c00700 1 --2- 172.21.15.22:0/703976871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 0x7f67b019e6e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67a40117e0 tx=0x7f67a4011ce0 comp rx=0 tx=0).stop 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:16.052+0000 7f67b7c00700 1 -- 172.21.15.22:0/703976871 shutdown_connections 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:16.052+0000 7f67b7c00700 1 --2- 172.21.15.22:0/703976871 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f6798039600 0x7f679803bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:16.052+0000 7f67b7c00700 1 --2- 172.21.15.22:0/703976871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b007de20 0x7f67b019e6e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:16.052+0000 7f67b7c00700 1 -- 172.21.15.22:0/703976871 >> 172.21.15.22:0/703976871 conn(0x7f67b0103b80 msgr2=0x7f67b0104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:16.052+0000 7f67b7c00700 1 -- 172.21.15.22:0/703976871 shutdown_connections 2024-04-20T21:23:16.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:16.052+0000 7f67b7c00700 1 -- 172.21.15.22:0/703976871 wait complete. 2024-04-20T21:23:17.060 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:16 smithi022 ceph-mon[16843]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:17.060 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:16 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:23:17.060 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:16 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/703976871' entity='client.admin' 2024-04-20T21:23:18.494 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.375+0000 7f3a6fc00700 1 Processor -- start 2024-04-20T21:23:18.494 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.376+0000 7f3a6fc00700 1 -- start start 2024-04-20T21:23:18.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.376+0000 7f3a6fc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 0x7f3a6807c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.377+0000 7f3a6fc00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6807c900 con 0x7f3a6807de80 2024-04-20T21:23:18.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.377+0000 7f3a6d200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 0x7f3a6807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.377+0000 7f3a6d200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 0x7f3a6807c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40894/0 (socket says 172.21.15.22:40894) 2024-04-20T21:23:18.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.377+0000 7f3a6d200700 1 -- 172.21.15.22:0/1916319564 learned_addr learned my addr 172.21.15.22:0/1916319564 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:18.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.378+0000 7f3a6d200700 1 -- 172.21.15.22:0/1916319564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6807ca80 con 0x7f3a6807de80 2024-04-20T21:23:18.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.378+0000 7f3a6d200700 1 --2- 172.21.15.22:0/1916319564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 0x7f3a6807c330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0098a0 tx=0x7f3a5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d379c774755ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.378+0000 7f3a67400700 1 -- 172.21.15.22:0/1916319564 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a5c004160 con 0x7f3a6807de80 2024-04-20T21:23:18.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.378+0000 7f3a67400700 1 -- 172.21.15.22:0/1916319564 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a5c004300 con 0x7f3a6807de80 2024-04-20T21:23:18.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.378+0000 7f3a67400700 1 -- 172.21.15.22:0/1916319564 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a5c011b20 con 0x7f3a6807de80 2024-04-20T21:23:18.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.379+0000 7f3a6fc00700 1 -- 172.21.15.22:0/1916319564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 msgr2=0x7f3a6807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.379+0000 7f3a6fc00700 1 --2- 172.21.15.22:0/1916319564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 0x7f3a6807c330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0098a0 tx=0x7f3a5c009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.379+0000 7f3a6fc00700 1 -- 172.21.15.22:0/1916319564 shutdown_connections 2024-04-20T21:23:18.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.379+0000 7f3a6fc00700 1 --2- 172.21.15.22:0/1916319564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 0x7f3a6807c330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.379+0000 7f3a6fc00700 1 -- 172.21.15.22:0/1916319564 >> 172.21.15.22:0/1916319564 conn(0x7f3a68103c10 msgr2=0x7f3a68106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:18.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.380+0000 7f3a6fc00700 1 -- 172.21.15.22:0/1916319564 shutdown_connections 2024-04-20T21:23:18.499 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.380+0000 7f3a6fc00700 1 -- 172.21.15.22:0/1916319564 wait complete. 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.381+0000 7f3a6fc00700 1 Processor -- start 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.381+0000 7f3a6fc00700 1 -- start start 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.381+0000 7f3a6fc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 0x7f3a681a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.381+0000 7f3a6fc00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6810a800 con 0x7f3a6807de80 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.382+0000 7f3a6d200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 0x7f3a681a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.382+0000 7f3a6d200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 0x7f3a681a2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40908/0 (socket says 172.21.15.22:40908) 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.382+0000 7f3a6d200700 1 -- 172.21.15.22:0/2305656464 learned_addr learned my addr 172.21.15.22:0/2305656464 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.382+0000 7f3a6d200700 1 -- 172.21.15.22:0/2305656464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a681a2f80 con 0x7f3a6807de80 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.383+0000 7f3a6d200700 1 --2- 172.21.15.22:0/2305656464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 0x7f3a681a2a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0117e0 tx=0x7f3a5c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.383+0000 7f3a65600700 1 -- 172.21.15.22:0/2305656464 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a5c003ba0 con 0x7f3a6807de80 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.383+0000 7f3a65600700 1 -- 172.21.15.22:0/2305656464 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a5c01a910 con 0x7f3a6807de80 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.383+0000 7f3a65600700 1 -- 172.21.15.22:0/2305656464 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a5c0248f0 con 0x7f3a6807de80 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.383+0000 7f3a6fc00700 1 -- 172.21.15.22:0/2305656464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a681a3210 con 0x7f3a6807de80 2024-04-20T21:23:18.500 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.383+0000 7f3a6fc00700 1 -- 172.21.15.22:0/2305656464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a681a3670 con 0x7f3a6807de80 2024-04-20T21:23:18.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.384+0000 7f3a6fc00700 1 -- 172.21.15.22:0/2305656464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a680640b0 con 0x7f3a6807de80 2024-04-20T21:23:18.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.385+0000 7f3a65600700 1 -- 172.21.15.22:0/2305656464 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3a5c022020 con 0x7f3a6807de80 2024-04-20T21:23:18.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.386+0000 7f3a65600700 1 --2- 172.21.15.22:0/2305656464 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f3a50039600 0x7f3a5003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.386+0000 7f3a65600700 1 -- 172.21.15.22:0/2305656464 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3a5c04dae0 con 0x7f3a6807de80 2024-04-20T21:23:18.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.386+0000 7f3a67e00700 1 --2- 172.21.15.22:0/2305656464 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f3a50039600 0x7f3a5003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.387+0000 7f3a67e00700 1 --2- 172.21.15.22:0/2305656464 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f3a50039600 0x7f3a5003bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a54006fd0 tx=0x7f3a54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.391+0000 7f3a65600700 1 -- 172.21.15.22:0/2305656464 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3a5c01ab70 con 0x7f3a6807de80 2024-04-20T21:23:18.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.626+0000 7f3a6fc00700 1 -- 172.21.15.22:0/2305656464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f3a68063660 con 0x7f3a6807de80 2024-04-20T21:23:18.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.632+0000 7f3a65600700 1 -- 172.21.15.22:0/2305656464 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f3a5c046e20 con 0x7f3a6807de80 2024-04-20T21:23:18.502 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.635+0000 7f3a6fc00700 1 -- 172.21.15.22:0/2305656464 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f3a50039600 msgr2=0x7f3a5003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.502 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.635+0000 7f3a6fc00700 1 --2- 172.21.15.22:0/2305656464 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f3a50039600 0x7f3a5003bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a54006fd0 tx=0x7f3a54006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:18.502 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.635+0000 7f3a6fc00700 1 -- 172.21.15.22:0/2305656464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 msgr2=0x7f3a681a2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.503 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.635+0000 7f3a6fc00700 1 --2- 172.21.15.22:0/2305656464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 0x7f3a681a2a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0117e0 tx=0x7f3a5c011ba0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.503 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.635+0000 7f3a6fc00700 1 -- 172.21.15.22:0/2305656464 shutdown_connections 2024-04-20T21:23:18.503 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.635+0000 7f3a6fc00700 1 --2- 172.21.15.22:0/2305656464 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f3a50039600 0x7f3a5003bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.503 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.635+0000 7f3a6fc00700 1 --2- 172.21.15.22:0/2305656464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a6807de80 0x7f3a681a2a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.503 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.635+0000 7f3a6fc00700 1 -- 172.21.15.22:0/2305656464 >> 172.21.15.22:0/2305656464 conn(0x7f3a68103c10 msgr2=0x7f3a681998e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:18.503 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.635+0000 7f3a6fc00700 1 -- 172.21.15.22:0/2305656464 shutdown_connections 2024-04-20T21:23:18.503 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:17.636+0000 7f3a6fc00700 1 -- 172.21.15.22:0/2305656464 wait complete. 2024-04-20T21:23:18.503 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling the dashboard module... 2024-04-20T21:23:18.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:18 smithi022 ceph-mon[16843]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:18.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:18 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2305656464' entity='client.admin' 2024-04-20T21:23:20.296 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:20 smithi022 ceph-mon[16843]: from='mgr.14120 172.21.15.22:0/360531129' entity='mgr.x' 2024-04-20T21:23:20.296 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:20 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/437123382' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-20T21:23:20.597 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:20 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: ignoring --setuser ceph since I am not root 2024-04-20T21:23:20.597 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:20 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: ignoring --setgroup ceph since I am not root 2024-04-20T21:23:20.597 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:20 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:20.597+0000 7f079538a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:23:20.858 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:20 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:20.857+0000 7f079538a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:23:20.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.442+0000 7f7e8b400700 1 Processor -- start 2024-04-20T21:23:20.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.442+0000 7f7e8b400700 1 -- start start 2024-04-20T21:23:20.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.442+0000 7f7e8b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 0x7f7e8410abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.442+0000 7f7e8b400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8410b1a0 con 0x7f7e8410a7b0 2024-04-20T21:23:20.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.443+0000 7f7e88a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 0x7f7e8410abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.443+0000 7f7e88a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 0x7f7e8410abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40924/0 (socket says 172.21.15.22:40924) 2024-04-20T21:23:20.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.443+0000 7f7e88a00700 1 -- 172.21.15.22:0/3455806394 learned_addr learned my addr 172.21.15.22:0/3455806394 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.443+0000 7f7e88a00700 1 -- 172.21.15.22:0/3455806394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8410b960 con 0x7f7e8410a7b0 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.444+0000 7f7e88a00700 1 --2- 172.21.15.22:0/3455806394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 0x7f7e8410abd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e780098a0 tx=0x7f7e78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef620435da80a14b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.444+0000 7f7e83400700 1 -- 172.21.15.22:0/3455806394 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e78004160 con 0x7f7e8410a7b0 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.444+0000 7f7e83400700 1 -- 172.21.15.22:0/3455806394 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e78004300 con 0x7f7e8410a7b0 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.444+0000 7f7e83400700 1 -- 172.21.15.22:0/3455806394 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e78011b20 con 0x7f7e8410a7b0 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.445+0000 7f7e8b400700 1 -- 172.21.15.22:0/3455806394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 msgr2=0x7f7e8410abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.445+0000 7f7e8b400700 1 --2- 172.21.15.22:0/3455806394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 0x7f7e8410abd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e780098a0 tx=0x7f7e78009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.445+0000 7f7e8b400700 1 -- 172.21.15.22:0/3455806394 shutdown_connections 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.445+0000 7f7e8b400700 1 --2- 172.21.15.22:0/3455806394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 0x7f7e8410abd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.445+0000 7f7e8b400700 1 -- 172.21.15.22:0/3455806394 >> 172.21.15.22:0/3455806394 conn(0x7f7e8407dd70 msgr2=0x7f7e841084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.445+0000 7f7e8b400700 1 -- 172.21.15.22:0/3455806394 shutdown_connections 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.445+0000 7f7e8b400700 1 -- 172.21.15.22:0/3455806394 wait complete. 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.446+0000 7f7e8b400700 1 Processor -- start 2024-04-20T21:23:20.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.447+0000 7f7e8b400700 1 -- start start 2024-04-20T21:23:20.990 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.447+0000 7f7e8b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 0x7f7e8419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.990 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.447+0000 7f7e8b400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8410ed10 con 0x7f7e8410a7b0 2024-04-20T21:23:20.990 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.447+0000 7f7e88a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 0x7f7e8419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.990 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.447+0000 7f7e88a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 0x7f7e8419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40936/0 (socket says 172.21.15.22:40936) 2024-04-20T21:23:20.990 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.447+0000 7f7e88a00700 1 -- 172.21.15.22:0/437123382 learned_addr learned my addr 172.21.15.22:0/437123382 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:20.990 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.448+0000 7f7e88a00700 1 -- 172.21.15.22:0/437123382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8419ec70 con 0x7f7e8410a7b0 2024-04-20T21:23:20.990 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.448+0000 7f7e88a00700 1 --2- 172.21.15.22:0/437123382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 0x7f7e8419e730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7e78000c00 tx=0x7f7e78011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:20.990 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.448+0000 7f7e81600700 1 -- 172.21.15.22:0/437123382 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e78003b60 con 0x7f7e8410a7b0 2024-04-20T21:23:20.990 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.448+0000 7f7e81600700 1 -- 172.21.15.22:0/437123382 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e7801a910 con 0x7f7e8410a7b0 2024-04-20T21:23:20.990 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.448+0000 7f7e81600700 1 -- 172.21.15.22:0/437123382 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e78024930 con 0x7f7e8410a7b0 2024-04-20T21:23:20.990 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.448+0000 7f7e8b400700 1 -- 172.21.15.22:0/437123382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8419ef00 con 0x7f7e8410a7b0 2024-04-20T21:23:20.990 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.449+0000 7f7e8b400700 1 -- 172.21.15.22:0/437123382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8419f360 con 0x7f7e8410a7b0 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.450+0000 7f7e8b400700 1 -- 172.21.15.22:0/437123382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e840640b0 con 0x7f7e8410a7b0 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.450+0000 7f7e81600700 1 -- 172.21.15.22:0/437123382 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7e78022020 con 0x7f7e8410a7b0 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.450+0000 7f7e81600700 1 --2- 172.21.15.22:0/437123382 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f7e6c039600 0x7f7e6c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.450+0000 7f7e81600700 1 -- 172.21.15.22:0/437123382 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7e7804d960 con 0x7f7e8410a7b0 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.451+0000 7f7e83e00700 1 --2- 172.21.15.22:0/437123382 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f7e6c039600 0x7f7e6c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.451+0000 7f7e83e00700 1 --2- 172.21.15.22:0/437123382 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f7e6c039600 0x7f7e6c03bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7e70006fd0 tx=0x7f7e70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.456+0000 7f7e81600700 1 -- 172.21.15.22:0/437123382 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7e7802ddb0 con 0x7f7e8410a7b0 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:19.751+0000 7f7e8b400700 1 -- 172.21.15.22:0/437123382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f7e841a1df0 con 0x7f7e8410a7b0 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:20.266+0000 7f7e81600700 1 -- 172.21.15.22:0/437123382 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f7e7800b3e0 con 0x7f7e8410a7b0 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:20.271+0000 7f7e81600700 1 -- 172.21.15.22:0/437123382 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f7e78024ad0 con 0x7f7e8410a7b0 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:20.272+0000 7f7e8b400700 1 -- 172.21.15.22:0/437123382 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f7e6c039600 msgr2=0x7f7e6c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:20.272+0000 7f7e8b400700 1 --2- 172.21.15.22:0/437123382 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f7e6c039600 0x7f7e6c03bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7e70006fd0 tx=0x7f7e70006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:20.272+0000 7f7e8b400700 1 -- 172.21.15.22:0/437123382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 msgr2=0x7f7e8419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:20.272+0000 7f7e8b400700 1 --2- 172.21.15.22:0/437123382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 0x7f7e8419e730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7e78000c00 tx=0x7f7e78011c10 comp rx=0 tx=0).stop 2024-04-20T21:23:20.992 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:20.272+0000 7f7e8b400700 1 -- 172.21.15.22:0/437123382 shutdown_connections 2024-04-20T21:23:20.992 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:20.272+0000 7f7e8b400700 1 --2- 172.21.15.22:0/437123382 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7f7e6c039600 0x7f7e6c03bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.992 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:20.272+0000 7f7e8b400700 1 --2- 172.21.15.22:0/437123382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e8410a7b0 0x7f7e8419e730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.992 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:20.272+0000 7f7e8b400700 1 -- 172.21.15.22:0/437123382 >> 172.21.15.22:0/437123382 conn(0x7f7e8407dd70 msgr2=0x7f7e84107bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:20.992 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:20.272+0000 7f7e8b400700 1 -- 172.21.15.22:0/437123382 shutdown_connections 2024-04-20T21:23:20.992 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:20.272+0000 7f7e8b400700 1 -- 172.21.15.22:0/437123382 wait complete. 2024-04-20T21:23:21.137 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:20 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:20.993+0000 7f079538a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:23:21.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:21 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/437123382' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-20T21:23:21.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:21 smithi022 ceph-mon[16843]: mgrmap e9: x(active, since 24s) 2024-04-20T21:23:21.657 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:21 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:21.405+0000 7f079538a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:23:21.657 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:21 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:21.524+0000 7f079538a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:23:22.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:22 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1526979577' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-20T21:23:22.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:23:22.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-20T21:23:22.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T21:23:22.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-20T21:23:22.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-20T21:23:22.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:23:22.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.572+0000 7fa9c3000700 1 Processor -- start 2024-04-20T21:23:22.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.572+0000 7fa9c3000700 1 -- start start 2024-04-20T21:23:22.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.573+0000 7fa9c3000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 0x7fa9bc07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.573+0000 7fa9c3000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc07c810 con 0x7fa9bc07de20 2024-04-20T21:23:22.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.573+0000 7fa9bbe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 0x7fa9bc07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.573+0000 7fa9bbe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 0x7fa9bc07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40968/0 (socket says 172.21.15.22:40968) 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.574+0000 7fa9bbe00700 1 -- 172.21.15.22:0/2575370916 learned_addr learned my addr 172.21.15.22:0/2575370916 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.574+0000 7fa9bbe00700 1 -- 172.21.15.22:0/2575370916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc07c990 con 0x7fa9bc07de20 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.574+0000 7fa9bbe00700 1 --2- 172.21.15.22:0/2575370916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 0x7fa9bc07c2d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0098a0 tx=0x7fa9ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968d3010c46afde3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.575+0000 7fa9baa00700 1 -- 172.21.15.22:0/2575370916 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ac004160 con 0x7fa9bc07de20 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.575+0000 7fa9baa00700 1 -- 172.21.15.22:0/2575370916 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac004300 con 0x7fa9bc07de20 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.575+0000 7fa9baa00700 1 -- 172.21.15.22:0/2575370916 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ac011b20 con 0x7fa9bc07de20 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.575+0000 7fa9c3000700 1 -- 172.21.15.22:0/2575370916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 msgr2=0x7fa9bc07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.575+0000 7fa9c3000700 1 --2- 172.21.15.22:0/2575370916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 0x7fa9bc07c2d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0098a0 tx=0x7fa9ac009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.576+0000 7fa9c3000700 1 -- 172.21.15.22:0/2575370916 shutdown_connections 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.576+0000 7fa9c3000700 1 --2- 172.21.15.22:0/2575370916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 0x7fa9bc07c2d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.576+0000 7fa9c3000700 1 -- 172.21.15.22:0/2575370916 >> 172.21.15.22:0/2575370916 conn(0x7fa9bc103b80 msgr2=0x7fa9bc105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.576+0000 7fa9c3000700 1 -- 172.21.15.22:0/2575370916 shutdown_connections 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.576+0000 7fa9c3000700 1 -- 172.21.15.22:0/2575370916 wait complete. 2024-04-20T21:23:22.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.577+0000 7fa9c3000700 1 Processor -- start 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.577+0000 7fa9c3000700 1 -- start start 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.577+0000 7fa9c3000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 0x7fa9bc19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.577+0000 7fa9c3000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc10eb20 con 0x7fa9bc07de20 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.578+0000 7fa9bbe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 0x7fa9bc19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.578+0000 7fa9bbe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 0x7fa9bc19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40976/0 (socket says 172.21.15.22:40976) 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.578+0000 7fa9bbe00700 1 -- 172.21.15.22:0/1526979577 learned_addr learned my addr 172.21.15.22:0/1526979577 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.578+0000 7fa9bbe00700 1 -- 172.21.15.22:0/1526979577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc19ebc0 con 0x7fa9bc07de20 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.578+0000 7fa9bbe00700 1 --2- 172.21.15.22:0/1526979577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 0x7fa9bc19e680 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0117e0 tx=0x7fa9ac011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.579+0000 7fa9b8c00700 1 -- 172.21.15.22:0/1526979577 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ac003c90 con 0x7fa9bc07de20 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.579+0000 7fa9b8c00700 1 -- 172.21.15.22:0/1526979577 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac01a910 con 0x7fa9bc07de20 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.579+0000 7fa9c3000700 1 -- 172.21.15.22:0/1526979577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9bc19ee50 con 0x7fa9bc07de20 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.579+0000 7fa9b8c00700 1 -- 172.21.15.22:0/1526979577 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ac024a10 con 0x7fa9bc07de20 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.579+0000 7fa9c3000700 1 -- 172.21.15.22:0/1526979577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9bc19f2b0 con 0x7fa9bc07de20 2024-04-20T21:23:22.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.580+0000 7fa9b8c00700 1 -- 172.21.15.22:0/1526979577 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fa9ac022020 con 0x7fa9bc07de20 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.580+0000 7fa9c3000700 1 -- 172.21.15.22:0/1526979577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9bc0640b0 con 0x7fa9bc07de20 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.580+0000 7fa9b8c00700 1 --2- 172.21.15.22:0/1526979577 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fa9a0039610 0x7fa9a003bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.580+0000 7fa9b8c00700 1 -- 172.21.15.22:0/1526979577 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa9ac04db60 con 0x7fa9bc07de20 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.580+0000 7fa9bb400700 1 -- 172.21.15.22:0/1526979577 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fa9a0039610 msgr2=0x7fa9a003bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2098569388 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.581+0000 7fa9bb400700 1 --2- 172.21.15.22:0/1526979577 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fa9a0039610 0x7fa9a003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.586+0000 7fa9b8c00700 1 -- 172.21.15.22:0/1526979577 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa9ac011e20 con 0x7fa9bc07de20 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.782+0000 7fa9bb400700 1 -- 172.21.15.22:0/1526979577 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fa9a0039610 msgr2=0x7fa9a003bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2098569388 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.782+0000 7fa9bb400700 1 --2- 172.21.15.22:0/1526979577 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fa9a0039610 0x7fa9a003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.875+0000 7fa9c3000700 1 -- 172.21.15.22:0/1526979577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa9bc1a1d20 con 0x7fa9bc07de20 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.876+0000 7fa9b8c00700 1 -- 172.21.15.22:0/1526979577 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fa9ac0172b0 con 0x7fa9bc07de20 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.879+0000 7fa9c3000700 1 -- 172.21.15.22:0/1526979577 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fa9a0039610 msgr2=0x7fa9a003bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.879+0000 7fa9c3000700 1 --2- 172.21.15.22:0/1526979577 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fa9a0039610 0x7fa9a003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.879+0000 7fa9c3000700 1 -- 172.21.15.22:0/1526979577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 msgr2=0x7fa9bc19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.879+0000 7fa9c3000700 1 --2- 172.21.15.22:0/1526979577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 0x7fa9bc19e680 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0117e0 tx=0x7fa9ac011ce0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.879+0000 7fa9c3000700 1 -- 172.21.15.22:0/1526979577 shutdown_connections 2024-04-20T21:23:22.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.879+0000 7fa9c3000700 1 --2- 172.21.15.22:0/1526979577 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fa9a0039610 0x7fa9a003bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.879+0000 7fa9c3000700 1 --2- 172.21.15.22:0/1526979577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9bc07de20 0x7fa9bc19e680 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.879+0000 7fa9c3000700 1 -- 172.21.15.22:0/1526979577 >> 172.21.15.22:0/1526979577 conn(0x7fa9bc103b80 msgr2=0x7fa9bc1046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:22.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.879+0000 7fa9c3000700 1 -- 172.21.15.22:0/1526979577 shutdown_connections 2024-04-20T21:23:22.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:21.880+0000 7fa9c3000700 1 -- 172.21.15.22:0/1526979577 wait complete. 2024-04-20T21:23:22.539 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-04-20T21:23:22.539 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 9... 2024-04-20T21:23:23.394 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:23 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:23.158+0000 7f079538a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:23:23.656 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:23 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:23.394+0000 7f079538a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:23:23.656 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:23 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:23.614+0000 7f079538a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:23:24.396 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:24 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:24.006+0000 7f079538a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:23:24.397 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:24 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:24.132+0000 7f079538a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:23:24.657 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:24 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:24.396+0000 7f079538a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:23:25.156 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:24 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:24.659+0000 7f079538a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:23:25.906 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:25 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:25.466+0000 7f079538a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:23:26.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:26 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:26.040+0000 7f079538a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:23:26.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:26 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:26.171+0000 7f079538a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:23:26.906 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:26 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:26.445+0000 7f079538a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:23:27.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:27 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:27.115+0000 7f079538a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:23:27.407 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:27 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:27.244+0000 7f079538a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:23:27.890 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:27 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:27.575+0000 7f079538a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:23:27.890 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:27 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:27.758+0000 7f079538a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:23:27.890 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:27 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:27.890+0000 7f079538a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:23:28.156 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:28 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:28.046+0000 7f079538a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:23:28.906 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:28 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:28.421+0000 7f079538a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:23:29.506 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:29 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:29.209+0000 7f079538a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:23:29.906 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:23:29 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:23:29.506+0000 7f079538a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:23:29.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:29 smithi022 ceph-mon[16843]: Active manager daemon x restarted 2024-04-20T21:23:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:29 smithi022 ceph-mon[16843]: Activating manager daemon x 2024-04-20T21:23:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:29 smithi022 ceph-mon[16843]: osdmap e3: 0 total, 0 up, 0 in 2024-04-20T21:23:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:29 smithi022 ceph-mon[16843]: mgrmap e10: x(active, starting, since 0.00477185s) 2024-04-20T21:23:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:23:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T21:23:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T21:23:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T21:23:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T21:23:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:29 smithi022 ceph-mon[16843]: Manager daemon x is now available 2024-04-20T21:23:30.810 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:30.810 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:23:30.810 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:23:30.810 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:30 smithi022 ceph-mon[16843]: mgrmap e11: x(active, since 1.00959s) 2024-04-20T21:23:31.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:23:31.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.121+0000 7fed2cc00700 1 Processor -- start 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.121+0000 7fed2cc00700 1 -- start start 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.122+0000 7fed2cc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 0x7fed2810b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.122+0000 7fed2cc00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed2807be20 con 0x7fed28108ec0 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.122+0000 7fed27400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 0x7fed2810b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.122+0000 7fed27400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 0x7fed2810b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35882/0 (socket says 172.21.15.22:35882) 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.122+0000 7fed27400700 1 -- 172.21.15.22:0/3068092358 learned_addr learned my addr 172.21.15.22:0/3068092358 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.123+0000 7fed27400700 1 -- 172.21.15.22:0/3068092358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2810b7f0 con 0x7fed28108ec0 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.123+0000 7fed27400700 1 --2- 172.21.15.22:0/3068092358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 0x7fed2810b2b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fed180098a0 tx=0x7fed18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4868b1a9812e685e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.123+0000 7fed26000700 1 -- 172.21.15.22:0/3068092358 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed18004160 con 0x7fed28108ec0 2024-04-20T21:23:31.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.123+0000 7fed26000700 1 -- 172.21.15.22:0/3068092358 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed18004300 con 0x7fed28108ec0 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.123+0000 7fed26000700 1 -- 172.21.15.22:0/3068092358 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed18011b20 con 0x7fed28108ec0 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.124+0000 7fed2cc00700 1 -- 172.21.15.22:0/3068092358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 msgr2=0x7fed2810b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.124+0000 7fed2cc00700 1 --2- 172.21.15.22:0/3068092358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 0x7fed2810b2b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fed180098a0 tx=0x7fed18009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.124+0000 7fed2cc00700 1 -- 172.21.15.22:0/3068092358 shutdown_connections 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.124+0000 7fed2cc00700 1 --2- 172.21.15.22:0/3068092358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 0x7fed2810b2b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.124+0000 7fed2cc00700 1 -- 172.21.15.22:0/3068092358 >> 172.21.15.22:0/3068092358 conn(0x7fed281028b0 msgr2=0x7fed28104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.124+0000 7fed2cc00700 1 -- 172.21.15.22:0/3068092358 shutdown_connections 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.124+0000 7fed2cc00700 1 -- 172.21.15.22:0/3068092358 wait complete. 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.125+0000 7fed2cc00700 1 Processor -- start 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.126+0000 7fed2cc00700 1 -- start start 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.126+0000 7fed2cc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 0x7fed2819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.126+0000 7fed2cc00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed2810c7f0 con 0x7fed28108ec0 2024-04-20T21:23:31.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.126+0000 7fed27400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 0x7fed2819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.126+0000 7fed27400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 0x7fed2819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35894/0 (socket says 172.21.15.22:35894) 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.126+0000 7fed27400700 1 -- 172.21.15.22:0/3438173030 learned_addr learned my addr 172.21.15.22:0/3438173030 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.126+0000 7fed27400700 1 -- 172.21.15.22:0/3438173030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2819eaf0 con 0x7fed28108ec0 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.127+0000 7fed27400700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 0x7fed2819e5b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed180116a0 tx=0x7fed18011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.127+0000 7fed1fe00700 1 -- 172.21.15.22:0/3438173030 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed18003a40 con 0x7fed28108ec0 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.127+0000 7fed1fe00700 1 -- 172.21.15.22:0/3438173030 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed18003be0 con 0x7fed28108ec0 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.127+0000 7fed2cc00700 1 -- 172.21.15.22:0/3438173030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2819ed80 con 0x7fed28108ec0 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.127+0000 7fed1fe00700 1 -- 172.21.15.22:0/3438173030 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed1801a750 con 0x7fed28108ec0 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.127+0000 7fed2cc00700 1 -- 172.21.15.22:0/3438173030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2819f1e0 con 0x7fed28108ec0 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.128+0000 7fed1fe00700 1 -- 172.21.15.22:0/3438173030 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fed18024aa0 con 0x7fed28108ec0 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.129+0000 7fed1fe00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 0x7fed0c03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.129+0000 7fed1fe00700 1 -- 172.21.15.22:0/3438173030 --> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fed0c03c1e0 con 0x7fed0c039610 2024-04-20T21:23:31.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.129+0000 7fed1fe00700 1 -- 172.21.15.22:0/3438173030 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fed1804dd40 con 0x7fed28108ec0 2024-04-20T21:23:31.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.129+0000 7fed26a00700 1 -- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 msgr2=0x7fed0c03bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2098569388 2024-04-20T21:23:31.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.129+0000 7fed26a00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 0x7fed0c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:23:31.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.330+0000 7fed26a00700 1 -- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 msgr2=0x7fed0c03bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2098569388 2024-04-20T21:23:31.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.330+0000 7fed26a00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 0x7fed0c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:23:31.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.731+0000 7fed26a00700 1 -- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 msgr2=0x7fed0c03bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2098569388 2024-04-20T21:23:31.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:23.731+0000 7fed26a00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 0x7fed0c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:23:31.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:24.532+0000 7fed26a00700 1 -- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 msgr2=0x7fed0c03bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2098569388 2024-04-20T21:23:31.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:24.532+0000 7fed26a00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 0x7fed0c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:23:31.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:26.134+0000 7fed26a00700 1 -- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 msgr2=0x7fed0c03bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2098569388 2024-04-20T21:23:31.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:26.134+0000 7fed26a00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 0x7fed0c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-20T21:23:31.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:29.338+0000 7fed26a00700 1 -- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 msgr2=0x7fed0c03bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2098569388 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:29.338+0000 7fed26a00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 0x7fed0c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:29.513+0000 7fed1fe00700 1 -- 172.21.15.22:0/3438173030 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fed18024430 con 0x7fed28108ec0 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:29.514+0000 7fed1fe00700 1 -- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 msgr2=0x7fed0c03bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:29.514+0000 7fed1fe00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 0x7fed0c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.516+0000 7fed1fe00700 1 -- 172.21.15.22:0/3438173030 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fed1804fe10 con 0x7fed28108ec0 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.516+0000 7fed1fe00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fed0c03cff0 0x7fed0c03f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.516+0000 7fed1fe00700 1 -- 172.21.15.22:0/3438173030 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fed0c03c1e0 con 0x7fed0c03cff0 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.517+0000 7fed26a00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fed0c03cff0 0x7fed0c03f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.519+0000 7fed26a00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fed0c03cff0 0x7fed0c03f3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fed14003a10 tx=0x7fed14009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.521+0000 7fed1fe00700 1 -- 172.21.15.22:0/3438173030 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fed0c03c1e0 con 0x7fed0c03cff0 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.530+0000 7fed2cc00700 1 -- 172.21.15.22:0/3438173030 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fed280640b0 con 0x7fed0c03cff0 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.530+0000 7fed1fe00700 1 -- 172.21.15.22:0/3438173030 <== mgr.14152 v2:172.21.15.22:6800/1911146506 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fed280640b0 con 0x7fed0c03cff0 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.530+0000 7fed2cc00700 1 -- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fed0c03cff0 msgr2=0x7fed0c03f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.530+0000 7fed2cc00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fed0c03cff0 0x7fed0c03f3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fed14003a10 tx=0x7fed14009270 comp rx=0 tx=0).stop 2024-04-20T21:23:31.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.530+0000 7fed2cc00700 1 -- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 msgr2=0x7fed2819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.530+0000 7fed2cc00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 0x7fed2819e5b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed180116a0 tx=0x7fed18011ba0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.531+0000 7fed2cc00700 1 -- 172.21.15.22:0/3438173030 shutdown_connections 2024-04-20T21:23:31.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.531+0000 7fed2cc00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed28108ec0 0x7fed2819e5b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.531+0000 7fed2cc00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fed0c03cff0 0x7fed0c03f3e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.531+0000 7fed2cc00700 1 --2- 172.21.15.22:0/3438173030 >> [v2:172.21.15.22:6800/2098569388,v1:172.21.15.22:6801/2098569388] conn(0x7fed0c039610 0x7fed0c03bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.531+0000 7fed2cc00700 1 -- 172.21.15.22:0/3438173030 >> 172.21.15.22:0/3438173030 conn(0x7fed281028b0 msgr2=0x7fed28103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.531+0000 7fed2cc00700 1 -- 172.21.15.22:0/3438173030 shutdown_connections 2024-04-20T21:23:31.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:30.531+0000 7fed2cc00700 1 -- 172.21.15.22:0/3438173030 wait complete. 2024-04-20T21:23:31.242 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 9 is available 2024-04-20T21:23:31.242 INFO:teuthology.orchestra.run.smithi022.stdout:Generating a dashboard self-signed certificate... 2024-04-20T21:23:31.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:31 smithi022 ceph-mon[16843]: [20/Apr/2024:21:23:30] ENGINE Bus STARTING 2024-04-20T21:23:31.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:31 smithi022 ceph-mon[16843]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-20T21:23:31.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:31 smithi022 ceph-mon[16843]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-20T21:23:31.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:31 smithi022 ceph-mon[16843]: [20/Apr/2024:21:23:30] ENGINE Serving on http://172.21.15.22:8765 2024-04-20T21:23:31.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:31 smithi022 ceph-mon[16843]: [20/Apr/2024:21:23:30] ENGINE Serving on https://172.21.15.22:7150 2024-04-20T21:23:31.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:31 smithi022 ceph-mon[16843]: [20/Apr/2024:21:23:30] ENGINE Bus STARTED 2024-04-20T21:23:32.882 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:32 smithi022 ceph-mon[16843]: mgrmap e12: x(active, since 2s) 2024-04-20T21:23:32.882 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:32 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:33.366 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-20T21:23:33.367 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.290+0000 7fdbe2a00700 1 Processor -- start 2024-04-20T21:23:33.367 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.290+0000 7fdbe2a00700 1 -- start start 2024-04-20T21:23:33.367 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.290+0000 7fdbe2a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 0x7fdbdc07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:33.367 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.290+0000 7fdbe2a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc07c810 con 0x7fdbdc07de20 2024-04-20T21:23:33.367 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.291+0000 7fdbdbe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 0x7fdbdc07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:33.367 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.291+0000 7fdbdbe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 0x7fdbdc07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35950/0 (socket says 172.21.15.22:35950) 2024-04-20T21:23:33.367 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.291+0000 7fdbdbe00700 1 -- 172.21.15.22:0/2849364528 learned_addr learned my addr 172.21.15.22:0/2849364528 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:33.367 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.291+0000 7fdbdbe00700 1 -- 172.21.15.22:0/2849364528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc07c990 con 0x7fdbdc07de20 2024-04-20T21:23:33.367 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.292+0000 7fdbdbe00700 1 --2- 172.21.15.22:0/2849364528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 0x7fdbdc07c2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0098a0 tx=0x7fdbcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5d1597c0f97d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:33.367 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.292+0000 7fdbdaa00700 1 -- 172.21.15.22:0/2849364528 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbcc004160 con 0x7fdbdc07de20 2024-04-20T21:23:33.367 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.292+0000 7fdbdaa00700 1 -- 172.21.15.22:0/2849364528 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbcc004300 con 0x7fdbdc07de20 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.292+0000 7fdbdaa00700 1 -- 172.21.15.22:0/2849364528 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbcc011b20 con 0x7fdbdc07de20 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.293+0000 7fdbe2a00700 1 -- 172.21.15.22:0/2849364528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 msgr2=0x7fdbdc07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.293+0000 7fdbe2a00700 1 --2- 172.21.15.22:0/2849364528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 0x7fdbdc07c2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0098a0 tx=0x7fdbcc009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.293+0000 7fdbe2a00700 1 -- 172.21.15.22:0/2849364528 shutdown_connections 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.293+0000 7fdbe2a00700 1 --2- 172.21.15.22:0/2849364528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 0x7fdbdc07c2d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.293+0000 7fdbe2a00700 1 -- 172.21.15.22:0/2849364528 >> 172.21.15.22:0/2849364528 conn(0x7fdbdc103b80 msgr2=0x7fdbdc105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.294+0000 7fdbe2a00700 1 -- 172.21.15.22:0/2849364528 shutdown_connections 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.294+0000 7fdbe2a00700 1 -- 172.21.15.22:0/2849364528 wait complete. 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.295+0000 7fdbe2a00700 1 Processor -- start 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.295+0000 7fdbe2a00700 1 -- start start 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.295+0000 7fdbe2a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 0x7fdbdc19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.295+0000 7fdbe2a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc10eb20 con 0x7fdbdc07de20 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.296+0000 7fdbdbe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 0x7fdbdc19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.296+0000 7fdbdbe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 0x7fdbdc19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35960/0 (socket says 172.21.15.22:35960) 2024-04-20T21:23:33.368 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.296+0000 7fdbdbe00700 1 -- 172.21.15.22:0/1448446698 learned_addr learned my addr 172.21.15.22:0/1448446698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.296+0000 7fdbdbe00700 1 -- 172.21.15.22:0/1448446698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc19ebc0 con 0x7fdbdc07de20 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.296+0000 7fdbdbe00700 1 --2- 172.21.15.22:0/1448446698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 0x7fdbdc19e680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0117e0 tx=0x7fdbcc011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.297+0000 7fdbd8c00700 1 -- 172.21.15.22:0/1448446698 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbcc01a430 con 0x7fdbdc07de20 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.297+0000 7fdbd8c00700 1 -- 172.21.15.22:0/1448446698 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbcc01aab0 con 0x7fdbdc07de20 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.297+0000 7fdbe2a00700 1 -- 172.21.15.22:0/1448446698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc19ee50 con 0x7fdbdc07de20 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.297+0000 7fdbd8c00700 1 -- 172.21.15.22:0/1448446698 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbcc024a60 con 0x7fdbdc07de20 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.297+0000 7fdbe2a00700 1 -- 172.21.15.22:0/1448446698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc19f2b0 con 0x7fdbdc07de20 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.299+0000 7fdbd2000700 1 -- 172.21.15.22:0/1448446698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb4005330 con 0x7fdbdc07de20 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.300+0000 7fdbd8c00700 1 -- 172.21.15.22:0/1448446698 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fdbcc022020 con 0x7fdbdc07de20 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.301+0000 7fdbd8c00700 1 --2- 172.21.15.22:0/1448446698 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdbc00396b0 0x7fdbc003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.301+0000 7fdbd8c00700 1 -- 172.21.15.22:0/1448446698 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdbcc04e8e0 con 0x7fdbdc07de20 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.301+0000 7fdbdb400700 1 --2- 172.21.15.22:0/1448446698 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdbc00396b0 0x7fdbc003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.302+0000 7fdbdb400700 1 --2- 172.21.15.22:0/1448446698 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdbc00396b0 0x7fdbc003bb70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8006fd0 tx=0x7fdbc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:33.369 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.306+0000 7fdbd8c00700 1 -- 172.21.15.22:0/1448446698 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdbcc02c3a0 con 0x7fdbdc07de20 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.551+0000 7fdbd2000700 1 -- 172.21.15.22:0/1448446698 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fdbb4000bc0 con 0x7fdbc00396b0 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.697+0000 7fdbd8c00700 1 -- 172.21.15.22:0/1448446698 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fdbb4000bc0 con 0x7fdbc00396b0 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.700+0000 7fdbd2000700 1 -- 172.21.15.22:0/1448446698 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdbc00396b0 msgr2=0x7fdbc003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.700+0000 7fdbd2000700 1 --2- 172.21.15.22:0/1448446698 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdbc00396b0 0x7fdbc003bb70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8006fd0 tx=0x7fdbc8006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.700+0000 7fdbd2000700 1 -- 172.21.15.22:0/1448446698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 msgr2=0x7fdbdc19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.700+0000 7fdbd2000700 1 --2- 172.21.15.22:0/1448446698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 0x7fdbdc19e680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0117e0 tx=0x7fdbcc011ce0 comp rx=0 tx=0).stop 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.700+0000 7fdbd2000700 1 -- 172.21.15.22:0/1448446698 shutdown_connections 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.700+0000 7fdbd2000700 1 --2- 172.21.15.22:0/1448446698 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdbc00396b0 0x7fdbc003bb70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.700+0000 7fdbd2000700 1 --2- 172.21.15.22:0/1448446698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbdc07de20 0x7fdbdc19e680 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.700+0000 7fdbd2000700 1 -- 172.21.15.22:0/1448446698 >> 172.21.15.22:0/1448446698 conn(0x7fdbdc103b80 msgr2=0x7fdbdc1046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.700+0000 7fdbd2000700 1 -- 172.21.15.22:0/1448446698 shutdown_connections 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:32.701+0000 7fdbd2000700 1 -- 172.21.15.22:0/1448446698 wait complete. 2024-04-20T21:23:33.370 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial admin user... 2024-04-20T21:23:33.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:33 smithi022 ceph-mon[16843]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:33.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:33 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:33.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:33 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:35.263 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:35.263 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:35 smithi022 ceph-mon[16843]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:35.263 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:35.276 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$t7arBe5bZ48SJZ7lbSxyY.PP6qhtkB6jWFwHZsPfnetzwE17YZDV2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713648214, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-20T21:23:35.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.031+0000 7fa9ae400700 1 Processor -- start 2024-04-20T21:23:35.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.031+0000 7fa9ae400700 1 -- start start 2024-04-20T21:23:35.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.032+0000 7fa9ae400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 0x7fa9a810b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:35.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.032+0000 7fa9ae400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a807bea0 con 0x7fa9a8108e10 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.032+0000 7fa9ad000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 0x7fa9a810b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.033+0000 7fa9ad000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 0x7fa9a810b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52580/0 (socket says 172.21.15.22:52580) 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.033+0000 7fa9ad000700 1 -- 172.21.15.22:0/3081005800 learned_addr learned my addr 172.21.15.22:0/3081005800 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.033+0000 7fa9ad000700 1 -- 172.21.15.22:0/3081005800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a810b740 con 0x7fa9a8108e10 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.033+0000 7fa9ad000700 1 --2- 172.21.15.22:0/3081005800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 0x7fa9a810b200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0098a0 tx=0x7fa99c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30957fe28bdc1f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.034+0000 7fa9a7400700 1 -- 172.21.15.22:0/3081005800 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99c004160 con 0x7fa9a8108e10 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.034+0000 7fa9a7400700 1 -- 172.21.15.22:0/3081005800 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa99c004300 con 0x7fa9a8108e10 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.034+0000 7fa9a7400700 1 -- 172.21.15.22:0/3081005800 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99c011b20 con 0x7fa9a8108e10 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.035+0000 7fa9ae400700 1 -- 172.21.15.22:0/3081005800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 msgr2=0x7fa9a810b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.035+0000 7fa9ae400700 1 --2- 172.21.15.22:0/3081005800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 0x7fa9a810b200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0098a0 tx=0x7fa99c009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.035+0000 7fa9ae400700 1 -- 172.21.15.22:0/3081005800 shutdown_connections 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.035+0000 7fa9ae400700 1 --2- 172.21.15.22:0/3081005800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 0x7fa9a810b200 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.035+0000 7fa9ae400700 1 -- 172.21.15.22:0/3081005800 >> 172.21.15.22:0/3081005800 conn(0x7fa9a8102860 msgr2=0x7fa9a8104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.035+0000 7fa9ae400700 1 -- 172.21.15.22:0/3081005800 shutdown_connections 2024-04-20T21:23:35.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.035+0000 7fa9ae400700 1 -- 172.21.15.22:0/3081005800 wait complete. 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.036+0000 7fa9ae400700 1 Processor -- start 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.036+0000 7fa9ae400700 1 -- start start 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.036+0000 7fa9ae400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 0x7fa9a819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.037+0000 7fa9ae400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a810c7b0 con 0x7fa9a8108e10 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.037+0000 7fa9ad000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 0x7fa9a819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.037+0000 7fa9ad000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 0x7fa9a819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52582/0 (socket says 172.21.15.22:52582) 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.037+0000 7fa9ad000700 1 -- 172.21.15.22:0/3557157949 learned_addr learned my addr 172.21.15.22:0/3557157949 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.037+0000 7fa9ad000700 1 -- 172.21.15.22:0/3557157949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a819ea80 con 0x7fa9a8108e10 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.038+0000 7fa9ad000700 1 --2- 172.21.15.22:0/3557157949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 0x7fa9a819e540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa99c011b60 tx=0x7fa99c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.038+0000 7fa9a5600700 1 -- 172.21.15.22:0/3557157949 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99c003b60 con 0x7fa9a8108e10 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.038+0000 7fa9a5600700 1 -- 172.21.15.22:0/3557157949 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa99c01a910 con 0x7fa9a8108e10 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.038+0000 7fa9a5600700 1 -- 172.21.15.22:0/3557157949 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99c024930 con 0x7fa9a8108e10 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.038+0000 7fa9ae400700 1 -- 172.21.15.22:0/3557157949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a819ed10 con 0x7fa9a8108e10 2024-04-20T21:23:35.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.038+0000 7fa9ae400700 1 -- 172.21.15.22:0/3557157949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a819f170 con 0x7fa9a8108e10 2024-04-20T21:23:35.283 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.040+0000 7fa9a5600700 1 -- 172.21.15.22:0/3557157949 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fa99c022020 con 0x7fa9a8108e10 2024-04-20T21:23:35.283 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.040+0000 7fa9ae400700 1 -- 172.21.15.22:0/3557157949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a80640b0 con 0x7fa9a8108e10 2024-04-20T21:23:35.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.040+0000 7fa9a5600700 1 --2- 172.21.15.22:0/3557157949 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa990039600 0x7fa99003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:35.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.040+0000 7fa9a5600700 1 -- 172.21.15.22:0/3557157949 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa99c04daf0 con 0x7fa9a8108e10 2024-04-20T21:23:35.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.041+0000 7fa9a7e00700 1 --2- 172.21.15.22:0/3557157949 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa990039600 0x7fa99003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:35.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.041+0000 7fa9a7e00700 1 --2- 172.21.15.22:0/3557157949 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa990039600 0x7fa99003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa994006fd0 tx=0x7fa994006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:35.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.046+0000 7fa9a5600700 1 -- 172.21.15.22:0/3557157949 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa99c00b3e0 con 0x7fa9a8108e10 2024-04-20T21:23:35.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.304+0000 7fa9ae400700 1 -- 172.21.15.22:0/3557157949 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa9a81a1ba0 con 0x7fa990039600 2024-04-20T21:23:35.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.567+0000 7fa9a5600700 1 -- 172.21.15.22:0/3557157949 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa9a81a1ba0 con 0x7fa990039600 2024-04-20T21:23:35.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.569+0000 7fa9ae400700 1 -- 172.21.15.22:0/3557157949 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa990039600 msgr2=0x7fa99003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:35.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.569+0000 7fa9ae400700 1 --2- 172.21.15.22:0/3557157949 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa990039600 0x7fa99003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa994006fd0 tx=0x7fa994006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:35.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.569+0000 7fa9ae400700 1 -- 172.21.15.22:0/3557157949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 msgr2=0x7fa9a819e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:35.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.569+0000 7fa9ae400700 1 --2- 172.21.15.22:0/3557157949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 0x7fa9a819e540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa99c011b60 tx=0x7fa99c011c40 comp rx=0 tx=0).stop 2024-04-20T21:23:35.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.570+0000 7fa9ae400700 1 -- 172.21.15.22:0/3557157949 shutdown_connections 2024-04-20T21:23:35.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.570+0000 7fa9ae400700 1 --2- 172.21.15.22:0/3557157949 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa990039600 0x7fa99003bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:35.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.570+0000 7fa9ae400700 1 --2- 172.21.15.22:0/3557157949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9a8108e10 0x7fa9a819e540 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:35.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.570+0000 7fa9ae400700 1 -- 172.21.15.22:0/3557157949 >> 172.21.15.22:0/3557157949 conn(0x7fa9a8102860 msgr2=0x7fa9a8103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:35.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.570+0000 7fa9ae400700 1 -- 172.21.15.22:0/3557157949 shutdown_connections 2024-04-20T21:23:35.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:34.570+0000 7fa9ae400700 1 -- 172.21.15.22:0/3557157949 wait complete. 2024-04-20T21:23:35.286 INFO:teuthology.orchestra.run.smithi022.stdout:Fetching dashboard port number... 2024-04-20T21:23:37.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:37 smithi022 ceph-mon[16843]: mgrmap e13: x(active, since 6s) 2024-04-20T21:23:37.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:37 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/600889503' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-20T21:23:37.527 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 8443 2024-04-20T21:23:37.528 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.022+0000 7fc82dc00700 1 Processor -- start 2024-04-20T21:23:37.528 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.023+0000 7fc82dc00700 1 -- start start 2024-04-20T21:23:37.528 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.023+0000 7fc82dc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 0x7fc82810f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:37.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.023+0000 7fc82dc00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc82810fa40 con 0x7fc82810f050 2024-04-20T21:23:37.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.023+0000 7fc826a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 0x7fc82810f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:37.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.024+0000 7fc826a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 0x7fc82810f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52592/0 (socket says 172.21.15.22:52592) 2024-04-20T21:23:37.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.024+0000 7fc826a00700 1 -- 172.21.15.22:0/764089598 learned_addr learned my addr 172.21.15.22:0/764089598 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:37.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.024+0000 7fc826a00700 1 -- 172.21.15.22:0/764089598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82810fd70 con 0x7fc82810f050 2024-04-20T21:23:37.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.024+0000 7fc826a00700 1 --2- 172.21.15.22:0/764089598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 0x7fc82810f470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8180098a0 tx=0x7fc818009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd733a29fa4781a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:37.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.025+0000 7fc825600700 1 -- 172.21.15.22:0/764089598 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc818004160 con 0x7fc82810f050 2024-04-20T21:23:37.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.025+0000 7fc825600700 1 -- 172.21.15.22:0/764089598 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc818004300 con 0x7fc82810f050 2024-04-20T21:23:37.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.025+0000 7fc825600700 1 -- 172.21.15.22:0/764089598 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc818011b20 con 0x7fc82810f050 2024-04-20T21:23:37.531 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.026+0000 7fc82dc00700 1 -- 172.21.15.22:0/764089598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 msgr2=0x7fc82810f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.026+0000 7fc82dc00700 1 --2- 172.21.15.22:0/764089598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 0x7fc82810f470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8180098a0 tx=0x7fc818009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.026+0000 7fc82dc00700 1 -- 172.21.15.22:0/764089598 shutdown_connections 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.026+0000 7fc82dc00700 1 --2- 172.21.15.22:0/764089598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 0x7fc82810f470 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.026+0000 7fc82dc00700 1 -- 172.21.15.22:0/764089598 >> 172.21.15.22:0/764089598 conn(0x7fc82807ddd0 msgr2=0x7fc8280801f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.026+0000 7fc82dc00700 1 -- 172.21.15.22:0/764089598 shutdown_connections 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.026+0000 7fc82dc00700 1 -- 172.21.15.22:0/764089598 wait complete. 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.027+0000 7fc82dc00700 1 Processor -- start 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.027+0000 7fc82dc00700 1 -- start start 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.027+0000 7fc82dc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 0x7fc8281a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.027+0000 7fc82dc00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc828113120 con 0x7fc82810f050 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.028+0000 7fc826a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 0x7fc8281a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.028+0000 7fc826a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 0x7fc8281a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52602/0 (socket says 172.21.15.22:52602) 2024-04-20T21:23:37.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.028+0000 7fc826a00700 1 -- 172.21.15.22:0/600889503 learned_addr learned my addr 172.21.15.22:0/600889503 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:37.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.028+0000 7fc826a00700 1 -- 172.21.15.22:0/600889503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8281a2f90 con 0x7fc82810f050 2024-04-20T21:23:37.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.028+0000 7fc826a00700 1 --2- 172.21.15.22:0/600889503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 0x7fc8281a2a50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc8180117e0 tx=0x7fc818011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:37.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.029+0000 7fc81f400700 1 -- 172.21.15.22:0/600889503 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc818003ba0 con 0x7fc82810f050 2024-04-20T21:23:37.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.029+0000 7fc81f400700 1 -- 172.21.15.22:0/600889503 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc81801a910 con 0x7fc82810f050 2024-04-20T21:23:37.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.029+0000 7fc82dc00700 1 -- 172.21.15.22:0/600889503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8281a3220 con 0x7fc82810f050 2024-04-20T21:23:37.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.029+0000 7fc81f400700 1 -- 172.21.15.22:0/600889503 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8180248f0 con 0x7fc82810f050 2024-04-20T21:23:37.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.029+0000 7fc82dc00700 1 -- 172.21.15.22:0/600889503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8281a3680 con 0x7fc82810f050 2024-04-20T21:23:37.535 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.030+0000 7fc82dc00700 1 -- 172.21.15.22:0/600889503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8280640b0 con 0x7fc82810f050 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.030+0000 7fc81f400700 1 -- 172.21.15.22:0/600889503 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fc818022020 con 0x7fc82810f050 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.031+0000 7fc81f400700 1 --2- 172.21.15.22:0/600889503 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc80c039660 0x7fc80c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.031+0000 7fc81f400700 1 -- 172.21.15.22:0/600889503 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc81804dcf0 con 0x7fc82810f050 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.031+0000 7fc826000700 1 --2- 172.21.15.22:0/600889503 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc80c039660 0x7fc80c03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.032+0000 7fc826000700 1 --2- 172.21.15.22:0/600889503 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc80c039660 0x7fc80c03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc814006fd0 tx=0x7fc814006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.036+0000 7fc81f400700 1 -- 172.21.15.22:0/600889503 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc81800f2e0 con 0x7fc82810f050 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.165+0000 7fc81f400700 1 -- 172.21.15.22:0/600889503 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc81802d3f0 con 0x7fc82810f050 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.274+0000 7fc82dc00700 1 -- 172.21.15.22:0/600889503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fc8281a6030 con 0x7fc82810f050 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.275+0000 7fc81f400700 1 -- 172.21.15.22:0/600889503 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fc81802be20 con 0x7fc82810f050 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.278+0000 7fc82dc00700 1 -- 172.21.15.22:0/600889503 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc80c039660 msgr2=0x7fc80c03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.278+0000 7fc82dc00700 1 --2- 172.21.15.22:0/600889503 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc80c039660 0x7fc80c03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc814006fd0 tx=0x7fc814006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.278+0000 7fc82dc00700 1 -- 172.21.15.22:0/600889503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 msgr2=0x7fc8281a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.278+0000 7fc82dc00700 1 --2- 172.21.15.22:0/600889503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 0x7fc8281a2a50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc8180117e0 tx=0x7fc818011ba0 comp rx=0 tx=0).stop 2024-04-20T21:23:37.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.278+0000 7fc82dc00700 1 -- 172.21.15.22:0/600889503 shutdown_connections 2024-04-20T21:23:37.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.278+0000 7fc82dc00700 1 --2- 172.21.15.22:0/600889503 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc80c039660 0x7fc80c03bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:37.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.278+0000 7fc82dc00700 1 --2- 172.21.15.22:0/600889503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc82810f050 0x7fc8281a2a50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:37.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.278+0000 7fc82dc00700 1 -- 172.21.15.22:0/600889503 >> 172.21.15.22:0/600889503 conn(0x7fc82807ddd0 msgr2=0x7fc82807f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:37.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.278+0000 7fc82dc00700 1 -- 172.21.15.22:0/600889503 shutdown_connections 2024-04-20T21:23:37.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:36.278+0000 7fc82dc00700 1 -- 172.21.15.22:0/600889503 wait complete. 2024-04-20T21:23:37.569 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T21:23:37.570 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-04-20T21:23:37.594 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T21:23:37.594 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-04-20T21:23:37.595 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-04-20T21:23:37.595 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-20T21:23:37.603 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph Dashboard is now available at: 2024-04-20T21:23:37.603 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:23:37.603 INFO:teuthology.orchestra.run.smithi022.stdout: URL: https://smithi022.front.sepia.ceph.com:8443/ 2024-04-20T21:23:37.603 INFO:teuthology.orchestra.run.smithi022.stdout: User: admin 2024-04-20T21:23:37.603 INFO:teuthology.orchestra.run.smithi022.stdout: Password: 9d2fxehcsx 2024-04-20T21:23:37.603 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:23:37.604 INFO:teuthology.orchestra.run.smithi022.stdout:Saving cluster configuration to /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config directory 2024-04-20T21:23:37.606 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling autotune for osd_memory_target 2024-04-20T21:23:39.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.481+0000 7f62acc00700 1 Processor -- start 2024-04-20T21:23:39.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.482+0000 7f62acc00700 1 -- start start 2024-04-20T21:23:39.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.482+0000 7f62acc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 0x7f62a810ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:39.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.482+0000 7f62acc00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f62a810b200 con 0x7f62a810a810 2024-04-20T21:23:39.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.482+0000 7f62a5e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 0x7f62a810ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:39.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.483+0000 7f62a5e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 0x7f62a810ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52612/0 (socket says 172.21.15.22:52612) 2024-04-20T21:23:39.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.483+0000 7f62a5e00700 1 -- 172.21.15.22:0/1344897546 learned_addr learned my addr 172.21.15.22:0/1344897546 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:39.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.483+0000 7f62a5e00700 1 -- 172.21.15.22:0/1344897546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a810b9c0 con 0x7f62a810a810 2024-04-20T21:23:39.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.484+0000 7f62a5e00700 1 --2- 172.21.15.22:0/1344897546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 0x7f62a810ac30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6298009e80 tx=0x7f629800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799a9f13461abb1e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:39.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.484+0000 7f62a5400700 1 -- 172.21.15.22:0/1344897546 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6298017070 con 0x7f62a810a810 2024-04-20T21:23:39.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.484+0000 7f62a5400700 1 -- 172.21.15.22:0/1344897546 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f629800b8c0 con 0x7f62a810a810 2024-04-20T21:23:39.359 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.484+0000 7f62a5400700 1 -- 172.21.15.22:0/1344897546 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62980045c0 con 0x7f62a810a810 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.485+0000 7f62acc00700 1 -- 172.21.15.22:0/1344897546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 msgr2=0x7f62a810ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.485+0000 7f62acc00700 1 --2- 172.21.15.22:0/1344897546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 0x7f62a810ac30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6298009e80 tx=0x7f629800b330 comp rx=0 tx=0).stop 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.485+0000 7f62acc00700 1 -- 172.21.15.22:0/1344897546 shutdown_connections 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.485+0000 7f62acc00700 1 --2- 172.21.15.22:0/1344897546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 0x7f62a810ac30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.485+0000 7f62acc00700 1 -- 172.21.15.22:0/1344897546 >> 172.21.15.22:0/1344897546 conn(0x7f62a807dce0 msgr2=0x7f62a81084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.485+0000 7f62acc00700 1 -- 172.21.15.22:0/1344897546 shutdown_connections 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.485+0000 7f62acc00700 1 -- 172.21.15.22:0/1344897546 wait complete. 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.486+0000 7f62acc00700 1 Processor -- start 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.487+0000 7f62acc00700 1 -- start start 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.487+0000 7f62acc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 0x7f62a819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.487+0000 7f62acc00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f62a810ed70 con 0x7f62a810a810 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.487+0000 7f62a5e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 0x7f62a819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.487+0000 7f62a5e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 0x7f62a819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52626/0 (socket says 172.21.15.22:52626) 2024-04-20T21:23:39.360 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.487+0000 7f62a5e00700 1 -- 172.21.15.22:0/639571226 learned_addr learned my addr 172.21.15.22:0/639571226 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.488+0000 7f62a5e00700 1 -- 172.21.15.22:0/639571226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a819ebc0 con 0x7f62a810a810 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.488+0000 7f62a5e00700 1 --2- 172.21.15.22:0/639571226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 0x7f62a819e680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6298009fd0 tx=0x7f6298003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.488+0000 7f629f400700 1 -- 172.21.15.22:0/639571226 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6298020020 con 0x7f62a810a810 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.488+0000 7f629f400700 1 -- 172.21.15.22:0/639571226 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f629800bba0 con 0x7f62a810a810 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.488+0000 7f629f400700 1 -- 172.21.15.22:0/639571226 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6298017040 con 0x7f62a810a810 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.488+0000 7f62acc00700 1 -- 172.21.15.22:0/639571226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a819ee50 con 0x7f62a810a810 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.488+0000 7f62acc00700 1 -- 172.21.15.22:0/639571226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a81a1b40 con 0x7f62a810a810 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.489+0000 7f62acc00700 1 -- 172.21.15.22:0/639571226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a80640b0 con 0x7f62a810a810 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.490+0000 7f629f400700 1 -- 172.21.15.22:0/639571226 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6298023020 con 0x7f62a810a810 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.490+0000 7f629f400700 1 --2- 172.21.15.22:0/639571226 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f628c039700 0x7f628c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.490+0000 7f629f400700 1 -- 172.21.15.22:0/639571226 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f629800f050 con 0x7f62a810a810 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.491+0000 7f629d400700 1 --2- 172.21.15.22:0/639571226 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f628c039700 0x7f628c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.491+0000 7f629d400700 1 --2- 172.21.15.22:0/639571226 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f628c039700 0x7f628c03bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6290006fd0 tx=0x7f6290006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:39.361 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.496+0000 7f629f400700 1 -- 172.21.15.22:0/639571226 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f629802a020 con 0x7f62a810a810 2024-04-20T21:23:39.362 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.722+0000 7f62acc00700 1 -- 172.21.15.22:0/639571226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f62a81a2120 con 0x7f62a810a810 2024-04-20T21:23:39.362 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.723+0000 7f629f400700 1 -- 172.21.15.22:0/639571226 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f629801bbb0 con 0x7f62a810a810 2024-04-20T21:23:39.362 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.727+0000 7f62acc00700 1 -- 172.21.15.22:0/639571226 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f628c039700 msgr2=0x7f628c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:39.362 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.727+0000 7f62acc00700 1 --2- 172.21.15.22:0/639571226 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f628c039700 0x7f628c03bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6290006fd0 tx=0x7f6290006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:39.362 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.727+0000 7f62acc00700 1 -- 172.21.15.22:0/639571226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 msgr2=0x7f62a819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:39.362 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.727+0000 7f62acc00700 1 --2- 172.21.15.22:0/639571226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 0x7f62a819e680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6298009fd0 tx=0x7f6298003d00 comp rx=0 tx=0).stop 2024-04-20T21:23:39.362 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.727+0000 7f62acc00700 1 -- 172.21.15.22:0/639571226 shutdown_connections 2024-04-20T21:23:39.362 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.727+0000 7f62acc00700 1 --2- 172.21.15.22:0/639571226 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f628c039700 0x7f628c03bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:39.362 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.727+0000 7f62acc00700 1 --2- 172.21.15.22:0/639571226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62a810a810 0x7f62a819e680 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:39.362 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.727+0000 7f62acc00700 1 -- 172.21.15.22:0/639571226 >> 172.21.15.22:0/639571226 conn(0x7f62a807dce0 msgr2=0x7f62a8107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:39.362 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.727+0000 7f62acc00700 1 -- 172.21.15.22:0/639571226 shutdown_connections 2024-04-20T21:23:39.362 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:38.727+0000 7f62acc00700 1 -- 172.21.15.22:0/639571226 wait complete. 2024-04-20T21:23:40.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.050+0000 7f08c4a00700 1 Processor -- start 2024-04-20T21:23:40.991 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.050+0000 7f08c4a00700 1 -- start start 2024-04-20T21:23:40.992 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.051+0000 7f08c4a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 0x7f08c010a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:40.992 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.051+0000 7f08c4a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08c010afb0 con 0x7f08c010a5c0 2024-04-20T21:23:40.993 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.051+0000 7f08be800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 0x7f08c010a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:40.993 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.051+0000 7f08be800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 0x7f08c010a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52636/0 (socket says 172.21.15.22:52636) 2024-04-20T21:23:40.993 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.051+0000 7f08be800700 1 -- 172.21.15.22:0/1425912187 learned_addr learned my addr 172.21.15.22:0/1425912187 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:40.993 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.052+0000 7f08be800700 1 -- 172.21.15.22:0/1425912187 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c010b770 con 0x7f08c010a5c0 2024-04-20T21:23:40.993 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.052+0000 7f08be800700 1 --2- 172.21.15.22:0/1425912187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 0x7f08c010a9e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f08b00098a0 tx=0x7f08b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd2af399a94229e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:40.999 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.052+0000 7f08bd400700 1 -- 172.21.15.22:0/1425912187 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08b0004160 con 0x7f08c010a5c0 2024-04-20T21:23:40.999 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.052+0000 7f08bd400700 1 -- 172.21.15.22:0/1425912187 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b0004300 con 0x7f08c010a5c0 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.052+0000 7f08bd400700 1 -- 172.21.15.22:0/1425912187 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08b0011b20 con 0x7f08c010a5c0 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.053+0000 7f08c4a00700 1 -- 172.21.15.22:0/1425912187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 msgr2=0x7f08c010a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.053+0000 7f08c4a00700 1 --2- 172.21.15.22:0/1425912187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 0x7f08c010a9e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f08b00098a0 tx=0x7f08b0009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.053+0000 7f08c4a00700 1 -- 172.21.15.22:0/1425912187 shutdown_connections 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.053+0000 7f08c4a00700 1 --2- 172.21.15.22:0/1425912187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 0x7f08c010a9e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.053+0000 7f08c4a00700 1 -- 172.21.15.22:0/1425912187 >> 172.21.15.22:0/1425912187 conn(0x7f08c007dbb0 msgr2=0x7f08c01084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.053+0000 7f08c4a00700 1 -- 172.21.15.22:0/1425912187 shutdown_connections 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.053+0000 7f08c4a00700 1 -- 172.21.15.22:0/1425912187 wait complete. 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.054+0000 7f08c4a00700 1 Processor -- start 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.054+0000 7f08c4a00700 1 -- start start 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.055+0000 7f08c4a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 0x7f08c019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.055+0000 7f08c4a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08c010eb20 con 0x7f08c010a5c0 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.055+0000 7f08be800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 0x7f08c019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:41.000 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.055+0000 7f08be800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 0x7f08c019e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52644/0 (socket says 172.21.15.22:52644) 2024-04-20T21:23:41.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.055+0000 7f08be800700 1 -- 172.21.15.22:0/3558574286 learned_addr learned my addr 172.21.15.22:0/3558574286 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:41.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.055+0000 7f08be800700 1 -- 172.21.15.22:0/3558574286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c019ea60 con 0x7f08c010a5c0 2024-04-20T21:23:41.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.056+0000 7f08be800700 1 --2- 172.21.15.22:0/3558574286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 0x7f08c019e520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f08b0000c00 tx=0x7f08b0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:41.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.056+0000 7f08b7400700 1 -- 172.21.15.22:0/3558574286 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08b0003b60 con 0x7f08c010a5c0 2024-04-20T21:23:41.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.056+0000 7f08b7400700 1 -- 172.21.15.22:0/3558574286 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b001a910 con 0x7f08c010a5c0 2024-04-20T21:23:41.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.056+0000 7f08b7400700 1 -- 172.21.15.22:0/3558574286 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08b0024930 con 0x7f08c010a5c0 2024-04-20T21:23:41.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.056+0000 7f08c4a00700 1 -- 172.21.15.22:0/3558574286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c019ecf0 con 0x7f08c010a5c0 2024-04-20T21:23:41.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.056+0000 7f08c4a00700 1 -- 172.21.15.22:0/3558574286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c019f150 con 0x7f08c010a5c0 2024-04-20T21:23:41.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.058+0000 7f08b7400700 1 -- 172.21.15.22:0/3558574286 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f08b0022020 con 0x7f08c010a5c0 2024-04-20T21:23:41.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.058+0000 7f08c4a00700 1 -- 172.21.15.22:0/3558574286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c00640b0 con 0x7f08c010a5c0 2024-04-20T21:23:41.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.058+0000 7f08b7400700 1 --2- 172.21.15.22:0/3558574286 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f08a4039700 0x7f08a403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:41.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.058+0000 7f08b7400700 1 -- 172.21.15.22:0/3558574286 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f08b004da70 con 0x7f08c010a5c0 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.058+0000 7f08bde00700 1 --2- 172.21.15.22:0/3558574286 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f08a4039700 0x7f08a403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.059+0000 7f08bde00700 1 --2- 172.21.15.22:0/3558574286 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f08a4039700 0x7f08a403bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f08ac006fd0 tx=0x7f08ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.064+0000 7f08b7400700 1 -- 172.21.15.22:0/3558574286 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f08b001ab70 con 0x7f08c010a5c0 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.417+0000 7f08c4a00700 1 -- 172.21.15.22:0/3558574286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f08c01a1b20 con 0x7f08c010a5c0 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.420+0000 7f08b7400700 1 -- 172.21.15.22:0/3558574286 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f08b002d3f0 con 0x7f08c010a5c0 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.423+0000 7f08c4a00700 1 -- 172.21.15.22:0/3558574286 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f08a4039700 msgr2=0x7f08a403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.423+0000 7f08c4a00700 1 --2- 172.21.15.22:0/3558574286 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f08a4039700 0x7f08a403bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f08ac006fd0 tx=0x7f08ac006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.423+0000 7f08c4a00700 1 -- 172.21.15.22:0/3558574286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 msgr2=0x7f08c019e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.423+0000 7f08c4a00700 1 --2- 172.21.15.22:0/3558574286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 0x7f08c019e520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f08b0000c00 tx=0x7f08b0011c10 comp rx=0 tx=0).stop 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.423+0000 7f08c4a00700 1 -- 172.21.15.22:0/3558574286 shutdown_connections 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.423+0000 7f08c4a00700 1 --2- 172.21.15.22:0/3558574286 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f08a4039700 0x7f08a403bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.423+0000 7f08c4a00700 1 --2- 172.21.15.22:0/3558574286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c010a5c0 0x7f08c019e520 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.423+0000 7f08c4a00700 1 -- 172.21.15.22:0/3558574286 >> 172.21.15.22:0/3558574286 conn(0x7f08c007dbb0 msgr2=0x7f08c0107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.424+0000 7f08c4a00700 1 -- 172.21.15.22:0/3558574286 shutdown_connections 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-04-20T21:23:40.424+0000 7f08c4a00700 1 -- 172.21.15.22:0/3558574286 wait complete. 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout:Or, if you are only running a single cluster on this host: 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-20T21:23:41.003 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:23:41.004 INFO:teuthology.orchestra.run.smithi022.stdout: ceph telemetry on 2024-04-20T21:23:41.004 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:23:41.004 INFO:teuthology.orchestra.run.smithi022.stdout:For more information see: 2024-04-20T21:23:41.004 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:23:41.004 INFO:teuthology.orchestra.run.smithi022.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-20T21:23:41.004 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:23:41.004 INFO:teuthology.orchestra.run.smithi022.stdout:Bootstrap complete. 2024-04-20T21:23:41.095 INFO:tasks.cephadm:Fetching config... 2024-04-20T21:23:41.095 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:23:41.095 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-20T21:23:41.192 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-20T21:23:41.192 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:23:41.192 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-20T21:23:41.238 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-20T21:23:41.238 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:23:41.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-20T21:23:41.351 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-20T21:23:41.351 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:23:41.351 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-20T21:23:41.446 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-20T21:23:41.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaAupTRE6xrMJ3Otr93e+WWUdSnuEhyzIx+TgCyj76BlkQyrXocGHEmuWElvD4l6YCSGPRQfd70Bejz3Z9ZgdptXZ/75ayPsOdg5g88L1w83p+d0pmPBX6GD2oW+6GjMQ+Dr0bClxQAXwIvuz87T+QxC0aUcO7jRRK3nkTKbTHN9ru/O/U7CWEPiJ6u2LYlDx/U3EDxbNI3/5rmZQZrN3pQEHFr06G36NqR20AuRspzSVtQbQMpBzfyrg12Vx2QK18zLmDSUpE7cqHtC3P6xzb7o5GJu95XB1KMDgujf2HBrcuSn/WRdUrJ6+Bv7HLKr17ihKScpJLkc+C96mSYmJl8zp1a05dnapCmhGOmVUNgrcaWFKZ9iadhRKIRiJO5ptBmvh73po+mNWjpXLlEDF+Uox9SIqtwb+wwR1oFvIM0cGcJvm6FRt+SqzswyyJQiski299vIhmil8nDTwd9KgUgXJ7FWkTKRZBCIv33rgh7WrOHL8d5Qr0vMOuVqYMSYk= ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T21:23:41.554 INFO:teuthology.orchestra.run.smithi022.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaAupTRE6xrMJ3Otr93e+WWUdSnuEhyzIx+TgCyj76BlkQyrXocGHEmuWElvD4l6YCSGPRQfd70Bejz3Z9ZgdptXZ/75ayPsOdg5g88L1w83p+d0pmPBX6GD2oW+6GjMQ+Dr0bClxQAXwIvuz87T+QxC0aUcO7jRRK3nkTKbTHN9ru/O/U7CWEPiJ6u2LYlDx/U3EDxbNI3/5rmZQZrN3pQEHFr06G36NqR20AuRspzSVtQbQMpBzfyrg12Vx2QK18zLmDSUpE7cqHtC3P6xzb7o5GJu95XB1KMDgujf2HBrcuSn/WRdUrJ6+Bv7HLKr17ihKScpJLkc+C96mSYmJl8zp1a05dnapCmhGOmVUNgrcaWFKZ9iadhRKIRiJO5ptBmvh73po+mNWjpXLlEDF+Uox9SIqtwb+wwR1oFvIM0cGcJvm6FRt+SqzswyyJQiski299vIhmil8nDTwd9KgUgXJ7FWkTKRZBCIv33rgh7WrOHL8d5Qr0vMOuVqYMSYk= ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:41.588 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:41 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3558574286' entity='client.admin' 2024-04-20T21:23:41.603 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaAupTRE6xrMJ3Otr93e+WWUdSnuEhyzIx+TgCyj76BlkQyrXocGHEmuWElvD4l6YCSGPRQfd70Bejz3Z9ZgdptXZ/75ayPsOdg5g88L1w83p+d0pmPBX6GD2oW+6GjMQ+Dr0bClxQAXwIvuz87T+QxC0aUcO7jRRK3nkTKbTHN9ru/O/U7CWEPiJ6u2LYlDx/U3EDxbNI3/5rmZQZrN3pQEHFr06G36NqR20AuRspzSVtQbQMpBzfyrg12Vx2QK18zLmDSUpE7cqHtC3P6xzb7o5GJu95XB1KMDgujf2HBrcuSn/WRdUrJ6+Bv7HLKr17ihKScpJLkc+C96mSYmJl8zp1a05dnapCmhGOmVUNgrcaWFKZ9iadhRKIRiJO5ptBmvh73po+mNWjpXLlEDF+Uox9SIqtwb+wwR1oFvIM0cGcJvm6FRt+SqzswyyJQiski299vIhmil8nDTwd9KgUgXJ7FWkTKRZBCIv33rgh7WrOHL8d5Qr0vMOuVqYMSYk= ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T21:23:41.643 INFO:teuthology.orchestra.run.smithi033.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaAupTRE6xrMJ3Otr93e+WWUdSnuEhyzIx+TgCyj76BlkQyrXocGHEmuWElvD4l6YCSGPRQfd70Bejz3Z9ZgdptXZ/75ayPsOdg5g88L1w83p+d0pmPBX6GD2oW+6GjMQ+Dr0bClxQAXwIvuz87T+QxC0aUcO7jRRK3nkTKbTHN9ru/O/U7CWEPiJ6u2LYlDx/U3EDxbNI3/5rmZQZrN3pQEHFr06G36NqR20AuRspzSVtQbQMpBzfyrg12Vx2QK18zLmDSUpE7cqHtC3P6xzb7o5GJu95XB1KMDgujf2HBrcuSn/WRdUrJ6+Bv7HLKr17ihKScpJLkc+C96mSYmJl8zp1a05dnapCmhGOmVUNgrcaWFKZ9iadhRKIRiJO5ptBmvh73po+mNWjpXLlEDF+Uox9SIqtwb+wwR1oFvIM0cGcJvm6FRt+SqzswyyJQiski299vIhmil8nDTwd9KgUgXJ7FWkTKRZBCIv33rgh7WrOHL8d5Qr0vMOuVqYMSYk= ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:41.656 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaAupTRE6xrMJ3Otr93e+WWUdSnuEhyzIx+TgCyj76BlkQyrXocGHEmuWElvD4l6YCSGPRQfd70Bejz3Z9ZgdptXZ/75ayPsOdg5g88L1w83p+d0pmPBX6GD2oW+6GjMQ+Dr0bClxQAXwIvuz87T+QxC0aUcO7jRRK3nkTKbTHN9ru/O/U7CWEPiJ6u2LYlDx/U3EDxbNI3/5rmZQZrN3pQEHFr06G36NqR20AuRspzSVtQbQMpBzfyrg12Vx2QK18zLmDSUpE7cqHtC3P6xzb7o5GJu95XB1KMDgujf2HBrcuSn/WRdUrJ6+Bv7HLKr17ihKScpJLkc+C96mSYmJl8zp1a05dnapCmhGOmVUNgrcaWFKZ9iadhRKIRiJO5ptBmvh73po+mNWjpXLlEDF+Uox9SIqtwb+wwR1oFvIM0cGcJvm6FRt+SqzswyyJQiski299vIhmil8nDTwd9KgUgXJ7FWkTKRZBCIv33rgh7WrOHL8d5Qr0vMOuVqYMSYk= ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T21:23:41.698 INFO:teuthology.orchestra.run.smithi122.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaAupTRE6xrMJ3Otr93e+WWUdSnuEhyzIx+TgCyj76BlkQyrXocGHEmuWElvD4l6YCSGPRQfd70Bejz3Z9ZgdptXZ/75ayPsOdg5g88L1w83p+d0pmPBX6GD2oW+6GjMQ+Dr0bClxQAXwIvuz87T+QxC0aUcO7jRRK3nkTKbTHN9ru/O/U7CWEPiJ6u2LYlDx/U3EDxbNI3/5rmZQZrN3pQEHFr06G36NqR20AuRspzSVtQbQMpBzfyrg12Vx2QK18zLmDSUpE7cqHtC3P6xzb7o5GJu95XB1KMDgujf2HBrcuSn/WRdUrJ6+Bv7HLKr17ihKScpJLkc+C96mSYmJl8zp1a05dnapCmhGOmVUNgrcaWFKZ9iadhRKIRiJO5ptBmvh73po+mNWjpXLlEDF+Uox9SIqtwb+wwR1oFvIM0cGcJvm6FRt+SqzswyyJQiski299vIhmil8nDTwd9KgUgXJ7FWkTKRZBCIv33rgh7WrOHL8d5Qr0vMOuVqYMSYk= ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:41.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-20T21:23:42.147 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:23:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.313+0000 7fd5efa00700 1 -- 172.21.15.22:0/2352106051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5e8074ee0 msgr2=0x7fd5e8075300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.313+0000 7fd5efa00700 1 --2- 172.21.15.22:0/2352106051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5e8074ee0 0x7fd5e8075300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009910 tx=0x7fd5dc009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.313+0000 7fd5efa00700 1 -- 172.21.15.22:0/2352106051 shutdown_connections 2024-04-20T21:23:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.313+0000 7fd5efa00700 1 --2- 172.21.15.22:0/2352106051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5e8074ee0 0x7fd5e8075300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.313+0000 7fd5efa00700 1 -- 172.21.15.22:0/2352106051 >> 172.21.15.22:0/2352106051 conn(0x7fd5e80fdaa0 msgr2=0x7fd5e80ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.313+0000 7fd5efa00700 1 -- 172.21.15.22:0/2352106051 shutdown_connections 2024-04-20T21:23:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.313+0000 7fd5efa00700 1 -- 172.21.15.22:0/2352106051 wait complete. 2024-04-20T21:23:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.314+0000 7fd5efa00700 1 Processor -- start 2024-04-20T21:23:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.314+0000 7fd5efa00700 1 -- start start 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.314+0000 7fd5efa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5e8074ee0 0x7fd5e819e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.314+0000 7fd5efa00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e810ec90 con 0x7fd5e8074ee0 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.315+0000 7fd5ed000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5e8074ee0 0x7fd5e819e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.315+0000 7fd5ed000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5e8074ee0 0x7fd5e819e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55410/0 (socket says 172.21.15.22:55410) 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.315+0000 7fd5ed000700 1 -- 172.21.15.22:0/862002362 learned_addr learned my addr 172.21.15.22:0/862002362 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.315+0000 7fd5ed000700 1 -- 172.21.15.22:0/862002362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e819e820 con 0x7fd5e8074ee0 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.316+0000 7fd5ed000700 1 --2- 172.21.15.22:0/862002362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5e8074ee0 0x7fd5e819e2e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc005950 tx=0x7fd5dc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.316+0000 7fd5e5600700 1 -- 172.21.15.22:0/862002362 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5dc026070 con 0x7fd5e8074ee0 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.316+0000 7fd5e5600700 1 -- 172.21.15.22:0/862002362 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5dc00bc30 con 0x7fd5e8074ee0 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.316+0000 7fd5efa00700 1 -- 172.21.15.22:0/862002362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e819eab0 con 0x7fd5e8074ee0 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.316+0000 7fd5e5600700 1 -- 172.21.15.22:0/862002362 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5dc003b30 con 0x7fd5e8074ee0 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.316+0000 7fd5efa00700 1 -- 172.21.15.22:0/862002362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e81a1750 con 0x7fd5e8074ee0 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.317+0000 7fd5e5600700 1 -- 172.21.15.22:0/862002362 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd5dc029070 con 0x7fd5e8074ee0 2024-04-20T21:23:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.317+0000 7fd5efa00700 1 -- 172.21.15.22:0/862002362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e810f3f0 con 0x7fd5e8074ee0 2024-04-20T21:23:44.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.318+0000 7fd5e5600700 1 --2- 172.21.15.22:0/862002362 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd5d0039650 0x7fd5d003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:44.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.318+0000 7fd5e5600700 1 -- 172.21.15.22:0/862002362 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd5dc01e070 con 0x7fd5e8074ee0 2024-04-20T21:23:44.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.318+0000 7fd5e7e00700 1 --2- 172.21.15.22:0/862002362 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd5d0039650 0x7fd5d003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:44.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.319+0000 7fd5e7e00700 1 --2- 172.21.15.22:0/862002362 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd5d0039650 0x7fd5d003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4006fd0 tx=0x7fd5d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:44.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.324+0000 7fd5e5600700 1 -- 172.21.15.22:0/862002362 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd5dc013070 con 0x7fd5e8074ee0 2024-04-20T21:23:44.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.562+0000 7fd5efa00700 1 -- 172.21.15.22:0/862002362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fd5e802df90 con 0x7fd5e8074ee0 2024-04-20T21:23:44.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.571+0000 7fd5e5600700 1 -- 172.21.15.22:0/862002362 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fd5dc015350 con 0x7fd5e8074ee0 2024-04-20T21:23:44.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.575+0000 7fd5efa00700 1 -- 172.21.15.22:0/862002362 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd5d0039650 msgr2=0x7fd5d003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:44.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.575+0000 7fd5efa00700 1 --2- 172.21.15.22:0/862002362 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd5d0039650 0x7fd5d003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4006fd0 tx=0x7fd5d4006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:44.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.575+0000 7fd5efa00700 1 -- 172.21.15.22:0/862002362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5e8074ee0 msgr2=0x7fd5e819e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:44.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.575+0000 7fd5efa00700 1 --2- 172.21.15.22:0/862002362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5e8074ee0 0x7fd5e819e2e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc005950 tx=0x7fd5dc005030 comp rx=0 tx=0).stop 2024-04-20T21:23:44.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.575+0000 7fd5efa00700 1 -- 172.21.15.22:0/862002362 shutdown_connections 2024-04-20T21:23:44.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.575+0000 7fd5efa00700 1 --2- 172.21.15.22:0/862002362 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd5d0039650 0x7fd5d003bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:44.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.575+0000 7fd5efa00700 1 --2- 172.21.15.22:0/862002362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5e8074ee0 0x7fd5e819e2e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:44.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.575+0000 7fd5efa00700 1 -- 172.21.15.22:0/862002362 >> 172.21.15.22:0/862002362 conn(0x7fd5e80fdaa0 msgr2=0x7fd5e80fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:44.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.575+0000 7fd5efa00700 1 -- 172.21.15.22:0/862002362 shutdown_connections 2024-04-20T21:23:44.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:44.575+0000 7fd5efa00700 1 -- 172.21.15.22:0/862002362 wait complete. 2024-04-20T21:23:45.329 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-20T21:23:45.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-20T21:23:45.780 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:23:45.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:45.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:45 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/862002362' entity='client.admin' 2024-04-20T21:23:45.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:45.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi022", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:23:45.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:45.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:45.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:47.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.767+0000 7f8718c00700 1 -- 172.21.15.22:0/4267048928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8714108c60 msgr2=0x7f8714109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:47.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.767+0000 7f8718c00700 1 --2- 172.21.15.22:0/4267048928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8714108c60 0x7f8714109080 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8704009910 tx=0x7f8704009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:47.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.768+0000 7f8718c00700 1 -- 172.21.15.22:0/4267048928 shutdown_connections 2024-04-20T21:23:47.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.768+0000 7f8718c00700 1 --2- 172.21.15.22:0/4267048928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8714108c60 0x7f8714109080 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:47.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.768+0000 7f8718c00700 1 -- 172.21.15.22:0/4267048928 >> 172.21.15.22:0/4267048928 conn(0x7f8714077b40 msgr2=0x7f8714079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:47.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.768+0000 7f8718c00700 1 -- 172.21.15.22:0/4267048928 shutdown_connections 2024-04-20T21:23:47.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.768+0000 7f8718c00700 1 -- 172.21.15.22:0/4267048928 wait complete. 2024-04-20T21:23:47.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.769+0000 7f8718c00700 1 Processor -- start 2024-04-20T21:23:47.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.769+0000 7f8718c00700 1 -- start start 2024-04-20T21:23:47.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.769+0000 7f8718c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8714108c60 0x7f871419e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:47.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.769+0000 7f8718c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f871410eb80 con 0x7f8714108c60 2024-04-20T21:23:47.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.770+0000 7f8711e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8714108c60 0x7f871419e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:47.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.770+0000 7f8711e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8714108c60 0x7f871419e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55422/0 (socket says 172.21.15.22:55422) 2024-04-20T21:23:47.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.770+0000 7f8711e00700 1 -- 172.21.15.22:0/4238544289 learned_addr learned my addr 172.21.15.22:0/4238544289 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:47.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.770+0000 7f8711e00700 1 -- 172.21.15.22:0/4238544289 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871419e730 con 0x7f8714108c60 2024-04-20T21:23:47.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.770+0000 7f8711e00700 1 --2- 172.21.15.22:0/4238544289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8714108c60 0x7f871419e1f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8704005850 tx=0x7f8704005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:47.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.771+0000 7f870b400700 1 -- 172.21.15.22:0/4238544289 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8704026070 con 0x7f8714108c60 2024-04-20T21:23:47.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.771+0000 7f870b400700 1 -- 172.21.15.22:0/4238544289 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87040044a0 con 0x7f8714108c60 2024-04-20T21:23:47.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.771+0000 7f870b400700 1 -- 172.21.15.22:0/4238544289 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8704003ab0 con 0x7f8714108c60 2024-04-20T21:23:47.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.771+0000 7f8718c00700 1 -- 172.21.15.22:0/4238544289 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871419e9c0 con 0x7f8714108c60 2024-04-20T21:23:47.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.771+0000 7f8718c00700 1 -- 172.21.15.22:0/4238544289 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87141a16b0 con 0x7f8714108c60 2024-04-20T21:23:47.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.772+0000 7f870b400700 1 -- 172.21.15.22:0/4238544289 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8704029070 con 0x7f8714108c60 2024-04-20T21:23:47.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.772+0000 7f8718c00700 1 -- 172.21.15.22:0/4238544289 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871410f300 con 0x7f8714108c60 2024-04-20T21:23:47.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.772+0000 7f870b400700 1 --2- 172.21.15.22:0/4238544289 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f86f80396b0 0x7f86f803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:47.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.773+0000 7f870b400700 1 -- 172.21.15.22:0/4238544289 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f870401e070 con 0x7f8714108c60 2024-04-20T21:23:47.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.773+0000 7f8709400700 1 --2- 172.21.15.22:0/4238544289 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f86f80396b0 0x7f86f803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:47.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.773+0000 7f8709400700 1 --2- 172.21.15.22:0/4238544289 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f86f80396b0 0x7f86f803bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f86fc006fd0 tx=0x7f86fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:47.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:47.778+0000 7f870b400700 1 -- 172.21.15.22:0/4238544289 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8704013070 con 0x7f8714108c60 2024-04-20T21:23:48.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:48.013+0000 7f8718c00700 1 -- 172.21.15.22:0/4238544289 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f871402e0a0 con 0x7f86f80396b0 2024-04-20T21:23:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:48.022+0000 7f870b400700 1 -- 172.21.15.22:0/4238544289 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f871402e0a0 con 0x7f86f80396b0 2024-04-20T21:23:48.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:48.025+0000 7f8718c00700 1 -- 172.21.15.22:0/4238544289 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f86f80396b0 msgr2=0x7f86f803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:48.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:48.025+0000 7f8718c00700 1 --2- 172.21.15.22:0/4238544289 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f86f80396b0 0x7f86f803bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f86fc006fd0 tx=0x7f86fc006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:48.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:48.025+0000 7f8718c00700 1 -- 172.21.15.22:0/4238544289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8714108c60 msgr2=0x7f871419e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:48.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:48.025+0000 7f8718c00700 1 --2- 172.21.15.22:0/4238544289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8714108c60 0x7f871419e1f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8704005850 tx=0x7f8704005030 comp rx=0 tx=0).stop 2024-04-20T21:23:48.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:48.025+0000 7f8718c00700 1 -- 172.21.15.22:0/4238544289 shutdown_connections 2024-04-20T21:23:48.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:48.025+0000 7f8718c00700 1 --2- 172.21.15.22:0/4238544289 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f86f80396b0 0x7f86f803bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:48.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:48.025+0000 7f8718c00700 1 --2- 172.21.15.22:0/4238544289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8714108c60 0x7f871419e1f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:48.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:48.025+0000 7f8718c00700 1 -- 172.21.15.22:0/4238544289 >> 172.21.15.22:0/4238544289 conn(0x7f8714077b40 msgr2=0x7f8714078690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:48.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:48.025+0000 7f8718c00700 1 -- 172.21.15.22:0/4238544289 shutdown_connections 2024-04-20T21:23:48.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:48.025+0000 7f8718c00700 1 -- 172.21.15.22:0/4238544289 wait complete. 2024-04-20T21:23:48.805 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi033 2024-04-20T21:23:48.806 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-20T21:23:48.806 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/etc/ceph/ceph.conf 2024-04-20T21:23:48.824 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-20T21:23:48.825 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:48.883 INFO:tasks.cephadm:Adding host smithi033 to orchestrator... 2024-04-20T21:23:48.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch host add smithi033 2024-04-20T21:23:49.319 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:23:49.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:49 smithi022 ceph-mon[16843]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:49.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:49 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:49.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:49 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:49.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:49 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:49.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:49 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:23:49.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:49 smithi022 ceph-mon[16843]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-20T21:23:50.022 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:50 smithi022 ceph-mon[16843]: Updating smithi022:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:50.022 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:50 smithi022 ceph-mon[16843]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:51.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:51 smithi022 ceph-mon[16843]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:51.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:51 smithi022 ceph-mon[16843]: Updating smithi022:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:23:51.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:51.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:51.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:51.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.428+0000 7f191ec00700 1 -- 172.21.15.22:0/769965646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918074df0 msgr2=0x7f19180751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.428+0000 7f191ec00700 1 --2- 172.21.15.22:0/769965646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918074df0 0x7f19180751d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1908009910 tx=0x7f1908009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.428+0000 7f191ec00700 1 -- 172.21.15.22:0/769965646 shutdown_connections 2024-04-20T21:23:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.428+0000 7f191ec00700 1 --2- 172.21.15.22:0/769965646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918074df0 0x7f19180751d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.428+0000 7f191ec00700 1 -- 172.21.15.22:0/769965646 >> 172.21.15.22:0/769965646 conn(0x7f19180fdf40 msgr2=0x7f1918100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.428+0000 7f191ec00700 1 -- 172.21.15.22:0/769965646 shutdown_connections 2024-04-20T21:23:51.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.428+0000 7f191ec00700 1 -- 172.21.15.22:0/769965646 wait complete. 2024-04-20T21:23:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.429+0000 7f191ec00700 1 Processor -- start 2024-04-20T21:23:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.429+0000 7f191ec00700 1 -- start start 2024-04-20T21:23:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.430+0000 7f191ec00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918074df0 0x7f191819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.430+0000 7f191ec00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f19181130f0 con 0x7f1918074df0 2024-04-20T21:23:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.430+0000 7f1917e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918074df0 0x7f191819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.430+0000 7f1917e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918074df0 0x7f191819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55440/0 (socket says 172.21.15.22:55440) 2024-04-20T21:23:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.430+0000 7f1917e00700 1 -- 172.21.15.22:0/2725868810 learned_addr learned my addr 172.21.15.22:0/2725868810 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.430+0000 7f1917e00700 1 -- 172.21.15.22:0/2725868810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191819eb30 con 0x7f1918074df0 2024-04-20T21:23:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.431+0000 7f1917e00700 1 --2- 172.21.15.22:0/2725868810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918074df0 0x7f191819e5f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1908006010 tx=0x7f1908005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.431+0000 7f1914c00700 1 -- 172.21.15.22:0/2725868810 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1908026070 con 0x7f1918074df0 2024-04-20T21:23:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.431+0000 7f191ec00700 1 -- 172.21.15.22:0/2725868810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191819edc0 con 0x7f1918074df0 2024-04-20T21:23:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.431+0000 7f1914c00700 1 -- 172.21.15.22:0/2725868810 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f190800bc30 con 0x7f1918074df0 2024-04-20T21:23:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.431+0000 7f1914c00700 1 -- 172.21.15.22:0/2725868810 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1908003ab0 con 0x7f1918074df0 2024-04-20T21:23:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.431+0000 7f191ec00700 1 -- 172.21.15.22:0/2725868810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191819f220 con 0x7f1918074df0 2024-04-20T21:23:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.432+0000 7f191ec00700 1 -- 172.21.15.22:0/2725868810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1918076530 con 0x7f1918074df0 2024-04-20T21:23:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.432+0000 7f1914c00700 1 -- 172.21.15.22:0/2725868810 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1908029070 con 0x7f1918074df0 2024-04-20T21:23:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.433+0000 7f1914c00700 1 --2- 172.21.15.22:0/2725868810 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f18fc039650 0x7f18fc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.433+0000 7f1914c00700 1 -- 172.21.15.22:0/2725868810 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f190801e070 con 0x7f1918074df0 2024-04-20T21:23:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.433+0000 7f1917400700 1 --2- 172.21.15.22:0/2725868810 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f18fc039650 0x7f18fc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.434+0000 7f1917400700 1 --2- 172.21.15.22:0/2725868810 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f18fc039650 0x7f18fc03bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1904006fd0 tx=0x7f1904006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:51.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.439+0000 7f1914c00700 1 -- 172.21.15.22:0/2725868810 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1908013070 con 0x7f1918074df0 2024-04-20T21:23:51.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:51.673+0000 7f191ec00700 1 -- 172.21.15.22:0/2725868810 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi033", "target": ["mon-mgr", ""]}) -- 0x7f191819f950 con 0x7f18fc039650 2024-04-20T21:23:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:53 smithi022 ceph-mon[16843]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:53 smithi022 ceph-mon[16843]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi033", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:53 smithi022 ceph-mon[16843]: Deploying cephadm binary to smithi033 2024-04-20T21:23:55.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:55 smithi022 ceph-mon[16843]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:56.544+0000 7f1914c00700 1 -- 172.21.15.22:0/2725868810 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f191819f950 con 0x7f18fc039650 2024-04-20T21:23:56.544 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi033' with addr '172.21.15.33' 2024-04-20T21:23:56.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:56.548+0000 7f191ec00700 1 -- 172.21.15.22:0/2725868810 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f18fc039650 msgr2=0x7f18fc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:56.548+0000 7f191ec00700 1 --2- 172.21.15.22:0/2725868810 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f18fc039650 0x7f18fc03bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1904006fd0 tx=0x7f1904006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:56.548+0000 7f191ec00700 1 -- 172.21.15.22:0/2725868810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918074df0 msgr2=0x7f191819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:56.548+0000 7f191ec00700 1 --2- 172.21.15.22:0/2725868810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918074df0 0x7f191819e5f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1908006010 tx=0x7f1908005050 comp rx=0 tx=0).stop 2024-04-20T21:23:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:56.548+0000 7f191ec00700 1 -- 172.21.15.22:0/2725868810 shutdown_connections 2024-04-20T21:23:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:56.548+0000 7f191ec00700 1 --2- 172.21.15.22:0/2725868810 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f18fc039650 0x7f18fc03bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:56.548+0000 7f191ec00700 1 --2- 172.21.15.22:0/2725868810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918074df0 0x7f191819e5f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:56.548+0000 7f191ec00700 1 -- 172.21.15.22:0/2725868810 >> 172.21.15.22:0/2725868810 conn(0x7f19180fdf40 msgr2=0x7f19181052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:56.548+0000 7f191ec00700 1 -- 172.21.15.22:0/2725868810 shutdown_connections 2024-04-20T21:23:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:56.549+0000 7f191ec00700 1 -- 172.21.15.22:0/2725868810 wait complete. 2024-04-20T21:23:57.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:57 smithi022 ceph-mon[16843]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:57.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:57 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:57.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:57 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:57.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-20T21:23:57.692 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:23:58.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:58 smithi022 ceph-mon[16843]: Added host smithi033 2024-04-20T21:23:58.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:58 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:58.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:58 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:23:59.352 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:23:59 smithi022 ceph-mon[16843]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:59.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.596+0000 7fdbec400700 1 -- 172.21.15.22:0/3542479649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe410a670 msgr2=0x7fdbe410aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:59.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.596+0000 7fdbec400700 1 --2- 172.21.15.22:0/3542479649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe410a670 0x7fdbe410aa50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009910 tx=0x7fdbd8009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:59.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.596+0000 7fdbec400700 1 -- 172.21.15.22:0/3542479649 shutdown_connections 2024-04-20T21:23:59.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.596+0000 7fdbec400700 1 --2- 172.21.15.22:0/3542479649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe410a670 0x7fdbe410aa50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:59.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.596+0000 7fdbec400700 1 -- 172.21.15.22:0/3542479649 >> 172.21.15.22:0/3542479649 conn(0x7fdbe4100160 msgr2=0x7fdbe4102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:59.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.596+0000 7fdbec400700 1 -- 172.21.15.22:0/3542479649 shutdown_connections 2024-04-20T21:23:59.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.596+0000 7fdbec400700 1 -- 172.21.15.22:0/3542479649 wait complete. 2024-04-20T21:23:59.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.598+0000 7fdbec400700 1 Processor -- start 2024-04-20T21:23:59.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.598+0000 7fdbec400700 1 -- start start 2024-04-20T21:23:59.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.598+0000 7fdbec400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe410a670 0x7fdbe419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:59.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.598+0000 7fdbec400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe410ecb0 con 0x7fdbe410a670 2024-04-20T21:23:59.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.599+0000 7fdbe9a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe410a670 0x7fdbe419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:59.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.599+0000 7fdbe9a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe410a670 0x7fdbe419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36548/0 (socket says 172.21.15.22:36548) 2024-04-20T21:23:59.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.599+0000 7fdbe9a00700 1 -- 172.21.15.22:0/2663598440 learned_addr learned my addr 172.21.15.22:0/2663598440 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:23:59.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.599+0000 7fdbe9a00700 1 -- 172.21.15.22:0/2663598440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe419a6f0 con 0x7fdbe410a670 2024-04-20T21:23:59.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.599+0000 7fdbe9a00700 1 --2- 172.21.15.22:0/2663598440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe410a670 0x7fdbe419a1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8006010 tx=0x7fdbd8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:59.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.600+0000 7fdbe2000700 1 -- 172.21.15.22:0/2663598440 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbd8026070 con 0x7fdbe410a670 2024-04-20T21:23:59.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.600+0000 7fdbe2000700 1 -- 172.21.15.22:0/2663598440 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd800bbd0 con 0x7fdbe410a670 2024-04-20T21:23:59.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.600+0000 7fdbec400700 1 -- 172.21.15.22:0/2663598440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe419a9e0 con 0x7fdbe410a670 2024-04-20T21:23:59.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.600+0000 7fdbe2000700 1 -- 172.21.15.22:0/2663598440 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbd8003910 con 0x7fdbe410a670 2024-04-20T21:23:59.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.600+0000 7fdbec400700 1 -- 172.21.15.22:0/2663598440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe419ea80 con 0x7fdbe410a670 2024-04-20T21:23:59.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.601+0000 7fdbec400700 1 -- 172.21.15.22:0/2663598440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe4105db0 con 0x7fdbe410a670 2024-04-20T21:23:59.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.601+0000 7fdbe2000700 1 -- 172.21.15.22:0/2663598440 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdbd8029070 con 0x7fdbe410a670 2024-04-20T21:23:59.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.602+0000 7fdbe2000700 1 --2- 172.21.15.22:0/2663598440 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdbcc039650 0x7fdbcc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:59.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.602+0000 7fdbe2000700 1 -- 172.21.15.22:0/2663598440 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdbd801e070 con 0x7fdbe410a670 2024-04-20T21:23:59.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.602+0000 7fdbe9000700 1 --2- 172.21.15.22:0/2663598440 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdbcc039650 0x7fdbcc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:59.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.603+0000 7fdbe9000700 1 --2- 172.21.15.22:0/2663598440 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdbcc039650 0x7fdbcc03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0006fd0 tx=0x7fdbd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:59.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.607+0000 7fdbe2000700 1 -- 172.21.15.22:0/2663598440 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdbd8013070 con 0x7fdbe410a670 2024-04-20T21:23:59.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.844+0000 7fdbec400700 1 -- 172.21.15.22:0/2663598440 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdbe419b510 con 0x7fdbcc039650 2024-04-20T21:23:59.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.847+0000 7fdbe2000700 1 -- 172.21.15.22:0/2663598440 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fdbe419b510 con 0x7fdbcc039650 2024-04-20T21:23:59.847 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:23:59.847 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}] 2024-04-20T21:23:59.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.850+0000 7fdbec400700 1 -- 172.21.15.22:0/2663598440 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdbcc039650 msgr2=0x7fdbcc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:59.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.850+0000 7fdbec400700 1 --2- 172.21.15.22:0/2663598440 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdbcc039650 0x7fdbcc03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0006fd0 tx=0x7fdbd0006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:59.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.850+0000 7fdbec400700 1 -- 172.21.15.22:0/2663598440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe410a670 msgr2=0x7fdbe419a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:59.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.850+0000 7fdbec400700 1 --2- 172.21.15.22:0/2663598440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe410a670 0x7fdbe419a1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8006010 tx=0x7fdbd8005030 comp rx=0 tx=0).stop 2024-04-20T21:23:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.851+0000 7fdbec400700 1 -- 172.21.15.22:0/2663598440 shutdown_connections 2024-04-20T21:23:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.851+0000 7fdbec400700 1 --2- 172.21.15.22:0/2663598440 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdbcc039650 0x7fdbcc03bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.851+0000 7fdbec400700 1 --2- 172.21.15.22:0/2663598440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbe410a670 0x7fdbe419a1b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.851+0000 7fdbec400700 1 -- 172.21.15.22:0/2663598440 >> 172.21.15.22:0/2663598440 conn(0x7fdbe4100160 msgr2=0x7fdbe4100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.851+0000 7fdbec400700 1 -- 172.21.15.22:0/2663598440 shutdown_connections 2024-04-20T21:23:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:23:59.851+0000 7fdbec400700 1 -- 172.21.15.22:0/2663598440 wait complete. 2024-04-20T21:24:00.625 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:00 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:00.702 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi122 2024-04-20T21:24:00.703 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-20T21:24:00.703 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/etc/ceph/ceph.conf 2024-04-20T21:24:00.721 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-20T21:24:00.721 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:24:00.778 INFO:tasks.cephadm:Adding host smithi122 to orchestrator... 2024-04-20T21:24:00.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch host add smithi122 2024-04-20T21:24:01.228 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:24:01.614 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:01 smithi022 ceph-mon[16843]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:01.614 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:01 smithi022 ceph-mon[16843]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:24:03.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.288+0000 7ff06fa00700 1 -- 172.21.15.22:0/674248202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068074df0 msgr2=0x7ff0680751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.288+0000 7ff06fa00700 1 --2- 172.21.15.22:0/674248202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068074df0 0x7ff0680751d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009910 tx=0x7ff05c009c20 comp rx=0 tx=0).stop 2024-04-20T21:24:03.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.288+0000 7ff06fa00700 1 -- 172.21.15.22:0/674248202 shutdown_connections 2024-04-20T21:24:03.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.288+0000 7ff06fa00700 1 --2- 172.21.15.22:0/674248202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068074df0 0x7ff0680751d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.288+0000 7ff06fa00700 1 -- 172.21.15.22:0/674248202 >> 172.21.15.22:0/674248202 conn(0x7ff0680fdf90 msgr2=0x7ff0681003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.288+0000 7ff06fa00700 1 -- 172.21.15.22:0/674248202 shutdown_connections 2024-04-20T21:24:03.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.289+0000 7ff06fa00700 1 -- 172.21.15.22:0/674248202 wait complete. 2024-04-20T21:24:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.290+0000 7ff06fa00700 1 Processor -- start 2024-04-20T21:24:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.290+0000 7ff06fa00700 1 -- start start 2024-04-20T21:24:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.290+0000 7ff06fa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068074df0 0x7ff06819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.291+0000 7ff06fa00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff068113100 con 0x7ff068074df0 2024-04-20T21:24:03.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.291+0000 7ff06d000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068074df0 0x7ff06819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.291+0000 7ff06d000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068074df0 0x7ff06819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50598/0 (socket says 172.21.15.22:50598) 2024-04-20T21:24:03.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.291+0000 7ff06d000700 1 -- 172.21.15.22:0/3007381638 learned_addr learned my addr 172.21.15.22:0/3007381638 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:24:03.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.291+0000 7ff06d000700 1 -- 172.21.15.22:0/3007381638 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06819eb10 con 0x7ff068074df0 2024-04-20T21:24:03.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.292+0000 7ff06d000700 1 --2- 172.21.15.22:0/3007381638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068074df0 0x7ff06819e5d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff05c006010 tx=0x7ff05c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.292+0000 7ff065600700 1 -- 172.21.15.22:0/3007381638 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff05c026070 con 0x7ff068074df0 2024-04-20T21:24:03.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.292+0000 7ff065600700 1 -- 172.21.15.22:0/3007381638 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff05c00bc30 con 0x7ff068074df0 2024-04-20T21:24:03.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.292+0000 7ff06fa00700 1 -- 172.21.15.22:0/3007381638 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06819eda0 con 0x7ff068074df0 2024-04-20T21:24:03.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.292+0000 7ff065600700 1 -- 172.21.15.22:0/3007381638 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff05c0039a0 con 0x7ff068074df0 2024-04-20T21:24:03.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.292+0000 7ff06fa00700 1 -- 172.21.15.22:0/3007381638 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06819f200 con 0x7ff068074df0 2024-04-20T21:24:03.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.293+0000 7ff06fa00700 1 -- 172.21.15.22:0/3007381638 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff068076530 con 0x7ff068074df0 2024-04-20T21:24:03.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.294+0000 7ff065600700 1 -- 172.21.15.22:0/3007381638 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff05c029070 con 0x7ff068074df0 2024-04-20T21:24:03.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.295+0000 7ff065600700 1 --2- 172.21.15.22:0/3007381638 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff0500396b0 0x7ff05003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.295+0000 7ff065600700 1 -- 172.21.15.22:0/3007381638 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff05c01e070 con 0x7ff068074df0 2024-04-20T21:24:03.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.295+0000 7ff067e00700 1 --2- 172.21.15.22:0/3007381638 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff0500396b0 0x7ff05003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.296+0000 7ff067e00700 1 --2- 172.21.15.22:0/3007381638 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff0500396b0 0x7ff05003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff054006fd0 tx=0x7ff054006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.301+0000 7ff065600700 1 -- 172.21.15.22:0/3007381638 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff05c013070 con 0x7ff068074df0 2024-04-20T21:24:03.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:03 smithi022 ceph-mon[16843]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:03.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:03.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:03.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:03.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:03.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:24:03.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:03.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:03.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:03.555+0000 7ff06fa00700 1 -- 172.21.15.22:0/3007381638 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi122", "target": ["mon-mgr", ""]}) -- 0x7ff06819f930 con 0x7ff0500396b0 2024-04-20T21:24:04.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:04 smithi022 ceph-mon[16843]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-20T21:24:04.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:04 smithi022 ceph-mon[16843]: Updating smithi033:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:04.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:04 smithi022 ceph-mon[16843]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:24:04.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:04.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:04.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:05.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:05 smithi022 ceph-mon[16843]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:05.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:05 smithi022 ceph-mon[16843]: Updating smithi033:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:24:05.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:05 smithi022 ceph-mon[16843]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi122", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:05.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:05 smithi022 ceph-mon[16843]: Deploying cephadm binary to smithi122 2024-04-20T21:24:07.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:07 smithi022 ceph-mon[16843]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:08.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:08.337+0000 7ff065600700 1 -- 172.21.15.22:0/3007381638 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff06819f930 con 0x7ff0500396b0 2024-04-20T21:24:08.337 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi122' with addr '172.21.15.122' 2024-04-20T21:24:08.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:08.341+0000 7ff06fa00700 1 -- 172.21.15.22:0/3007381638 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff0500396b0 msgr2=0x7ff05003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:08.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:08.341+0000 7ff06fa00700 1 --2- 172.21.15.22:0/3007381638 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff0500396b0 0x7ff05003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff054006fd0 tx=0x7ff054006e40 comp rx=0 tx=0).stop 2024-04-20T21:24:08.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:08.341+0000 7ff06fa00700 1 -- 172.21.15.22:0/3007381638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068074df0 msgr2=0x7ff06819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:08.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:08.341+0000 7ff06fa00700 1 --2- 172.21.15.22:0/3007381638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068074df0 0x7ff06819e5d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff05c006010 tx=0x7ff05c005030 comp rx=0 tx=0).stop 2024-04-20T21:24:08.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:08.341+0000 7ff06fa00700 1 -- 172.21.15.22:0/3007381638 shutdown_connections 2024-04-20T21:24:08.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:08.341+0000 7ff06fa00700 1 --2- 172.21.15.22:0/3007381638 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff0500396b0 0x7ff05003bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:08.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:08.341+0000 7ff06fa00700 1 --2- 172.21.15.22:0/3007381638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068074df0 0x7ff06819e5d0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff05c006010 tx=0x7ff05c005030 comp rx=0 tx=0).stop 2024-04-20T21:24:08.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:08.341+0000 7ff06fa00700 1 -- 172.21.15.22:0/3007381638 >> 172.21.15.22:0/3007381638 conn(0x7ff0680fdf90 msgr2=0x7ff0680ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:08.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:08.342+0000 7ff06fa00700 1 -- 172.21.15.22:0/3007381638 shutdown_connections 2024-04-20T21:24:08.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:08.342+0000 7ff06fa00700 1 -- 172.21.15.22:0/3007381638 wait complete. 2024-04-20T21:24:09.093 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-20T21:24:09.556 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:24:09.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:09 smithi022 ceph-mon[16843]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:09 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:09 smithi022 ceph-mon[16843]: Added host smithi122 2024-04-20T21:24:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:09 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:09 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:10.733 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:10 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:10.733 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:10 smithi022 ceph-mon[16843]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:11.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.667+0000 7fdb12200700 1 -- 172.21.15.22:0/2970450598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb0c10a680 msgr2=0x7fdb0c10aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:11.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.667+0000 7fdb12200700 1 --2- 172.21.15.22:0/2970450598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb0c10a680 0x7fdb0c10aa60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009910 tx=0x7fdafc009c20 comp rx=0 tx=0).stop 2024-04-20T21:24:11.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.667+0000 7fdb12200700 1 -- 172.21.15.22:0/2970450598 shutdown_connections 2024-04-20T21:24:11.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.667+0000 7fdb12200700 1 --2- 172.21.15.22:0/2970450598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb0c10a680 0x7fdb0c10aa60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:11.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.667+0000 7fdb12200700 1 -- 172.21.15.22:0/2970450598 >> 172.21.15.22:0/2970450598 conn(0x7fdb0c100170 msgr2=0x7fdb0c102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:11.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.667+0000 7fdb12200700 1 -- 172.21.15.22:0/2970450598 shutdown_connections 2024-04-20T21:24:11.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.667+0000 7fdb12200700 1 -- 172.21.15.22:0/2970450598 wait complete. 2024-04-20T21:24:11.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.669+0000 7fdb12200700 1 Processor -- start 2024-04-20T21:24:11.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.669+0000 7fdb12200700 1 -- start start 2024-04-20T21:24:11.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.670+0000 7fdb12200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb0c19a360 0x7fdb0c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:11.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.670+0000 7fdb12200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c10ecc0 con 0x7fdb0c19a360 2024-04-20T21:24:11.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.670+0000 7fdb0b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb0c19a360 0x7fdb0c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:11.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.670+0000 7fdb0b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb0c19a360 0x7fdb0c19a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50610/0 (socket says 172.21.15.22:50610) 2024-04-20T21:24:11.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.670+0000 7fdb0b400700 1 -- 172.21.15.22:0/1310321295 learned_addr learned my addr 172.21.15.22:0/1310321295 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:24:11.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.671+0000 7fdb0b400700 1 -- 172.21.15.22:0/1310321295 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0c19ac80 con 0x7fdb0c19a360 2024-04-20T21:24:11.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.671+0000 7fdb0b400700 1 --2- 172.21.15.22:0/1310321295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb0c19a360 0x7fdb0c19a740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdafc005950 tx=0x7fdafc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:11.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.671+0000 7fdb03e00700 1 -- 172.21.15.22:0/1310321295 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdafc026070 con 0x7fdb0c19a360 2024-04-20T21:24:11.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.672+0000 7fdb03e00700 1 -- 172.21.15.22:0/1310321295 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdafc00b6f0 con 0x7fdb0c19a360 2024-04-20T21:24:11.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.672+0000 7fdb12200700 1 -- 172.21.15.22:0/1310321295 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0c19af70 con 0x7fdb0c19a360 2024-04-20T21:24:11.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.672+0000 7fdb03e00700 1 -- 172.21.15.22:0/1310321295 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdafc0038a0 con 0x7fdb0c19a360 2024-04-20T21:24:11.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.672+0000 7fdb12200700 1 -- 172.21.15.22:0/1310321295 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0c19ee00 con 0x7fdb0c19a360 2024-04-20T21:24:11.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.673+0000 7fdb12200700 1 -- 172.21.15.22:0/1310321295 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb0c105dc0 con 0x7fdb0c19a360 2024-04-20T21:24:11.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.673+0000 7fdb03e00700 1 -- 172.21.15.22:0/1310321295 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdafc029070 con 0x7fdb0c19a360 2024-04-20T21:24:11.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.673+0000 7fdb03e00700 1 --2- 172.21.15.22:0/1310321295 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdaf00396b0 0x7fdaf003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:11.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.673+0000 7fdb03e00700 1 -- 172.21.15.22:0/1310321295 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdafc01e070 con 0x7fdb0c19a360 2024-04-20T21:24:11.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.674+0000 7fdb0aa00700 1 --2- 172.21.15.22:0/1310321295 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdaf00396b0 0x7fdaf003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:11.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.674+0000 7fdb0aa00700 1 --2- 172.21.15.22:0/1310321295 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdaf00396b0 0x7fdaf003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8006fd0 tx=0x7fdaf8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:11.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.679+0000 7fdb03e00700 1 -- 172.21.15.22:0/1310321295 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdafc013070 con 0x7fdb0c19a360 2024-04-20T21:24:11.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.913+0000 7fdb12200700 1 -- 172.21.15.22:0/1310321295 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb0c19b730 con 0x7fdaf00396b0 2024-04-20T21:24:11.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.915+0000 7fdb03e00700 1 -- 172.21.15.22:0/1310321295 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fdb0c19b730 con 0x7fdaf00396b0 2024-04-20T21:24:11.914 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:24:11.914 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}] 2024-04-20T21:24:11.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.918+0000 7fdb12200700 1 -- 172.21.15.22:0/1310321295 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdaf00396b0 msgr2=0x7fdaf003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:11.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.918+0000 7fdb12200700 1 --2- 172.21.15.22:0/1310321295 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdaf00396b0 0x7fdaf003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8006fd0 tx=0x7fdaf8006e40 comp rx=0 tx=0).stop 2024-04-20T21:24:11.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.918+0000 7fdb12200700 1 -- 172.21.15.22:0/1310321295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb0c19a360 msgr2=0x7fdb0c19a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:11.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.918+0000 7fdb12200700 1 --2- 172.21.15.22:0/1310321295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb0c19a360 0x7fdb0c19a740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdafc005950 tx=0x7fdafc005030 comp rx=0 tx=0).stop 2024-04-20T21:24:11.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.918+0000 7fdb12200700 1 -- 172.21.15.22:0/1310321295 shutdown_connections 2024-04-20T21:24:11.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.918+0000 7fdb12200700 1 --2- 172.21.15.22:0/1310321295 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdaf00396b0 0x7fdaf003bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:11.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.918+0000 7fdb12200700 1 --2- 172.21.15.22:0/1310321295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb0c19a360 0x7fdb0c19a740 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:11.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.918+0000 7fdb12200700 1 -- 172.21.15.22:0/1310321295 >> 172.21.15.22:0/1310321295 conn(0x7fdb0c100170 msgr2=0x7fdb0c10bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:11.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.919+0000 7fdb12200700 1 -- 172.21.15.22:0/1310321295 shutdown_connections 2024-04-20T21:24:11.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:11.919+0000 7fdb12200700 1 -- 172.21.15.22:0/1310321295 wait complete. 2024-04-20T21:24:12.280 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:12 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:12.629 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-20T21:24:12.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-20T21:24:13.096 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:24:13.379 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:13 smithi022 ceph-mon[16843]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:13.379 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:13 smithi022 ceph-mon[16843]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:24:15.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.053+0000 7f6f69a00700 1 -- 172.21.15.22:0/2750759779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f64069e70 msgr2=0x7f6f6406a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:15.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.053+0000 7f6f69a00700 1 --2- 172.21.15.22:0/2750759779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f64069e70 0x7f6f6406a250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009910 tx=0x7f6f54009c20 comp rx=0 tx=0).stop 2024-04-20T21:24:15.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.053+0000 7f6f69a00700 1 -- 172.21.15.22:0/2750759779 shutdown_connections 2024-04-20T21:24:15.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.053+0000 7f6f69a00700 1 --2- 172.21.15.22:0/2750759779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f64069e70 0x7f6f6406a250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:15.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.053+0000 7f6f69a00700 1 -- 172.21.15.22:0/2750759779 >> 172.21.15.22:0/2750759779 conn(0x7f6f64076eb0 msgr2=0x7f6f640772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:15.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.053+0000 7f6f69a00700 1 -- 172.21.15.22:0/2750759779 shutdown_connections 2024-04-20T21:24:15.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.053+0000 7f6f69a00700 1 -- 172.21.15.22:0/2750759779 wait complete. 2024-04-20T21:24:15.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.054+0000 7f6f69a00700 1 Processor -- start 2024-04-20T21:24:15.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.055+0000 7f6f69a00700 1 -- start start 2024-04-20T21:24:15.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.055+0000 7f6f69a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f64069e70 0x7f6f64103680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:15.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.055+0000 7f6f69a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6410ecf0 con 0x7f6f64069e70 2024-04-20T21:24:15.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.056+0000 7f6f62a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f64069e70 0x7f6f64103680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:15.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.056+0000 7f6f62a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f64069e70 0x7f6f64103680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51644/0 (socket says 172.21.15.22:51644) 2024-04-20T21:24:15.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.056+0000 7f6f62a00700 1 -- 172.21.15.22:0/1248166355 learned_addr learned my addr 172.21.15.22:0/1248166355 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:24:15.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.056+0000 7f6f62a00700 1 -- 172.21.15.22:0/1248166355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f641075e0 con 0x7f6f64069e70 2024-04-20T21:24:15.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.056+0000 7f6f62a00700 1 --2- 172.21.15.22:0/1248166355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f64069e70 0x7f6f64103680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f54006010 tx=0x7f6f54005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:15.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.057+0000 7f6f5b400700 1 -- 172.21.15.22:0/1248166355 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f54026070 con 0x7f6f64069e70 2024-04-20T21:24:15.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.057+0000 7f6f5b400700 1 -- 172.21.15.22:0/1248166355 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5400bc30 con 0x7f6f64069e70 2024-04-20T21:24:15.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.057+0000 7f6f69a00700 1 -- 172.21.15.22:0/1248166355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f64103bc0 con 0x7f6f64069e70 2024-04-20T21:24:15.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.057+0000 7f6f5b400700 1 -- 172.21.15.22:0/1248166355 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f540039d0 con 0x7f6f64069e70 2024-04-20T21:24:15.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.057+0000 7f6f69a00700 1 -- 172.21.15.22:0/1248166355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f64104020 con 0x7f6f64069e70 2024-04-20T21:24:15.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.058+0000 7f6f5b400700 1 -- 172.21.15.22:0/1248166355 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6f54029070 con 0x7f6f64069e70 2024-04-20T21:24:15.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.058+0000 7f6f69a00700 1 -- 172.21.15.22:0/1248166355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f64101980 con 0x7f6f64069e70 2024-04-20T21:24:15.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.059+0000 7f6f5b400700 1 --2- 172.21.15.22:0/1248166355 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f6f480396b0 0x7f6f4803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:15.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.059+0000 7f6f5b400700 1 -- 172.21.15.22:0/1248166355 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f5401e070 con 0x7f6f64069e70 2024-04-20T21:24:15.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.059+0000 7f6f62000700 1 --2- 172.21.15.22:0/1248166355 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f6f480396b0 0x7f6f4803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:15.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.060+0000 7f6f62000700 1 --2- 172.21.15.22:0/1248166355 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f6f480396b0 0x7f6f4803bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6f50006fd0 tx=0x7f6f50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:15.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.065+0000 7f6f5b400700 1 -- 172.21.15.22:0/1248166355 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6f54013070 con 0x7f6f64069e70 2024-04-20T21:24:15.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:15.358+0000 7f6f69a00700 1 -- 172.21.15.22:0/1248166355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f6f640640e0 con 0x7f6f64069e70 2024-04-20T21:24:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:15 smithi022 ceph-mon[16843]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:24:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:15 smithi022 ceph-mon[16843]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-20T21:24:16.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:16.196+0000 7f6f5b400700 1 -- 172.21.15.22:0/1248166355 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f6f640640e0 con 0x7f6f64069e70 2024-04-20T21:24:16.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:16.200+0000 7f6f69a00700 1 -- 172.21.15.22:0/1248166355 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f6f480396b0 msgr2=0x7f6f4803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:16.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:16.200+0000 7f6f69a00700 1 --2- 172.21.15.22:0/1248166355 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f6f480396b0 0x7f6f4803bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6f50006fd0 tx=0x7f6f50006e40 comp rx=0 tx=0).stop 2024-04-20T21:24:16.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:16.200+0000 7f6f69a00700 1 -- 172.21.15.22:0/1248166355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f64069e70 msgr2=0x7f6f64103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:16.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:16.200+0000 7f6f69a00700 1 --2- 172.21.15.22:0/1248166355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f64069e70 0x7f6f64103680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f54006010 tx=0x7f6f54005030 comp rx=0 tx=0).stop 2024-04-20T21:24:16.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:16.200+0000 7f6f69a00700 1 -- 172.21.15.22:0/1248166355 shutdown_connections 2024-04-20T21:24:16.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:16.200+0000 7f6f69a00700 1 --2- 172.21.15.22:0/1248166355 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f6f480396b0 0x7f6f4803bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:16.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:16.200+0000 7f6f69a00700 1 --2- 172.21.15.22:0/1248166355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f64069e70 0x7f6f64103680 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:16.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:16.200+0000 7f6f69a00700 1 -- 172.21.15.22:0/1248166355 >> 172.21.15.22:0/1248166355 conn(0x7f6f64076eb0 msgr2=0x7f6f641099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:16.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:16.201+0000 7f6f69a00700 1 -- 172.21.15.22:0/1248166355 shutdown_connections 2024-04-20T21:24:16.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:16.201+0000 7f6f69a00700 1 -- 172.21.15.22:0/1248166355 wait complete. 2024-04-20T21:24:16.201 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2024-04-20T21:24:16.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:16 smithi022 ceph-mon[16843]: Updating smithi122:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:16.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:16 smithi022 ceph-mon[16843]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:24:16.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:16 smithi022 ceph-mon[16843]: Updating smithi122:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:24:16.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:16 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1248166355' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-20T21:24:16.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:16.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:16.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:17.035 INFO:tasks.cephadm:Adding mon.a on smithi022 2024-04-20T21:24:17.035 INFO:tasks.cephadm:Adding mon.b on smithi033 2024-04-20T21:24:17.035 INFO:tasks.cephadm:Adding mon.c on smithi122 2024-04-20T21:24:17.036 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch apply mon '3;smithi022:172.21.15.22=a;smithi033:172.21.15.33=b;smithi122:172.21.15.122=c' 2024-04-20T21:24:17.273 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:17.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:17 smithi022 ceph-mon[16843]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:17.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:17 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1248166355' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-20T21:24:17.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:17 smithi022 ceph-mon[16843]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:24:18.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.880+0000 7f1af4714700 1 -- 172.21.15.122:0/4235386846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1aec1084f0 msgr2=0x7f1aec1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:18.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.880+0000 7f1af4714700 1 --2- 172.21.15.122:0/4235386846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1aec1084f0 0x7f1aec1088d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009910 tx=0x7f1ae8009c20 comp rx=0 tx=0).stop 2024-04-20T21:24:18.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.880+0000 7f1af4714700 1 -- 172.21.15.122:0/4235386846 shutdown_connections 2024-04-20T21:24:18.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.880+0000 7f1af4714700 1 --2- 172.21.15.122:0/4235386846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1aec1084f0 0x7f1aec1088d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:18.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.880+0000 7f1af4714700 1 -- 172.21.15.122:0/4235386846 >> 172.21.15.122:0/4235386846 conn(0x7f1aec0fdf90 msgr2=0x7f1aec1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:18.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.880+0000 7f1af4714700 1 -- 172.21.15.122:0/4235386846 shutdown_connections 2024-04-20T21:24:18.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.880+0000 7f1af4714700 1 -- 172.21.15.122:0/4235386846 wait complete. 2024-04-20T21:24:18.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.881+0000 7f1af4714700 1 Processor -- start 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.881+0000 7f1af4714700 1 -- start start 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.881+0000 7f1af4714700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1aec1084f0 0x7f1aec197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.881+0000 7f1af4714700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1aec10caa0 con 0x7f1aec1084f0 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.882+0000 7f1af24b0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1aec1084f0 0x7f1aec197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.882+0000 7f1af24b0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1aec1084f0 0x7f1aec197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.122:40808/0 (socket says 172.21.15.122:40808) 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.882+0000 7f1af24b0700 1 -- 172.21.15.122:0/2320951014 learned_addr learned my addr 172.21.15.122:0/2320951014 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.882+0000 7f1af24b0700 1 -- 172.21.15.122:0/2320951014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aec1984e0 con 0x7f1aec1084f0 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.883+0000 7f1af24b0700 1 --2- 172.21.15.122:0/2320951014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1aec1084f0 0x7f1aec197fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8005850 tx=0x7f1ae8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.883+0000 7f1ae37fe700 1 -- 172.21.15.122:0/2320951014 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ae8026070 con 0x7f1aec1084f0 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.883+0000 7f1af4714700 1 -- 172.21.15.122:0/2320951014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aec1987d0 con 0x7f1aec1084f0 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.883+0000 7f1af4714700 1 -- 172.21.15.122:0/2320951014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aec19c870 con 0x7f1aec1084f0 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.883+0000 7f1ae37fe700 1 -- 172.21.15.122:0/2320951014 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae800bbd0 con 0x7f1aec1084f0 2024-04-20T21:24:18.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.883+0000 7f1ae37fe700 1 -- 172.21.15.122:0/2320951014 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ae80039b0 con 0x7f1aec1084f0 2024-04-20T21:24:18.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.884+0000 7f1af4714700 1 -- 172.21.15.122:0/2320951014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad0005330 con 0x7f1aec1084f0 2024-04-20T21:24:18.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.885+0000 7f1ae37fe700 1 -- 172.21.15.122:0/2320951014 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1ae8029070 con 0x7f1aec1084f0 2024-04-20T21:24:18.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.885+0000 7f1ae37fe700 1 --2- 172.21.15.122:0/2320951014 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f1ad8039650 0x7f1ad803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:18.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.885+0000 7f1ae37fe700 1 -- 172.21.15.122:0/2320951014 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1ae801e070 con 0x7f1aec1084f0 2024-04-20T21:24:18.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.886+0000 7f1af1caf700 1 --2- 172.21.15.122:0/2320951014 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f1ad8039650 0x7f1ad803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:18.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.887+0000 7f1af1caf700 1 --2- 172.21.15.122:0/2320951014 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f1ad8039650 0x7f1ad803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1adc006fd0 tx=0x7f1adc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:18.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:18.901+0000 7f1ae37fe700 1 -- 172.21.15.122:0/2320951014 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1ae8013070 con 0x7f1aec1084f0 2024-04-20T21:24:19.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:19.110+0000 7f1af4714700 1 -- 172.21.15.122:0/2320951014 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi033:172.21.15.33=b;smithi122:172.21.15.122=c", "target": ["mon-mgr", ""]}) -- 0x7f1ad0000c80 con 0x7f1ad8039650 2024-04-20T21:24:19.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:19.125+0000 7f1ae37fe700 1 -- 172.21.15.122:0/2320951014 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1ad0000c80 con 0x7f1ad8039650 2024-04-20T21:24:19.124 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled mon update... 2024-04-20T21:24:19.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:19.127+0000 7f1af4714700 1 -- 172.21.15.122:0/2320951014 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f1ad8039650 msgr2=0x7f1ad803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:19.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:19.127+0000 7f1af4714700 1 --2- 172.21.15.122:0/2320951014 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f1ad8039650 0x7f1ad803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1adc006fd0 tx=0x7f1adc006e40 comp rx=0 tx=0).stop 2024-04-20T21:24:19.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:19.127+0000 7f1af4714700 1 -- 172.21.15.122:0/2320951014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1aec1084f0 msgr2=0x7f1aec197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:19.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:19.127+0000 7f1af4714700 1 --2- 172.21.15.122:0/2320951014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1aec1084f0 0x7f1aec197fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8005850 tx=0x7f1ae8005030 comp rx=0 tx=0).stop 2024-04-20T21:24:19.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:19.127+0000 7f1af4714700 1 -- 172.21.15.122:0/2320951014 shutdown_connections 2024-04-20T21:24:19.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:19.127+0000 7f1af4714700 1 --2- 172.21.15.122:0/2320951014 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f1ad8039650 0x7f1ad803bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:19.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:19.127+0000 7f1af4714700 1 --2- 172.21.15.122:0/2320951014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1aec1084f0 0x7f1aec197fa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:19.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:19.127+0000 7f1af4714700 1 -- 172.21.15.122:0/2320951014 >> 172.21.15.122:0/2320951014 conn(0x7f1aec0fdf90 msgr2=0x7f1aec109fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:19.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:19.127+0000 7f1af4714700 1 -- 172.21.15.122:0/2320951014 shutdown_connections 2024-04-20T21:24:19.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:19.127+0000 7f1af4714700 1 -- 172.21.15.122:0/2320951014 wait complete. 2024-04-20T21:24:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:19 smithi022 ceph-mon[16843]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:19.716 DEBUG:teuthology.orchestra.run.smithi033:mon.b> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.b.service 2024-04-20T21:24:19.719 DEBUG:teuthology.orchestra.run.smithi122:mon.c> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.c.service 2024-04-20T21:24:19.721 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-20T21:24:19.721 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-20T21:24:20.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:20 smithi022 ceph-mon[16843]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi033:172.21.15.33=b;smithi122:172.21.15.122=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:20.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:20 smithi022 ceph-mon[16843]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi033:172.21.15.33=b;smithi122:172.21.15.122=c;count:3 2024-04-20T21:24:20.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:20 smithi022 ceph-mon[16843]: Deploying daemon mon.c on smithi122 2024-04-20T21:24:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:21.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:21 smithi022 ceph-mon[16843]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:23.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:23 smithi022 ceph-mon[16843]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:23.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.665+0000 7f4c35616700 1 -- 172.21.15.122:0/3738067145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30106340 msgr2=0x7f4c30106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:23.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.665+0000 7f4c35616700 1 --2- 172.21.15.122:0/3738067145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30106340 0x7f4c30106720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009910 tx=0x7f4c20009c20 comp rx=0 tx=0).stop 2024-04-20T21:24:23.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.665+0000 7f4c35616700 1 -- 172.21.15.122:0/3738067145 shutdown_connections 2024-04-20T21:24:23.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.665+0000 7f4c35616700 1 --2- 172.21.15.122:0/3738067145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30106340 0x7f4c30106720 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:23.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.665+0000 7f4c35616700 1 -- 172.21.15.122:0/3738067145 >> 172.21.15.122:0/3738067145 conn(0x7f4c30077ad0 msgr2=0x7f4c30077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:23.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.665+0000 7f4c35616700 1 -- 172.21.15.122:0/3738067145 shutdown_connections 2024-04-20T21:24:23.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.665+0000 7f4c35616700 1 -- 172.21.15.122:0/3738067145 wait complete. 2024-04-20T21:24:23.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.666+0000 7f4c35616700 1 Processor -- start 2024-04-20T21:24:23.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.666+0000 7f4c35616700 1 -- start start 2024-04-20T21:24:23.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.666+0000 7f4c35616700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30106340 0x7f4c30198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:23.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.666+0000 7f4c35616700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3010bac0 con 0x7f4c30106340 2024-04-20T21:24:23.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.667+0000 7f4c2effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30106340 0x7f4c30198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:23.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.667+0000 7f4c2effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30106340 0x7f4c30198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.122:34810/0 (socket says 172.21.15.122:34810) 2024-04-20T21:24:23.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.667+0000 7f4c2effd700 1 -- 172.21.15.122:0/4189227477 learned_addr learned my addr 172.21.15.122:0/4189227477 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-20T21:24:23.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.668+0000 7f4c2effd700 1 -- 172.21.15.122:0/4189227477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c301985c0 con 0x7f4c30106340 2024-04-20T21:24:23.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.668+0000 7f4c2effd700 1 --2- 172.21.15.122:0/4189227477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30106340 0x7f4c30198080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4c20005850 tx=0x7f4c20005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:23.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.668+0000 7f4c17fff700 1 -- 172.21.15.122:0/4189227477 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c20026070 con 0x7f4c30106340 2024-04-20T21:24:23.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.668+0000 7f4c35616700 1 -- 172.21.15.122:0/4189227477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c30198850 con 0x7f4c30106340 2024-04-20T21:24:23.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.668+0000 7f4c17fff700 1 -- 172.21.15.122:0/4189227477 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c2000bc30 con 0x7f4c30106340 2024-04-20T21:24:23.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.668+0000 7f4c17fff700 1 -- 172.21.15.122:0/4189227477 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c20003a10 con 0x7f4c30106340 2024-04-20T21:24:23.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.668+0000 7f4c35616700 1 -- 172.21.15.122:0/4189227477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c30198cb0 con 0x7f4c30106340 2024-04-20T21:24:23.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.670+0000 7f4c35616700 1 -- 172.21.15.122:0/4189227477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c30102b00 con 0x7f4c30106340 2024-04-20T21:24:23.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.670+0000 7f4c17fff700 1 -- 172.21.15.122:0/4189227477 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4c20029070 con 0x7f4c30106340 2024-04-20T21:24:23.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.670+0000 7f4c17fff700 1 --2- 172.21.15.122:0/4189227477 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4c18039650 0x7f4c1803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:23.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.670+0000 7f4c17fff700 1 -- 172.21.15.122:0/4189227477 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4c2001e070 con 0x7f4c30106340 2024-04-20T21:24:23.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.671+0000 7f4c2e7fc700 1 --2- 172.21.15.122:0/4189227477 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4c18039650 0x7f4c1803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:23.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.672+0000 7f4c2e7fc700 1 --2- 172.21.15.122:0/4189227477 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4c18039650 0x7f4c1803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006fd0 tx=0x7f4c24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:23.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.685+0000 7f4c17fff700 1 -- 172.21.15.122:0/4189227477 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4c20013070 con 0x7f4c30106340 2024-04-20T21:24:23.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.981+0000 7f4c35616700 1 -- 172.21.15.122:0/4189227477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4c301993e0 con 0x7f4c30106340 2024-04-20T21:24:23.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.982+0000 7f4c17fff700 1 -- 172.21.15.122:0/4189227477 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f4c20028d80 con 0x7f4c30106340 2024-04-20T21:24:23.982 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:24:23.982 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":1,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","modified":"2024-04-20T21:22:09.923834Z","created":"2024-04-20T21:22:09.923834Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T21:24:23.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.985+0000 7f4c35616700 1 -- 172.21.15.122:0/4189227477 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4c18039650 msgr2=0x7f4c1803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:23.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.985+0000 7f4c35616700 1 --2- 172.21.15.122:0/4189227477 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4c18039650 0x7f4c1803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006fd0 tx=0x7f4c24006e40 comp rx=0 tx=0).stop 2024-04-20T21:24:23.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.985+0000 7f4c35616700 1 -- 172.21.15.122:0/4189227477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30106340 msgr2=0x7f4c30198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:23.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.985+0000 7f4c35616700 1 --2- 172.21.15.122:0/4189227477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30106340 0x7f4c30198080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4c20005850 tx=0x7f4c20005030 comp rx=0 tx=0).stop 2024-04-20T21:24:23.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.985+0000 7f4c35616700 1 -- 172.21.15.122:0/4189227477 shutdown_connections 2024-04-20T21:24:23.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.985+0000 7f4c35616700 1 --2- 172.21.15.122:0/4189227477 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4c18039650 0x7f4c1803bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:23.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.985+0000 7f4c35616700 1 --2- 172.21.15.122:0/4189227477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c30106340 0x7f4c30198080 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:23.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.985+0000 7f4c35616700 1 -- 172.21.15.122:0/4189227477 >> 172.21.15.122:0/4189227477 conn(0x7f4c30077ad0 msgr2=0x7f4c300fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:23.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.985+0000 7f4c35616700 1 -- 172.21.15.122:0/4189227477 shutdown_connections 2024-04-20T21:24:23.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:23.985+0000 7f4c35616700 1 -- 172.21.15.122:0/4189227477 wait complete. 2024-04-20T21:24:23.985 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 1 2024-04-20T21:24:25.757 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-20T21:24:25.757 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-20T21:24:26.009 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:24:29.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:29.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:24:29.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: mon.a calling monitor election 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: mon.c calling monitor election 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: monmap epoch 2 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: last_changed 2024-04-20T21:24:24.065673+0000 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: created 2024-04-20T21:22:09.923834+0000 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: min_mon_release 19 (squid) 2024-04-20T21:24:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: election_strategy: 1 2024-04-20T21:24:29.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-04-20T21:24:29.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-20T21:24:29.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: fsmap 2024-04-20T21:24:29.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:24:29.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: mgrmap e13: x(active, since 59s) 2024-04-20T21:24:29.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: overall HEALTH_OK 2024-04-20T21:24:29.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:29.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:30.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.506+0000 7f28453d1700 1 -- 172.21.15.122:0/2906950491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28280035c0 msgr2=0x7f2828005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:30.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.506+0000 7f28453d1700 1 --2- 172.21.15.122:0/2906950491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28280035c0 0x7f2828005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f284006acc0 tx=0x7f283000feb0 comp rx=0 tx=0).stop 2024-04-20T21:24:30.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.506+0000 7f28453d1700 1 -- 172.21.15.122:0/2906950491 shutdown_connections 2024-04-20T21:24:30.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.506+0000 7f28453d1700 1 --2- 172.21.15.122:0/2906950491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28280035c0 0x7f2828005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:30.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.506+0000 7f28453d1700 1 -- 172.21.15.122:0/2906950491 >> 172.21.15.122:0/2906950491 conn(0x7f28400fcf90 msgr2=0x7f28400ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:30.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.506+0000 7f28453d1700 1 -- 172.21.15.122:0/2906950491 shutdown_connections 2024-04-20T21:24:30.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.506+0000 7f28453d1700 1 -- 172.21.15.122:0/2906950491 wait complete. 2024-04-20T21:24:30.506 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:24:30.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.507+0000 7f28453d1700 1 Processor -- start 2024-04-20T21:24:30.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.507+0000 7f28453d1700 1 -- start start 2024-04-20T21:24:30.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.507+0000 7f28453d1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28280035c0 0x7f2840198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:30.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.507+0000 7f28453d1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28401986c0 0x7f284019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:30.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.507+0000 7f28453d1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f284010a620 con 0x7f28401986c0 2024-04-20T21:24:30.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.507+0000 7f28453d1700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f284010a4a0 con 0x7f28280035c0 2024-04-20T21:24:30.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.508+0000 7f283effd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28280035c0 0x7f2840198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:30.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.508+0000 7f283effd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28280035c0 0x7f2840198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50102/0 (socket says 172.21.15.122:50102) 2024-04-20T21:24:30.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.508+0000 7f283effd700 1 -- 172.21.15.122:0/1720159782 learned_addr learned my addr 172.21.15.122:0/1720159782 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-20T21:24:30.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.508+0000 7f283effd700 1 -- 172.21.15.122:0/1720159782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28280035c0 msgr2=0x7f2840198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-20T21:24:30.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.508+0000 7f283effd700 1 -- 172.21.15.122:0/1720159782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28280035c0 msgr2=0x7f2840198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T21:24:30.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.508+0000 7f283effd700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28280035c0 0x7f2840198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T21:24:30.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.508+0000 7f283effd700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28280035c0 0x7f2840198180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:24:30.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.508+0000 7f283e7fc700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28401986c0 0x7f284019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:30.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.509+0000 7f283e7fc700 1 -- 172.21.15.122:0/1720159782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28280035c0 msgr2=0x7f2840198180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:24:30.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.509+0000 7f283e7fc700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28280035c0 0x7f2840198180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:30.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.509+0000 7f283e7fc700 1 -- 172.21.15.122:0/1720159782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2840198d00 con 0x7f28401986c0 2024-04-20T21:24:30.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.509+0000 7f283e7fc700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28401986c0 0x7f284019ca50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f283400b4d0 tx=0x7f283400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:30.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.509+0000 7f2827fff700 1 -- 172.21.15.122:0/1720159782 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2834011630 con 0x7f28401986c0 2024-04-20T21:24:30.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.509+0000 7f28453d1700 1 -- 172.21.15.122:0/1720159782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284019cf90 con 0x7f28401986c0 2024-04-20T21:24:30.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.510+0000 7f28453d1700 1 -- 172.21.15.122:0/1720159782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284019d4f0 con 0x7f28401986c0 2024-04-20T21:24:30.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.510+0000 7f2827fff700 1 -- 172.21.15.122:0/1720159782 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2834011cb0 con 0x7f28401986c0 2024-04-20T21:24:30.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.510+0000 7f2827fff700 1 -- 172.21.15.122:0/1720159782 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f283401ae20 con 0x7f28401986c0 2024-04-20T21:24:30.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.511+0000 7f28453d1700 1 -- 172.21.15.122:0/1720159782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284006b4c0 con 0x7f28401986c0 2024-04-20T21:24:30.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.511+0000 7f2827fff700 1 -- 172.21.15.122:0/1720159782 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2834018020 con 0x7f28401986c0 2024-04-20T21:24:30.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.511+0000 7f2827fff700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f282c0393f0 0x7f282c03b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:30.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.511+0000 7f2827fff700 1 -- 172.21.15.122:0/1720159782 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f283404c260 con 0x7f28401986c0 2024-04-20T21:24:30.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.512+0000 7f283effd700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f282c0393f0 0x7f282c03b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:30.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.513+0000 7f283effd700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f282c0393f0 0x7f282c03b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f283000fa60 tx=0x7f2830005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:30.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.526+0000 7f2827fff700 1 -- 172.21.15.122:0/1720159782 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f283401f020 con 0x7f28401986c0 2024-04-20T21:24:30.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:30.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:30.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:30.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:30 smithi022 ceph-mon[16843]: Deploying daemon mon.b on smithi033 2024-04-20T21:24:30.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:24:30.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.812+0000 7f28453d1700 1 -- 172.21.15.122:0/1720159782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2840199570 con 0x7f28401986c0 2024-04-20T21:24:30.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.813+0000 7f2827fff700 1 -- 172.21.15.122:0/1720159782 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f2840199570 con 0x7f28401986c0 2024-04-20T21:24:30.815 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:24:30.815 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":2,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","modified":"2024-04-20T21:24:24.065673Z","created":"2024-04-20T21:22:09.923834Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-20T21:24:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.815+0000 7f28453d1700 1 -- 172.21.15.122:0/1720159782 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f282c0393f0 msgr2=0x7f282c03b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.815+0000 7f28453d1700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f282c0393f0 0x7f282c03b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f283000fa60 tx=0x7f2830005960 comp rx=0 tx=0).stop 2024-04-20T21:24:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.815+0000 7f28453d1700 1 -- 172.21.15.122:0/1720159782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28401986c0 msgr2=0x7f284019ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.815+0000 7f28453d1700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28401986c0 0x7f284019ca50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f283400b4d0 tx=0x7f283400b890 comp rx=0 tx=0).stop 2024-04-20T21:24:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.815+0000 7f28453d1700 1 -- 172.21.15.122:0/1720159782 shutdown_connections 2024-04-20T21:24:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.815+0000 7f28453d1700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f282c0393f0 0x7f282c03b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.815+0000 7f28453d1700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28280035c0 0x7f2840198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.815+0000 7f28453d1700 1 --2- 172.21.15.122:0/1720159782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28401986c0 0x7f284019ca50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.815+0000 7f28453d1700 1 -- 172.21.15.122:0/1720159782 >> 172.21.15.122:0/1720159782 conn(0x7f28400fcf90 msgr2=0x7f28400fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.815+0000 7f28453d1700 1 -- 172.21.15.122:0/1720159782 shutdown_connections 2024-04-20T21:24:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:30.815+0000 7f28453d1700 1 -- 172.21.15.122:0/1720159782 wait complete. 2024-04-20T21:24:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 2 2024-04-20T21:24:31.337 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:31 smithi022 ceph-mon[16843]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:31.338 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:31 smithi022 ceph-mon[16843]: from='client.? 172.21.15.122:0/1720159782' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:24:31.338 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:24:31 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:24:31.066+0000 7f0751800700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-20T21:24:31.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:31 smithi122 ceph-mon[22147]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:31.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:31 smithi122 ceph-mon[22147]: from='client.? 172.21.15.122:0/1720159782' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:24:32.404 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-20T21:24:32.404 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-20T21:24:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:24:32.953 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:32 smithi033 ceph-mon[21626]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-20T21:24:33.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:32 smithi033 bash[21607]: 2ea070e847cc8cf1e26fdec595c3d8ebac83f31882853d8eb0460bec2694ee81 2024-04-20T21:24:33.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:32 smithi033 systemd[1]: Started Ceph mon.b for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:24:38.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: mon.c calling monitor election 2024-04-20T21:24:38.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:24:38.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: mon.a calling monitor election 2024-04-20T21:24:38.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:38.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: monmap epoch 3 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: last_changed 2024-04-20T21:24:32.945232+0000 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: created 2024-04-20T21:22:09.923834+0000 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: min_mon_release 19 (squid) 2024-04-20T21:24:38.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: election_strategy: 1 2024-04-20T21:24:38.277 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-04-20T21:24:38.277 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-20T21:24:38.277 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: 2: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-04-20T21:24:38.277 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: fsmap 2024-04-20T21:24:38.277 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:24:38.277 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: mgrmap e13: x(active, since 68s) 2024-04-20T21:24:38.277 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: overall HEALTH_OK 2024-04-20T21:24:38.277 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:38.277 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:38.277 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:38.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: mon.c calling monitor election 2024-04-20T21:24:38.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:24:38.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: mon.a calling monitor election 2024-04-20T21:24:38.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:38.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:24:38.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:38.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: monmap epoch 3 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: last_changed 2024-04-20T21:24:32.945232+0000 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: created 2024-04-20T21:22:09.923834+0000 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: min_mon_release 19 (squid) 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: election_strategy: 1 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: 2: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-04-20T21:24:38.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: fsmap 2024-04-20T21:24:38.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:24:38.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: mgrmap e13: x(active, since 68s) 2024-04-20T21:24:38.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: overall HEALTH_OK 2024-04-20T21:24:38.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:38.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:38.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:38.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.567+0000 7f8337c72700 1 -- 172.21.15.122:0/1787148483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f831c0035c0 msgr2=0x7f831c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.567+0000 7f8337c72700 1 --2- 172.21.15.122:0/1787148483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f831c0035c0 0x7f831c005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f83301024c0 tx=0x7f832000fe60 comp rx=0 tx=0).stop 2024-04-20T21:24:38.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.567+0000 7f8337c72700 1 -- 172.21.15.122:0/1787148483 shutdown_connections 2024-04-20T21:24:38.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.567+0000 7f8337c72700 1 --2- 172.21.15.122:0/1787148483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f831c0035c0 0x7f831c005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.567+0000 7f8337c72700 1 -- 172.21.15.122:0/1787148483 >> 172.21.15.122:0/1787148483 conn(0x7f83300fe040 msgr2=0x7f8330100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:38.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.567+0000 7f8337c72700 1 -- 172.21.15.122:0/1787148483 shutdown_connections 2024-04-20T21:24:38.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.568+0000 7f8337c72700 1 -- 172.21.15.122:0/1787148483 wait complete. 2024-04-20T21:24:38.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.568+0000 7f8337c72700 1 Processor -- start 2024-04-20T21:24:38.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.568+0000 7f8337c72700 1 -- start start 2024-04-20T21:24:38.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.568+0000 7f8337c72700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f831c0035c0 0x7f8330198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.568+0000 7f8337c72700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8330198750 0x7f833019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.568+0000 7f8337c72700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f833019cfd0 0x7f833019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.568+0000 7f8337c72700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f833010a5e0 con 0x7f831c0035c0 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.568+0000 7f8337c72700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f833010a460 con 0x7f833019cfd0 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.568+0000 7f8337c72700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f833010a760 con 0x7f8330198750 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f833520d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8330198750 0x7f833019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f833520d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8330198750 0x7f833019ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36802/0 (socket says 172.21.15.122:36802) 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f833520d700 1 -- 172.21.15.122:0/2235023461 learned_addr learned my addr 172.21.15.122:0/2235023461 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f833520d700 1 -- 172.21.15.122:0/2235023461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8330198750 msgr2=0x7f833019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f833520d700 1 -- 172.21.15.122:0/2235023461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8330198750 msgr2=0x7f833019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f833520d700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8330198750 0x7f833019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f833620f700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f833019cfd0 0x7f833019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f833520d700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8330198750 0x7f833019ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f8335a0e700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f831c0035c0 0x7f8330198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f833620f700 1 -- 172.21.15.122:0/2235023461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f833019cfd0 msgr2=0x7f833019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-20T21:24:38.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f833620f700 1 -- 172.21.15.122:0/2235023461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f833019cfd0 msgr2=0x7f833019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f833620f700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f833019cfd0 0x7f833019d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f833620f700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f833019cfd0 0x7f833019d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f8335a0e700 1 -- 172.21.15.122:0/2235023461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8330198750 msgr2=0x7f833019ca90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f8335a0e700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8330198750 0x7f833019ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f8335a0e700 1 -- 172.21.15.122:0/2235023461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f833019cfd0 msgr2=0x7f833019d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f8335a0e700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f833019cfd0 0x7f833019d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f8335a0e700 1 -- 172.21.15.122:0/2235023461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83301a4d40 con 0x7f831c0035c0 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.569+0000 7f8335a0e700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f831c0035c0 0x7f8330198210 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f832001f000 tx=0x7f8320005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.570+0000 7f8326ffd700 1 -- 172.21.15.122:0/2235023461 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832002b020 con 0x7f831c0035c0 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.570+0000 7f8326ffd700 1 -- 172.21.15.122:0/2235023461 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8320004110 con 0x7f831c0035c0 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.570+0000 7f8326ffd700 1 -- 172.21.15.122:0/2235023461 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320022040 con 0x7f831c0035c0 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.570+0000 7f8337c72700 1 -- 172.21.15.122:0/2235023461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83301a4f10 con 0x7f831c0035c0 2024-04-20T21:24:38.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.570+0000 7f8337c72700 1 -- 172.21.15.122:0/2235023461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83301a5440 con 0x7f831c0035c0 2024-04-20T21:24:38.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.571+0000 7f8337c72700 1 -- 172.21.15.122:0/2235023461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83300640b0 con 0x7f831c0035c0 2024-04-20T21:24:38.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.571+0000 7f8326ffd700 1 -- 172.21.15.122:0/2235023461 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f832002e020 con 0x7f831c0035c0 2024-04-20T21:24:38.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.571+0000 7f8326ffd700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8314039ae0 0x7f831403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.571+0000 7f8326ffd700 1 -- 172.21.15.122:0/2235023461 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f832004f240 con 0x7f831c0035c0 2024-04-20T21:24:38.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.572+0000 7f833520d700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8314039ae0 0x7f831403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.572+0000 7f833520d700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8314039ae0 0x7f831403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f832c009180 tx=0x7f832c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:38.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.579+0000 7f8326ffd700 1 -- 172.21.15.122:0/2235023461 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8320035020 con 0x7f831c0035c0 2024-04-20T21:24:38.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.871+0000 7f8337c72700 1 -- 172.21.15.122:0/2235023461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f83301a56a0 con 0x7f831c0035c0 2024-04-20T21:24:38.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.872+0000 7f8326ffd700 1 -- 172.21.15.122:0/2235023461 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f8320038350 con 0x7f831c0035c0 2024-04-20T21:24:38.871 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-20T21:24:38.871 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":3,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","modified":"2024-04-20T21:24:32.945232Z","created":"2024-04-20T21:22:09.923834Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-20T21:24:38.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.875+0000 7f8337c72700 1 -- 172.21.15.122:0/2235023461 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8314039ae0 msgr2=0x7f831403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.875+0000 7f8337c72700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8314039ae0 0x7f831403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f832c009180 tx=0x7f832c008040 comp rx=0 tx=0).stop 2024-04-20T21:24:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.875+0000 7f8337c72700 1 -- 172.21.15.122:0/2235023461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f831c0035c0 msgr2=0x7f8330198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.875+0000 7f8337c72700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f831c0035c0 0x7f8330198210 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f832001f000 tx=0x7f8320005be0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.875+0000 7f8337c72700 1 -- 172.21.15.122:0/2235023461 shutdown_connections 2024-04-20T21:24:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.875+0000 7f8337c72700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8314039ae0 0x7f831403bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.875+0000 7f8337c72700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f833019cfd0 0x7f833019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.875+0000 7f8337c72700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f831c0035c0 0x7f8330198210 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.875+0000 7f8337c72700 1 --2- 172.21.15.122:0/2235023461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8330198750 0x7f833019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.875+0000 7f8337c72700 1 -- 172.21.15.122:0/2235023461 >> 172.21.15.122:0/2235023461 conn(0x7f83300fe040 msgr2=0x7f83300ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.875+0000 7f8337c72700 1 -- 172.21.15.122:0/2235023461 shutdown_connections 2024-04-20T21:24:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:38.875+0000 7f8337c72700 1 -- 172.21.15.122:0/2235023461 wait complete. 2024-04-20T21:24:38.876 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 3 2024-04-20T21:24:39.186 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:39 smithi122 ceph-mon[22147]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:39.187 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:39.187 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:39.187 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:39 smithi122 ceph-mon[22147]: from='client.? 172.21.15.122:0/2235023461' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:24:39.187 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:39.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:39 smithi022 ceph-mon[16843]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:39.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:39.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:39.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:39 smithi022 ceph-mon[16843]: from='client.? 172.21.15.122:0/2235023461' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:24:39.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:39.429 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-20T21:24:39.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-20T21:24:39.877 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:24:40.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: mon.c calling monitor election 2024-04-20T21:24:40.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:24:40.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: mon.a calling monitor election 2024-04-20T21:24:40.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:40.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:24:40.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:40.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:40.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:40.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: monmap epoch 3 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: last_changed 2024-04-20T21:24:32.945232+0000 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: created 2024-04-20T21:22:09.923834+0000 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: min_mon_release 19 (squid) 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: election_strategy: 1 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: 2: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: fsmap 2024-04-20T21:24:40.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:24:40.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: mgrmap e13: x(active, since 68s) 2024-04-20T21:24:40.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: overall HEALTH_OK 2024-04-20T21:24:40.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:40.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:40.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:40.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:40.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:40.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:40.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='client.? 172.21.15.122:0/2235023461' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:24:40.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:41.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: mon.b calling monitor election 2024-04-20T21:24:41.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:41.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: mon.b calling monitor election 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: mon.c calling monitor election 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: mon.a calling monitor election 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: monmap epoch 3 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: last_changed 2024-04-20T21:24:32.945232+0000 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: created 2024-04-20T21:22:09.923834+0000 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: min_mon_release 19 (squid) 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: election_strategy: 1 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: 2: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-04-20T21:24:41.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: fsmap 2024-04-20T21:24:41.027 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:24:41.027 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: mgrmap e13: x(active, since 70s) 2024-04-20T21:24:41.027 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: overall HEALTH_OK 2024-04-20T21:24:41.027 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.027 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.027 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:41.027 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:41.027 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-20T21:24:41.027 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-20T21:24:41.027 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-20T21:24:41.027 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:40 smithi122 ceph-mon[22147]: Updating smithi122:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:41.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: mon.b calling monitor election 2024-04-20T21:24:41.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:41.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: mon.b calling monitor election 2024-04-20T21:24:41.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: mon.c calling monitor election 2024-04-20T21:24:41.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: mon.a calling monitor election 2024-04-20T21:24:41.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-20T21:24:41.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: monmap epoch 3 2024-04-20T21:24:41.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:24:41.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: last_changed 2024-04-20T21:24:32.945232+0000 2024-04-20T21:24:41.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: created 2024-04-20T21:22:09.923834+0000 2024-04-20T21:24:41.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: min_mon_release 19 (squid) 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: election_strategy: 1 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: 2: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: fsmap 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: mgrmap e13: x(active, since 70s) 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: overall HEALTH_OK 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-20T21:24:41.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-20T21:24:41.056 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-20T21:24:41.056 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:40 smithi033 ceph-mon[21626]: Updating smithi122:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: mon.b calling monitor election 2024-04-20T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: mon.b calling monitor election 2024-04-20T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: mon.c calling monitor election 2024-04-20T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: mon.a calling monitor election 2024-04-20T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-20T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: monmap epoch 3 2024-04-20T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:24:41.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: last_changed 2024-04-20T21:24:32.945232+0000 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: created 2024-04-20T21:22:09.923834+0000 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: min_mon_release 19 (squid) 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: election_strategy: 1 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: 0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: 2: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: fsmap 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: mgrmap e13: x(active, since 70s) 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: overall HEALTH_OK 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:41.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:41.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-20T21:24:41.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-20T21:24:41.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-20T21:24:41.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:40 smithi022 ceph-mon[16843]: Updating smithi122:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:41.792 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:41 smithi022 ceph-mon[16843]: Updating smithi033:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:41.792 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.792 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:41 smithi022 ceph-mon[16843]: Updating smithi022:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:41.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:41.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.980+0000 7fbddc200700 1 -- 172.21.15.22:0/416032927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdd4074e20 msgr2=0x7fbdd4075200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.980+0000 7fbddc200700 1 --2- 172.21.15.22:0/416032927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdd4074e20 0x7fbdd4075200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009910 tx=0x7fbdc8009c20 comp rx=0 tx=0).stop 2024-04-20T21:24:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.980+0000 7fbddc200700 1 -- 172.21.15.22:0/416032927 shutdown_connections 2024-04-20T21:24:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.980+0000 7fbddc200700 1 --2- 172.21.15.22:0/416032927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdd4074e20 0x7fbdd4075200 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.980+0000 7fbddc200700 1 -- 172.21.15.22:0/416032927 >> 172.21.15.22:0/416032927 conn(0x7fbdd40fdf90 msgr2=0x7fbdd41003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.981+0000 7fbddc200700 1 -- 172.21.15.22:0/416032927 shutdown_connections 2024-04-20T21:24:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.981+0000 7fbddc200700 1 -- 172.21.15.22:0/416032927 wait complete. 2024-04-20T21:24:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.982+0000 7fbddc200700 1 Processor -- start 2024-04-20T21:24:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.982+0000 7fbddc200700 1 -- start start 2024-04-20T21:24:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.982+0000 7fbddc200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd4074e20 0x7fbdd419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.982+0000 7fbddc200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdd419a950 0x7fbdd419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.982+0000 7fbddc200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdd419f220 0x7fbdd419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.982+0000 7fbddc200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd410ed00 con 0x7fbdd419f220 2024-04-20T21:24:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.982+0000 7fbddc200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd410eb80 con 0x7fbdd4074e20 2024-04-20T21:24:41.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.982+0000 7fbddc200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd410ee80 con 0x7fbdd419a950 2024-04-20T21:24:41.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.983+0000 7fbdd9800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd4074e20 0x7fbdd419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.983+0000 7fbdd9800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd4074e20 0x7fbdd419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60536/0 (socket says 172.21.15.22:60536) 2024-04-20T21:24:41.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.983+0000 7fbdd9800700 1 -- 172.21.15.22:0/57356188 learned_addr learned my addr 172.21.15.22:0/57356188 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:24:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.983+0000 7fbdd8e00700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdd419a950 0x7fbdd419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.983+0000 7fbdda200700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdd419f220 0x7fbdd419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.983+0000 7fbdd8e00700 1 -- 172.21.15.22:0/57356188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd4074e20 msgr2=0x7fbdd419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.983+0000 7fbdd8e00700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd4074e20 0x7fbdd419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.983+0000 7fbdd8e00700 1 -- 172.21.15.22:0/57356188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdd419f220 msgr2=0x7fbdd419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.983+0000 7fbdd8e00700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdd419f220 0x7fbdd419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.983+0000 7fbdd8e00700 1 -- 172.21.15.22:0/57356188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd41a6f90 con 0x7fbdd419a950 2024-04-20T21:24:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.983+0000 7fbdd9800700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd4074e20 0x7fbdd419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:24:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.984+0000 7fbdd8e00700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdd419a950 0x7fbdd419ece0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000b550 tx=0x7fbdc000b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.984+0000 7fbdd2000700 1 -- 172.21.15.22:0/57356188 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0004230 con 0x7fbdd419a950 2024-04-20T21:24:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.984+0000 7fbdd2000700 1 -- 172.21.15.22:0/57356188 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc00043d0 con 0x7fbdd419a950 2024-04-20T21:24:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.984+0000 7fbddc200700 1 -- 172.21.15.22:0/57356188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd41a71c0 con 0x7fbdd419a950 2024-04-20T21:24:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.984+0000 7fbdd2000700 1 -- 172.21.15.22:0/57356188 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0011b20 con 0x7fbdd419a950 2024-04-20T21:24:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.984+0000 7fbddc200700 1 -- 172.21.15.22:0/57356188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd41a7750 con 0x7fbdd419a950 2024-04-20T21:24:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.985+0000 7fbddc200700 1 -- 172.21.15.22:0/57356188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd40640b0 con 0x7fbdd419a950 2024-04-20T21:24:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.986+0000 7fbdd2000700 1 -- 172.21.15.22:0/57356188 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbdc0011d00 con 0x7fbdd419a950 2024-04-20T21:24:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.986+0000 7fbdd2000700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fbdbc039ae0 0x7fbdbc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.986+0000 7fbdd2000700 1 -- 172.21.15.22:0/57356188 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbdc004c370 con 0x7fbdd419a950 2024-04-20T21:24:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.986+0000 7fbdd9800700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fbdbc039ae0 0x7fbdbc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.987+0000 7fbdd9800700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fbdbc039ae0 0x7fbdbc03bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009750 tx=0x7fbdc800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:41.992+0000 7fbdd2000700 1 -- 172.21.15.22:0/57356188 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbdc0020500 con 0x7fbdd419a950 2024-04-20T21:24:42.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:41 smithi122 ceph-mon[22147]: Updating smithi033:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:42.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:41 smithi122 ceph-mon[22147]: Updating smithi022:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:42.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:42.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:41 smithi033 ceph-mon[21626]: Updating smithi033:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:42.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:41 smithi033 ceph-mon[21626]: Updating smithi022:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:42.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:42.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:42.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.234+0000 7fbddc200700 1 -- 172.21.15.22:0/57356188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fbdd41a7a10 con 0x7fbdd419a950 2024-04-20T21:24:42.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.235+0000 7fbdd2000700 1 -- 172.21.15.22:0/57356188 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fbdc00202a0 con 0x7fbdd419a950 2024-04-20T21:24:42.235 INFO:teuthology.orchestra.run.smithi022.stdout:# minimal ceph.conf for 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:24:42.235 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-04-20T21:24:42.235 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:24:42.235 INFO:teuthology.orchestra.run.smithi022.stdout: mon_host = [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] 2024-04-20T21:24:42.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.238+0000 7fbddc200700 1 -- 172.21.15.22:0/57356188 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fbdbc039ae0 msgr2=0x7fbdbc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.238+0000 7fbddc200700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fbdbc039ae0 0x7fbdbc03bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009750 tx=0x7fbdc800b540 comp rx=0 tx=0).stop 2024-04-20T21:24:42.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.238+0000 7fbddc200700 1 -- 172.21.15.22:0/57356188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdd419a950 msgr2=0x7fbdd419ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.238+0000 7fbddc200700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdd419a950 0x7fbdd419ece0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000b550 tx=0x7fbdc000b860 comp rx=0 tx=0).stop 2024-04-20T21:24:42.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.238+0000 7fbddc200700 1 -- 172.21.15.22:0/57356188 shutdown_connections 2024-04-20T21:24:42.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.238+0000 7fbddc200700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fbdbc039ae0 0x7fbdbc03bfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.238+0000 7fbddc200700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdd419f220 0x7fbdd419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.238+0000 7fbddc200700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd4074e20 0x7fbdd419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.238+0000 7fbddc200700 1 --2- 172.21.15.22:0/57356188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdd419a950 0x7fbdd419ece0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.238+0000 7fbddc200700 1 -- 172.21.15.22:0/57356188 >> 172.21.15.22:0/57356188 conn(0x7fbdd40fdf90 msgr2=0x7fbdd41068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:42.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.238+0000 7fbddc200700 1 -- 172.21.15.22:0/57356188 shutdown_connections 2024-04-20T21:24:42.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:42.238+0000 7fbddc200700 1 -- 172.21.15.22:0/57356188 wait complete. 2024-04-20T21:24:43.014 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-20T21:24:43.014 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:24:43.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T21:24:43.104 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:42 smithi022 ceph-mon[16843]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:43.104 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:43.104 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:43.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:43.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:43.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:42 smithi022 ceph-mon[16843]: Reconfiguring mon.a (unknown last config time)... 2024-04-20T21:24:43.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:43.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:43.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:43.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:42 smithi022 ceph-mon[16843]: Reconfiguring daemon mon.a on smithi022 2024-04-20T21:24:43.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:42 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/57356188' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:43.107 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:24:43.107 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:24:43.196 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:42 smithi122 ceph-mon[22147]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:43.196 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:43.196 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:43.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:43.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:43.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:42 smithi122 ceph-mon[22147]: Reconfiguring mon.a (unknown last config time)... 2024-04-20T21:24:43.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:43.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:43.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:43.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:42 smithi122 ceph-mon[22147]: Reconfiguring daemon mon.a on smithi022 2024-04-20T21:24:43.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:42 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/57356188' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:43.240 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-20T21:24:43.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T21:24:43.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:42 smithi033 ceph-mon[21626]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:43.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:43.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:43.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:43.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:43.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:42 smithi033 ceph-mon[21626]: Reconfiguring mon.a (unknown last config time)... 2024-04-20T21:24:43.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:43.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:43.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:43.269 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:42 smithi033 ceph-mon[21626]: Reconfiguring daemon mon.a on smithi022 2024-04-20T21:24:43.269 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:42 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/57356188' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:43.273 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-20T21:24:43.273 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:24:43.343 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-20T21:24:43.343 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T21:24:43.372 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-20T21:24:43.372 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:24:43.437 INFO:tasks.cephadm:Adding mgr.x on smithi022 2024-04-20T21:24:43.437 INFO:tasks.cephadm:Adding mgr.y on smithi033 2024-04-20T21:24:43.437 INFO:tasks.cephadm:Adding mgr.z on smithi122 2024-04-20T21:24:43.438 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi022=x;smithi033=y;smithi122=z' 2024-04-20T21:24:43.721 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:24:45.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.267+0000 7ff081c61700 1 -- 172.21.15.122:0/2644500543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff07c103080 msgr2=0x7ff07c105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.267+0000 7ff081c61700 1 --2- 172.21.15.122:0/2644500543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff07c103080 0x7ff07c105470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009910 tx=0x7ff06c009c20 comp rx=0 tx=0).stop 2024-04-20T21:24:45.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.267+0000 7ff081c61700 1 -- 172.21.15.122:0/2644500543 shutdown_connections 2024-04-20T21:24:45.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.267+0000 7ff081c61700 1 --2- 172.21.15.122:0/2644500543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff07c103080 0x7ff07c105470 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.267+0000 7ff081c61700 1 -- 172.21.15.122:0/2644500543 >> 172.21.15.122:0/2644500543 conn(0x7ff07c0fca90 msgr2=0x7ff07c0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:45.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.267+0000 7ff081c61700 1 -- 172.21.15.122:0/2644500543 shutdown_connections 2024-04-20T21:24:45.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.267+0000 7ff081c61700 1 -- 172.21.15.122:0/2644500543 wait complete. 2024-04-20T21:24:45.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.267+0000 7ff081c61700 1 Processor -- start 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.267+0000 7ff081c61700 1 -- start start 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff081c61700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff07c103080 0x7ff07c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff081c61700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff07c198310 0x7ff07c19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff081c61700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff07c198790 0x7ff07c19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff081c61700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c108620 con 0x7ff07c103080 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff081c61700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1084a0 con 0x7ff07c198790 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff081c61700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1087a0 con 0x7ff07c198310 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff07affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff07c198310 0x7ff07c19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff07affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff07c198310 0x7ff07c19d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52420/0 (socket says 172.21.15.122:52420) 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff07affd700 1 -- 172.21.15.122:0/3326920472 learned_addr learned my addr 172.21.15.122:0/3326920472 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff07bfff700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff07c198790 0x7ff07c19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff07b7fe700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff07c103080 0x7ff07c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff07affd700 1 -- 172.21.15.122:0/3326920472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff07c198790 msgr2=0x7ff07c19f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff07affd700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff07c198790 0x7ff07c19f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff07affd700 1 -- 172.21.15.122:0/3326920472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff07c103080 msgr2=0x7ff07c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff07affd700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff07c103080 0x7ff07c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff07affd700 1 -- 172.21.15.122:0/3326920472 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c1a01e0 con 0x7ff07c198310 2024-04-20T21:24:45.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff07b7fe700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff07c103080 0x7ff07c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:45.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.268+0000 7ff07affd700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff07c198310 0x7ff07c19d380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff07000b780 tx=0x7ff07000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.269+0000 7ff078ff9700 1 -- 172.21.15.122:0/3326920472 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070004230 con 0x7ff07c198310 2024-04-20T21:24:45.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.269+0000 7ff078ff9700 1 -- 172.21.15.122:0/3326920472 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0700043d0 con 0x7ff07c198310 2024-04-20T21:24:45.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.269+0000 7ff081c61700 1 -- 172.21.15.122:0/3326920472 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07c1a04d0 con 0x7ff07c198310 2024-04-20T21:24:45.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.269+0000 7ff078ff9700 1 -- 172.21.15.122:0/3326920472 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070011c90 con 0x7ff07c198310 2024-04-20T21:24:45.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.269+0000 7ff081c61700 1 -- 172.21.15.122:0/3326920472 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07c1a0ae0 con 0x7ff07c198310 2024-04-20T21:24:45.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.270+0000 7ff078ff9700 1 -- 172.21.15.122:0/3326920472 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff07001a400 con 0x7ff07c198310 2024-04-20T21:24:45.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.270+0000 7ff081c61700 1 -- 172.21.15.122:0/3326920472 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff07c0640b0 con 0x7ff07c198310 2024-04-20T21:24:45.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.270+0000 7ff078ff9700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff064039ae0 0x7ff06403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.270+0000 7ff078ff9700 1 -- 172.21.15.122:0/3326920472 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff07004c310 con 0x7ff07c198310 2024-04-20T21:24:45.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.270+0000 7ff07b7fe700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff064039ae0 0x7ff06403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.271+0000 7ff07b7fe700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff064039ae0 0x7ff06403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009750 tx=0x7ff06c0052d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.275+0000 7ff078ff9700 1 -- 172.21.15.122:0/3326920472 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff0700151c0 con 0x7ff07c198310 2024-04-20T21:24:45.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.485+0000 7ff081c61700 1 -- 172.21.15.122:0/3326920472 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi033=y;smithi122=z", "target": ["mon-mgr", ""]}) -- 0x7ff07c062f20 con 0x7ff064039ae0 2024-04-20T21:24:45.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.499+0000 7ff078ff9700 1 -- 172.21.15.122:0/3326920472 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff07c062f20 con 0x7ff064039ae0 2024-04-20T21:24:45.498 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled mgr update... 2024-04-20T21:24:45.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.500+0000 7ff081c61700 1 -- 172.21.15.122:0/3326920472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff064039ae0 msgr2=0x7ff06403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.501+0000 7ff081c61700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff064039ae0 0x7ff06403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009750 tx=0x7ff06c0052d0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.501+0000 7ff081c61700 1 -- 172.21.15.122:0/3326920472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff07c198310 msgr2=0x7ff07c19d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.501+0000 7ff081c61700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff07c198310 0x7ff07c19d380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff07000b780 tx=0x7ff07000ba90 comp rx=0 tx=0).stop 2024-04-20T21:24:45.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.501+0000 7ff081c61700 1 -- 172.21.15.122:0/3326920472 shutdown_connections 2024-04-20T21:24:45.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.501+0000 7ff081c61700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff064039ae0 0x7ff06403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.501+0000 7ff081c61700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff07c198790 0x7ff07c19f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.501+0000 7ff081c61700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff07c103080 0x7ff07c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.501+0000 7ff081c61700 1 --2- 172.21.15.122:0/3326920472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff07c198310 0x7ff07c19d380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.501+0000 7ff081c61700 1 -- 172.21.15.122:0/3326920472 >> 172.21.15.122:0/3326920472 conn(0x7ff07c0fca90 msgr2=0x7ff07c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:45.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.501+0000 7ff081c61700 1 -- 172.21.15.122:0/3326920472 shutdown_connections 2024-04-20T21:24:45.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:24:45.501+0000 7ff081c61700 1 -- 172.21.15.122:0/3326920472 wait complete. 2024-04-20T21:24:45.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:45 smithi022 ceph-mon[16843]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:45.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:45.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:45.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:45.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:45.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:45.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:45 smithi122 ceph-mon[22147]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:45.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:45 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:45.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:45 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:45.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:45 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:45.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:45 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:45.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:45 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:45.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:45 smithi033 ceph-mon[21626]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:45.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:45 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:45.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:45 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:45.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:45 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:45.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:45 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:45.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:45 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:46.102 DEBUG:teuthology.orchestra.run.smithi033:mgr.y> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.y.service 2024-04-20T21:24:46.104 DEBUG:teuthology.orchestra.run.smithi122:mgr.z> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.z.service 2024-04-20T21:24:46.106 INFO:tasks.cephadm:Deploying OSDs... 2024-04-20T21:24:46.106 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:24:46.107 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T21:24:46.168 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T21:24:46.168 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2024-04-20T21:24:46.224 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T21:24:46.225 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:46.225 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-20T21:24:46.225 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:46.225 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:46.225 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-20 21:23:42.726992336 +0000 2024-04-20T21:24:46.225 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-20 21:13:49.840375107 +0000 2024-04-20T21:24:46.225 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-20 21:13:49.840375107 +0000 2024-04-20T21:24:46.225 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-04-20 21:13:49.840375107 +0000 2024-04-20T21:24:46.225 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T21:24:46.309 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-04-20T21:24:46.309 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-04-20T21:24:46.309 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000284198 s, 1.8 MB/s 2024-04-20T21:24:46.313 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T21:24:46.371 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2024-04-20T21:24:46.426 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T21:24:46.426 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:46.426 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-20T21:24:46.426 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:46.426 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:46.427 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-20 21:23:42.803991331 +0000 2024-04-20T21:24:46.427 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-20 21:13:50.461366919 +0000 2024-04-20T21:24:46.427 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-20 21:13:50.461366919 +0000 2024-04-20T21:24:46.427 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-04-20 21:13:50.461366919 +0000 2024-04-20T21:24:46.427 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T21:24:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-04-20T21:24:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-04-20T21:24:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000277723 s, 1.8 MB/s 2024-04-20T21:24:46.515 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T21:24:46.572 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2024-04-20T21:24:46.627 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T21:24:46.627 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:46.627 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-20T21:24:46.627 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:46.628 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:46.628 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-20 21:23:42.901990053 +0000 2024-04-20T21:24:46.628 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-20 21:13:51.083358718 +0000 2024-04-20T21:24:46.628 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-20 21:13:51.083358718 +0000 2024-04-20T21:24:46.628 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-04-20 21:13:51.083358718 +0000 2024-04-20T21:24:46.628 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T21:24:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:46 smithi022 ceph-mon[16843]: Reconfiguring mon.b (monmap changed)... 2024-04-20T21:24:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:46 smithi022 ceph-mon[16843]: Reconfiguring daemon mon.b on smithi033 2024-04-20T21:24:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:46.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:46.705 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:46 smithi122 ceph-mon[22147]: Reconfiguring mon.b (monmap changed)... 2024-04-20T21:24:46.705 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:46 smithi122 ceph-mon[22147]: Reconfiguring daemon mon.b on smithi033 2024-04-20T21:24:46.705 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:46.705 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:46.705 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:46.705 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:46.705 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:46.705 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:46.716 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-04-20T21:24:46.716 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-04-20T21:24:46.716 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000287523 s, 1.8 MB/s 2024-04-20T21:24:46.720 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T21:24:46.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:46 smithi033 ceph-mon[21626]: Reconfiguring mon.b (monmap changed)... 2024-04-20T21:24:46.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:46 smithi033 ceph-mon[21626]: Reconfiguring daemon mon.b on smithi033 2024-04-20T21:24:46.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:46.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:46.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:46.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:46.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:46.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:46.822 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2024-04-20T21:24:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T21:24:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:46.875 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-20T21:24:46.875 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:46.875 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:46.875 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-20 21:23:43.008988656 +0000 2024-04-20T21:24:46.876 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-20 21:13:51.736350109 +0000 2024-04-20T21:24:46.876 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-20 21:13:51.736350109 +0000 2024-04-20T21:24:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-04-20 21:13:51.736350109 +0000 2024-04-20T21:24:46.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T21:24:46.958 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-04-20T21:24:46.958 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-04-20T21:24:46.958 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000286329 s, 1.8 MB/s 2024-04-20T21:24:46.962 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T21:24:47.022 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-20T21:24:47.022 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T21:24:47.042 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T21:24:47.042 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-04-20T21:24:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T21:24:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:47.100 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-20T21:24:47.100 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:47.100 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:47.100 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-20 21:24:32.415000722 +0000 2024-04-20T21:24:47.100 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-20 21:13:56.072474535 +0000 2024-04-20T21:24:47.100 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-20 21:13:56.072474535 +0000 2024-04-20T21:24:47.100 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-20 21:13:56.072474535 +0000 2024-04-20T21:24:47.101 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T21:24:47.166 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-20T21:24:47.166 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-20T21:24:47.166 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000282631 s, 1.8 MB/s 2024-04-20T21:24:47.167 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T21:24:47.226 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-04-20T21:24:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T21:24:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:47.284 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-20T21:24:47.284 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:47.284 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:47.284 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-20 21:24:32.415000722 +0000 2024-04-20T21:24:47.284 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-20 21:13:56.440468449 +0000 2024-04-20T21:24:47.284 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-20 21:13:56.440468449 +0000 2024-04-20T21:24:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-20 21:13:56.440468449 +0000 2024-04-20T21:24:47.285 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T21:24:47.349 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-20T21:24:47.349 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-20T21:24:47.349 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000206208 s, 2.5 MB/s 2024-04-20T21:24:47.350 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T21:24:47.408 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-04-20T21:24:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T21:24:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:47.465 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-20T21:24:47.465 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:47.465 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:47.465 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-20 21:24:32.416000706 +0000 2024-04-20T21:24:47.465 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-20 21:13:56.801462479 +0000 2024-04-20T21:24:47.465 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-20 21:13:56.801462479 +0000 2024-04-20T21:24:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-20 21:13:56.801462479 +0000 2024-04-20T21:24:47.466 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T21:24:47.526 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:47 smithi033 ceph-mon[21626]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi033=y;smithi122=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:47.527 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:47 smithi033 ceph-mon[21626]: Saving service mgr spec with placement smithi022=x;smithi033=y;smithi122=z;count:3 2024-04-20T21:24:47.527 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:47 smithi033 ceph-mon[21626]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:47.527 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:47 smithi033 ceph-mon[21626]: Reconfiguring mon.c (monmap changed)... 2024-04-20T21:24:47.527 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:47 smithi033 ceph-mon[21626]: Reconfiguring daemon mon.c on smithi122 2024-04-20T21:24:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-20T21:24:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-20T21:24:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000198438 s, 2.6 MB/s 2024-04-20T21:24:47.530 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T21:24:47.588 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-04-20T21:24:47.624 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:47 smithi122 ceph-mon[22147]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi033=y;smithi122=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:47.624 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:47 smithi122 ceph-mon[22147]: Saving service mgr spec with placement smithi022=x;smithi033=y;smithi122=z;count:3 2024-04-20T21:24:47.624 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:47 smithi122 ceph-mon[22147]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:47.624 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:47 smithi122 ceph-mon[22147]: Reconfiguring mon.c (monmap changed)... 2024-04-20T21:24:47.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:47 smithi122 ceph-mon[22147]: Reconfiguring daemon mon.c on smithi122 2024-04-20T21:24:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T21:24:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:47.644 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-20T21:24:47.644 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:47.644 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:47.644 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-20 21:24:32.416000706 +0000 2024-04-20T21:24:47.644 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-20 21:13:57.169456393 +0000 2024-04-20T21:24:47.644 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-20 21:13:57.169456393 +0000 2024-04-20T21:24:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-20 21:13:57.169456393 +0000 2024-04-20T21:24:47.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T21:24:47.708 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-20T21:24:47.708 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-20T21:24:47.708 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000205388 s, 2.5 MB/s 2024-04-20T21:24:47.709 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T21:24:47.766 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-20T21:24:47.767 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T21:24:47.781 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T21:24:47.781 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-04-20T21:24:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T21:24:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:47.836 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-20T21:24:47.836 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:47.836 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:47.836 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-20 21:24:22.834115579 +0000 2024-04-20T21:24:47.836 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-20 21:13:55.776722641 +0000 2024-04-20T21:24:47.837 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-20 21:13:55.776722641 +0000 2024-04-20T21:24:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-20 21:13:55.776722641 +0000 2024-04-20T21:24:47.837 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T21:24:47.899 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-20T21:24:47.900 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-20T21:24:47.900 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000269946 s, 1.9 MB/s 2024-04-20T21:24:47.901 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T21:24:47.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:47 smithi022 ceph-mon[16843]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi033=y;smithi122=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:47.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:47 smithi022 ceph-mon[16843]: Saving service mgr spec with placement smithi022=x;smithi033=y;smithi122=z;count:3 2024-04-20T21:24:47.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:47 smithi022 ceph-mon[16843]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:47.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:47 smithi022 ceph-mon[16843]: Reconfiguring mon.c (monmap changed)... 2024-04-20T21:24:47.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:47 smithi022 ceph-mon[16843]: Reconfiguring daemon mon.c on smithi122 2024-04-20T21:24:47.956 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-04-20T21:24:48.011 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T21:24:48.012 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:48.012 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-20T21:24:48.012 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:48.012 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:48.012 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-20 21:24:22.834115579 +0000 2024-04-20T21:24:48.012 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-20 21:13:56.133714472 +0000 2024-04-20T21:24:48.012 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-20 21:13:56.133714472 +0000 2024-04-20T21:24:48.012 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-20 21:13:56.133714472 +0000 2024-04-20T21:24:48.012 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T21:24:48.074 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-20T21:24:48.074 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-20T21:24:48.074 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.00017998 s, 2.8 MB/s 2024-04-20T21:24:48.076 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T21:24:48.131 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-04-20T21:24:48.185 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T21:24:48.186 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:48.186 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-20T21:24:48.186 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:48.186 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:48.186 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-20 21:24:22.835115556 +0000 2024-04-20T21:24:48.186 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-20 21:13:56.479706554 +0000 2024-04-20T21:24:48.186 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-20 21:13:56.479706554 +0000 2024-04-20T21:24:48.186 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-20 21:13:56.479706554 +0000 2024-04-20T21:24:48.186 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T21:24:48.248 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-20T21:24:48.248 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-20T21:24:48.248 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000187916 s, 2.7 MB/s 2024-04-20T21:24:48.250 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T21:24:48.305 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-04-20T21:24:48.359 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T21:24:48.359 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:48.359 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-20T21:24:48.359 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:48.360 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:48.360 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-20 21:24:22.835115556 +0000 2024-04-20T21:24:48.360 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-20 21:13:56.813698911 +0000 2024-04-20T21:24:48.360 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-20 21:13:56.813698911 +0000 2024-04-20T21:24:48.360 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-20 21:13:56.813698911 +0000 2024-04-20T21:24:48.360 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T21:24:48.422 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-20T21:24:48.422 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-20T21:24:48.422 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.00016892 s, 3.0 MB/s 2024-04-20T21:24:48.423 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T21:24:48.478 INFO:tasks.cephadm:Deploying osd.0 on smithi022 with /dev/vg_nvme/lv_4... 2024-04-20T21:24:48.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T21:24:48.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:48 smithi122 ceph-mon[22147]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:48.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:48.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:48.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:48.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:48.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:48.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:48.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:48.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T21:24:48.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:48.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:48.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:48 smithi122 ceph-mon[22147]: Deploying daemon mgr.z on smithi122 2024-04-20T21:24:48.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:48 smithi022 ceph-mon[16843]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:48.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:48.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:48.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:48.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:48.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:48.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:48.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:48.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T21:24:48.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:48.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:48.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:48 smithi022 ceph-mon[16843]: Deploying daemon mgr.z on smithi122 2024-04-20T21:24:48.926 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:24:49.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:48 smithi033 ceph-mon[21626]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:49.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:49.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:49.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:49.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:49.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:49.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:49.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:49.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T21:24:49.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:49.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:49.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:48 smithi033 ceph-mon[21626]: Deploying daemon mgr.z on smithi122 2024-04-20T21:24:49.959 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:49 smithi122 systemd[1]: Starting Ceph mgr.z for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:24:50.228 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:49 smithi122 podman[24005]: 2024-04-20 21:24:49.863603964 +0000 UTC m=+0.018095740 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:24:50.228 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:49 smithi122 podman[24005]: 2024-04-20 21:24:49.968928537 +0000 UTC m=+0.123420289 container create 9a11fad32beeb9637ed9476ba1bed300af6d8c776073f1ee9beaa967a512cff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:24:50.228 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:50 smithi122 podman[24005]: 2024-04-20 21:24:50.171103573 +0000 UTC m=+0.325595327 container init 9a11fad32beeb9637ed9476ba1bed300af6d8c776073f1ee9beaa967a512cff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-20T21:24:50.229 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:50 smithi122 podman[24005]: 2024-04-20 21:24:50.173553184 +0000 UTC m=+0.328047156 container start 9a11fad32beeb9637ed9476ba1bed300af6d8c776073f1ee9beaa967a512cff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416) 2024-04-20T21:24:50.229 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:50 smithi122 ceph-mgr[24024]: -- 172.21.15.122:0/2192402423 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a7ca69c5a0 con 0x55a7ca671000 2024-04-20T21:24:50.229 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:50 smithi122 bash[24005]: 9a11fad32beeb9637ed9476ba1bed300af6d8c776073f1ee9beaa967a512cff3 2024-04-20T21:24:50.509 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:50 smithi122 systemd[1]: Started Ceph mgr.z for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:24:50.509 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:50.326+0000 7f7f5a7d3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:24:50.509 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:50.509+0000 7f7f5a7d3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:24:50.775 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:50.610+0000 7f7f5a7d3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:24:51.275 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:50.916+0000 7f7f5a7d3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:24:51.275 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:51 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:50.999+0000 7f7f5a7d3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:24:51.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:51 smithi033 ceph-mon[21626]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:51.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:51.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:51.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:51.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:51.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T21:24:51.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:51.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:51.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:51 smithi033 ceph-mon[21626]: Deploying daemon mgr.y on smithi033 2024-04-20T21:24:51.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:51 smithi022 ceph-mon[16843]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:51.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:51.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:51.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:51.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:51.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T21:24:51.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:51.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:51.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:51 smithi022 ceph-mon[16843]: Deploying daemon mgr.y on smithi033 2024-04-20T21:24:51.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:51 smithi122 ceph-mon[22147]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:51.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:51.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:51.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:51.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:51.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T21:24:51.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:51.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:51.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:51 smithi122 ceph-mon[22147]: Deploying daemon mgr.y on smithi033 2024-04-20T21:24:51.808 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T21:24:51.808 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T21:24:51.808 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-04-20T21:24:51.809 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-04-20T21:24:51.809 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0377783 s, 278 MB/s 2024-04-20T21:24:51.809 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-04-20T21:24:52.521 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:52.187+0000 7f7f5a7d3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:24:52.521 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:52.355+0000 7f7f5a7d3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:24:52.521 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:52.522+0000 7f7f5a7d3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:24:52.553 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:52 smithi033 systemd[1]: Starting Ceph mgr.y for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:24:52.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_4 2024-04-20T21:24:52.936 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:52 smithi033 podman[22988]: 2024-04-20 21:24:52.582541928 +0000 UTC m=+0.018226530 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:24:52.937 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:52 smithi033 podman[22988]: 2024-04-20 21:24:52.68303549 +0000 UTC m=+0.118720115 container create f64fc78265e2e7446da8cbf1b18c0bdedd390e3f6d3a267c4e8fb274afd849c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-20T21:24:52.937 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:52 smithi033 podman[22988]: 2024-04-20 21:24:52.910206596 +0000 UTC m=+0.345891179 container init f64fc78265e2e7446da8cbf1b18c0bdedd390e3f6d3a267c4e8fb274afd849c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:24:52.937 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:52 smithi033 podman[22988]: 2024-04-20 21:24:52.912743169 +0000 UTC m=+0.348427762 container start f64fc78265e2e7446da8cbf1b18c0bdedd390e3f6d3a267c4e8fb274afd849c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-20T21:24:53.096 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:52.805+0000 7f7f5a7d3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:24:53.096 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:52.895+0000 7f7f5a7d3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:24:53.096 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:53.096+0000 7f7f5a7d3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:24:53.197 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:24:53.267 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:52 smithi033 ceph-mgr[23007]: -- 172.21.15.33:0/2021615833 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d00d6ca5a0 con 0x55d00d69f000 2024-04-20T21:24:53.267 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:53 smithi033 bash[22988]: f64fc78265e2e7446da8cbf1b18c0bdedd390e3f6d3a267c4e8fb274afd849c3 2024-04-20T21:24:53.267 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:53 smithi033 systemd[1]: Started Ceph mgr.y for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:24:53.267 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:53 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:53.070+0000 7fa4f954a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:24:53.267 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:53 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:53.268+0000 7fa4f954a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:24:53.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:53.286+0000 7f7f5a7d3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:24:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:53 smithi122 ceph-mon[22147]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:53 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:53 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:53 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:53 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:53.553 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:53 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:53.381+0000 7fa4f954a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:24:53.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:53 smithi033 ceph-mon[21626]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:53.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:53 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:53.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:53 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:53.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:53 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:53.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:53 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:53.557 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:53 smithi022 ceph-mon[16843]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:53.557 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:53 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:53.557 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:53 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:53.557 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:53 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:53.557 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:53 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:54.053 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:53 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:53.722+0000 7fa4f954a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:24:54.053 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:53 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:53.813+0000 7fa4f954a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:24:54.232 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:53.930+0000 7f7f5a7d3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:24:54.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:54 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:54.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:54 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:54.375+0000 7f7f5a7d3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:24:54.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:54 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:54.472+0000 7f7f5a7d3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:24:54.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:54 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:54.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:54 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:55.025 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:54 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:54.674+0000 7f7f5a7d3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:24:55.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.355+0000 7ff61ea00700 1 -- 172.21.15.22:0/2360203418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff618101380 msgr2=0x7ff6181017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.355+0000 7ff61ea00700 1 --2- 172.21.15.22:0/2360203418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff618101380 0x7ff6181017a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff608009990 tx=0x7ff608009ca0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.355+0000 7ff61ea00700 1 -- 172.21.15.22:0/2360203418 shutdown_connections 2024-04-20T21:24:55.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.355+0000 7ff61ea00700 1 --2- 172.21.15.22:0/2360203418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6181026a0 0x7ff618102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.355+0000 7ff61ea00700 1 --2- 172.21.15.22:0/2360203418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff618101380 0x7ff6181017a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.355+0000 7ff61ea00700 1 --2- 172.21.15.22:0/2360203418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff618101ce0 0x7ff618102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.355+0000 7ff61ea00700 1 -- 172.21.15.22:0/2360203418 >> 172.21.15.22:0/2360203418 conn(0x7ff6180fcf40 msgr2=0x7ff6180ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:55.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.355+0000 7ff61ea00700 1 -- 172.21.15.22:0/2360203418 shutdown_connections 2024-04-20T21:24:55.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.355+0000 7ff61ea00700 1 -- 172.21.15.22:0/2360203418 wait complete. 2024-04-20T21:24:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.357+0000 7ff61ea00700 1 Processor -- start 2024-04-20T21:24:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.357+0000 7ff61ea00700 1 -- start start 2024-04-20T21:24:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.357+0000 7ff61ea00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff618101380 0x7ff61819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.357+0000 7ff61ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff618101ce0 0x7ff61819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff61ea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6181026a0 0x7ff6181a2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff61ea00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff618110890 con 0x7ff6181026a0 2024-04-20T21:24:55.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff61ea00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff618110710 con 0x7ff618101ce0 2024-04-20T21:24:55.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff61ea00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff618110a10 con 0x7ff618101380 2024-04-20T21:24:55.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff617e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff618101380 0x7ff61819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:55.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff617e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff618101380 0x7ff61819ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46848/0 (socket says 172.21.15.22:46848) 2024-04-20T21:24:55.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff617e00700 1 -- 172.21.15.22:0/1224230505 learned_addr learned my addr 172.21.15.22:0/1224230505 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:24:55.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff617400700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff618101ce0 0x7ff61819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:55.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff61ca00700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6181026a0 0x7ff6181a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:55.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff617e00700 1 -- 172.21.15.22:0/1224230505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff618101ce0 msgr2=0x7ff61819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff617e00700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff618101ce0 0x7ff61819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff617e00700 1 -- 172.21.15.22:0/1224230505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6181026a0 msgr2=0x7ff6181a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff617e00700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6181026a0 0x7ff6181a2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff617e00700 1 -- 172.21.15.22:0/1224230505 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6181a2ad0 con 0x7ff618101380 2024-04-20T21:24:55.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.358+0000 7ff617400700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff618101ce0 0x7ff61819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:24:55.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.359+0000 7ff617e00700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff618101380 0x7ff61819ae70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff6080095c0 tx=0x7ff60800bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:55.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.359+0000 7ff614c00700 1 -- 172.21.15.22:0/1224230505 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608026070 con 0x7ff618101380 2024-04-20T21:24:55.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.359+0000 7ff61ea00700 1 -- 172.21.15.22:0/1224230505 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6181a2d60 con 0x7ff618101380 2024-04-20T21:24:55.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.359+0000 7ff614c00700 1 -- 172.21.15.22:0/1224230505 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff608003d70 con 0x7ff618101380 2024-04-20T21:24:55.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.359+0000 7ff614c00700 1 -- 172.21.15.22:0/1224230505 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608021410 con 0x7ff618101380 2024-04-20T21:24:55.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.359+0000 7ff61ea00700 1 -- 172.21.15.22:0/1224230505 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6181a31b0 con 0x7ff618101380 2024-04-20T21:24:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.360+0000 7ff614c00700 1 -- 172.21.15.22:0/1224230505 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff608029070 con 0x7ff618101380 2024-04-20T21:24:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.360+0000 7ff61ea00700 1 -- 172.21.15.22:0/1224230505 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff618069740 con 0x7ff618101380 2024-04-20T21:24:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.361+0000 7ff614c00700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5fc039890 0x7ff5fc03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.361+0000 7ff614c00700 1 -- 172.21.15.22:0/1224230505 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff60804d890 con 0x7ff618101380 2024-04-20T21:24:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.361+0000 7ff617400700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5fc039890 0x7ff5fc03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:55.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.362+0000 7ff617400700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5fc039890 0x7ff5fc03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff604011490 tx=0x7ff604009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:55.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.366+0000 7ff614c00700 1 -- 172.21.15.22:0/1224230505 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff608013070 con 0x7ff618101380 2024-04-20T21:24:55.468 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:55.116+0000 7fa4f954a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:24:55.468 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:55.292+0000 7fa4f954a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:24:55.468 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:55.469+0000 7fa4f954a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:24:55.468 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-mon[21626]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:55.468 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.468 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.469 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.469 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.469 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.469 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:55.469 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:55.469 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.469 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:55.469 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:55.469 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:55.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:55.225+0000 7f7f5a7d3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:24:55.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:55.315+0000 7f7f5a7d3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:24:55.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-mon[22147]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:55.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:55.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:55.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:55.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:55.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:55.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:55.607+0000 7ff61ea00700 1 -- 172.21.15.22:0/1224230505 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff618062e90 con 0x7ff5fc039890 2024-04-20T21:24:55.606 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:55 smithi022 ceph-mon[16843]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:55.606 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:55 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:55 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:55 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:55 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:55 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:55 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:55.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:55 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:55.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:55 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:55.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:55 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:55.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:55 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:55.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:55 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:55.891 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:55.556+0000 7f7f5a7d3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:24:55.891 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:55.686+0000 7f7f5a7d3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:24:55.891 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:55.778+0000 7f7f5a7d3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:24:55.891 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:55.891+0000 7f7f5a7d3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:24:56.053 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:55.767+0000 7fa4f954a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:24:56.053 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:55 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:55.862+0000 7fa4f954a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:24:56.323 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:56 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:56.072+0000 7fa4f954a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:24:56.323 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:56 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:56.271+0000 7fa4f954a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:24:56.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:56.190+0000 7f7f5a7d3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:24:56.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:56 smithi122 ceph-mon[22147]: Reconfiguring mgr.x (unknown last config time)... 2024-04-20T21:24:56.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:56 smithi122 ceph-mon[22147]: Reconfiguring daemon mgr.x on smithi022 2024-04-20T21:24:56.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:56 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:24:56.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:56 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:24:56.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:56 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:56.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:56 smithi022 ceph-mon[16843]: Reconfiguring mgr.x (unknown last config time)... 2024-04-20T21:24:56.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:56 smithi022 ceph-mon[16843]: Reconfiguring daemon mgr.x on smithi022 2024-04-20T21:24:56.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:56 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:24:56.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:56 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:24:56.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:56 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:56.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:56 smithi033 ceph-mon[21626]: Reconfiguring mgr.x (unknown last config time)... 2024-04-20T21:24:56.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:56 smithi033 ceph-mon[21626]: Reconfiguring daemon mgr.x on smithi022 2024-04-20T21:24:56.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:56 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:24:56.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:56 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:24:56.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:56 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:57.275 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:56.783+0000 7f7f5a7d3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:24:57.275 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:24:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:24:56.995+0000 7f7f5a7d3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:24:57.303 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:56 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:56.909+0000 7fa4f954a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:24:57.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:24:57.336+0000 7ff614c00700 1 -- 172.21.15.22:0/1224230505 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7ff608009d10 con 0x7ff618101380 2024-04-20T21:24:57.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:57.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:57.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.122:0/4032988678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T21:24:57.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: Standby manager daemon z started 2024-04-20T21:24:57.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.122:0/4032988678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:24:57.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.122:0/4032988678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T21:24:57.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.122:0/4032988678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:24:57.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:57.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:57.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:57.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:57.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:57.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:57 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:57.655 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:57.347+0000 7fa4f954a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:24:57.655 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:57.447+0000 7fa4f954a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:24:57.655 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:57.656+0000 7fa4f954a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:24:57.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:57.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:57.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.122:0/4032988678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T21:24:57.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: Standby manager daemon z started 2024-04-20T21:24:57.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.122:0/4032988678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:24:57.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.122:0/4032988678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T21:24:57.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.122:0/4032988678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:24:57.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:57.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:57.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:57.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:57.656 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:57.657 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:57 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.122:0/4032988678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T21:24:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: Standby manager daemon z started 2024-04-20T21:24:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.122:0/4032988678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:24:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.122:0/4032988678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T21:24:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.122:0/4032988678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:24:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:57.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:57.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:57 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:24:58.552 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:58 smithi033 ceph-mon[21626]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-20T21:24:58.552 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:58 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T21:24:58.552 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:58 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:58.206+0000 7fa4f954a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:24:58.552 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:58 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:58.301+0000 7fa4f954a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:24:58.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:58 smithi022 ceph-mon[16843]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-20T21:24:58.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:58 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T21:24:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:58 smithi122 ceph-mon[22147]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-20T21:24:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:58 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T21:24:58.803 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:58 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:58.553+0000 7fa4f954a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:24:58.803 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:58 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:58.690+0000 7fa4f954a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:24:58.803 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:58 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:58.788+0000 7fa4f954a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:24:59.218 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:58 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:58.908+0000 7fa4f954a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:24:59.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:24:59 smithi033 ceph-mon[21626]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:59.553 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:59 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:59.219+0000 7fa4f954a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:24:59.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:24:59 smithi022 ceph-mon[16843]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:59.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:24:59 smithi122 ceph-mon[22147]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:00.303 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:24:59 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:24:59.847+0000 7fa4f954a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:25:00.304 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:25:00 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:25:00.070+0000 7fa4f954a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:25:00.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:00.366+0000 7ff614c00700 1 -- 172.21.15.22:0/1224230505 <== mon.1 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7ff60800f460 con 0x7ff618101380 2024-04-20T21:25:00.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:00 smithi022 ceph-mon[16843]: Standby manager daemon y started 2024-04-20T21:25:00.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:00 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.33:0/2775606008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T21:25:00.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:00 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.33:0/2775606008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:25:00.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:00 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.33:0/2775606008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T21:25:00.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:00 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.33:0/2775606008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:25:00.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:00 smithi122 ceph-mon[22147]: Standby manager daemon y started 2024-04-20T21:25:00.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:00 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.33:0/2775606008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T21:25:00.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:00 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.33:0/2775606008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:25:00.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:00 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.33:0/2775606008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T21:25:00.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:00 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.33:0/2775606008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:25:00.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:00 smithi033 ceph-mon[21626]: Standby manager daemon y started 2024-04-20T21:25:00.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:00 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.33:0/2775606008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T21:25:00.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:00 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.33:0/2775606008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:25:00.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:00 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.33:0/2775606008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T21:25:00.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:00 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.33:0/2775606008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:25:01.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:01 smithi022 ceph-mon[16843]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:01.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:01 smithi022 ceph-mon[16843]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-20T21:25:01.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T21:25:01.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:01 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3101216427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67f06b5e-0fdc-4d2a-9355-fc7aa718d77e"}]: dispatch 2024-04-20T21:25:01.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:01 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67f06b5e-0fdc-4d2a-9355-fc7aa718d77e"}]: dispatch 2024-04-20T21:25:01.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:01 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67f06b5e-0fdc-4d2a-9355-fc7aa718d77e"}]': finished 2024-04-20T21:25:01.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:01 smithi022 ceph-mon[16843]: osdmap e5: 1 total, 0 up, 1 in 2024-04-20T21:25:01.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:01.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:01 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3727713496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:01 smithi122 ceph-mon[22147]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:01 smithi122 ceph-mon[22147]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-20T21:25:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T21:25:01.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:01 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3101216427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67f06b5e-0fdc-4d2a-9355-fc7aa718d77e"}]: dispatch 2024-04-20T21:25:01.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:01 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67f06b5e-0fdc-4d2a-9355-fc7aa718d77e"}]: dispatch 2024-04-20T21:25:01.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:01 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67f06b5e-0fdc-4d2a-9355-fc7aa718d77e"}]': finished 2024-04-20T21:25:01.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:01 smithi122 ceph-mon[22147]: osdmap e5: 1 total, 0 up, 1 in 2024-04-20T21:25:01.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:01.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:01 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3727713496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:01.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:01 smithi033 ceph-mon[21626]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:01.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:01 smithi033 ceph-mon[21626]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-20T21:25:01.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T21:25:01.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:01 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3101216427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67f06b5e-0fdc-4d2a-9355-fc7aa718d77e"}]: dispatch 2024-04-20T21:25:01.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:01 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67f06b5e-0fdc-4d2a-9355-fc7aa718d77e"}]: dispatch 2024-04-20T21:25:01.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:01 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67f06b5e-0fdc-4d2a-9355-fc7aa718d77e"}]': finished 2024-04-20T21:25:01.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:01 smithi033 ceph-mon[21626]: osdmap e5: 1 total, 0 up, 1 in 2024-04-20T21:25:01.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:01.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:01 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3727713496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:03.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:03 smithi022 ceph-mon[16843]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:03.769 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:03 smithi033 ceph-mon[21626]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:03.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:03 smithi122 ceph-mon[22147]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:05.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:05 smithi022 ceph-mon[16843]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:05.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:05 smithi122 ceph-mon[22147]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:05.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:05 smithi033 ceph-mon[21626]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:07 smithi022 ceph-mon[16843]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:07 smithi122 ceph-mon[22147]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:07.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:07 smithi033 ceph-mon[21626]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:09 smithi022 ceph-mon[16843]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:09 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T21:25:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:09 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:09 smithi122 ceph-mon[22147]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:09 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T21:25:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:09 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:09.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:09 smithi033 ceph-mon[21626]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:09.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:09 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T21:25:09.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:09 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:10.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:10 smithi022 ceph-mon[16843]: Deploying daemon osd.0 on smithi022 2024-04-20T21:25:10.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:10 smithi122 ceph-mon[22147]: Deploying daemon osd.0 on smithi022 2024-04-20T21:25:10.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:10 smithi033 ceph-mon[21626]: Deploying daemon osd.0 on smithi022 2024-04-20T21:25:11.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:11 smithi022 ceph-mon[16843]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:11.676 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:11 smithi122 ceph-mon[22147]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:11.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:11 smithi033 ceph-mon[21626]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:13.644 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:13 smithi022 ceph-mon[16843]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:13.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:13 smithi122 ceph-mon[22147]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:13.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:13 smithi033 ceph-mon[21626]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:15 smithi022 ceph-mon[16843]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:15 smithi122 ceph-mon[22147]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:15.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:15 smithi033 ceph-mon[21626]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:17.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:17 smithi022 ceph-mon[16843]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:17 smithi122 ceph-mon[22147]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:17.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:17 smithi033 ceph-mon[21626]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:18.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:18.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:18.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:18.739 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:18 smithi022 ceph-mon[16843]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:18 smithi122 ceph-mon[22147]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:18.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:18.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:18.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:18.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:18 smithi033 ceph-mon[21626]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:20.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:20 smithi022 ceph-mon[16843]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:21.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:20 smithi122 ceph-mon[22147]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:21.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:20 smithi033 ceph-mon[21626]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:22.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:21 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:22.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:21 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:22.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:21 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:22.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:21 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:22.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:21 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:22.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:21 smithi122 ceph-mon[22147]: from='osd.0 [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T21:25:22.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:21 smithi122 ceph-mon[22147]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T21:25:22.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:21 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:22.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:21 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:22.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:21 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:22.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:21 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:22.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:21 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:22.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:21 smithi033 ceph-mon[21626]: from='osd.0 [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T21:25:22.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:21 smithi033 ceph-mon[21626]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T21:25:22.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:21 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:22.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:21 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:22.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:21 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:22.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:21 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:22.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:21 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:22.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:21 smithi022 ceph-mon[16843]: from='osd.0 [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T21:25:22.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:21 smithi022 ceph-mon[16843]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T21:25:22.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.247+0000 7ff614c00700 1 -- 172.21.15.22:0/1224230505 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff618062e90 con 0x7ff5fc039890 2024-04-20T21:25:22.247 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 0 on host 'smithi022' 2024-04-20T21:25:22.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.251+0000 7ff61ea00700 1 -- 172.21.15.22:0/1224230505 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5fc039890 msgr2=0x7ff5fc03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:22.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.251+0000 7ff61ea00700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5fc039890 0x7ff5fc03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff604011490 tx=0x7ff604009260 comp rx=0 tx=0).stop 2024-04-20T21:25:22.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.251+0000 7ff61ea00700 1 -- 172.21.15.22:0/1224230505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff618101380 msgr2=0x7ff61819ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:22.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.251+0000 7ff61ea00700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff618101380 0x7ff61819ae70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff6080095c0 tx=0x7ff60800bb90 comp rx=0 tx=0).stop 2024-04-20T21:25:22.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.251+0000 7ff61ea00700 1 -- 172.21.15.22:0/1224230505 shutdown_connections 2024-04-20T21:25:22.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.251+0000 7ff61ea00700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5fc039890 0x7ff5fc03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:22.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.251+0000 7ff61ea00700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6181026a0 0x7ff6181a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:22.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.251+0000 7ff61ea00700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff618101380 0x7ff61819ae70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:22.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.251+0000 7ff61ea00700 1 --2- 172.21.15.22:0/1224230505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff618101ce0 0x7ff61819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:22.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.251+0000 7ff61ea00700 1 -- 172.21.15.22:0/1224230505 >> 172.21.15.22:0/1224230505 conn(0x7ff6180fcf40 msgr2=0x7ff6180fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:22.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.251+0000 7ff61ea00700 1 -- 172.21.15.22:0/1224230505 shutdown_connections 2024-04-20T21:25:22.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:25:22.251+0000 7ff61ea00700 1 -- 172.21.15.22:0/1224230505 wait complete. 2024-04-20T21:25:22.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:22 smithi022 ceph-mon[16843]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:22.895 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:22 smithi022 ceph-mon[16843]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T21:25:22.895 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:22 smithi022 ceph-mon[16843]: from='osd.0 [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:25:22.895 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:22 smithi022 ceph-mon[16843]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T21:25:22.895 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:22 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:22.895 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:22 smithi022 ceph-mon[16843]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:25:22.895 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:22 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:22.895 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:22 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:22.895 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:22 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:22.941 DEBUG:teuthology.orchestra.run.smithi022:osd.0> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.0.service 2024-04-20T21:25:22.945 INFO:tasks.cephadm:Deploying osd.1 on smithi033 with /dev/vg_nvme/lv_4... 2024-04-20T21:25:22.945 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T21:25:23.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:22 smithi033 ceph-mon[21626]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:23.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:22 smithi033 ceph-mon[21626]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T21:25:23.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:22 smithi033 ceph-mon[21626]: from='osd.0 [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:25:23.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:22 smithi033 ceph-mon[21626]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T21:25:23.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:22 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:23.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:22 smithi033 ceph-mon[21626]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:25:23.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:22 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:23.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:22 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:23.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:22 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:23.209 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:25:23.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:22 smithi122 ceph-mon[22147]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:23.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:22 smithi122 ceph-mon[22147]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T21:25:23.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:22 smithi122 ceph-mon[22147]: from='osd.0 [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:25:23.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:22 smithi122 ceph-mon[22147]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T21:25:23.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:22 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:23.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:22 smithi122 ceph-mon[22147]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:25:23.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:22 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:23.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:22 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:23.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:22 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:23.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:23 smithi033 ceph-mon[21626]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-20T21:25:23.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:23 smithi033 ceph-mon[21626]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T21:25:23.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:23 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:23.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:23 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:24.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:23 smithi022 ceph-mon[16843]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-20T21:25:24.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:23 smithi022 ceph-mon[16843]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T21:25:24.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:23 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:24.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:23 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:24.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:23 smithi122 ceph-mon[22147]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-20T21:25:24.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:23 smithi122 ceph-mon[22147]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T21:25:24.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:23 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:24.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:23 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:25.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:24 smithi033 ceph-mon[21626]: purged_snaps scrub starts 2024-04-20T21:25:25.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:24 smithi033 ceph-mon[21626]: purged_snaps scrub ok 2024-04-20T21:25:25.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:24 smithi033 ceph-mon[21626]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:25.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:24 smithi033 ceph-mon[21626]: osd.0 [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324] boot 2024-04-20T21:25:25.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:24 smithi033 ceph-mon[21626]: osdmap e8: 1 total, 1 up, 1 in 2024-04-20T21:25:25.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:24 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:25.057 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T21:25:25.057 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T21:25:25.057 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-20T21:25:25.057 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-20T21:25:25.057 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014196 s, 739 MB/s 2024-04-20T21:25:25.057 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-20T21:25:25.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:24 smithi022 ceph-mon[16843]: purged_snaps scrub starts 2024-04-20T21:25:25.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:24 smithi022 ceph-mon[16843]: purged_snaps scrub ok 2024-04-20T21:25:25.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:24 smithi022 ceph-mon[16843]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:25.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:24 smithi022 ceph-mon[16843]: osd.0 [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324] boot 2024-04-20T21:25:25.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:24 smithi022 ceph-mon[16843]: osdmap e8: 1 total, 1 up, 1 in 2024-04-20T21:25:25.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:24 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:25.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:24 smithi122 ceph-mon[22147]: purged_snaps scrub starts 2024-04-20T21:25:25.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:24 smithi122 ceph-mon[22147]: purged_snaps scrub ok 2024-04-20T21:25:25.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:24 smithi122 ceph-mon[22147]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:25:25.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:24 smithi122 ceph-mon[22147]: osd.0 [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324] boot 2024-04-20T21:25:25.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:24 smithi122 ceph-mon[22147]: osdmap e8: 1 total, 1 up, 1 in 2024-04-20T21:25:25.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:24 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:25:25.660 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_4 2024-04-20T21:25:25.908 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:25:26.030 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:25 smithi033 ceph-mon[21626]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T21:25:26.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:25 smithi022 ceph-mon[16843]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T21:25:26.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:25 smithi122 ceph-mon[22147]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T21:25:26.944 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:26 smithi033 ceph-mon[21626]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:27.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:26 smithi022 ceph-mon[16843]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:27.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:26 smithi122 ceph-mon[22147]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.461+0000 7f2ac6a71700 1 -- 172.21.15.33:0/3207056166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac0105e90 msgr2=0x7f2ac0108280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.461+0000 7f2ac6a71700 1 --2- 172.21.15.33:0/3207056166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac0105e90 0x7f2ac0108280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009990 tx=0x7f2aac009ca0 comp rx=0 tx=0).stop 2024-04-20T21:25:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.461+0000 7f2ac6a71700 1 -- 172.21.15.33:0/3207056166 shutdown_connections 2024-04-20T21:25:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.461+0000 7f2ac6a71700 1 --2- 172.21.15.33:0/3207056166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ac01087c0 0x7f2ac010ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.461+0000 7f2ac6a71700 1 --2- 172.21.15.33:0/3207056166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ac0103560 0x7f2ac0105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.461+0000 7f2ac6a71700 1 --2- 172.21.15.33:0/3207056166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac0105e90 0x7f2ac0108280 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.461+0000 7f2ac6a71700 1 -- 172.21.15.33:0/3207056166 >> 172.21.15.33:0/3207056166 conn(0x7f2ac00fcf70 msgr2=0x7f2ac00ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.461+0000 7f2ac6a71700 1 -- 172.21.15.33:0/3207056166 shutdown_connections 2024-04-20T21:25:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.461+0000 7f2ac6a71700 1 -- 172.21.15.33:0/3207056166 wait complete. 2024-04-20T21:25:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.461+0000 7f2ac6a71700 1 Processor -- start 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.461+0000 7f2ac6a71700 1 -- start start 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2ac6a71700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ac0103560 0x7f2ac0102ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2ac6a71700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac0105e90 0x7f2ac0101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2ac6a71700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ac01087c0 0x7f2ac0101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2ac6a71700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac010e7a0 con 0x7f2ac01087c0 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2ac6a71700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac010e620 con 0x7f2ac0105e90 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2ac6a71700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac010e920 con 0x7f2ac0103560 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2abffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac0105e90 0x7f2ac0101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2abffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac0105e90 0x7f2ac0101510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51082/0 (socket says 172.21.15.33:51082) 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2abffff700 1 -- 172.21.15.33:0/207849196 learned_addr learned my addr 172.21.15.33:0/207849196 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2ac480d700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ac0103560 0x7f2ac0102ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2abffff700 1 -- 172.21.15.33:0/207849196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ac0103560 msgr2=0x7f2ac0102ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2abffff700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ac0103560 0x7f2ac0102ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2abffff700 1 -- 172.21.15.33:0/207849196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ac01087c0 msgr2=0x7f2ac0101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2abffff700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ac01087c0 0x7f2ac0101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2abffff700 1 -- 172.21.15.33:0/207849196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac0102300 con 0x7f2ac0105e90 2024-04-20T21:25:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2abffff700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac0105e90 0x7f2ac0101510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2aac005910 tx=0x7f2aac0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2abdffb700 1 -- 172.21.15.33:0/207849196 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac027070 con 0x7f2ac0105e90 2024-04-20T21:25:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2abdffb700 1 -- 172.21.15.33:0/207849196 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aac022470 con 0x7f2ac0105e90 2024-04-20T21:25:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.462+0000 7f2ac6a71700 1 -- 172.21.15.33:0/207849196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac01a2f20 con 0x7f2ac0105e90 2024-04-20T21:25:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.463+0000 7f2abdffb700 1 -- 172.21.15.33:0/207849196 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00f6c0 con 0x7f2ac0105e90 2024-04-20T21:25:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.463+0000 7f2ac6a71700 1 -- 172.21.15.33:0/207849196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac01a3340 con 0x7f2ac0105e90 2024-04-20T21:25:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.464+0000 7f2ac6a71700 1 -- 172.21.15.33:0/207849196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac002df90 con 0x7f2ac0105e90 2024-04-20T21:25:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.466+0000 7f2abdffb700 1 -- 172.21.15.33:0/207849196 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f2aac02a070 con 0x7f2ac0105e90 2024-04-20T21:25:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.466+0000 7f2abdffb700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2ab00a3c30 0x7f2ab00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.466+0000 7f2abdffb700 1 -- 172.21.15.33:0/207849196 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f2aac0cfa50 con 0x7f2ac0105e90 2024-04-20T21:25:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.467+0000 7f2ac480d700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2ab00a3c30 0x7f2ab00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.468+0000 7f2ac480d700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2ab00a3c30 0x7f2ab00a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009a10 tx=0x7f2ab4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.470+0000 7f2abdffb700 1 -- 172.21.15.33:0/207849196 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2aac013070 con 0x7f2ac0105e90 2024-04-20T21:25:27.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:27.688+0000 7f2ac6a71700 1 -- 172.21.15.33:0/207849196 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2ac0062e90 con 0x7f2ab00a3c30 2024-04-20T21:25:28.633 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:28.633 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: Detected new or changed devices on smithi022 2024-04-20T21:25:28.633 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:28.633 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:28.633 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:25:28.633 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi022 to 17094M 2024-04-20T21:25:28.633 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:28.633 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:28.634 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:28.634 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:28.634 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:25:28.634 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:25:28.634 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:25:28.634 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: Detected new or changed devices on smithi022 2024-04-20T21:25:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:25:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi022 to 17094M 2024-04-20T21:25:28.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:28.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:28.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:28.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:28.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:25:28.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:25:28.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:25:28.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: Detected new or changed devices on smithi022 2024-04-20T21:25:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:25:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi022 to 17094M 2024-04-20T21:25:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:25:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:25:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:25:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:30.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:30 smithi022 ceph-mon[16843]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:30.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:30 smithi022 ceph-mon[16843]: from='client.? 172.21.15.33:0/2936803875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "728fe9c7-163a-4b08-b7fe-5cd2e76aaa99"}]: dispatch 2024-04-20T21:25:30.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:30 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "728fe9c7-163a-4b08-b7fe-5cd2e76aaa99"}]: dispatch 2024-04-20T21:25:30.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:30 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "728fe9c7-163a-4b08-b7fe-5cd2e76aaa99"}]': finished 2024-04-20T21:25:30.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:30 smithi022 ceph-mon[16843]: osdmap e10: 2 total, 1 up, 2 in 2024-04-20T21:25:30.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:30 smithi122 ceph-mon[22147]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:30 smithi122 ceph-mon[22147]: from='client.? 172.21.15.33:0/2936803875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "728fe9c7-163a-4b08-b7fe-5cd2e76aaa99"}]: dispatch 2024-04-20T21:25:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:30 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "728fe9c7-163a-4b08-b7fe-5cd2e76aaa99"}]: dispatch 2024-04-20T21:25:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:30 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "728fe9c7-163a-4b08-b7fe-5cd2e76aaa99"}]': finished 2024-04-20T21:25:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:30 smithi122 ceph-mon[22147]: osdmap e10: 2 total, 1 up, 2 in 2024-04-20T21:25:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:31.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:30 smithi033 ceph-mon[21626]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:31.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:30 smithi033 ceph-mon[21626]: from='client.? 172.21.15.33:0/2936803875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "728fe9c7-163a-4b08-b7fe-5cd2e76aaa99"}]: dispatch 2024-04-20T21:25:31.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:30 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "728fe9c7-163a-4b08-b7fe-5cd2e76aaa99"}]: dispatch 2024-04-20T21:25:31.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:30 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "728fe9c7-163a-4b08-b7fe-5cd2e76aaa99"}]': finished 2024-04-20T21:25:31.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:30 smithi033 ceph-mon[21626]: osdmap e10: 2 total, 1 up, 2 in 2024-04-20T21:25:31.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:31.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:31 smithi022 ceph-mon[16843]: from='client.? 172.21.15.33:0/603300060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:32.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:31 smithi122 ceph-mon[22147]: from='client.? 172.21.15.33:0/603300060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:32.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:31 smithi033 ceph-mon[21626]: from='client.? 172.21.15.33:0/603300060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:32.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:32 smithi022 ceph-mon[16843]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:33.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:32 smithi122 ceph-mon[22147]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:33.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:32 smithi033 ceph-mon[21626]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:34.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:34 smithi022 ceph-mon[16843]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:34.918 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:34 smithi033 ceph-mon[21626]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:35.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:34 smithi122 ceph-mon[22147]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:36.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:36 smithi033 ceph-mon[21626]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:36.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:36 smithi022 ceph-mon[16843]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:37.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:36 smithi122 ceph-mon[22147]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:38.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:38 smithi033 ceph-mon[21626]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:38.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:38 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T21:25:38.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:38 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:38.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:38 smithi033 ceph-mon[21626]: Deploying daemon osd.1 on smithi033 2024-04-20T21:25:38.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:38 smithi022 ceph-mon[16843]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:38.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T21:25:38.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:38.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:38 smithi022 ceph-mon[16843]: Deploying daemon osd.1 on smithi033 2024-04-20T21:25:39.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:38 smithi122 ceph-mon[22147]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:39.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T21:25:39.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:39.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:38 smithi122 ceph-mon[22147]: Deploying daemon osd.1 on smithi033 2024-04-20T21:25:40.754 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:40 smithi033 ceph-mon[21626]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:40.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:40 smithi022 ceph-mon[16843]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:41.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:40 smithi122 ceph-mon[22147]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:42.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:42 smithi033 ceph-mon[21626]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:42.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:42 smithi022 ceph-mon[16843]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:43.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:42 smithi122 ceph-mon[22147]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:45.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:44 smithi122 ceph-mon[22147]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:45.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:45.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:45.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:45.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:44 smithi033 ceph-mon[21626]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:45.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:45.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:45.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:45.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:44 smithi022 ceph-mon[16843]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:45.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:45.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:45.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:47.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:46 smithi122 ceph-mon[22147]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:47.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:47.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:47.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:47.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:47.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:47.047 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:46 smithi033 ceph-mon[21626]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:47.048 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:47.048 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:47.048 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:47.048 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:47.048 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:47.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:46 smithi022 ceph-mon[16843]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:47.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:47.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:47.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:47.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:47.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.425+0000 7f2abdffb700 1 -- 172.21.15.33:0/207849196 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2ac0062e90 con 0x7f2ab00a3c30 2024-04-20T21:25:47.424 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 1 on host 'smithi033' 2024-04-20T21:25:47.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.427+0000 7f2ac6a71700 1 -- 172.21.15.33:0/207849196 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2ab00a3c30 msgr2=0x7f2ab00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:47.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.427+0000 7f2ac6a71700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2ab00a3c30 0x7f2ab00a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009a10 tx=0x7f2ab4009190 comp rx=0 tx=0).stop 2024-04-20T21:25:47.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.427+0000 7f2ac6a71700 1 -- 172.21.15.33:0/207849196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac0105e90 msgr2=0x7f2ac0101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:47.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.427+0000 7f2ac6a71700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac0105e90 0x7f2ac0101510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2aac005910 tx=0x7f2aac0047b0 comp rx=0 tx=0).stop 2024-04-20T21:25:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.427+0000 7f2ac6a71700 1 -- 172.21.15.33:0/207849196 shutdown_connections 2024-04-20T21:25:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.427+0000 7f2ac6a71700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2ab00a3c30 0x7f2ab00a60f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.427+0000 7f2ac6a71700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ac01087c0 0x7f2ac0101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.427+0000 7f2ac6a71700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ac0103560 0x7f2ac0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.427+0000 7f2ac6a71700 1 --2- 172.21.15.33:0/207849196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ac0105e90 0x7f2ac0101510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.427+0000 7f2ac6a71700 1 -- 172.21.15.33:0/207849196 >> 172.21.15.33:0/207849196 conn(0x7f2ac00fcf70 msgr2=0x7f2ac0103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.427+0000 7f2ac6a71700 1 -- 172.21.15.33:0/207849196 shutdown_connections 2024-04-20T21:25:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:25:47.427+0000 7f2ac6a71700 1 -- 172.21.15.33:0/207849196 wait complete. 2024-04-20T21:25:48.053 DEBUG:teuthology.orchestra.run.smithi033:osd.1> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.1.service 2024-04-20T21:25:48.056 INFO:tasks.cephadm:Deploying osd.2 on smithi122 with /dev/vg_nvme/lv_4... 2024-04-20T21:25:48.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T21:25:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:25:48.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:48.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:48.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:48.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:48 smithi022 ceph-mon[16843]: from='osd.1 [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T21:25:48.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:48 smithi022 ceph-mon[16843]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T21:25:48.723 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:48.723 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:48.723 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:48.723 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:48 smithi122 ceph-mon[22147]: from='osd.1 [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T21:25:48.723 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:48 smithi122 ceph-mon[22147]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T21:25:48.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:48.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:48.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:48.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:48 smithi033 ceph-mon[21626]: from='osd.1 [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T21:25:48.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:48 smithi033 ceph-mon[21626]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T21:25:49.585 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:49 smithi033 ceph-mon[21626]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:49.585 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:49 smithi033 ceph-mon[21626]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T21:25:49.585 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:49 smithi033 ceph-mon[21626]: from='osd.1 [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:25:49.585 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:49 smithi033 ceph-mon[21626]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T21:25:49.585 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:49 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:49.585 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:49 smithi033 ceph-mon[21626]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:25:49.683 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:49 smithi122 ceph-mon[22147]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:49.683 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:49 smithi122 ceph-mon[22147]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T21:25:49.684 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:49 smithi122 ceph-mon[22147]: from='osd.1 [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:25:49.684 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:49 smithi122 ceph-mon[22147]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T21:25:49.684 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:49 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:49.684 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:49 smithi122 ceph-mon[22147]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:25:49.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:49 smithi022 ceph-mon[16843]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:49.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:49 smithi022 ceph-mon[16843]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T21:25:49.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:49 smithi022 ceph-mon[16843]: from='osd.1 [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:25:49.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:49 smithi022 ceph-mon[16843]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T21:25:49.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:49 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:49.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:49 smithi022 ceph-mon[16843]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:25:50.242 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T21:25:50.242 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T21:25:50.242 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-20T21:25:50.242 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-20T21:25:50.242 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110469 s, 949 MB/s 2024-04-20T21:25:50.242 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-20T21:25:50.559 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:50 smithi122 ceph-mon[22147]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-20T21:25:50.559 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:50 smithi122 ceph-mon[22147]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T21:25:50.559 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:50.559 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:50.559 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:50 smithi122 ceph-mon[22147]: from='osd.1 ' entity='osd.1' 2024-04-20T21:25:50.559 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:50.729 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:50 smithi033 ceph-mon[21626]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-20T21:25:50.730 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:50 smithi033 ceph-mon[21626]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T21:25:50.730 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:50.730 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:50.730 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:50 smithi033 ceph-mon[21626]: from='osd.1 ' entity='osd.1' 2024-04-20T21:25:50.730 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:50.730 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:25:50 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1[26204]: 2024-04-20T21:25:50.320+0000 7fe9e157c700 -1 osd.1 0 waiting for initial osdmap 2024-04-20T21:25:50.730 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:25:50 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1[26204]: 2024-04-20T21:25:50.415+0000 7fe9d8b66700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:25:50.861 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_4 2024-04-20T21:25:50.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:50 smithi022 ceph-mon[16843]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-20T21:25:50.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:50 smithi022 ceph-mon[16843]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T21:25:50.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:50.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:50.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:50 smithi022 ceph-mon[16843]: from='osd.1 ' entity='osd.1' 2024-04-20T21:25:50.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:51.101 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:25:51.512 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:51 smithi122 ceph-mon[22147]: purged_snaps scrub starts 2024-04-20T21:25:51.512 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:51 smithi122 ceph-mon[22147]: purged_snaps scrub ok 2024-04-20T21:25:51.512 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:51 smithi122 ceph-mon[22147]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:51.512 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:51.512 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:51.512 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:25:51.512 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:51.512 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:51.512 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:51.513 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:51.513 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:51.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:51 smithi033 ceph-mon[21626]: purged_snaps scrub starts 2024-04-20T21:25:51.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:51 smithi033 ceph-mon[21626]: purged_snaps scrub ok 2024-04-20T21:25:51.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:51 smithi033 ceph-mon[21626]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:51.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:51.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:51.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:25:51.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:51.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:51.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:51.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:51.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:51.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:51 smithi022 ceph-mon[16843]: purged_snaps scrub starts 2024-04-20T21:25:51.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:51 smithi022 ceph-mon[16843]: purged_snaps scrub ok 2024-04-20T21:25:51.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:51 smithi022 ceph-mon[16843]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:51.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:51.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:51.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:25:51.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:51.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:51.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:51.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:25:51.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:52.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.588+0000 7f32ff87e700 1 -- 172.21.15.122:0/1685780281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32f8102ab0 msgr2=0x7f32f8102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.588+0000 7f32ff87e700 1 --2- 172.21.15.122:0/1685780281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32f8102ab0 0x7f32f8102ed0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009990 tx=0x7f32f4009ca0 comp rx=0 tx=0).stop 2024-04-20T21:25:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.588+0000 7f32ff87e700 1 -- 172.21.15.122:0/1685780281 shutdown_connections 2024-04-20T21:25:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.588+0000 7f32ff87e700 1 --2- 172.21.15.122:0/1685780281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f8104650 0x7f32f810af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.588+0000 7f32ff87e700 1 --2- 172.21.15.122:0/1685780281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32f8102ab0 0x7f32f8102ed0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.588+0000 7f32ff87e700 1 --2- 172.21.15.122:0/1685780281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f8103cb0 0x7f32f8104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.588+0000 7f32ff87e700 1 -- 172.21.15.122:0/1685780281 >> 172.21.15.122:0/1685780281 conn(0x7f32f80fe050 msgr2=0x7f32f8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32ff87e700 1 -- 172.21.15.122:0/1685780281 shutdown_connections 2024-04-20T21:25:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32ff87e700 1 -- 172.21.15.122:0/1685780281 wait complete. 2024-04-20T21:25:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32ff87e700 1 Processor -- start 2024-04-20T21:25:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32ff87e700 1 -- start start 2024-04-20T21:25:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32ff87e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32f8102ab0 0x7f32f8198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32ff87e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f8103cb0 0x7f32f8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32ff87e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f8104650 0x7f32f81a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32ff87e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f32f810e710 con 0x7f32f8103cb0 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32ff87e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f32f810e590 con 0x7f32f8104650 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32ff87e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32f810e890 con 0x7f32f8102ab0 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32fd61a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32f8102ab0 0x7f32f8198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32fd61a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32f8102ab0 0x7f32f8198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36120/0 (socket says 172.21.15.122:36120) 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.589+0000 7f32fd61a700 1 -- 172.21.15.122:0/1855056532 learned_addr learned my addr 172.21.15.122:0/1855056532 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.590+0000 7f32fde1b700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f8104650 0x7f32f81a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.590+0000 7f32fd61a700 1 -- 172.21.15.122:0/1855056532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f8104650 msgr2=0x7f32f81a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.590+0000 7f32fd61a700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f8104650 0x7f32f81a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.590+0000 7f32fd61a700 1 -- 172.21.15.122:0/1855056532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f8103cb0 msgr2=0x7f32f8199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.590+0000 7f32fd61a700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f8103cb0 0x7f32f8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.590+0000 7f32fd61a700 1 -- 172.21.15.122:0/1855056532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f81a09f0 con 0x7f32f8102ab0 2024-04-20T21:25:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.590+0000 7f32fd61a700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32f8102ab0 0x7f32f8198df0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f32f40039c0 tx=0x7f32f40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:52.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.590+0000 7f32ee7fc700 1 -- 172.21.15.122:0/1855056532 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4003c70 con 0x7f32f8102ab0 2024-04-20T21:25:52.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.590+0000 7f32ee7fc700 1 -- 172.21.15.122:0/1855056532 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f4004830 con 0x7f32f8102ab0 2024-04-20T21:25:52.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.590+0000 7f32ee7fc700 1 -- 172.21.15.122:0/1855056532 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4005440 con 0x7f32f8102ab0 2024-04-20T21:25:52.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.590+0000 7f32ff87e700 1 -- 172.21.15.122:0/1855056532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f81a0c80 con 0x7f32f8102ab0 2024-04-20T21:25:52.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.590+0000 7f32ff87e700 1 -- 172.21.15.122:0/1855056532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f81a10d0 con 0x7f32f8102ab0 2024-04-20T21:25:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.591+0000 7f32ff87e700 1 -- 172.21.15.122:0/1855056532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f802df90 con 0x7f32f8102ab0 2024-04-20T21:25:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.592+0000 7f32ee7fc700 1 -- 172.21.15.122:0/1855056532 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f32f4003e10 con 0x7f32f8102ab0 2024-04-20T21:25:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.593+0000 7f32ee7fc700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f32e40a3c30 0x7f32e40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.593+0000 7f32ee7fc700 1 -- 172.21.15.122:0/1855056532 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f32f40d0080 con 0x7f32f8102ab0 2024-04-20T21:25:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.593+0000 7f32fce19700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f32e40a3c30 0x7f32e40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.594+0000 7f32fce19700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f32e40a3c30 0x7f32e40a60f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f32e80097b0 tx=0x7f32e8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:52.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.597+0000 7f32ee7fc700 1 -- 172.21.15.122:0/1855056532 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f32f4013070 con 0x7f32f8102ab0 2024-04-20T21:25:52.719 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:52 smithi033 ceph-mon[21626]: Detected new or changed devices on smithi033 2024-04-20T21:25:52.719 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:52 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi033 to 17093M 2024-04-20T21:25:52.719 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:52 smithi033 ceph-mon[21626]: osd.1 [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441] boot 2024-04-20T21:25:52.719 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:52 smithi033 ceph-mon[21626]: osdmap e13: 2 total, 2 up, 2 in 2024-04-20T21:25:52.719 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:52 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:52.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:52 smithi122 ceph-mon[22147]: Detected new or changed devices on smithi033 2024-04-20T21:25:52.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:52 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi033 to 17093M 2024-04-20T21:25:52.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:52 smithi122 ceph-mon[22147]: osd.1 [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441] boot 2024-04-20T21:25:52.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:52 smithi122 ceph-mon[22147]: osdmap e13: 2 total, 2 up, 2 in 2024-04-20T21:25:52.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:52 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:52.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:25:52.799+0000 7f32ff87e700 1 -- 172.21.15.122:0/1855056532 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f32f8062e90 con 0x7f32e40a3c30 2024-04-20T21:25:52.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:52 smithi022 ceph-mon[16843]: Detected new or changed devices on smithi033 2024-04-20T21:25:52.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:52 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi033 to 17093M 2024-04-20T21:25:52.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:52 smithi022 ceph-mon[16843]: osd.1 [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441] boot 2024-04-20T21:25:52.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:52 smithi022 ceph-mon[16843]: osdmap e13: 2 total, 2 up, 2 in 2024-04-20T21:25:52.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:52 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:53.699 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:53 smithi033 ceph-mon[21626]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:53.699 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:53 smithi033 ceph-mon[21626]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T21:25:53.700 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:53 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:25:53.700 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:53 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:25:53.700 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:53 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:53.713 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:53 smithi122 ceph-mon[22147]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:53.713 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:53 smithi122 ceph-mon[22147]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T21:25:53.713 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:53 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:25:53.713 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:53 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:25:53.713 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:53 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:53.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:53 smithi022 ceph-mon[16843]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:53.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:53 smithi022 ceph-mon[16843]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T21:25:53.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:53 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:25:53.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:53 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:25:53.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:53 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:54.605 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:54 smithi122 ceph-mon[22147]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:25:54.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:54 smithi033 ceph-mon[21626]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:25:54.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:54 smithi022 ceph-mon[16843]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:25:55.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:55 smithi122 ceph-mon[22147]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:55.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:55 smithi033 ceph-mon[21626]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:55.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:55 smithi022 ceph-mon[16843]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:56 smithi122 ceph-mon[22147]: from='client.? 172.21.15.122:0/379982581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdddd7e5-5a5e-4d68-9849-420d9f891c98"}]: dispatch 2024-04-20T21:25:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:56 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdddd7e5-5a5e-4d68-9849-420d9f891c98"}]: dispatch 2024-04-20T21:25:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:56 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdddd7e5-5a5e-4d68-9849-420d9f891c98"}]': finished 2024-04-20T21:25:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:56 smithi122 ceph-mon[22147]: osdmap e15: 3 total, 2 up, 3 in 2024-04-20T21:25:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:56 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:56 smithi122 ceph-mon[22147]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:56 smithi122 ceph-mon[22147]: from='client.? 172.21.15.122:0/1257594907' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:56.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:56 smithi033 ceph-mon[21626]: from='client.? 172.21.15.122:0/379982581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdddd7e5-5a5e-4d68-9849-420d9f891c98"}]: dispatch 2024-04-20T21:25:56.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:56 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdddd7e5-5a5e-4d68-9849-420d9f891c98"}]: dispatch 2024-04-20T21:25:56.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:56 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdddd7e5-5a5e-4d68-9849-420d9f891c98"}]': finished 2024-04-20T21:25:56.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:56 smithi033 ceph-mon[21626]: osdmap e15: 3 total, 2 up, 3 in 2024-04-20T21:25:56.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:56 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:56.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:56 smithi033 ceph-mon[21626]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:56.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:56 smithi033 ceph-mon[21626]: from='client.? 172.21.15.122:0/1257594907' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:56.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:56 smithi022 ceph-mon[16843]: from='client.? 172.21.15.122:0/379982581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdddd7e5-5a5e-4d68-9849-420d9f891c98"}]: dispatch 2024-04-20T21:25:56.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:56 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdddd7e5-5a5e-4d68-9849-420d9f891c98"}]: dispatch 2024-04-20T21:25:56.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:56 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdddd7e5-5a5e-4d68-9849-420d9f891c98"}]': finished 2024-04-20T21:25:56.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:56 smithi022 ceph-mon[16843]: osdmap e15: 3 total, 2 up, 3 in 2024-04-20T21:25:56.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:56 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:56.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:56 smithi022 ceph-mon[16843]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:56.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:56 smithi022 ceph-mon[16843]: from='client.? 172.21.15.122:0/1257594907' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:58.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:25:58 smithi022 ceph-mon[16843]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:59.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:25:58 smithi122 ceph-mon[22147]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:59.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:25:58 smithi033 ceph-mon[21626]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:00.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:00 smithi022 ceph-mon[16843]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:00.911 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:00 smithi122 ceph-mon[22147]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:01.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:00 smithi033 ceph-mon[21626]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:02.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:02 smithi022 ceph-mon[16843]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:03.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:02 smithi122 ceph-mon[22147]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:03.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:02 smithi033 ceph-mon[21626]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:03.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T21:26:03.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:03.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:03 smithi022 ceph-mon[16843]: Deploying daemon osd.2 on smithi122 2024-04-20T21:26:03.977 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:03 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T21:26:03.977 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:03 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:03.977 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:03 smithi122 ceph-mon[22147]: Deploying daemon osd.2 on smithi122 2024-04-20T21:26:04.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:03 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T21:26:04.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:03 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:04.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:03 smithi033 ceph-mon[21626]: Deploying daemon osd.2 on smithi122 2024-04-20T21:26:04.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:04 smithi022 ceph-mon[16843]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:04.955 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:04 smithi122 ceph-mon[22147]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:05.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:04 smithi033 ceph-mon[21626]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:06.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:06 smithi022 ceph-mon[16843]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:06.942 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:06 smithi122 ceph-mon[22147]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:07.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:06 smithi033 ceph-mon[21626]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:08 smithi122 ceph-mon[22147]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:08.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:08 smithi022 ceph-mon[16843]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:08.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:08.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:08.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:09.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:08 smithi033 ceph-mon[21626]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:09.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:09.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:09.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:10.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:10 smithi022 ceph-mon[16843]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:11.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:10 smithi122 ceph-mon[22147]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:11.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:10 smithi033 ceph-mon[21626]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:11.484 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 2 on host 'smithi122' 2024-04-20T21:26:11.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.485+0000 7f32ee7fc700 1 -- 172.21.15.122:0/1855056532 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f32f8062e90 con 0x7f32e40a3c30 2024-04-20T21:26:11.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.487+0000 7f32ff87e700 1 -- 172.21.15.122:0/1855056532 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f32e40a3c30 msgr2=0x7f32e40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:11.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.487+0000 7f32ff87e700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f32e40a3c30 0x7f32e40a60f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f32e80097b0 tx=0x7f32e8006d20 comp rx=0 tx=0).stop 2024-04-20T21:26:11.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.487+0000 7f32ff87e700 1 -- 172.21.15.122:0/1855056532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32f8102ab0 msgr2=0x7f32f8198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:11.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.487+0000 7f32ff87e700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32f8102ab0 0x7f32f8198df0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f32f40039c0 tx=0x7f32f40047b0 comp rx=0 tx=0).stop 2024-04-20T21:26:11.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.487+0000 7f32ff87e700 1 -- 172.21.15.122:0/1855056532 shutdown_connections 2024-04-20T21:26:11.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.487+0000 7f32ff87e700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f32e40a3c30 0x7f32e40a60f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:11.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.487+0000 7f32ff87e700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f8104650 0x7f32f81a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:11.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.487+0000 7f32ff87e700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32f8102ab0 0x7f32f8198df0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:11.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.487+0000 7f32ff87e700 1 --2- 172.21.15.122:0/1855056532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f8103cb0 0x7f32f8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:11.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.487+0000 7f32ff87e700 1 -- 172.21.15.122:0/1855056532 >> 172.21.15.122:0/1855056532 conn(0x7f32f80fe050 msgr2=0x7f32f8106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:11.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.487+0000 7f32ff87e700 1 -- 172.21.15.122:0/1855056532 shutdown_connections 2024-04-20T21:26:11.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:26:11.487+0000 7f32ff87e700 1 -- 172.21.15.122:0/1855056532 wait complete. 2024-04-20T21:26:12.088 DEBUG:teuthology.orchestra.run.smithi122:osd.2> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.2.service 2024-04-20T21:26:12.090 INFO:tasks.cephadm:Deploying osd.3 on smithi022 with /dev/vg_nvme/lv_3... 2024-04-20T21:26:12.091 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T21:26:12.554 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:26:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:12 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:12 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:12 smithi022 ceph-mon[16843]: from='osd.2 [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T21:26:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:12 smithi022 ceph-mon[16843]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T21:26:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:12 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:12 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:12.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:12 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:12.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:12 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:12.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:12 smithi122 ceph-mon[22147]: from='osd.2 [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T21:26:12.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:12 smithi122 ceph-mon[22147]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T21:26:12.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:12 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:12.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:12 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:12.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:12 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:12.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:12 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:12.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:12 smithi033 ceph-mon[21626]: from='osd.2 [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T21:26:12.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:12 smithi033 ceph-mon[21626]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T21:26:12.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:12 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:12.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:12 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:13.726 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:13 smithi022 ceph-mon[16843]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:13.726 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:13 smithi022 ceph-mon[16843]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T21:26:13.726 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:13 smithi022 ceph-mon[16843]: from='osd.2 [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:26:13.726 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:13 smithi022 ceph-mon[16843]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T21:26:13.726 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:13 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:26:13.727 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:13 smithi022 ceph-mon[16843]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:26:13.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:13 smithi122 ceph-mon[22147]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:13.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:13 smithi122 ceph-mon[22147]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T21:26:13.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:13 smithi122 ceph-mon[22147]: from='osd.2 [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:26:13.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:13 smithi122 ceph-mon[22147]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T21:26:13.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:13 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:26:13.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:13 smithi122 ceph-mon[22147]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:26:13.776 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:26:13 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2[27302]: 2024-04-20T21:26:13.450+0000 7f5cb41f6700 -1 osd.2 0 waiting for initial osdmap 2024-04-20T21:26:13.776 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:26:13 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2[27302]: 2024-04-20T21:26:13.454+0000 7f5caffe9700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:26:13.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:13 smithi033 ceph-mon[21626]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:13.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:13 smithi033 ceph-mon[21626]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T21:26:13.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:13 smithi033 ceph-mon[21626]: from='osd.2 [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:26:13.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:13 smithi033 ceph-mon[21626]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T21:26:13.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:13 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:26:13.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:13 smithi033 ceph-mon[21626]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:26:14.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:14 smithi022 ceph-mon[16843]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-20T21:26:14.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:14 smithi022 ceph-mon[16843]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T21:26:14.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:26:14.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:26:14.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:14 smithi122 ceph-mon[22147]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-20T21:26:14.752 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:14 smithi122 ceph-mon[22147]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T21:26:14.752 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:26:14.752 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:26:14.808 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:14 smithi033 ceph-mon[21626]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-20T21:26:14.808 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:14 smithi033 ceph-mon[21626]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T21:26:14.808 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:26:14.809 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:26:15.323 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T21:26:15.323 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T21:26:15.323 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-04-20T21:26:15.323 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-04-20T21:26:15.323 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0391923 s, 268 MB/s 2024-04-20T21:26:15.323 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-04-20T21:26:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: purged_snaps scrub starts 2024-04-20T21:26:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: purged_snaps scrub ok 2024-04-20T21:26:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: osd.2 [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534] boot 2024-04-20T21:26:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: osdmap e18: 3 total, 3 up, 3 in 2024-04-20T21:26:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:26:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: Detected new or changed devices on smithi122 2024-04-20T21:26:15.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:15.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:15.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:15.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi122 to 17088M 2024-04-20T21:26:15.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:15.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:15.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:15.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: purged_snaps scrub starts 2024-04-20T21:26:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: purged_snaps scrub ok 2024-04-20T21:26:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: osd.2 [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534] boot 2024-04-20T21:26:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: osdmap e18: 3 total, 3 up, 3 in 2024-04-20T21:26:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:26:15.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: Detected new or changed devices on smithi122 2024-04-20T21:26:15.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:15.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:15.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:15.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi122 to 17088M 2024-04-20T21:26:15.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:15.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:15.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:15.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:15.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: purged_snaps scrub starts 2024-04-20T21:26:15.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: purged_snaps scrub ok 2024-04-20T21:26:15.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:26:15.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: osd.2 [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534] boot 2024-04-20T21:26:15.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: osdmap e18: 3 total, 3 up, 3 in 2024-04-20T21:26:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:26:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: Detected new or changed devices on smithi122 2024-04-20T21:26:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi122 to 17088M 2024-04-20T21:26:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:16.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_3 2024-04-20T21:26:16.477 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:26:16.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:16 smithi022 ceph-mon[16843]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T21:26:16.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:16 smithi022 ceph-mon[16843]: pgmap v92: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:16.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:26:16.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:16.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:16.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:16.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:16.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:16 smithi122 ceph-mon[22147]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T21:26:16.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:16 smithi122 ceph-mon[22147]: pgmap v92: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:16.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:26:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:16 smithi033 ceph-mon[21626]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T21:26:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:16 smithi033 ceph-mon[21626]: pgmap v92: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:26:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:16.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:16.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:17.740 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:17 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T21:26:17.740 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:17 smithi022 ceph-mon[16843]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T21:26:17.740 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:17 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:26:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:17 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T21:26:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:17 smithi122 ceph-mon[22147]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T21:26:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:17 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:26:17.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:17 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T21:26:17.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:17 smithi033 ceph-mon[21626]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T21:26:17.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:17 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:26:18.038 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:26:17 smithi022 sudo[33381]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T21:26:18.039 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:26:17 smithi022 sudo[33381]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:26:18.039 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:26:17 smithi022 sudo[33381]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:26:18.039 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:26:17 smithi022 sudo[33381]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:26:18.039 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:26:17 smithi022 sudo[33386]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T21:26:18.039 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:26:17 smithi022 sudo[33386]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:26:18.039 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:26:17 smithi022 sudo[33386]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:26:18.039 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:26:17 smithi022 sudo[33386]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:26:18.160 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:26:17 smithi033 sudo[30311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T21:26:18.160 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:26:17 smithi033 sudo[30311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:26:18.160 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:26:17 smithi033 sudo[30311]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:26:18.160 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:26:17 smithi033 sudo[30311]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:26:18.160 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:26:17 smithi033 sudo[30314]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T21:26:18.161 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:26:17 smithi033 sudo[30314]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:26:18.161 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:26:17 smithi033 sudo[30314]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:26:18.161 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:26:17 smithi033 sudo[30314]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:26:18.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 sudo[31344]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T21:26:18.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 sudo[31344]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:26:18.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 sudo[31344]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:26:18.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 sudo[31344]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:26:18.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 sudo[31347]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T21:26:18.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 sudo[31347]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:26:18.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 sudo[31347]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:26:18.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 sudo[31347]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:26:18.276 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:26:17 smithi122 sudo[31338]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T21:26:18.276 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:26:17 smithi122 sudo[31338]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:26:18.276 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:26:17 smithi122 sudo[31338]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:26:18.276 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:26:17 smithi122 sudo[31338]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:26:18.276 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:26:17 smithi122 sudo[31341]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T21:26:18.276 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:26:17 smithi122 sudo[31341]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:26:18.276 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:26:17 smithi122 sudo[31341]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:26:18.276 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:26:17 smithi122 sudo[31341]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:26:18.290 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 sudo[33407]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T21:26:18.290 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 sudo[33407]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:26:18.290 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 sudo[33407]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:26:18.290 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 sudo[33407]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:26:18.290 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 sudo[33410]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T21:26:18.291 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 sudo[33410]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:26:18.291 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 sudo[33410]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:26:18.291 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 sudo[33410]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:26:18.497 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 sudo[30317]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T21:26:18.497 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 sudo[30317]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:26:18.497 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 sudo[30317]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:26:18.497 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 sudo[30317]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:26:18.498 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 sudo[30320]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T21:26:18.498 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 sudo[30320]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:26:18.498 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 sudo[30320]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:26:18.498 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 sudo[30320]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:26:18.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.565+0000 7f698ac00700 1 -- 172.21.15.22:0/1437830480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6984104cf0 msgr2=0x7f6984105110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:18.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.565+0000 7f698ac00700 1 --2- 172.21.15.22:0/1437830480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6984104cf0 0x7f6984105110 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f69740099c0 tx=0x7f6974009cd0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.566+0000 7f698ac00700 1 -- 172.21.15.22:0/1437830480 shutdown_connections 2024-04-20T21:26:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.566+0000 7f698ac00700 1 --2- 172.21.15.22:0/1437830480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984106890 0x7f698410d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.566+0000 7f698ac00700 1 --2- 172.21.15.22:0/1437830480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6984104cf0 0x7f6984105110 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.566+0000 7f698ac00700 1 --2- 172.21.15.22:0/1437830480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6984105ef0 0x7f6984106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.566+0000 7f698ac00700 1 -- 172.21.15.22:0/1437830480 >> 172.21.15.22:0/1437830480 conn(0x7f69841002b0 msgr2=0x7f69841026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.566+0000 7f698ac00700 1 -- 172.21.15.22:0/1437830480 shutdown_connections 2024-04-20T21:26:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.566+0000 7f698ac00700 1 -- 172.21.15.22:0/1437830480 wait complete. 2024-04-20T21:26:18.568 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T21:26:18.568 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T21:26:18.569 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:18.569 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:18.569 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:18.569 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:18.569 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:26:18.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:26:18.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:26:18.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:26:18.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:26:18.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:26:18.571 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:26:18.571 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:26:18.571 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:26:18.571 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:26:18.572 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:26:18.572 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:26:18.572 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:26:18.572 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:26:18.572 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:18 smithi022 ceph-mon[16843]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:26:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.567+0000 7f698ac00700 1 Processor -- start 2024-04-20T21:26:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.567+0000 7f698ac00700 1 -- start start 2024-04-20T21:26:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.568+0000 7f698ac00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984104cf0 0x7f698419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.568+0000 7f698ac00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6984105ef0 0x7f698419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.568+0000 7f698ac00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6984106890 0x7f69841a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.568+0000 7f698ac00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6984110900 con 0x7f6984105ef0 2024-04-20T21:26:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.568+0000 7f698ac00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6984110780 con 0x7f6984106890 2024-04-20T21:26:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.568+0000 7f698ac00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6984110a80 con 0x7f6984104cf0 2024-04-20T21:26:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.568+0000 7f6983e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984104cf0 0x7f698419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.568+0000 7f6983e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984104cf0 0x7f698419aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:49240/0 (socket says 172.21.15.22:49240) 2024-04-20T21:26:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.568+0000 7f6983e00700 1 -- 172.21.15.22:0/2891874415 learned_addr learned my addr 172.21.15.22:0/2891874415 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:26:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.568+0000 7f697be00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6984105ef0 0x7f698419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.568+0000 7f6988c00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6984106890 0x7f69841a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.569+0000 7f6983e00700 1 -- 172.21.15.22:0/2891874415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6984106890 msgr2=0x7f69841a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.569+0000 7f6983e00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6984106890 0x7f69841a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.569+0000 7f6983e00700 1 -- 172.21.15.22:0/2891874415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6984105ef0 msgr2=0x7f698419b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.569+0000 7f6983e00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6984105ef0 0x7f698419b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.569+0000 7f6983e00700 1 -- 172.21.15.22:0/2891874415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69841a2b40 con 0x7f6984104cf0 2024-04-20T21:26:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.569+0000 7f6983e00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984104cf0 0x7f698419aee0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f697400fa00 tx=0x7f697400fa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.569+0000 7f6981600700 1 -- 172.21.15.22:0/2891874415 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974027070 con 0x7f6984104cf0 2024-04-20T21:26:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.569+0000 7f6981600700 1 -- 172.21.15.22:0/2891874415 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6974022950 con 0x7f6984104cf0 2024-04-20T21:26:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.569+0000 7f698ac00700 1 -- 172.21.15.22:0/2891874415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69841a2dd0 con 0x7f6984104cf0 2024-04-20T21:26:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.569+0000 7f6981600700 1 -- 172.21.15.22:0/2891874415 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697400fe60 con 0x7f6984104cf0 2024-04-20T21:26:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.569+0000 7f698ac00700 1 -- 172.21.15.22:0/2891874415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69841a31c0 con 0x7f6984104cf0 2024-04-20T21:26:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.571+0000 7f698ac00700 1 -- 172.21.15.22:0/2891874415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6984069740 con 0x7f6984104cf0 2024-04-20T21:26:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.572+0000 7f6981600700 1 -- 172.21.15.22:0/2891874415 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f697402a070 con 0x7f6984104cf0 2024-04-20T21:26:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.574+0000 7f6981600700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f69680a3c30 0x7f69680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.574+0000 7f6981600700 1 -- 172.21.15.22:0/2891874415 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3237+0+0 (secure 0 0 0) 0x7f69740cfb50 con 0x7f6984104cf0 2024-04-20T21:26:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.574+0000 7f697be00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f69680a3c30 0x7f69680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.575+0000 7f697be00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f69680a3c30 0x7f69680a60f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f696c009c60 tx=0x7f696c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:18.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.578+0000 7f6981600700 1 -- 172.21.15.22:0/2891874415 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6974013070 con 0x7f6984104cf0 2024-04-20T21:26:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T21:26:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T21:26:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:18.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:26:18.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:26:18.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:26:18.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:26:18.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:26:18.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:26:18.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:26:18.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:26:18.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:26:18.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:26:18.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:26:18.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:26:18.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:26:18.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:26:18.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:18 smithi122 ceph-mon[22147]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:26:18.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T21:26:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T21:26:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:26:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:26:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:26:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:26:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:26:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:26:18.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:26:18.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:26:18.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:26:18.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:26:18.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:26:18.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:26:18.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:26:18.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:26:18.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:18 smithi033 ceph-mon[21626]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:26:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:18.822+0000 7f698ac00700 1 -- 172.21.15.22:0/2891874415 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6984062e90 con 0x7f69680a3c30 2024-04-20T21:26:19.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:19 smithi122 ceph-mon[22147]: osdmap e22: 3 total, 3 up, 3 in 2024-04-20T21:26:19.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:19 smithi122 ceph-mon[22147]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:26:19.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:19 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:26:19.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:19 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:26:19.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:19 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:19.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:19 smithi022 ceph-mon[16843]: osdmap e22: 3 total, 3 up, 3 in 2024-04-20T21:26:19.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:19 smithi022 ceph-mon[16843]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:26:19.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:26:19.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:26:19.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:19.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:19 smithi033 ceph-mon[21626]: osdmap e22: 3 total, 3 up, 3 in 2024-04-20T21:26:19.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:19 smithi033 ceph-mon[21626]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:26:19.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:19 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:26:19.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:19 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:26:19.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:19 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:20.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:20.542+0000 7f6981600700 1 -- 172.21.15.22:0/2891874415 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6974022470 con 0x7f6984104cf0 2024-04-20T21:26:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:20 smithi022 ceph-mon[16843]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:20.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:20 smithi033 ceph-mon[21626]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:21.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:20 smithi122 ceph-mon[22147]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:21.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:21 smithi022 ceph-mon[16843]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-20T21:26:21.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:21 smithi033 ceph-mon[21626]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-20T21:26:22.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:21 smithi122 ceph-mon[22147]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-20T21:26:22.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:22 smithi033 ceph-mon[21626]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:22.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:22 smithi022 ceph-mon[16843]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:23.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:22 smithi122 ceph-mon[22147]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:23.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:23 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1342318968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9989d732-245e-417b-9a43-83c77374159d"}]: dispatch 2024-04-20T21:26:23.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:23 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1342318968' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9989d732-245e-417b-9a43-83c77374159d"}]': finished 2024-04-20T21:26:23.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:23 smithi033 ceph-mon[21626]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T21:26:23.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:23 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:23.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:23 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1342318968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9989d732-245e-417b-9a43-83c77374159d"}]: dispatch 2024-04-20T21:26:23.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:23 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1342318968' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9989d732-245e-417b-9a43-83c77374159d"}]': finished 2024-04-20T21:26:23.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:23 smithi022 ceph-mon[16843]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T21:26:23.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:23 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:24.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:23 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1342318968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9989d732-245e-417b-9a43-83c77374159d"}]: dispatch 2024-04-20T21:26:24.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:23 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1342318968' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9989d732-245e-417b-9a43-83c77374159d"}]': finished 2024-04-20T21:26:24.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:23 smithi122 ceph-mon[22147]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T21:26:24.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:23 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:24.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:24 smithi022 ceph-mon[16843]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:24.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:24 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1927254583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:26:25.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:24 smithi122 ceph-mon[22147]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:25.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:24 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1927254583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:26:25.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:24 smithi033 ceph-mon[21626]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:25.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:24 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1927254583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:26:26.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:26 smithi022 ceph-mon[16843]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:27.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:26 smithi122 ceph-mon[22147]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:27.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:26 smithi033 ceph-mon[21626]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:28.887 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:28 smithi022 ceph-mon[16843]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:28 smithi122 ceph-mon[22147]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:29.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:28 smithi033 ceph-mon[21626]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:30.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:30 smithi022 ceph-mon[16843]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:30 smithi122 ceph-mon[22147]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:31.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:30 smithi033 ceph-mon[21626]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:32.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:32 smithi022 ceph-mon[16843]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:32.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:32 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T21:26:32.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:32 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:32.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:32 smithi022 ceph-mon[16843]: Deploying daemon osd.3 on smithi022 2024-04-20T21:26:33.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:32 smithi122 ceph-mon[22147]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:33.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:32 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T21:26:33.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:32 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:33.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:32 smithi122 ceph-mon[22147]: Deploying daemon osd.3 on smithi022 2024-04-20T21:26:33.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:32 smithi033 ceph-mon[21626]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:33.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:32 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T21:26:33.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:32 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:33.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:32 smithi033 ceph-mon[21626]: Deploying daemon osd.3 on smithi022 2024-04-20T21:26:34.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:34 smithi022 ceph-mon[16843]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:35.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:34 smithi122 ceph-mon[22147]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:35.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:34 smithi033 ceph-mon[21626]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:36.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:36 smithi022 ceph-mon[16843]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:37.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:36 smithi122 ceph-mon[22147]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:37.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:36 smithi033 ceph-mon[21626]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:38.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:38 smithi022 ceph-mon[16843]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:39.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:38 smithi122 ceph-mon[22147]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:39.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:38 smithi033 ceph-mon[21626]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:41.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:40 smithi022 ceph-mon[16843]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:41.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:40 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:41.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:40 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:41.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:40 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:41.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:40 smithi122 ceph-mon[22147]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:41.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:40 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:41.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:40 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:41.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:40 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:41.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:40 smithi033 ceph-mon[21626]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:41.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:40 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:41.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:40 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:41.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:40 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:43.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:42 smithi022 ceph-mon[16843]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:43.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:42 smithi122 ceph-mon[22147]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:43.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:42 smithi033 ceph-mon[21626]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:44.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:44.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:44.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:44.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:44.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:44.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:44 smithi022 ceph-mon[16843]: from='osd.3 [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T21:26:44.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:44 smithi022 ceph-mon[16843]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T21:26:44.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:44.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:44.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:44.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:44.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:44.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:44 smithi122 ceph-mon[22147]: from='osd.3 [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T21:26:44.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:44 smithi122 ceph-mon[22147]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T21:26:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:44.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:44.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:44.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:44.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:44.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:44 smithi033 ceph-mon[21626]: from='osd.3 [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T21:26:44.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:44 smithi033 ceph-mon[21626]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T21:26:44.721 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 3 on host 'smithi022' 2024-04-20T21:26:44.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.720+0000 7f6981600700 1 -- 172.21.15.22:0/2891874415 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6984062e90 con 0x7f69680a3c30 2024-04-20T21:26:44.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.725+0000 7f698ac00700 1 -- 172.21.15.22:0/2891874415 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f69680a3c30 msgr2=0x7f69680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.725+0000 7f698ac00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f69680a3c30 0x7f69680a60f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f696c009c60 tx=0x7f696c009260 comp rx=0 tx=0).stop 2024-04-20T21:26:44.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.725+0000 7f698ac00700 1 -- 172.21.15.22:0/2891874415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984104cf0 msgr2=0x7f698419aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.725+0000 7f698ac00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984104cf0 0x7f698419aee0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f697400fa00 tx=0x7f697400fa30 comp rx=0 tx=0).stop 2024-04-20T21:26:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.725+0000 7f698ac00700 1 -- 172.21.15.22:0/2891874415 shutdown_connections 2024-04-20T21:26:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.725+0000 7f698ac00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f69680a3c30 0x7f69680a60f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.725+0000 7f698ac00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6984106890 0x7f69841a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.725+0000 7f698ac00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984104cf0 0x7f698419aee0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.725+0000 7f698ac00700 1 --2- 172.21.15.22:0/2891874415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6984105ef0 0x7f698419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.725+0000 7f698ac00700 1 -- 172.21.15.22:0/2891874415 >> 172.21.15.22:0/2891874415 conn(0x7f69841002b0 msgr2=0x7f69841091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.725+0000 7f698ac00700 1 -- 172.21.15.22:0/2891874415 shutdown_connections 2024-04-20T21:26:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:26:44.725+0000 7f698ac00700 1 -- 172.21.15.22:0/2891874415 wait complete. 2024-04-20T21:26:45.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:45 smithi022 ceph-mon[16843]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:45.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:45 smithi022 ceph-mon[16843]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T21:26:45.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:45 smithi022 ceph-mon[16843]: from='osd.3 [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:26:45.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:45 smithi022 ceph-mon[16843]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T21:26:45.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:45.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:45 smithi022 ceph-mon[16843]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:26:45.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:45.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:45.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:45.443 DEBUG:teuthology.orchestra.run.smithi022:osd.3> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.3.service 2024-04-20T21:26:45.447 INFO:tasks.cephadm:Deploying osd.4 on smithi033 with /dev/vg_nvme/lv_3... 2024-04-20T21:26:45.448 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T21:26:45.477 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:45 smithi033 ceph-mon[21626]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:45.478 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:45 smithi033 ceph-mon[21626]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T21:26:45.478 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:45 smithi033 ceph-mon[21626]: from='osd.3 [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:26:45.478 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:45 smithi033 ceph-mon[21626]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T21:26:45.478 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:45 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:45.478 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:45 smithi033 ceph-mon[21626]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:26:45.478 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:45 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:45.478 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:45 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:45.478 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:45 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:45.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:45 smithi122 ceph-mon[22147]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:45.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:45 smithi122 ceph-mon[22147]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T21:26:45.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:45 smithi122 ceph-mon[22147]: from='osd.3 [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:26:45.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:45 smithi122 ceph-mon[22147]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T21:26:45.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:45 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:45.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:45 smithi122 ceph-mon[22147]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:26:45.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:45 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:45.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:45 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:45.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:45 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:45.701 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:26:46.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:46 smithi022 ceph-mon[16843]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-20T21:26:46.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:46 smithi022 ceph-mon[16843]: osdmap e25: 4 total, 3 up, 4 in 2024-04-20T21:26:46.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:46.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:46 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:46.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:46 smithi122 ceph-mon[22147]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-20T21:26:46.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:46 smithi122 ceph-mon[22147]: osdmap e25: 4 total, 3 up, 4 in 2024-04-20T21:26:46.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:46.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:46 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:46 smithi033 ceph-mon[21626]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-20T21:26:46.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:46 smithi033 ceph-mon[21626]: osdmap e25: 4 total, 3 up, 4 in 2024-04-20T21:26:46.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:46.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:46 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:47.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:47 smithi033 ceph-mon[21626]: purged_snaps scrub starts 2024-04-20T21:26:47.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:47 smithi033 ceph-mon[21626]: purged_snaps scrub ok 2024-04-20T21:26:47.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:47 smithi033 ceph-mon[21626]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:47.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:47 smithi033 ceph-mon[21626]: osd.3 [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045] boot 2024-04-20T21:26:47.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:47 smithi033 ceph-mon[21626]: osdmap e26: 4 total, 4 up, 4 in 2024-04-20T21:26:47.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:47 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:47.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:47 smithi033 ceph-mon[21626]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T21:26:47.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:47 smithi022 ceph-mon[16843]: purged_snaps scrub starts 2024-04-20T21:26:47.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:47 smithi022 ceph-mon[16843]: purged_snaps scrub ok 2024-04-20T21:26:47.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:47 smithi022 ceph-mon[16843]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:47.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:47 smithi022 ceph-mon[16843]: osd.3 [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045] boot 2024-04-20T21:26:47.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:47 smithi022 ceph-mon[16843]: osdmap e26: 4 total, 4 up, 4 in 2024-04-20T21:26:47.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:47 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:47.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:47 smithi022 ceph-mon[16843]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T21:26:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:47 smithi122 ceph-mon[22147]: purged_snaps scrub starts 2024-04-20T21:26:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:47 smithi122 ceph-mon[22147]: purged_snaps scrub ok 2024-04-20T21:26:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:47 smithi122 ceph-mon[22147]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:47 smithi122 ceph-mon[22147]: osd.3 [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045] boot 2024-04-20T21:26:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:47 smithi122 ceph-mon[22147]: osdmap e26: 4 total, 4 up, 4 in 2024-04-20T21:26:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:47 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:47 smithi122 ceph-mon[22147]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T21:26:47.613 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T21:26:47.613 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T21:26:47.614 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-20T21:26:47.614 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-20T21:26:47.614 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111179 s, 943 MB/s 2024-04-20T21:26:47.614 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-20T21:26:48.251 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_3 2024-04-20T21:26:48.544 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:26:49.521 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:49 smithi033 ceph-mon[21626]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:49.521 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:49 smithi033 ceph-mon[21626]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T21:26:49.521 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:49 smithi033 ceph-mon[21626]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T21:26:49.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:49 smithi122 ceph-mon[22147]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:49.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:49 smithi122 ceph-mon[22147]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T21:26:49.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:49 smithi122 ceph-mon[22147]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T21:26:49.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:49 smithi022 ceph-mon[16843]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:49.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:49 smithi022 ceph-mon[16843]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T21:26:49.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:49 smithi022 ceph-mon[16843]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T21:26:50.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.079+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1789759916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d010a9d0 msgr2=0x7fa7d010ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.079+0000 7fa7d51b5700 1 --2- 172.21.15.33:0/1789759916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d010a9d0 0x7fa7d010ce40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009990 tx=0x7fa7c0009ca0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.079+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1789759916 shutdown_connections 2024-04-20T21:26:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.079+0000 7fa7d51b5700 1 --2- 172.21.15.33:0/1789759916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d010a9d0 0x7fa7d010ce40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.079+0000 7fa7d51b5700 1 --2- 172.21.15.33:0/1789759916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7d0076a00 0x7fa7d0074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.079+0000 7fa7d51b5700 1 --2- 172.21.15.33:0/1789759916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d0075430 0x7fa7d00758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.079+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1789759916 >> 172.21.15.33:0/1789759916 conn(0x7fa7d00fe070 msgr2=0x7fa7d01004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.079+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1789759916 shutdown_connections 2024-04-20T21:26:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.079+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1789759916 wait complete. 2024-04-20T21:26:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.080+0000 7fa7d51b5700 1 Processor -- start 2024-04-20T21:26:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.080+0000 7fa7d51b5700 1 -- start start 2024-04-20T21:26:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.080+0000 7fa7d51b5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7d0075430 0x7fa7d019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.080+0000 7fa7d51b5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d0076a00 0x7fa7d019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.080+0000 7fa7d51b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d010a9d0 0x7fa7d01a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.080+0000 7fa7d51b5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d0110970 con 0x7fa7d0075430 2024-04-20T21:26:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.080+0000 7fa7d51b5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01107f0 con 0x7fa7d010a9d0 2024-04-20T21:26:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.080+0000 7fa7d51b5700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d0110af0 con 0x7fa7d0076a00 2024-04-20T21:26:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.080+0000 7fa7cf59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d010a9d0 0x7fa7d01a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.080+0000 7fa7cf59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d010a9d0 0x7fa7d01a2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48450/0 (socket says 172.21.15.33:48450) 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.080+0000 7fa7cf59e700 1 -- 172.21.15.33:0/1008726672 learned_addr learned my addr 172.21.15.33:0/1008726672 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.080+0000 7fa7ce59c700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d0076a00 0x7fa7d019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.081+0000 7fa7cf59e700 1 -- 172.21.15.33:0/1008726672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d0076a00 msgr2=0x7fa7d019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.081+0000 7fa7cf59e700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d0076a00 0x7fa7d019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.081+0000 7fa7cf59e700 1 -- 172.21.15.33:0/1008726672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7d0075430 msgr2=0x7fa7d019af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.081+0000 7fa7cf59e700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7d0075430 0x7fa7d019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.081+0000 7fa7cf59e700 1 -- 172.21.15.33:0/1008726672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d01a2bd0 con 0x7fa7d010a9d0 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.081+0000 7fa7cf59e700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d010a9d0 0x7fa7d01a2530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00059d0 tx=0x7fa7c0004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.081+0000 7fa7c7fff700 1 -- 172.21.15.33:0/1008726672 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0026070 con 0x7fa7d010a9d0 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.081+0000 7fa7c7fff700 1 -- 172.21.15.33:0/1008726672 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7c00041d0 con 0x7fa7d010a9d0 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.081+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1008726672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d01a2e60 con 0x7fa7d010a9d0 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.081+0000 7fa7c7fff700 1 -- 172.21.15.33:0/1008726672 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00054f0 con 0x7fa7d010a9d0 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.081+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1008726672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d01a3330 con 0x7fa7d010a9d0 2024-04-20T21:26:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.082+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1008726672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d0069740 con 0x7fa7d010a9d0 2024-04-20T21:26:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.083+0000 7fa7c7fff700 1 -- 172.21.15.33:0/1008726672 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa7c0029070 con 0x7fa7d010a9d0 2024-04-20T21:26:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.084+0000 7fa7c7fff700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa7bc0a3d40 0x7fa7bc0a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.084+0000 7fa7c7fff700 1 -- 172.21.15.33:0/1008726672 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7fa7c00cfd70 con 0x7fa7d010a9d0 2024-04-20T21:26:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.084+0000 7fa7ced9d700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa7bc0a3d40 0x7fa7bc0a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.085+0000 7fa7ced9d700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa7bc0a3d40 0x7fa7bc0a6200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800ab90 tx=0x7fa7c8009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.089+0000 7fa7c7fff700 1 -- 172.21.15.33:0/1008726672 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa7c005dff0 con 0x7fa7d010a9d0 2024-04-20T21:26:50.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:26:50.310+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1008726672 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa7d0062e90 con 0x7fa7bc0a3d40 2024-04-20T21:26:51.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:51.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: osdmap e30: 4 total, 4 up, 4 in 2024-04-20T21:26:51.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:26:51.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:26:51.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:26:51.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:51.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: Detected new or changed devices on smithi022 2024-04-20T21:26:51.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:51.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:51.159 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:51.159 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:51.159 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi022 to 8547M 2024-04-20T21:26:51.159 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:51.159 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:51.159 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:51.159 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:51.191 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: osdmap e30: 4 total, 4 up, 4 in 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: Detected new or changed devices on smithi022 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi022 to 8547M 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:51.192 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:51.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:51.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: osdmap e30: 4 total, 4 up, 4 in 2024-04-20T21:26:51.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:26:51.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:26:51.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:26:51.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:51.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: Detected new or changed devices on smithi022 2024-04-20T21:26:51.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:51.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:51.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:51.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:51.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi022 to 8547M 2024-04-20T21:26:51.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:51.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:51.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:51.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:26:53.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:52 smithi022 ceph-mon[16843]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:53.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:52 smithi122 ceph-mon[22147]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:53.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:52 smithi033 ceph-mon[21626]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:54.278 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:53 smithi122 ceph-mon[22147]: from='client.? 172.21.15.33:0/2117695190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ff6d17-2779-4462-bb8b-1b50e117c78c"}]: dispatch 2024-04-20T21:26:54.278 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:53 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ff6d17-2779-4462-bb8b-1b50e117c78c"}]: dispatch 2024-04-20T21:26:54.278 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:53 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12ff6d17-2779-4462-bb8b-1b50e117c78c"}]': finished 2024-04-20T21:26:54.278 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:53 smithi122 ceph-mon[22147]: osdmap e31: 5 total, 4 up, 5 in 2024-04-20T21:26:54.278 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:53 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:54.278 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:53 smithi122 ceph-mon[22147]: from='client.? 172.21.15.33:0/559708304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:26:54.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:53 smithi033 ceph-mon[21626]: from='client.? 172.21.15.33:0/2117695190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ff6d17-2779-4462-bb8b-1b50e117c78c"}]: dispatch 2024-04-20T21:26:54.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:53 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ff6d17-2779-4462-bb8b-1b50e117c78c"}]: dispatch 2024-04-20T21:26:54.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:53 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12ff6d17-2779-4462-bb8b-1b50e117c78c"}]': finished 2024-04-20T21:26:54.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:53 smithi033 ceph-mon[21626]: osdmap e31: 5 total, 4 up, 5 in 2024-04-20T21:26:54.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:53 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:54.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:53 smithi033 ceph-mon[21626]: from='client.? 172.21.15.33:0/559708304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:26:54.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:53 smithi022 ceph-mon[16843]: from='client.? 172.21.15.33:0/2117695190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ff6d17-2779-4462-bb8b-1b50e117c78c"}]: dispatch 2024-04-20T21:26:54.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:53 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ff6d17-2779-4462-bb8b-1b50e117c78c"}]: dispatch 2024-04-20T21:26:54.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:53 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12ff6d17-2779-4462-bb8b-1b50e117c78c"}]': finished 2024-04-20T21:26:54.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:53 smithi022 ceph-mon[16843]: osdmap e31: 5 total, 4 up, 5 in 2024-04-20T21:26:54.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:53 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:54.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:53 smithi022 ceph-mon[16843]: from='client.? 172.21.15.33:0/559708304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:26:55.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:55 smithi033 ceph-mon[21626]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-20T21:26:55.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:55 smithi022 ceph-mon[16843]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-20T21:26:55.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:55 smithi122 ceph-mon[22147]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-20T21:26:57.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:57 smithi033 ceph-mon[21626]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-20T21:26:57.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:57 smithi022 ceph-mon[16843]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-20T21:26:57.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:57 smithi122 ceph-mon[22147]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-20T21:26:59.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:26:59 smithi033 ceph-mon[21626]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T21:26:59.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:26:59 smithi022 ceph-mon[16843]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T21:26:59.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:26:59 smithi122 ceph-mon[22147]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T21:27:01.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:01 smithi033 ceph-mon[21626]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-20T21:27:01.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T21:27:01.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:01.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:01 smithi022 ceph-mon[16843]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-20T21:27:01.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T21:27:01.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:01.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:01 smithi122 ceph-mon[22147]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-20T21:27:01.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T21:27:01.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:02.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:02 smithi033 ceph-mon[21626]: Deploying daemon osd.4 on smithi033 2024-04-20T21:27:02.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:02 smithi022 ceph-mon[16843]: Deploying daemon osd.4 on smithi033 2024-04-20T21:27:02.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:02 smithi122 ceph-mon[22147]: Deploying daemon osd.4 on smithi033 2024-04-20T21:27:03.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:03 smithi022 ceph-mon[16843]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:27:03.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:03 smithi033 ceph-mon[21626]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:27:03.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:03 smithi122 ceph-mon[22147]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:27:05.400 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:05 smithi033 ceph-mon[21626]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:05.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:05 smithi022 ceph-mon[16843]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:05.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:05 smithi122 ceph-mon[22147]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:07.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:07 smithi033 ceph-mon[21626]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:07.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:07 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:07.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:07 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:07.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:07 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:07 smithi022 ceph-mon[16843]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:07 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:07 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:07 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:07 smithi122 ceph-mon[22147]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:07 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:07 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:07 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:09.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:09 smithi033 ceph-mon[21626]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:09.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:09 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:09.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:09 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:09.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:09 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:09.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:09 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:09.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:09 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:09 smithi022 ceph-mon[16843]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:09 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:09 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:09 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:09 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:09 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:09 smithi122 ceph-mon[22147]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:09 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:09 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:09 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:09 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:09 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.050+0000 7fa7c7fff700 1 -- 172.21.15.33:0/1008726672 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa7d0062e90 con 0x7fa7bc0a3d40 2024-04-20T21:27:10.049 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 4 on host 'smithi033' 2024-04-20T21:27:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.053+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1008726672 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa7bc0a3d40 msgr2=0x7fa7bc0a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.053+0000 7fa7d51b5700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa7bc0a3d40 0x7fa7bc0a6200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800ab90 tx=0x7fa7c8009250 comp rx=0 tx=0).stop 2024-04-20T21:27:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.053+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1008726672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d010a9d0 msgr2=0x7fa7d01a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.053+0000 7fa7d51b5700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d010a9d0 0x7fa7d01a2530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00059d0 tx=0x7fa7c0004790 comp rx=0 tx=0).stop 2024-04-20T21:27:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.053+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1008726672 shutdown_connections 2024-04-20T21:27:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.053+0000 7fa7d51b5700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa7bc0a3d40 0x7fa7bc0a6200 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.053+0000 7fa7d51b5700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d010a9d0 0x7fa7d01a2530 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.053+0000 7fa7d51b5700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7d0075430 0x7fa7d019af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.053+0000 7fa7d51b5700 1 --2- 172.21.15.33:0/1008726672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d0076a00 0x7fa7d019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.053+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1008726672 >> 172.21.15.33:0/1008726672 conn(0x7fa7d00fe070 msgr2=0x7fa7d0106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.053+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1008726672 shutdown_connections 2024-04-20T21:27:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:27:10.053+0000 7fa7d51b5700 1 -- 172.21.15.33:0/1008726672 wait complete. 2024-04-20T21:27:10.681 DEBUG:teuthology.orchestra.run.smithi033:osd.4> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.4.service 2024-04-20T21:27:10.683 INFO:tasks.cephadm:Deploying osd.5 on smithi122 with /dev/vg_nvme/lv_3... 2024-04-20T21:27:10.683 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T21:27:10.926 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:27:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:11 smithi122 ceph-mon[22147]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:11 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:11 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:11 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:11 smithi122 ceph-mon[22147]: from='osd.4 [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T21:27:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:11 smithi122 ceph-mon[22147]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T21:27:11.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:11 smithi033 ceph-mon[21626]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:11.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:11 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:11.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:11 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:11.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:11 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:11.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:11 smithi033 ceph-mon[21626]: from='osd.4 [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T21:27:11.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:11 smithi033 ceph-mon[21626]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T21:27:11.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:11 smithi022 ceph-mon[16843]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:11.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:11 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:11.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:11 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:11.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:11 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:11.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:11 smithi022 ceph-mon[16843]: from='osd.4 [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T21:27:11.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:11 smithi022 ceph-mon[16843]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T21:27:12.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:12 smithi122 ceph-mon[22147]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T21:27:12.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:12 smithi122 ceph-mon[22147]: from='osd.4 [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:27:12.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:12 smithi122 ceph-mon[22147]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T21:27:12.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:12 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:12.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:12 smithi122 ceph-mon[22147]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:27:12.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:12 smithi033 ceph-mon[21626]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T21:27:12.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:12 smithi033 ceph-mon[21626]: from='osd.4 [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:27:12.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:12 smithi033 ceph-mon[21626]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T21:27:12.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:12 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:12.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:12 smithi033 ceph-mon[21626]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:27:12.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:12 smithi022 ceph-mon[16843]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T21:27:12.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:12 smithi022 ceph-mon[16843]: from='osd.4 [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:27:12.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:12 smithi022 ceph-mon[16843]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T21:27:12.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:12 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:12.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:12 smithi022 ceph-mon[16843]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:27:12.867 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T21:27:12.867 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T21:27:12.867 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-20T21:27:12.867 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-20T21:27:12.867 INFO:teuthology.orchestra.run.smithi122.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0134858 s, 778 MB/s 2024-04-20T21:27:12.867 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-20T21:27:13.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:13 smithi033 ceph-mon[21626]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:13.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:13 smithi033 ceph-mon[21626]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-20T21:27:13.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:13 smithi033 ceph-mon[21626]: osdmap e33: 5 total, 4 up, 5 in 2024-04-20T21:27:13.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:13 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:13.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:13 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:13.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:13 smithi033 ceph-mon[21626]: from='osd.4 ' entity='osd.4' 2024-04-20T21:27:13.306 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:27:12 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:27:12.990+0000 7f989e94b700 -1 osd.4 0 waiting for initial osdmap 2024-04-20T21:27:13.306 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:27:13 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:27:13.086+0000 7f9896736700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:27:13.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:13 smithi022 ceph-mon[16843]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:13.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:13 smithi022 ceph-mon[16843]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-20T21:27:13.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:13 smithi022 ceph-mon[16843]: osdmap e33: 5 total, 4 up, 5 in 2024-04-20T21:27:13.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:13 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:13.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:13 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:13.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:13 smithi022 ceph-mon[16843]: from='osd.4 ' entity='osd.4' 2024-04-20T21:27:13.523 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:13 smithi122 ceph-mon[22147]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:13.523 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:13 smithi122 ceph-mon[22147]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-20T21:27:13.523 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:13 smithi122 ceph-mon[22147]: osdmap e33: 5 total, 4 up, 5 in 2024-04-20T21:27:13.523 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:13 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:13.523 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:13 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:13.523 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:13 smithi122 ceph-mon[22147]: from='osd.4 ' entity='osd.4' 2024-04-20T21:27:13.549 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_3 2024-04-20T21:27:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:27:14.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:14 smithi033 ceph-mon[21626]: purged_snaps scrub starts 2024-04-20T21:27:14.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:14 smithi033 ceph-mon[21626]: purged_snaps scrub ok 2024-04-20T21:27:14.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:14.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:14.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:14.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:14.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:14.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:14.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:14.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:14.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:14.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:14.184 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:14 smithi122 ceph-mon[22147]: purged_snaps scrub starts 2024-04-20T21:27:14.184 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:14 smithi122 ceph-mon[22147]: purged_snaps scrub ok 2024-04-20T21:27:14.184 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:14.184 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:14.184 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:14.185 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:14.185 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:14.185 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:14.185 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:14.185 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:14.185 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:14.185 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:14.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:14 smithi022 ceph-mon[16843]: purged_snaps scrub starts 2024-04-20T21:27:14.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:14 smithi022 ceph-mon[16843]: purged_snaps scrub ok 2024-04-20T21:27:14.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:14.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:14.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:14.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:14.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:14.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:14.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:14.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:14.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:14.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:15.299 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:15 smithi122 ceph-mon[22147]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:15.299 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:15 smithi122 ceph-mon[22147]: Detected new or changed devices on smithi033 2024-04-20T21:27:15.300 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:15 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi033 to 8546M 2024-04-20T21:27:15.300 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:15 smithi122 ceph-mon[22147]: osd.4 [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998] boot 2024-04-20T21:27:15.300 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:15 smithi122 ceph-mon[22147]: osdmap e34: 5 total, 5 up, 5 in 2024-04-20T21:27:15.300 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:15.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:15 smithi033 ceph-mon[21626]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:15.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:15 smithi033 ceph-mon[21626]: Detected new or changed devices on smithi033 2024-04-20T21:27:15.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:15 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi033 to 8546M 2024-04-20T21:27:15.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:15 smithi033 ceph-mon[21626]: osd.4 [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998] boot 2024-04-20T21:27:15.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:15 smithi033 ceph-mon[21626]: osdmap e34: 5 total, 5 up, 5 in 2024-04-20T21:27:15.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:15.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:15 smithi022 ceph-mon[16843]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:27:15.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:15 smithi022 ceph-mon[16843]: Detected new or changed devices on smithi033 2024-04-20T21:27:15.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:15 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi033 to 8546M 2024-04-20T21:27:15.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:15 smithi022 ceph-mon[16843]: osd.4 [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998] boot 2024-04-20T21:27:15.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:15 smithi022 ceph-mon[16843]: osdmap e34: 5 total, 5 up, 5 in 2024-04-20T21:27:15.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:27:15.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.511+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2203810502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7db8102630 msgr2=0x7f7db8109ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:15.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.511+0000 7f7dbcd19700 1 --2- 172.21.15.122:0/2203810502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7db8102630 0x7f7db8109ef0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7db000b0d0 tx=0x7f7db000b3e0 comp rx=0 tx=0).stop 2024-04-20T21:27:15.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.512+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2203810502 shutdown_connections 2024-04-20T21:27:15.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.512+0000 7f7dbcd19700 1 --2- 172.21.15.122:0/2203810502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7db8102630 0x7f7db8109ef0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:15.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.512+0000 7f7dbcd19700 1 --2- 172.21.15.122:0/2203810502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7db8102f50 0x7f7db8103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:15.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.512+0000 7f7dbcd19700 1 --2- 172.21.15.122:0/2203810502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8101c90 0x7f7db81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:15.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.512+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2203810502 >> 172.21.15.122:0/2203810502 conn(0x7f7db80fc6c0 msgr2=0x7f7db80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:15.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.512+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2203810502 shutdown_connections 2024-04-20T21:27:15.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.512+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2203810502 wait complete. 2024-04-20T21:27:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.512+0000 7f7dbcd19700 1 Processor -- start 2024-04-20T21:27:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.512+0000 7f7dbcd19700 1 -- start start 2024-04-20T21:27:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.512+0000 7f7dbcd19700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7db8101c90 0x7f7db81016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7dbcd19700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8102630 0x7f7db819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7dbcd19700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7db8102f50 0x7f7db81a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7dbcd19700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7db810d640 con 0x7f7db8101c90 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7dbcd19700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7db810d4c0 con 0x7f7db8102630 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7dbcd19700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7db810d7c0 con 0x7f7db8102f50 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7db6d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7db8102f50 0x7f7db81a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7db6d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7db8102f50 0x7f7db81a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34440/0 (socket says 172.21.15.122:34440) 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7db6d9d700 1 -- 172.21.15.122:0/2381235909 learned_addr learned my addr 172.21.15.122:0/2381235909 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7db5d9b700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8102630 0x7f7db819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7db6d9d700 1 -- 172.21.15.122:0/2381235909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8102630 msgr2=0x7f7db819fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7db6d9d700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8102630 0x7f7db819fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7db6d9d700 1 -- 172.21.15.122:0/2381235909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7db8101c90 msgr2=0x7f7db81016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7db6d9d700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7db8101c90 0x7f7db81016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7db6d9d700 1 -- 172.21.15.122:0/2381235909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db81a2b60 con 0x7f7db8102f50 2024-04-20T21:27:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7db6d9d700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7db8102f50 0x7f7db81a2380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7db0006070 tx=0x7f7db0007dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7da77fe700 1 -- 172.21.15.122:0/2381235909 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0021020 con 0x7f7db8102f50 2024-04-20T21:27:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2381235909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db81a2df0 con 0x7f7db8102f50 2024-04-20T21:27:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7da77fe700 1 -- 172.21.15.122:0/2381235909 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7db001d950 con 0x7f7db8102f50 2024-04-20T21:27:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7da77fe700 1 -- 172.21.15.122:0/2381235909 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0005510 con 0x7f7db8102f50 2024-04-20T21:27:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.513+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2381235909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db81a33d0 con 0x7f7db8102f50 2024-04-20T21:27:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.514+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2381235909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db8069740 con 0x7f7db8102f50 2024-04-20T21:27:15.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.515+0000 7f7da77fe700 1 -- 172.21.15.122:0/2381235909 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7db0024020 con 0x7f7db8102f50 2024-04-20T21:27:15.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.516+0000 7f7da77fe700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7da00a3c40 0x7f7da00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:15.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.516+0000 7f7da77fe700 1 -- 172.21.15.122:0/2381235909 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f7db000f020 con 0x7f7db8102f50 2024-04-20T21:27:15.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.516+0000 7f7db659c700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7da00a3c40 0x7f7da00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:15.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.517+0000 7f7db659c700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7da00a3c40 0x7f7da00a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7db806af80 tx=0x7f7da8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:15.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.521+0000 7f7da77fe700 1 -- 172.21.15.122:0/2381235909 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7db009e140 con 0x7f7db8102f50 2024-04-20T21:27:15.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:15.728+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2381235909 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7db8062e90 con 0x7f7da00a3c40 2024-04-20T21:27:16.366 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:16 smithi122 ceph-mon[22147]: osdmap e35: 5 total, 5 up, 5 in 2024-04-20T21:27:16.366 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:27:16.366 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:27:16.366 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:16.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:16 smithi022 ceph-mon[16843]: osdmap e35: 5 total, 5 up, 5 in 2024-04-20T21:27:16.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:27:16.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:27:16.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:16.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:16 smithi033 ceph-mon[21626]: osdmap e35: 5 total, 5 up, 5 in 2024-04-20T21:27:16.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:27:16.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:27:16.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:17.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:17 smithi122 ceph-mon[22147]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:17.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:17 smithi122 ceph-mon[22147]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:27:17.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:17 smithi022 ceph-mon[16843]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:17.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:17 smithi022 ceph-mon[16843]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:27:17.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:17 smithi033 ceph-mon[21626]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:17.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:17 smithi033 ceph-mon[21626]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:27:19.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:19 smithi022 ceph-mon[16843]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:19.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:19 smithi022 ceph-mon[16843]: from='client.? 172.21.15.122:0/3097064895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a6256a-db56-4eab-bd2c-3f7a499cb166"}]: dispatch 2024-04-20T21:27:19.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:19 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a6256a-db56-4eab-bd2c-3f7a499cb166"}]: dispatch 2024-04-20T21:27:19.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:19 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70a6256a-db56-4eab-bd2c-3f7a499cb166"}]': finished 2024-04-20T21:27:19.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:19 smithi022 ceph-mon[16843]: osdmap e36: 6 total, 5 up, 6 in 2024-04-20T21:27:19.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:19.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:19 smithi122 ceph-mon[22147]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:19.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:19 smithi122 ceph-mon[22147]: from='client.? 172.21.15.122:0/3097064895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a6256a-db56-4eab-bd2c-3f7a499cb166"}]: dispatch 2024-04-20T21:27:19.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:19 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a6256a-db56-4eab-bd2c-3f7a499cb166"}]: dispatch 2024-04-20T21:27:19.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:19 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70a6256a-db56-4eab-bd2c-3f7a499cb166"}]': finished 2024-04-20T21:27:19.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:19 smithi122 ceph-mon[22147]: osdmap e36: 6 total, 5 up, 6 in 2024-04-20T21:27:19.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:19 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:19.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:19 smithi033 ceph-mon[21626]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:19.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:19 smithi033 ceph-mon[21626]: from='client.? 172.21.15.122:0/3097064895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a6256a-db56-4eab-bd2c-3f7a499cb166"}]: dispatch 2024-04-20T21:27:19.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:19 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a6256a-db56-4eab-bd2c-3f7a499cb166"}]: dispatch 2024-04-20T21:27:19.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:19 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70a6256a-db56-4eab-bd2c-3f7a499cb166"}]': finished 2024-04-20T21:27:19.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:19 smithi033 ceph-mon[21626]: osdmap e36: 6 total, 5 up, 6 in 2024-04-20T21:27:19.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:19 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:20.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:20 smithi022 ceph-mon[16843]: from='client.? 172.21.15.122:0/693116958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:27:20.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:20 smithi122 ceph-mon[22147]: from='client.? 172.21.15.122:0/693116958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:27:20.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:20 smithi033 ceph-mon[21626]: from='client.? 172.21.15.122:0/693116958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:27:21.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:21 smithi022 ceph-mon[16843]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:21.454 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:21 smithi033 ceph-mon[21626]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:21.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:21 smithi122 ceph-mon[22147]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:23.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:23 smithi122 ceph-mon[22147]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:23.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:23 smithi022 ceph-mon[16843]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:23.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:23 smithi033 ceph-mon[21626]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:25.340 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:25 smithi122 ceph-mon[22147]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:25.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:25 smithi022 ceph-mon[16843]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:25.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:25 smithi033 ceph-mon[21626]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:26.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:26 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T21:27:26.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:26 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:26.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:26 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T21:27:26.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:26 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:26.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:26 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T21:27:26.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:26 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:27.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:27 smithi122 ceph-mon[22147]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:27.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:27 smithi122 ceph-mon[22147]: Deploying daemon osd.5 on smithi122 2024-04-20T21:27:27.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:27 smithi022 ceph-mon[16843]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:27.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:27 smithi022 ceph-mon[16843]: Deploying daemon osd.5 on smithi122 2024-04-20T21:27:27.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:27 smithi033 ceph-mon[21626]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:27.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:27 smithi033 ceph-mon[21626]: Deploying daemon osd.5 on smithi122 2024-04-20T21:27:29.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:29 smithi122 ceph-mon[22147]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:29.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:29 smithi022 ceph-mon[16843]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:29.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:29 smithi033 ceph-mon[21626]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:31.376 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:31 smithi122 ceph-mon[22147]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:31.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:31 smithi022 ceph-mon[16843]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:31.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:31 smithi033 ceph-mon[21626]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:32 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:32.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:32 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:32.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:32 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:32.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:32 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:32.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:32 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:32.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:32 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:32.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:32 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:32.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:32 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:32.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:32 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:33.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:33 smithi122 ceph-mon[22147]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:33.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:33 smithi033 ceph-mon[21626]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:33.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:33 smithi022 ceph-mon[16843]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:35.321 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 5 on host 'smithi122' 2024-04-20T21:27:35.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.322+0000 7f7da77fe700 1 -- 172.21.15.122:0/2381235909 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7db8062e90 con 0x7f7da00a3c40 2024-04-20T21:27:35.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.324+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2381235909 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7da00a3c40 msgr2=0x7f7da00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:35.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.324+0000 7f7dbcd19700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7da00a3c40 0x7f7da00a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7db806af80 tx=0x7f7da8006d20 comp rx=0 tx=0).stop 2024-04-20T21:27:35.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.324+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2381235909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7db8102f50 msgr2=0x7f7db81a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:35.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.324+0000 7f7dbcd19700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7db8102f50 0x7f7db81a2380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7db0006070 tx=0x7f7db0007dc0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.324+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2381235909 shutdown_connections 2024-04-20T21:27:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.324+0000 7f7dbcd19700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7da00a3c40 0x7f7da00a6100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.324+0000 7f7dbcd19700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7db8102f50 0x7f7db81a2380 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.324+0000 7f7dbcd19700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7db8101c90 0x7f7db81016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.324+0000 7f7dbcd19700 1 --2- 172.21.15.122:0/2381235909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db8102630 0x7f7db819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.324+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2381235909 >> 172.21.15.122:0/2381235909 conn(0x7f7db80fc6c0 msgr2=0x7f7db8105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.324+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2381235909 shutdown_connections 2024-04-20T21:27:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:27:35.324+0000 7f7dbcd19700 1 -- 172.21.15.122:0/2381235909 wait complete. 2024-04-20T21:27:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:35 smithi122 ceph-mon[22147]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:35 smithi122 ceph-mon[22147]: from='osd.5 [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T21:27:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:35 smithi122 ceph-mon[22147]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T21:27:35.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:35 smithi022 ceph-mon[16843]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:35.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:35.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:35.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:35.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:35 smithi022 ceph-mon[16843]: from='osd.5 [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T21:27:35.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:35 smithi022 ceph-mon[16843]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T21:27:35.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:35 smithi033 ceph-mon[21626]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:35.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:35.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:35.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:35.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:35.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:35 smithi033 ceph-mon[21626]: from='osd.5 [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T21:27:35.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:35 smithi033 ceph-mon[21626]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T21:27:36.010 DEBUG:teuthology.orchestra.run.smithi122:osd.5> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.5.service 2024-04-20T21:27:36.012 INFO:tasks.cephadm:Deploying osd.6 on smithi022 with /dev/vg_nvme/lv_2... 2024-04-20T21:27:36.012 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T21:27:36.496 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:27:37.513 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:37 smithi122 ceph-mon[22147]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:37.513 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:37 smithi122 ceph-mon[22147]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T21:27:37.513 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:37 smithi122 ceph-mon[22147]: from='osd.5 [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:27:37.514 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:37 smithi122 ceph-mon[22147]: osdmap e37: 6 total, 5 up, 6 in 2024-04-20T21:27:37.514 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:37 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:37.514 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:37 smithi122 ceph-mon[22147]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:27:37.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:37 smithi033 ceph-mon[21626]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:37.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:37 smithi033 ceph-mon[21626]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T21:27:37.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:37 smithi033 ceph-mon[21626]: from='osd.5 [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:27:37.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:37 smithi033 ceph-mon[21626]: osdmap e37: 6 total, 5 up, 6 in 2024-04-20T21:27:37.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:37 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:37.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:37 smithi033 ceph-mon[21626]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:27:37.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:37 smithi022 ceph-mon[16843]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:37.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:37 smithi022 ceph-mon[16843]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T21:27:37.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:37 smithi022 ceph-mon[16843]: from='osd.5 [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:27:37.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:37 smithi022 ceph-mon[16843]: osdmap e37: 6 total, 5 up, 6 in 2024-04-20T21:27:37.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:37 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:37.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:37 smithi022 ceph-mon[16843]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:27:38.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:38 smithi122 ceph-mon[22147]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-20T21:27:38.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:38 smithi122 ceph-mon[22147]: osdmap e38: 6 total, 5 up, 6 in 2024-04-20T21:27:38.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:38.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:38 smithi122 ceph-mon[22147]: from='osd.5 ' entity='osd.5' 2024-04-20T21:27:38.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:38.526 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:27:38 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:27:38.176+0000 7f8ddfa0f700 -1 osd.5 0 waiting for initial osdmap 2024-04-20T21:27:38.526 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:27:38 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:27:38.323+0000 7f8dd77fa700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:27:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:38 smithi033 ceph-mon[21626]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-20T21:27:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:38 smithi033 ceph-mon[21626]: osdmap e38: 6 total, 5 up, 6 in 2024-04-20T21:27:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:38 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:38 smithi033 ceph-mon[21626]: from='osd.5 ' entity='osd.5' 2024-04-20T21:27:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:38 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:38.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:38 smithi022 ceph-mon[16843]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-20T21:27:38.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:38 smithi022 ceph-mon[16843]: osdmap e38: 6 total, 5 up, 6 in 2024-04-20T21:27:38.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:38.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:38 smithi022 ceph-mon[16843]: from='osd.5 ' entity='osd.5' 2024-04-20T21:27:38.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T21:27:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T21:27:39.488 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-04-20T21:27:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-04-20T21:27:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0342248 s, 306 MB/s 2024-04-20T21:27:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-04-20T21:27:39.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: purged_snaps scrub starts 2024-04-20T21:27:39.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: purged_snaps scrub ok 2024-04-20T21:27:39.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: pgmap v152: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:39.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: osdmap e39: 6 total, 5 up, 6 in 2024-04-20T21:27:39.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:39.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:39.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:39.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:39.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:39.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:39.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:39.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:39.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:39.555 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:39.594 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: purged_snaps scrub starts 2024-04-20T21:27:39.594 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: purged_snaps scrub ok 2024-04-20T21:27:39.594 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: pgmap v152: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:39.594 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: osdmap e39: 6 total, 5 up, 6 in 2024-04-20T21:27:39.594 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:39.594 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:39.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:39.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:39.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:39.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:39.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:39.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:39.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:39.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:39.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: purged_snaps scrub starts 2024-04-20T21:27:39.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: purged_snaps scrub ok 2024-04-20T21:27:39.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: pgmap v152: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:39.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: osdmap e39: 6 total, 5 up, 6 in 2024-04-20T21:27:39.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:39.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:39.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:39.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:39.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:39.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:39.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:39.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:39.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:27:39.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:40.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_2 2024-04-20T21:27:40.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:40 smithi022 ceph-mon[16843]: Detected new or changed devices on smithi122 2024-04-20T21:27:40.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:40 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi122 to 8544M 2024-04-20T21:27:40.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:40 smithi022 ceph-mon[16843]: osd.5 [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470] boot 2024-04-20T21:27:40.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:40 smithi022 ceph-mon[16843]: osdmap e40: 6 total, 6 up, 6 in 2024-04-20T21:27:40.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:40 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:40.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:40 smithi122 ceph-mon[22147]: Detected new or changed devices on smithi122 2024-04-20T21:27:40.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:40 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi122 to 8544M 2024-04-20T21:27:40.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:40 smithi122 ceph-mon[22147]: osd.5 [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470] boot 2024-04-20T21:27:40.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:40 smithi122 ceph-mon[22147]: osdmap e40: 6 total, 6 up, 6 in 2024-04-20T21:27:40.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:40 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:40.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:40 smithi033 ceph-mon[21626]: Detected new or changed devices on smithi122 2024-04-20T21:27:40.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:40 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi122 to 8544M 2024-04-20T21:27:40.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:40 smithi033 ceph-mon[21626]: osd.5 [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470] boot 2024-04-20T21:27:40.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:40 smithi033 ceph-mon[21626]: osdmap e40: 6 total, 6 up, 6 in 2024-04-20T21:27:40.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:40 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:27:40.934 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:27:41.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:41 smithi022 ceph-mon[16843]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:41.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:41 smithi022 ceph-mon[16843]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T21:27:41.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:41 smithi122 ceph-mon[22147]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:41.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:41 smithi122 ceph-mon[22147]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T21:27:41.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:41 smithi033 ceph-mon[21626]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:27:41.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:41 smithi033 ceph-mon[21626]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T21:27:42.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:42 smithi022 ceph-mon[16843]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T21:27:42.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:42 smithi122 ceph-mon[22147]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T21:27:42.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:42 smithi033 ceph-mon[21626]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T21:27:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.914+0000 7f374be00700 1 -- 172.21.15.22:0/1413695765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3744103cc0 msgr2=0x7f3744104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.914+0000 7f374be00700 1 --2- 172.21.15.22:0/1413695765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3744103cc0 0x7f3744104120 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f37300099c0 tx=0x7f3730009cd0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.915+0000 7f374be00700 1 -- 172.21.15.22:0/1413695765 shutdown_connections 2024-04-20T21:27:42.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.915+0000 7f374be00700 1 --2- 172.21.15.22:0/1413695765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3744104660 0x7f374410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.915+0000 7f374be00700 1 --2- 172.21.15.22:0/1413695765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3744102ac0 0x7f3744102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.915+0000 7f374be00700 1 --2- 172.21.15.22:0/1413695765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3744103cc0 0x7f3744104120 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.915+0000 7f374be00700 1 -- 172.21.15.22:0/1413695765 >> 172.21.15.22:0/1413695765 conn(0x7f37440fe020 msgr2=0x7f37441004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:42.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.915+0000 7f374be00700 1 -- 172.21.15.22:0/1413695765 shutdown_connections 2024-04-20T21:27:42.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.915+0000 7f374be00700 1 -- 172.21.15.22:0/1413695765 wait complete. 2024-04-20T21:27:42.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.917+0000 7f374be00700 1 Processor -- start 2024-04-20T21:27:42.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.917+0000 7f374be00700 1 -- start start 2024-04-20T21:27:42.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.918+0000 7f374be00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3744102ac0 0x7f3744198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.918+0000 7f374be00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3744103cc0 0x7f37441992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.918+0000 7f374be00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3744104660 0x7f37441a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.918+0000 7f374be00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f374410e6f0 con 0x7f3744102ac0 2024-04-20T21:27:42.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.918+0000 7f374be00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f374410e570 con 0x7f3744104660 2024-04-20T21:27:42.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.918+0000 7f374be00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f374410e870 con 0x7f3744103cc0 2024-04-20T21:27:42.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.918+0000 7f3749400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3744102ac0 0x7f3744198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.918+0000 7f3749400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3744102ac0 0x7f3744198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35312/0 (socket says 172.21.15.22:35312) 2024-04-20T21:27:42.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.918+0000 7f3749400700 1 -- 172.21.15.22:0/3194099186 learned_addr learned my addr 172.21.15.22:0/3194099186 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:27:42.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.918+0000 7f3748a00700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3744103cc0 0x7f37441992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.918+0000 7f3749e00700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3744104660 0x7f37441a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.919+0000 7f3749400700 1 -- 172.21.15.22:0/3194099186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3744103cc0 msgr2=0x7f37441992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.919+0000 7f3749400700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3744103cc0 0x7f37441992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.919+0000 7f3749400700 1 -- 172.21.15.22:0/3194099186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3744104660 msgr2=0x7f37441a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.919+0000 7f3749400700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3744104660 0x7f37441a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.919+0000 7f3749400700 1 -- 172.21.15.22:0/3194099186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37441a09c0 con 0x7f3744102ac0 2024-04-20T21:27:42.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.919+0000 7f3748a00700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3744103cc0 0x7f37441992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:27:42.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.919+0000 7f3749400700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3744102ac0 0x7f3744198d60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f373400e980 tx=0x7f373400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:42.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.919+0000 7f3742000700 1 -- 172.21.15.22:0/3194099186 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734004230 con 0x7f3744102ac0 2024-04-20T21:27:42.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.919+0000 7f3742000700 1 -- 172.21.15.22:0/3194099186 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37340043d0 con 0x7f3744102ac0 2024-04-20T21:27:42.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.919+0000 7f374be00700 1 -- 172.21.15.22:0/3194099186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37441a0cb0 con 0x7f3744102ac0 2024-04-20T21:27:42.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.919+0000 7f3742000700 1 -- 172.21.15.22:0/3194099186 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734003dc0 con 0x7f3744102ac0 2024-04-20T21:27:42.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.919+0000 7f374be00700 1 -- 172.21.15.22:0/3194099186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37441a11a0 con 0x7f3744102ac0 2024-04-20T21:27:42.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.921+0000 7f374be00700 1 -- 172.21.15.22:0/3194099186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3744069740 con 0x7f3744102ac0 2024-04-20T21:27:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.923+0000 7f3742000700 1 -- 172.21.15.22:0/3194099186 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f373400c810 con 0x7f3744102ac0 2024-04-20T21:27:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.924+0000 7f3742000700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f37280a3c40 0x7f37280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.924+0000 7f3742000700 1 -- 172.21.15.22:0/3194099186 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7f37340cf8e0 con 0x7f3744102ac0 2024-04-20T21:27:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.925+0000 7f3748a00700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f37280a3c40 0x7f37280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.925+0000 7f3748a00700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f37280a3c40 0x7f37280a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3730000c00 tx=0x7f3730005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:42.927+0000 7f3742000700 1 -- 172.21.15.22:0/3194099186 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f373409d0e0 con 0x7f3744102ac0 2024-04-20T21:27:43.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:27:43.171+0000 7f374be00700 1 -- 172.21.15.22:0/3194099186 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3744062e90 con 0x7f37280a3c40 2024-04-20T21:27:43.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:43 smithi022 ceph-mon[16843]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:43.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:43 smithi022 ceph-mon[16843]: osdmap e43: 6 total, 6 up, 6 in 2024-04-20T21:27:43.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:43 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:27:43.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:43 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:27:43.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:43 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:43.676 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:43 smithi122 ceph-mon[22147]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:43.676 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:43 smithi122 ceph-mon[22147]: osdmap e43: 6 total, 6 up, 6 in 2024-04-20T21:27:43.676 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:43 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:27:43.676 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:43 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:27:43.676 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:43 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:43.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:43 smithi033 ceph-mon[21626]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:43.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:43 smithi033 ceph-mon[21626]: osdmap e43: 6 total, 6 up, 6 in 2024-04-20T21:27:43.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:43 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:27:43.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:43 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:27:43.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:43 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:44.472 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:44 smithi022 ceph-mon[16843]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:27:44.472 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:44 smithi022 ceph-mon[16843]: osdmap e44: 6 total, 6 up, 6 in 2024-04-20T21:27:44.688 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:44 smithi122 ceph-mon[22147]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:27:44.688 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:44 smithi122 ceph-mon[22147]: osdmap e44: 6 total, 6 up, 6 in 2024-04-20T21:27:44.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:44 smithi033 ceph-mon[21626]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:27:44.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:44 smithi033 ceph-mon[21626]: osdmap e44: 6 total, 6 up, 6 in 2024-04-20T21:27:45.633 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:45 smithi022 ceph-mon[16843]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T21:27:45.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:45 smithi122 ceph-mon[22147]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T21:27:45.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:45 smithi033 ceph-mon[21626]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T21:27:47.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:47 smithi022 ceph-mon[16843]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-20T21:27:47.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:47 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/4294320962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "283c2314-3dbb-4f48-964d-127ae967d38f"}]: dispatch 2024-04-20T21:27:47.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:47 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "283c2314-3dbb-4f48-964d-127ae967d38f"}]: dispatch 2024-04-20T21:27:47.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:47 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "283c2314-3dbb-4f48-964d-127ae967d38f"}]': finished 2024-04-20T21:27:47.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:47 smithi022 ceph-mon[16843]: osdmap e45: 7 total, 6 up, 7 in 2024-04-20T21:27:47.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:47 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:47.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:47 smithi122 ceph-mon[22147]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-20T21:27:47.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:47 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/4294320962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "283c2314-3dbb-4f48-964d-127ae967d38f"}]: dispatch 2024-04-20T21:27:47.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:47 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "283c2314-3dbb-4f48-964d-127ae967d38f"}]: dispatch 2024-04-20T21:27:47.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:47 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "283c2314-3dbb-4f48-964d-127ae967d38f"}]': finished 2024-04-20T21:27:47.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:47 smithi122 ceph-mon[22147]: osdmap e45: 7 total, 6 up, 7 in 2024-04-20T21:27:47.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:47 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:47.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:47 smithi033 ceph-mon[21626]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-04-20T21:27:47.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:47 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/4294320962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "283c2314-3dbb-4f48-964d-127ae967d38f"}]: dispatch 2024-04-20T21:27:47.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:47 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "283c2314-3dbb-4f48-964d-127ae967d38f"}]: dispatch 2024-04-20T21:27:47.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:47 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "283c2314-3dbb-4f48-964d-127ae967d38f"}]': finished 2024-04-20T21:27:47.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:47 smithi033 ceph-mon[21626]: osdmap e45: 7 total, 6 up, 7 in 2024-04-20T21:27:47.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:47 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:48.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:48 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2622931583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:27:48.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:48 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2622931583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:27:48.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:48 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2622931583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:27:49.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:49 smithi022 ceph-mon[16843]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:27:49.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:49 smithi122 ceph-mon[22147]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:27:49.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:49 smithi033 ceph-mon[21626]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:27:51.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:51 smithi022 ceph-mon[16843]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-20T21:27:51.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:51 smithi122 ceph-mon[22147]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-20T21:27:51.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:51 smithi033 ceph-mon[21626]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-20T21:27:53.651 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:53 smithi022 ceph-mon[16843]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:53.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:53 smithi122 ceph-mon[22147]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:53.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:53 smithi033 ceph-mon[21626]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:55.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:55 smithi022 ceph-mon[16843]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:55.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:55 smithi122 ceph-mon[22147]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:55.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:55 smithi033 ceph-mon[21626]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:57.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:57 smithi022 ceph-mon[16843]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:57.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:57 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-20T21:27:57.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:57 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:57 smithi122 ceph-mon[22147]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:57 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-20T21:27:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:57 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:57.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:57 smithi033 ceph-mon[21626]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:57.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:57 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-20T21:27:57.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:57 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:58.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:58 smithi022 ceph-mon[16843]: Deploying daemon osd.6 on smithi022 2024-04-20T21:27:58.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:58 smithi122 ceph-mon[22147]: Deploying daemon osd.6 on smithi022 2024-04-20T21:27:58.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:58 smithi033 ceph-mon[21626]: Deploying daemon osd.6 on smithi022 2024-04-20T21:27:59.608 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:27:59 smithi022 ceph-mon[16843]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:59.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:27:59 smithi122 ceph-mon[22147]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:59.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:27:59 smithi033 ceph-mon[21626]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:01 smithi122 ceph-mon[22147]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:01.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:01 smithi033 ceph-mon[21626]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:01.889 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:01 smithi022 ceph-mon[16843]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:03.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:03 smithi122 ceph-mon[22147]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:03.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:03 smithi033 ceph-mon[21626]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:03.900 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:03 smithi022 ceph-mon[16843]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:05.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:05 smithi122 ceph-mon[22147]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:05.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:05.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:05.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:05.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:05 smithi033 ceph-mon[21626]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:05.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:05.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:05.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:05.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:05 smithi022 ceph-mon[16843]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:05.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:05.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:05.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:07.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:07 smithi022 ceph-mon[16843]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:07 smithi122 ceph-mon[22147]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:07.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:07 smithi033 ceph-mon[21626]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:09 smithi122 ceph-mon[22147]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:09 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:09 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:09 smithi122 ceph-mon[22147]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T21:28:09.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:09 smithi033 ceph-mon[21626]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:09.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:09 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:09.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:09 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:09.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:09 smithi033 ceph-mon[21626]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T21:28:09.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:09 smithi022 ceph-mon[16843]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:09.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:09 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:09.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:09 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:09.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:09 smithi022 ceph-mon[16843]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T21:28:10.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.023+0000 7f3742000700 1 -- 172.21.15.22:0/3194099186 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3744062e90 con 0x7f37280a3c40 2024-04-20T21:28:10.023 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 6 on host 'smithi022' 2024-04-20T21:28:10.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.026+0000 7f374be00700 1 -- 172.21.15.22:0/3194099186 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f37280a3c40 msgr2=0x7f37280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:10.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.026+0000 7f374be00700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f37280a3c40 0x7f37280a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3730000c00 tx=0x7f3730005fb0 comp rx=0 tx=0).stop 2024-04-20T21:28:10.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.026+0000 7f374be00700 1 -- 172.21.15.22:0/3194099186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3744102ac0 msgr2=0x7f3744198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:10.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.026+0000 7f374be00700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3744102ac0 0x7f3744198d60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f373400e980 tx=0x7f373400ec90 comp rx=0 tx=0).stop 2024-04-20T21:28:10.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.026+0000 7f374be00700 1 -- 172.21.15.22:0/3194099186 shutdown_connections 2024-04-20T21:28:10.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.026+0000 7f374be00700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f37280a3c40 0x7f37280a6100 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:10.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.026+0000 7f374be00700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3744104660 0x7f37441a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:10.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.026+0000 7f374be00700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3744102ac0 0x7f3744198d60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:10.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.026+0000 7f374be00700 1 --2- 172.21.15.22:0/3194099186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3744103cc0 0x7f37441992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:10.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.026+0000 7f374be00700 1 -- 172.21.15.22:0/3194099186 >> 172.21.15.22:0/3194099186 conn(0x7f37440fe020 msgr2=0x7f3744107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:10.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.027+0000 7f374be00700 1 -- 172.21.15.22:0/3194099186 shutdown_connections 2024-04-20T21:28:10.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:28:10.027+0000 7f374be00700 1 -- 172.21.15.22:0/3194099186 wait complete. 2024-04-20T21:28:10.801 DEBUG:teuthology.orchestra.run.smithi022:osd.6> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.6.service 2024-04-20T21:28:10.806 INFO:tasks.cephadm:Deploying osd.7 on smithi033 with /dev/vg_nvme/lv_2... 2024-04-20T21:28:10.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T21:28:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:28:11.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:10 smithi022 ceph-mon[16843]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:11.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:10 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:11.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:10 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:11.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:10 smithi022 ceph-mon[16843]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-20T21:28:11.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:10 smithi022 ceph-mon[16843]: osdmap e46: 7 total, 6 up, 7 in 2024-04-20T21:28:11.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:10 smithi022 ceph-mon[16843]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:28:11.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:10 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:11.185 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:10 smithi033 ceph-mon[21626]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:11.185 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:10 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:11.185 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:10 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:11.186 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:10 smithi033 ceph-mon[21626]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-20T21:28:11.186 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:10 smithi033 ceph-mon[21626]: osdmap e46: 7 total, 6 up, 7 in 2024-04-20T21:28:11.186 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:10 smithi033 ceph-mon[21626]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:28:11.186 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:10 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:11.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:10 smithi122 ceph-mon[22147]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:10 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:10 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:10 smithi122 ceph-mon[22147]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-20T21:28:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:10 smithi122 ceph-mon[22147]: osdmap e46: 7 total, 6 up, 7 in 2024-04-20T21:28:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:10 smithi122 ceph-mon[22147]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:28:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:10 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:12.087 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:12 smithi022 ceph-mon[16843]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-20T21:28:12.087 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:12 smithi022 ceph-mon[16843]: osdmap e47: 7 total, 6 up, 7 in 2024-04-20T21:28:12.087 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:12 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:12.087 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:12 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:12.088 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:28:12 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:28:12.042+0000 7fb564c00700 -1 osd.6 0 waiting for initial osdmap 2024-04-20T21:28:12.088 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:28:12 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:28:12.087+0000 7fb55aa00700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:28:12.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:12 smithi122 ceph-mon[22147]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-20T21:28:12.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:12 smithi122 ceph-mon[22147]: osdmap e47: 7 total, 6 up, 7 in 2024-04-20T21:28:12.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:12 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:12.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:12 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:12.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:12 smithi033 ceph-mon[21626]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-20T21:28:12.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:12 smithi033 ceph-mon[21626]: osdmap e47: 7 total, 6 up, 7 in 2024-04-20T21:28:12.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:12 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:12.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:12 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T21:28:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T21:28:13.231 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-20T21:28:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-20T21:28:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111919 s, 937 MB/s 2024-04-20T21:28:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-20T21:28:13.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:13 smithi122 ceph-mon[22147]: purged_snaps scrub starts 2024-04-20T21:28:13.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:13 smithi122 ceph-mon[22147]: purged_snaps scrub ok 2024-04-20T21:28:13.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:13 smithi122 ceph-mon[22147]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:13.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:13 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:13.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:13 smithi122 ceph-mon[22147]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' 2024-04-20T21:28:13.297 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:13 smithi033 ceph-mon[21626]: purged_snaps scrub starts 2024-04-20T21:28:13.297 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:13 smithi033 ceph-mon[21626]: purged_snaps scrub ok 2024-04-20T21:28:13.297 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:13 smithi033 ceph-mon[21626]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:13.298 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:13 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:13.298 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:13 smithi033 ceph-mon[21626]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' 2024-04-20T21:28:13.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:13 smithi022 ceph-mon[16843]: purged_snaps scrub starts 2024-04-20T21:28:13.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:13 smithi022 ceph-mon[16843]: purged_snaps scrub ok 2024-04-20T21:28:13.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:13 smithi022 ceph-mon[16843]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:13.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:13 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:13.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:13 smithi022 ceph-mon[16843]: from='osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286]' entity='osd.6' 2024-04-20T21:28:13.845 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_2 2024-04-20T21:28:14.091 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:28:14.217 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:14.217 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:14 smithi033 ceph-mon[21626]: osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286] boot 2024-04-20T21:28:14.217 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:14 smithi033 ceph-mon[21626]: osdmap e48: 7 total, 7 up, 7 in 2024-04-20T21:28:14.217 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:14 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:14.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:14.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:14 smithi122 ceph-mon[22147]: osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286] boot 2024-04-20T21:28:14.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:14 smithi122 ceph-mon[22147]: osdmap e48: 7 total, 7 up, 7 in 2024-04-20T21:28:14.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:14 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:14.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:14.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:14 smithi022 ceph-mon[16843]: osd.6 [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286] boot 2024-04-20T21:28:14.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:14 smithi022 ceph-mon[16843]: osdmap e48: 7 total, 7 up, 7 in 2024-04-20T21:28:14.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:14 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:28:15.223 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:15 smithi033 ceph-mon[21626]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:15.223 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:15 smithi033 ceph-mon[21626]: osdmap e49: 7 total, 7 up, 7 in 2024-04-20T21:28:15.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:15 smithi022 ceph-mon[16843]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:15.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:15 smithi022 ceph-mon[16843]: osdmap e49: 7 total, 7 up, 7 in 2024-04-20T21:28:15.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:15 smithi122 ceph-mon[22147]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:28:15.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:15 smithi122 ceph-mon[22147]: osdmap e49: 7 total, 7 up, 7 in 2024-04-20T21:28:15.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.753+0000 7ff669b34700 1 -- 172.21.15.33:0/1052832829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664103d80 msgr2=0x7ff6641041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:15.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.753+0000 7ff669b34700 1 --2- 172.21.15.33:0/1052832829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664103d80 0x7ff6641041e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff658009990 tx=0x7ff658009ca0 comp rx=0 tx=0).stop 2024-04-20T21:28:15.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.753+0000 7ff669b34700 1 -- 172.21.15.33:0/1052832829 shutdown_connections 2024-04-20T21:28:15.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.753+0000 7ff669b34700 1 --2- 172.21.15.33:0/1052832829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff664104720 0x7ff66410afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:15.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.753+0000 7ff669b34700 1 --2- 172.21.15.33:0/1052832829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff664100da0 0x7ff6641011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:15.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.753+0000 7ff669b34700 1 --2- 172.21.15.33:0/1052832829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664103d80 0x7ff6641041e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:15.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.753+0000 7ff669b34700 1 -- 172.21.15.33:0/1052832829 >> 172.21.15.33:0/1052832829 conn(0x7ff6640fc700 msgr2=0x7ff6640feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:15.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.753+0000 7ff669b34700 1 -- 172.21.15.33:0/1052832829 shutdown_connections 2024-04-20T21:28:15.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.753+0000 7ff669b34700 1 -- 172.21.15.33:0/1052832829 wait complete. 2024-04-20T21:28:15.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.754+0000 7ff669b34700 1 Processor -- start 2024-04-20T21:28:15.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.754+0000 7ff669b34700 1 -- start start 2024-04-20T21:28:15.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.754+0000 7ff669b34700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff664100da0 0x7ff664198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:15.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff669b34700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff664103d80 0x7ff664199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:15.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff669b34700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664104720 0x7ff6641a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:15.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff669b34700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff66410e7b0 con 0x7ff664100da0 2024-04-20T21:28:15.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff663fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664104720 0x7ff6641a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:15.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff669b34700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff66410e630 con 0x7ff664104720 2024-04-20T21:28:15.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff669b34700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff66410e930 con 0x7ff664103d80 2024-04-20T21:28:15.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff663fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664104720 0x7ff6641a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51836/0 (socket says 172.21.15.33:51836) 2024-04-20T21:28:15.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff663fff700 1 -- 172.21.15.33:0/945797744 learned_addr learned my addr 172.21.15.33:0/945797744 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff662ffd700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff664103d80 0x7ff664199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff6637fe700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff664100da0 0x7ff664198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff663fff700 1 -- 172.21.15.33:0/945797744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff664103d80 msgr2=0x7ff664199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff663fff700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff664103d80 0x7ff664199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff663fff700 1 -- 172.21.15.33:0/945797744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff664100da0 msgr2=0x7ff664198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff663fff700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff664100da0 0x7ff664198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff663fff700 1 -- 172.21.15.33:0/945797744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6641a0a80 con 0x7ff664104720 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff663fff700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664104720 0x7ff6641a03e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00b970 tx=0x7ff65c00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.755+0000 7ff660ff9700 1 -- 172.21.15.33:0/945797744 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c019070 con 0x7ff664104720 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.756+0000 7ff660ff9700 1 -- 172.21.15.33:0/945797744 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65c00ce10 con 0x7ff664104720 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.756+0000 7ff669b34700 1 -- 172.21.15.33:0/945797744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6641a0d70 con 0x7ff664104720 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.756+0000 7ff660ff9700 1 -- 172.21.15.33:0/945797744 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c0073d0 con 0x7ff664104720 2024-04-20T21:28:15.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.756+0000 7ff669b34700 1 -- 172.21.15.33:0/945797744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6641a1280 con 0x7ff664104720 2024-04-20T21:28:15.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.757+0000 7ff669b34700 1 -- 172.21.15.33:0/945797744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff650005360 con 0x7ff664104720 2024-04-20T21:28:15.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.758+0000 7ff660ff9700 1 -- 172.21.15.33:0/945797744 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff65c01c070 con 0x7ff664104720 2024-04-20T21:28:15.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.759+0000 7ff660ff9700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff64c0a3c40 0x7ff64c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.759+0000 7ff660ff9700 1 -- 172.21.15.33:0/945797744 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7ff65c0cea70 con 0x7ff664104720 2024-04-20T21:28:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.759+0000 7ff6637fe700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff64c0a3c40 0x7ff64c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.760+0000 7ff6637fe700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff64c0a3c40 0x7ff64c0a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff654006fd0 tx=0x7ff654008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.764+0000 7ff660ff9700 1 -- 172.21.15.33:0/945797744 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff65c09c0c0 con 0x7ff664104720 2024-04-20T21:28:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:15.984+0000 7ff669b34700 1 -- 172.21.15.33:0/945797744 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff650000bf0 con 0x7ff64c0a3c40 2024-04-20T21:28:16.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: Detected new or changed devices on smithi022 2024-04-20T21:28:16.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi022 to 5698M 2024-04-20T21:28:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:16.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:28:16.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:28:16.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:28:16.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:16 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:16.793 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: Detected new or changed devices on smithi022 2024-04-20T21:28:16.793 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.793 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.793 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:16.793 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:16.794 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:16.794 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi022 to 5698M 2024-04-20T21:28:16.794 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.794 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:16.794 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:16.794 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.794 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:16.794 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:16.794 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:16.794 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:16.794 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.795 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:28:16.795 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:28:16.795 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:28:16.795 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:16 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:16.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: Detected new or changed devices on smithi022 2024-04-20T21:28:16.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi022 to 5698M 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:16.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:16.909 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:16.909 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:28:16.909 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:28:16.909 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:28:16.909 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:16 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:18.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:18 smithi022 ceph-mon[16843]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:19.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:18 smithi122 ceph-mon[22147]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:19.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:18 smithi033 ceph-mon[21626]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:20.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:19 smithi122 ceph-mon[22147]: from='client.? 172.21.15.33:0/2976864239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "986f4852-193c-413d-8ffb-2faba217f4ce"}]: dispatch 2024-04-20T21:28:20.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:19 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "986f4852-193c-413d-8ffb-2faba217f4ce"}]: dispatch 2024-04-20T21:28:20.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:19 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "986f4852-193c-413d-8ffb-2faba217f4ce"}]': finished 2024-04-20T21:28:20.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:19 smithi122 ceph-mon[22147]: osdmap e50: 8 total, 7 up, 8 in 2024-04-20T21:28:20.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:19 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:20.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:19 smithi122 ceph-mon[22147]: from='client.? 172.21.15.33:0/997185946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:28:20.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:19 smithi033 ceph-mon[21626]: from='client.? 172.21.15.33:0/2976864239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "986f4852-193c-413d-8ffb-2faba217f4ce"}]: dispatch 2024-04-20T21:28:20.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:19 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "986f4852-193c-413d-8ffb-2faba217f4ce"}]: dispatch 2024-04-20T21:28:20.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:19 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "986f4852-193c-413d-8ffb-2faba217f4ce"}]': finished 2024-04-20T21:28:20.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:19 smithi033 ceph-mon[21626]: osdmap e50: 8 total, 7 up, 8 in 2024-04-20T21:28:20.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:19 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:20.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:19 smithi033 ceph-mon[21626]: from='client.? 172.21.15.33:0/997185946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:28:20.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:19 smithi022 ceph-mon[16843]: from='client.? 172.21.15.33:0/2976864239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "986f4852-193c-413d-8ffb-2faba217f4ce"}]: dispatch 2024-04-20T21:28:20.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:19 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "986f4852-193c-413d-8ffb-2faba217f4ce"}]: dispatch 2024-04-20T21:28:20.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:19 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "986f4852-193c-413d-8ffb-2faba217f4ce"}]': finished 2024-04-20T21:28:20.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:19 smithi022 ceph-mon[16843]: osdmap e50: 8 total, 7 up, 8 in 2024-04-20T21:28:20.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:20.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:19 smithi022 ceph-mon[16843]: from='client.? 172.21.15.33:0/997185946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:28:21.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:20 smithi122 ceph-mon[22147]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:21.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:20 smithi033 ceph-mon[21626]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:21.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:20 smithi022 ceph-mon[16843]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:22.967 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:22 smithi033 ceph-mon[21626]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:23.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:22 smithi122 ceph-mon[22147]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:23.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:22 smithi022 ceph-mon[16843]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:25.008 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:24 smithi033 ceph-mon[21626]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:25.024 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:24 smithi122 ceph-mon[22147]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:25.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:24 smithi022 ceph-mon[16843]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:27.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:26 smithi122 ceph-mon[22147]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:27.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:26 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-20T21:28:27.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:26 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:27.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:26 smithi122 ceph-mon[22147]: Deploying daemon osd.7 on smithi033 2024-04-20T21:28:27.034 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:26 smithi033 ceph-mon[21626]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:27.034 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:26 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-20T21:28:27.034 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:26 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:27.034 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:26 smithi033 ceph-mon[21626]: Deploying daemon osd.7 on smithi033 2024-04-20T21:28:27.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:26 smithi022 ceph-mon[16843]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:27.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:26 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-20T21:28:27.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:26 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:27.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:26 smithi022 ceph-mon[16843]: Deploying daemon osd.7 on smithi033 2024-04-20T21:28:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:28 smithi122 ceph-mon[22147]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:29.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:28 smithi033 ceph-mon[21626]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:29.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:28 smithi022 ceph-mon[16843]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:30.910 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:30 smithi033 ceph-mon[21626]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:30 smithi122 ceph-mon[22147]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:31.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:30 smithi022 ceph-mon[16843]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:33.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:32 smithi022 ceph-mon[16843]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:33.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:32 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:33.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:32 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:33.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:32 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:33.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:32 smithi122 ceph-mon[22147]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:33.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:32 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:33.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:32 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:33.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:32 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:33.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:32 smithi033 ceph-mon[21626]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:33.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:32 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:33.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:32 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:33.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:32 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:34.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:34 smithi022 ceph-mon[16843]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:34.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:34 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:35.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:34 smithi122 ceph-mon[22147]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:35.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:34 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:35.280 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:34 smithi033 ceph-mon[21626]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:35.280 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:34 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:35.666 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 7 on host 'smithi033' 2024-04-20T21:28:35.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.667+0000 7ff660ff9700 1 -- 172.21.15.33:0/945797744 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff650000bf0 con 0x7ff64c0a3c40 2024-04-20T21:28:35.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.669+0000 7ff669b34700 1 -- 172.21.15.33:0/945797744 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff64c0a3c40 msgr2=0x7ff64c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:35.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.669+0000 7ff669b34700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff64c0a3c40 0x7ff64c0a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff654006fd0 tx=0x7ff654008040 comp rx=0 tx=0).stop 2024-04-20T21:28:35.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.669+0000 7ff669b34700 1 -- 172.21.15.33:0/945797744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664104720 msgr2=0x7ff6641a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:35.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.669+0000 7ff669b34700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664104720 0x7ff6641a03e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00b970 tx=0x7ff65c00bc80 comp rx=0 tx=0).stop 2024-04-20T21:28:35.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.669+0000 7ff669b34700 1 -- 172.21.15.33:0/945797744 shutdown_connections 2024-04-20T21:28:35.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.669+0000 7ff669b34700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff64c0a3c40 0x7ff64c0a6100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:35.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.669+0000 7ff669b34700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff664104720 0x7ff6641a03e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:35.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.669+0000 7ff669b34700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff664100da0 0x7ff664198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:35.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.669+0000 7ff669b34700 1 --2- 172.21.15.33:0/945797744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff664103d80 0x7ff664199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:35.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.669+0000 7ff669b34700 1 -- 172.21.15.33:0/945797744 >> 172.21.15.33:0/945797744 conn(0x7ff6640fc700 msgr2=0x7ff6641070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:35.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.670+0000 7ff669b34700 1 -- 172.21.15.33:0/945797744 shutdown_connections 2024-04-20T21:28:35.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:28:35.670+0000 7ff669b34700 1 -- 172.21.15.33:0/945797744 wait complete. 2024-04-20T21:28:36.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:36.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:36.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:36.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:36.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:36.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:36.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:36.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:35 smithi022 ceph-mon[16843]: from='osd.7 [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T21:28:36.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:35 smithi022 ceph-mon[16843]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T21:28:36.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:36.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:36.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:36.268 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:36.269 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:36.269 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:36.269 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:36.269 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:35 smithi033 ceph-mon[21626]: from='osd.7 [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T21:28:36.269 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:35 smithi033 ceph-mon[21626]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T21:28:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:35 smithi122 ceph-mon[22147]: from='osd.7 [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T21:28:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:35 smithi122 ceph-mon[22147]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T21:28:36.295 DEBUG:teuthology.orchestra.run.smithi033:osd.7> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.7.service 2024-04-20T21:28:36.297 INFO:tasks.cephadm:Deploying osd.8 on smithi122 with /dev/vg_nvme/lv_2... 2024-04-20T21:28:36.297 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T21:28:36.538 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:28:36.945 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:36 smithi122 ceph-mon[22147]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:36.945 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:36 smithi122 ceph-mon[22147]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-20T21:28:36.945 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:36 smithi122 ceph-mon[22147]: from='osd.7 [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:28:36.945 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:36 smithi122 ceph-mon[22147]: osdmap e51: 8 total, 7 up, 8 in 2024-04-20T21:28:36.945 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:36 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:36.945 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:36 smithi122 ceph-mon[22147]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:28:37.109 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:36 smithi033 ceph-mon[21626]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:37.109 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:36 smithi033 ceph-mon[21626]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-20T21:28:37.109 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:36 smithi033 ceph-mon[21626]: from='osd.7 [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:28:37.109 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:36 smithi033 ceph-mon[21626]: osdmap e51: 8 total, 7 up, 8 in 2024-04-20T21:28:37.109 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:36 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:37.109 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:36 smithi033 ceph-mon[21626]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:28:37.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:36 smithi022 ceph-mon[16843]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:37.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:36 smithi022 ceph-mon[16843]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-20T21:28:37.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:36 smithi022 ceph-mon[16843]: from='osd.7 [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:28:37.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:36 smithi022 ceph-mon[16843]: osdmap e51: 8 total, 7 up, 8 in 2024-04-20T21:28:37.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:36 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:37.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:36 smithi022 ceph-mon[16843]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:28:37.649 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:28:37 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:28:37.638+0000 7fc751472700 -1 osd.7 0 waiting for initial osdmap 2024-04-20T21:28:37.649 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:28:37 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:28:37.650+0000 7fc74d265700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:28:38.429 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T21:28:38.429 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T21:28:38.430 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-20T21:28:38.430 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-20T21:28:38.430 INFO:teuthology.orchestra.run.smithi122.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110459 s, 949 MB/s 2024-04-20T21:28:38.430 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-20T21:28:38.773 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:38 smithi033 ceph-mon[21626]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:38.773 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:38 smithi033 ceph-mon[21626]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-20T21:28:38.773 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:38 smithi033 ceph-mon[21626]: osdmap e52: 8 total, 7 up, 8 in 2024-04-20T21:28:38.773 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:38 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:38.773 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:38 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:38.821 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:38 smithi122 ceph-mon[22147]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:38.821 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:38 smithi122 ceph-mon[22147]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-20T21:28:38.821 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:38 smithi122 ceph-mon[22147]: osdmap e52: 8 total, 7 up, 8 in 2024-04-20T21:28:38.821 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:38.821 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:38.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:38 smithi022 ceph-mon[16843]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:38.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:38 smithi022 ceph-mon[16843]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-20T21:28:38.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:38 smithi022 ceph-mon[16843]: osdmap e52: 8 total, 7 up, 8 in 2024-04-20T21:28:38.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:38.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:39.124 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_2 2024-04-20T21:28:39.364 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:28:39.750 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: purged_snaps scrub starts 2024-04-20T21:28:39.750 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: purged_snaps scrub ok 2024-04-20T21:28:39.750 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:39.750 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: osd.7 [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982] boot 2024-04-20T21:28:39.750 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: osdmap e53: 8 total, 8 up, 8 in 2024-04-20T21:28:39.750 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:39.750 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: osdmap e54: 8 total, 8 up, 8 in 2024-04-20T21:28:39.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: Detected new or changed devices on smithi033 2024-04-20T21:28:39.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:39.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:39.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:39.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:39.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:39.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi033 to 5697M 2024-04-20T21:28:39.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:39.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:39.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:39.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:39 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:39.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: purged_snaps scrub starts 2024-04-20T21:28:39.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: purged_snaps scrub ok 2024-04-20T21:28:39.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:39.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: osd.7 [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982] boot 2024-04-20T21:28:39.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: osdmap e53: 8 total, 8 up, 8 in 2024-04-20T21:28:39.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:39.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: osdmap e54: 8 total, 8 up, 8 in 2024-04-20T21:28:39.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: Detected new or changed devices on smithi033 2024-04-20T21:28:39.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:39.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:39.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:39.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:39.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:39.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi033 to 5697M 2024-04-20T21:28:39.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:39.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:39.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:39.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:39 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:40.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: purged_snaps scrub starts 2024-04-20T21:28:40.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: purged_snaps scrub ok 2024-04-20T21:28:40.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:40.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: osd.7 [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982] boot 2024-04-20T21:28:40.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: osdmap e53: 8 total, 8 up, 8 in 2024-04-20T21:28:40.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:40.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: osdmap e54: 8 total, 8 up, 8 in 2024-04-20T21:28:40.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: Detected new or changed devices on smithi033 2024-04-20T21:28:40.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:40.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:40.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:40.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:40.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:40.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi033 to 5697M 2024-04-20T21:28:40.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:40.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:40.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:40.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:39 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.985+0000 7fc25f64d700 1 -- 172.21.15.122:0/1065209719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258102ab0 msgr2=0x7fc258102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.985+0000 7fc25f64d700 1 --2- 172.21.15.122:0/1065209719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258102ab0 0x7fc258102ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc254009990 tx=0x7fc254009ca0 comp rx=0 tx=0).stop 2024-04-20T21:28:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 -- 172.21.15.122:0/1065209719 shutdown_connections 2024-04-20T21:28:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 --2- 172.21.15.122:0/1065209719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc258104650 0x7fc25810af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 --2- 172.21.15.122:0/1065209719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258102ab0 0x7fc258102ed0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 --2- 172.21.15.122:0/1065209719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258103cb0 0x7fc258104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 -- 172.21.15.122:0/1065209719 >> 172.21.15.122:0/1065209719 conn(0x7fc2580fe050 msgr2=0x7fc258100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 -- 172.21.15.122:0/1065209719 shutdown_connections 2024-04-20T21:28:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 -- 172.21.15.122:0/1065209719 wait complete. 2024-04-20T21:28:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 Processor -- start 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 -- start start 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc258102ab0 0x7fc2580767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258103cb0 0x7fc258074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258104650 0x7fc258075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc25810e710 con 0x7fc258104650 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc25810e590 con 0x7fc258102ab0 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.986+0000 7fc25f64d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc25810e890 con 0x7fc258103cb0 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc25cbe8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258103cb0 0x7fc258074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc25cbe8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258103cb0 0x7fc258074e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41574/0 (socket says 172.21.15.122:41574) 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc25cbe8700 1 -- 172.21.15.122:0/1424769160 learned_addr learned my addr 172.21.15.122:0/1424769160 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc25d3e9700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc258102ab0 0x7fc2580767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc25cbe8700 1 -- 172.21.15.122:0/1424769160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc258102ab0 msgr2=0x7fc2580767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc25cbe8700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc258102ab0 0x7fc2580767f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc25cbe8700 1 -- 172.21.15.122:0/1424769160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258104650 msgr2=0x7fc258075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc25cbe8700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258104650 0x7fc258075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:40.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc25cbe8700 1 -- 172.21.15.122:0/1424769160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc258075c30 con 0x7fc258103cb0 2024-04-20T21:28:40.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc25cbe8700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258103cb0 0x7fc258074e40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc24800b530 tx=0x7fc24800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:40.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc24e7fc700 1 -- 172.21.15.122:0/1424769160 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248015020 con 0x7fc258103cb0 2024-04-20T21:28:40.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc24e7fc700 1 -- 172.21.15.122:0/1424769160 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc248011d10 con 0x7fc258103cb0 2024-04-20T21:28:40.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc25f64d700 1 -- 172.21.15.122:0/1424769160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2581a2f10 con 0x7fc258103cb0 2024-04-20T21:28:40.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc24e7fc700 1 -- 172.21.15.122:0/1424769160 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24800f2e0 con 0x7fc258103cb0 2024-04-20T21:28:40.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.987+0000 7fc25f64d700 1 -- 172.21.15.122:0/1424769160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2581a33f0 con 0x7fc258103cb0 2024-04-20T21:28:40.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.988+0000 7fc25f64d700 1 -- 172.21.15.122:0/1424769160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc258069740 con 0x7fc258103cb0 2024-04-20T21:28:40.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.989+0000 7fc24e7fc700 1 -- 172.21.15.122:0/1424769160 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc248018020 con 0x7fc258103cb0 2024-04-20T21:28:40.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.990+0000 7fc24e7fc700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc2440a3c40 0x7fc2440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:40.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.990+0000 7fc24e7fc700 1 -- 172.21.15.122:0/1424769160 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5426+0+0 (secure 0 0 0) 0x7fc2480cf120 con 0x7fc258103cb0 2024-04-20T21:28:40.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.990+0000 7fc25d3e9700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc2440a3c40 0x7fc2440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:40.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.991+0000 7fc25d3e9700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc2440a3c40 0x7fc2440a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc254003890 tx=0x7fc254005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:40.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:40.994+0000 7fc24e7fc700 1 -- 172.21.15.122:0/1424769160 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc24801f020 con 0x7fc258103cb0 2024-04-20T21:28:41.024 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:40 smithi122 ceph-mon[22147]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:41.024 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:40 smithi122 ceph-mon[22147]: osdmap e55: 8 total, 8 up, 8 in 2024-04-20T21:28:41.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:28:41.202+0000 7fc25f64d700 1 -- 172.21.15.122:0/1424769160 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc258062e90 con 0x7fc2440a3c40 2024-04-20T21:28:41.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:40 smithi033 ceph-mon[21626]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:41.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:40 smithi033 ceph-mon[21626]: osdmap e55: 8 total, 8 up, 8 in 2024-04-20T21:28:41.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:40 smithi022 ceph-mon[16843]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:28:41.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:40 smithi022 ceph-mon[16843]: osdmap e55: 8 total, 8 up, 8 in 2024-04-20T21:28:42.110 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:41 smithi122 ceph-mon[22147]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:28:42.110 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:28:42.110 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:28:42.110 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:42.110 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:41 smithi122 ceph-mon[22147]: osdmap e56: 8 total, 8 up, 8 in 2024-04-20T21:28:42.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:41 smithi033 ceph-mon[21626]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:28:42.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:28:42.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:28:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:41 smithi033 ceph-mon[21626]: osdmap e56: 8 total, 8 up, 8 in 2024-04-20T21:28:42.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:41 smithi022 ceph-mon[16843]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:28:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:28:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:28:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:41 smithi022 ceph-mon[16843]: osdmap e56: 8 total, 8 up, 8 in 2024-04-20T21:28:42.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:42 smithi122 ceph-mon[22147]: pgmap v202: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-20T21:28:42.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:42 smithi122 ceph-mon[22147]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-20T21:28:42.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:42 smithi122 ceph-mon[22147]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-20T21:28:42.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:42 smithi122 ceph-mon[22147]: osdmap e57: 8 total, 8 up, 8 in 2024-04-20T21:28:43.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:42 smithi033 ceph-mon[21626]: pgmap v202: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-20T21:28:43.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:42 smithi033 ceph-mon[21626]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-20T21:28:43.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:42 smithi033 ceph-mon[21626]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-20T21:28:43.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:42 smithi033 ceph-mon[21626]: osdmap e57: 8 total, 8 up, 8 in 2024-04-20T21:28:43.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:42 smithi022 ceph-mon[16843]: pgmap v202: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-20T21:28:43.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:42 smithi022 ceph-mon[16843]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-20T21:28:43.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:42 smithi022 ceph-mon[16843]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-20T21:28:43.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:42 smithi022 ceph-mon[16843]: osdmap e57: 8 total, 8 up, 8 in 2024-04-20T21:28:44.203 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:43 smithi122 ceph-mon[22147]: from='client.? 172.21.15.122:0/1253870105' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a6f0da8-369c-4576-9105-13a02e09b03d"}]: dispatch 2024-04-20T21:28:44.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:43 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a6f0da8-369c-4576-9105-13a02e09b03d"}]: dispatch 2024-04-20T21:28:44.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:43 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a6f0da8-369c-4576-9105-13a02e09b03d"}]': finished 2024-04-20T21:28:44.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:43 smithi122 ceph-mon[22147]: osdmap e58: 9 total, 8 up, 9 in 2024-04-20T21:28:44.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:43 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:44.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:43 smithi033 ceph-mon[21626]: from='client.? 172.21.15.122:0/1253870105' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a6f0da8-369c-4576-9105-13a02e09b03d"}]: dispatch 2024-04-20T21:28:44.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:43 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a6f0da8-369c-4576-9105-13a02e09b03d"}]: dispatch 2024-04-20T21:28:44.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:43 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a6f0da8-369c-4576-9105-13a02e09b03d"}]': finished 2024-04-20T21:28:44.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:43 smithi033 ceph-mon[21626]: osdmap e58: 9 total, 8 up, 9 in 2024-04-20T21:28:44.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:43 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:44.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:43 smithi022 ceph-mon[16843]: from='client.? 172.21.15.122:0/1253870105' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a6f0da8-369c-4576-9105-13a02e09b03d"}]: dispatch 2024-04-20T21:28:44.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:43 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a6f0da8-369c-4576-9105-13a02e09b03d"}]: dispatch 2024-04-20T21:28:44.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:43 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a6f0da8-369c-4576-9105-13a02e09b03d"}]': finished 2024-04-20T21:28:44.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:43 smithi022 ceph-mon[16843]: osdmap e58: 9 total, 8 up, 9 in 2024-04-20T21:28:44.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:43 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:45.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:44 smithi022 ceph-mon[16843]: pgmap v204: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-20T21:28:45.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:44 smithi022 ceph-mon[16843]: from='client.? 172.21.15.122:0/369276136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:28:45.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:44 smithi122 ceph-mon[22147]: pgmap v204: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-20T21:28:45.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:44 smithi122 ceph-mon[22147]: from='client.? 172.21.15.122:0/369276136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:28:45.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:44 smithi033 ceph-mon[21626]: pgmap v204: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-04-20T21:28:45.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:44 smithi033 ceph-mon[21626]: from='client.? 172.21.15.122:0/369276136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:28:46.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:46 smithi122 ceph-mon[22147]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-20T21:28:46.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:46 smithi122 ceph-mon[22147]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-20T21:28:46.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:46 smithi122 ceph-mon[22147]: Cluster is now healthy 2024-04-20T21:28:46.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:46 smithi033 ceph-mon[21626]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-20T21:28:46.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:46 smithi033 ceph-mon[21626]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-20T21:28:46.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:46 smithi033 ceph-mon[21626]: Cluster is now healthy 2024-04-20T21:28:46.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:46 smithi022 ceph-mon[16843]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-20T21:28:46.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:46 smithi022 ceph-mon[16843]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-20T21:28:46.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:46 smithi022 ceph-mon[16843]: Cluster is now healthy 2024-04-20T21:28:47.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:47 smithi122 ceph-mon[22147]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:47.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:47 smithi033 ceph-mon[21626]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:47.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:47 smithi022 ceph-mon[16843]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:49.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:49 smithi122 ceph-mon[22147]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:49.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:49 smithi033 ceph-mon[21626]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:49.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:49 smithi022 ceph-mon[16843]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:51.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:51 smithi122 ceph-mon[22147]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:51.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:51 smithi033 ceph-mon[21626]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:51.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:51 smithi022 ceph-mon[16843]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:52.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:52 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-20T21:28:52.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:52 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:52.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:52 smithi033 ceph-mon[21626]: Deploying daemon osd.8 on smithi122 2024-04-20T21:28:52.380 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:52 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-20T21:28:52.380 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:52 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:52.380 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:52 smithi122 ceph-mon[22147]: Deploying daemon osd.8 on smithi122 2024-04-20T21:28:52.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:52 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-20T21:28:52.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:52 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:52.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:52 smithi022 ceph-mon[16843]: Deploying daemon osd.8 on smithi122 2024-04-20T21:28:53.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:53 smithi033 ceph-mon[21626]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:53.404 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:53 smithi122 ceph-mon[22147]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:53.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:53 smithi022 ceph-mon[16843]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:55.294 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:55 smithi122 ceph-mon[22147]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:55.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:55 smithi033 ceph-mon[21626]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:55.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:55 smithi022 ceph-mon[16843]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:57.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:57 smithi122 ceph-mon[22147]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:57.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:57 smithi033 ceph-mon[21626]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:57.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:57 smithi022 ceph-mon[16843]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:58.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:58 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:58.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:58 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:58.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:58 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:58.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:58 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:58.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:58 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:58.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:58 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:58.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:58 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:58.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:58 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:58.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:58 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:28:59.383 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:28:59 smithi122 ceph-mon[22147]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:59.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:28:59 smithi022 ceph-mon[16843]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:59.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:28:59 smithi033 ceph-mon[21626]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:29:00.641 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 8 on host 'smithi122' 2024-04-20T21:29:00.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.642+0000 7fc24e7fc700 1 -- 172.21.15.122:0/1424769160 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc258062e90 con 0x7fc2440a3c40 2024-04-20T21:29:00.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.644+0000 7fc25f64d700 1 -- 172.21.15.122:0/1424769160 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc2440a3c40 msgr2=0x7fc2440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:00.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.644+0000 7fc25f64d700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc2440a3c40 0x7fc2440a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc254003890 tx=0x7fc254005fd0 comp rx=0 tx=0).stop 2024-04-20T21:29:00.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.644+0000 7fc25f64d700 1 -- 172.21.15.122:0/1424769160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258103cb0 msgr2=0x7fc258074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:00.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.644+0000 7fc25f64d700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258103cb0 0x7fc258074e40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc24800b530 tx=0x7fc24800b840 comp rx=0 tx=0).stop 2024-04-20T21:29:00.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.644+0000 7fc25f64d700 1 -- 172.21.15.122:0/1424769160 shutdown_connections 2024-04-20T21:29:00.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.644+0000 7fc25f64d700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc2440a3c40 0x7fc2440a6100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:00.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.644+0000 7fc25f64d700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258104650 0x7fc258075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:00.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.644+0000 7fc25f64d700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc258102ab0 0x7fc2580767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:00.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.644+0000 7fc25f64d700 1 --2- 172.21.15.122:0/1424769160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258103cb0 0x7fc258074e40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:00.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.644+0000 7fc25f64d700 1 -- 172.21.15.122:0/1424769160 >> 172.21.15.122:0/1424769160 conn(0x7fc2580fe050 msgr2=0x7fc258107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:00.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.644+0000 7fc25f64d700 1 -- 172.21.15.122:0/1424769160 shutdown_connections 2024-04-20T21:29:00.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:29:00.644+0000 7fc25f64d700 1 -- 172.21.15.122:0/1424769160 wait complete. 2024-04-20T21:29:01.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:01 smithi122 ceph-mon[22147]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:29:01.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:01.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:01.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:01.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:01.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:01 smithi122 ceph-mon[22147]: from='osd.8 [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T21:29:01.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:01 smithi122 ceph-mon[22147]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T21:29:01.338 DEBUG:teuthology.orchestra.run.smithi122:osd.8> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.8.service 2024-04-20T21:29:01.341 INFO:tasks.cephadm:Deploying osd.9 on smithi022 with /dev/vg_nvme/lv_1... 2024-04-20T21:29:01.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-20T21:29:01.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:01 smithi022 ceph-mon[16843]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:29:01.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:01.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:01.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:01.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:01.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:01 smithi022 ceph-mon[16843]: from='osd.8 [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T21:29:01.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:01 smithi022 ceph-mon[16843]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T21:29:01.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:01 smithi033 ceph-mon[21626]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:29:01.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:01.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:01.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:01.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:01.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:01 smithi033 ceph-mon[21626]: from='osd.8 [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T21:29:01.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:01 smithi033 ceph-mon[21626]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T21:29:01.810 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:29:02.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:02 smithi022 ceph-mon[16843]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:29:02.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:02 smithi022 ceph-mon[16843]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-20T21:29:02.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:02 smithi022 ceph-mon[16843]: from='osd.8 [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:29:02.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:02 smithi022 ceph-mon[16843]: osdmap e59: 9 total, 8 up, 9 in 2024-04-20T21:29:02.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:02 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:29:02.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:02 smithi022 ceph-mon[16843]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:29:03.024 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:02 smithi122 ceph-mon[22147]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:29:03.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:02 smithi122 ceph-mon[22147]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-20T21:29:03.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:02 smithi122 ceph-mon[22147]: from='osd.8 [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:29:03.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:02 smithi122 ceph-mon[22147]: osdmap e59: 9 total, 8 up, 9 in 2024-04-20T21:29:03.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:02 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:29:03.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:02 smithi122 ceph-mon[22147]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:29:03.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:02 smithi033 ceph-mon[21626]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:29:03.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:02 smithi033 ceph-mon[21626]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-20T21:29:03.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:02 smithi033 ceph-mon[21626]: from='osd.8 [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:29:03.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:02 smithi033 ceph-mon[21626]: osdmap e59: 9 total, 8 up, 9 in 2024-04-20T21:29:03.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:02 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:29:03.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:02 smithi033 ceph-mon[21626]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:29:04.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:03 smithi122 ceph-mon[22147]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-20T21:29:04.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:03 smithi122 ceph-mon[22147]: osdmap e60: 9 total, 8 up, 9 in 2024-04-20T21:29:04.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:03 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:29:04.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:03 smithi122 ceph-mon[22147]: from='osd.8 ' entity='osd.8' 2024-04-20T21:29:04.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:03 smithi122 ceph-mon[22147]: osdmap e61: 9 total, 8 up, 9 in 2024-04-20T21:29:04.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:03 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:29:04.025 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:29:03 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:29:03.632+0000 7f53d4d42700 -1 osd.8 0 waiting for initial osdmap 2024-04-20T21:29:04.026 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:29:03 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:29:03.793+0000 7f53ccb2d700 -1 osd.8 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:29:04.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:03 smithi033 ceph-mon[21626]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-20T21:29:04.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:03 smithi033 ceph-mon[21626]: osdmap e60: 9 total, 8 up, 9 in 2024-04-20T21:29:04.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:03 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:29:04.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:03 smithi033 ceph-mon[21626]: from='osd.8 ' entity='osd.8' 2024-04-20T21:29:04.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:03 smithi033 ceph-mon[21626]: osdmap e61: 9 total, 8 up, 9 in 2024-04-20T21:29:04.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:03 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:29:04.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:03 smithi022 ceph-mon[16843]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-20T21:29:04.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:03 smithi022 ceph-mon[16843]: osdmap e60: 9 total, 8 up, 9 in 2024-04-20T21:29:04.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:29:04.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:03 smithi022 ceph-mon[16843]: from='osd.8 ' entity='osd.8' 2024-04-20T21:29:04.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:03 smithi022 ceph-mon[16843]: osdmap e61: 9 total, 8 up, 9 in 2024-04-20T21:29:04.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:29:04.626 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-20T21:29:04.626 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-20T21:29:04.626 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-04-20T21:29:04.626 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-04-20T21:29:04.626 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0382372 s, 274 MB/s 2024-04-20T21:29:04.626 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-04-20T21:29:04.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: purged_snaps scrub starts 2024-04-20T21:29:04.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: purged_snaps scrub ok 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: Detected new or changed devices on smithi122 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi122 to 5696M 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:04.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:04.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:04.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:04.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: osd.8 [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556] boot 2024-04-20T21:29:04.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: osdmap e62: 9 total, 9 up, 9 in 2024-04-20T21:29:04.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:29:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: purged_snaps scrub starts 2024-04-20T21:29:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: purged_snaps scrub ok 2024-04-20T21:29:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:29:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: Detected new or changed devices on smithi122 2024-04-20T21:29:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi122 to 5696M 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: osd.8 [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556] boot 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: osdmap e62: 9 total, 9 up, 9 in 2024-04-20T21:29:04.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:29:05.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: purged_snaps scrub starts 2024-04-20T21:29:05.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: purged_snaps scrub ok 2024-04-20T21:29:05.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:29:05.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: Detected new or changed devices on smithi122 2024-04-20T21:29:05.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:05.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:05.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:05.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:05.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:05.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi122 to 5696M 2024-04-20T21:29:05.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:05.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:05.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:05.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:05.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:05.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:05.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:05.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:05.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: osd.8 [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556] boot 2024-04-20T21:29:05.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: osdmap e62: 9 total, 9 up, 9 in 2024-04-20T21:29:05.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:29:05.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_1 2024-04-20T21:29:05.880 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:29:07.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:06 smithi122 ceph-mon[22147]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:07.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:06 smithi122 ceph-mon[22147]: osdmap e63: 9 total, 9 up, 9 in 2024-04-20T21:29:07.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:06 smithi022 ceph-mon[16843]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:07.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:06 smithi022 ceph-mon[16843]: osdmap e63: 9 total, 9 up, 9 in 2024-04-20T21:29:07.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:06 smithi033 ceph-mon[21626]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:07.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:06 smithi033 ceph-mon[21626]: osdmap e63: 9 total, 9 up, 9 in 2024-04-20T21:29:07.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.900+0000 7fb8cfe00700 1 -- 172.21.15.22:0/2796765117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8d0105df0 msgr2=0x7fb8d0106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:07.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.900+0000 7fb8cfe00700 1 --2- 172.21.15.22:0/2796765117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8d0105df0 0x7fb8d0106250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80099c0 tx=0x7fb8b8009cd0 comp rx=0 tx=0).stop 2024-04-20T21:29:07.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.900+0000 7fb8cfe00700 1 -- 172.21.15.22:0/2796765117 shutdown_connections 2024-04-20T21:29:07.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.900+0000 7fb8cfe00700 1 --2- 172.21.15.22:0/2796765117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8d0106790 0x7fb8d010d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:07.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.900+0000 7fb8cfe00700 1 --2- 172.21.15.22:0/2796765117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d0104bf0 0x7fb8d0105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:07.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.900+0000 7fb8cfe00700 1 --2- 172.21.15.22:0/2796765117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8d0105df0 0x7fb8d0106250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:07.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.900+0000 7fb8cfe00700 1 -- 172.21.15.22:0/2796765117 >> 172.21.15.22:0/2796765117 conn(0x7fb8d0100170 msgr2=0x7fb8d01025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:07.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.900+0000 7fb8cfe00700 1 -- 172.21.15.22:0/2796765117 shutdown_connections 2024-04-20T21:29:07.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.900+0000 7fb8cfe00700 1 -- 172.21.15.22:0/2796765117 wait complete. 2024-04-20T21:29:07.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.901+0000 7fb8cfe00700 1 Processor -- start 2024-04-20T21:29:07.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.902+0000 7fb8cfe00700 1 -- start start 2024-04-20T21:29:07.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.902+0000 7fb8cfe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d0104bf0 0x7fb8d019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:07.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.902+0000 7fb8cfe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8d0105df0 0x7fb8d019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:07.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.902+0000 7fb8cfe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8d0106790 0x7fb8d01a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:07.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.902+0000 7fb8cfe00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0110800 con 0x7fb8d0106790 2024-04-20T21:29:07.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.902+0000 7fb8cfe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0110680 con 0x7fb8d0105df0 2024-04-20T21:29:07.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.902+0000 7fb8cfe00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0110980 con 0x7fb8d0104bf0 2024-04-20T21:29:07.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.902+0000 7fb8c7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8d0105df0 0x7fb8d019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:07.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.902+0000 7fb8cea00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d0104bf0 0x7fb8d019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:07.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.902+0000 7fb8c7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8d0105df0 0x7fb8d019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:40122/0 (socket says 172.21.15.22:40122) 2024-04-20T21:29:07.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.902+0000 7fb8c7e00700 1 -- 172.21.15.22:0/3819431921 learned_addr learned my addr 172.21.15.22:0/3819431921 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:29:07.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.903+0000 7fb8cf400700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8d0106790 0x7fb8d01a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:07.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.903+0000 7fb8cea00700 1 -- 172.21.15.22:0/3819431921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8d0105df0 msgr2=0x7fb8d019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:07.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.903+0000 7fb8cea00700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8d0105df0 0x7fb8d019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:07.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.903+0000 7fb8cea00700 1 -- 172.21.15.22:0/3819431921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8d0106790 msgr2=0x7fb8d01a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:07.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.903+0000 7fb8cea00700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8d0106790 0x7fb8d01a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:07.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.903+0000 7fb8cea00700 1 -- 172.21.15.22:0/3819431921 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d01a2a10 con 0x7fb8d0104bf0 2024-04-20T21:29:07.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.903+0000 7fb8cea00700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d0104bf0 0x7fb8d019adb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000d770 tx=0x7fb8c000da80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:07.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.904+0000 7fb8c7400700 1 -- 172.21.15.22:0/3819431921 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0016070 con 0x7fb8d0104bf0 2024-04-20T21:29:07.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.904+0000 7fb8c7400700 1 -- 172.21.15.22:0/3819431921 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c000fb70 con 0x7fb8d0104bf0 2024-04-20T21:29:07.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.904+0000 7fb8cfe00700 1 -- 172.21.15.22:0/3819431921 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d01a2d00 con 0x7fb8d0104bf0 2024-04-20T21:29:07.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.904+0000 7fb8c7400700 1 -- 172.21.15.22:0/3819431921 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0003c20 con 0x7fb8d0104bf0 2024-04-20T21:29:07.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.904+0000 7fb8cfe00700 1 -- 172.21.15.22:0/3819431921 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d01a3190 con 0x7fb8d0104bf0 2024-04-20T21:29:07.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.905+0000 7fb8cfe00700 1 -- 172.21.15.22:0/3819431921 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d0069740 con 0x7fb8d0104bf0 2024-04-20T21:29:07.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.907+0000 7fb8c7400700 1 -- 172.21.15.22:0/3819431921 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb8c0004220 con 0x7fb8d0104bf0 2024-04-20T21:29:07.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.908+0000 7fb8c7400700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb8b40a3c40 0x7fb8b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:07.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.909+0000 7fb8c7400700 1 -- 172.21.15.22:0/3819431921 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5829+0+0 (secure 0 0 0) 0x7fb8c00cf3c0 con 0x7fb8d0104bf0 2024-04-20T21:29:07.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.909+0000 7fb8c7e00700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb8b40a3c40 0x7fb8b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:07.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.909+0000 7fb8c7e00700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb8b40a3c40 0x7fb8b40a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80038c0 tx=0x7fb8b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:07.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:07.913+0000 7fb8c7400700 1 -- 172.21.15.22:0/3819431921 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb8c009c6b0 con 0x7fb8d0104bf0 2024-04-20T21:29:08.024 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:07 smithi122 ceph-mon[22147]: osdmap e64: 9 total, 9 up, 9 in 2024-04-20T21:29:08.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:07 smithi033 ceph-mon[21626]: osdmap e64: 9 total, 9 up, 9 in 2024-04-20T21:29:08.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:08.154+0000 7fb8cfe00700 1 -- 172.21.15.22:0/3819431921 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb8d0062e90 con 0x7fb8b40a3c40 2024-04-20T21:29:08.154 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:07 smithi022 ceph-mon[16843]: osdmap e64: 9 total, 9 up, 9 in 2024-04-20T21:29:09.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:08 smithi122 ceph-mon[22147]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:09.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:08 smithi122 ceph-mon[22147]: osdmap e65: 9 total, 9 up, 9 in 2024-04-20T21:29:09.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:08 smithi122 ceph-mon[22147]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:29:09.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:29:09.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:29:09.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:09.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:08 smithi033 ceph-mon[21626]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:09.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:08 smithi033 ceph-mon[21626]: osdmap e65: 9 total, 9 up, 9 in 2024-04-20T21:29:09.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:08 smithi033 ceph-mon[21626]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:29:09.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:29:09.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:29:09.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:09.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:08 smithi022 ceph-mon[16843]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:09.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:08 smithi022 ceph-mon[16843]: osdmap e65: 9 total, 9 up, 9 in 2024-04-20T21:29:09.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:08 smithi022 ceph-mon[16843]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:29:09.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:29:09.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:29:09.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:10.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:09 smithi122 ceph-mon[22147]: osdmap e66: 9 total, 9 up, 9 in 2024-04-20T21:29:10.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:09 smithi033 ceph-mon[21626]: osdmap e66: 9 total, 9 up, 9 in 2024-04-20T21:29:10.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:09 smithi022 ceph-mon[16843]: osdmap e66: 9 total, 9 up, 9 in 2024-04-20T21:29:11.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:10 smithi122 ceph-mon[22147]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:11.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:10 smithi022 ceph-mon[16843]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:11.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:10 smithi033 ceph-mon[21626]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:12.948 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:12 smithi022 ceph-mon[16843]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-20T21:29:12.949 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:12 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/809067164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c35e9ebb-ebc5-4547-9490-09c2944b3e9c"}]: dispatch 2024-04-20T21:29:12.949 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:12 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/809067164' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c35e9ebb-ebc5-4547-9490-09c2944b3e9c"}]': finished 2024-04-20T21:29:12.949 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:12 smithi022 ceph-mon[16843]: osdmap e67: 10 total, 9 up, 10 in 2024-04-20T21:29:12.949 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:12 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:13.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:12 smithi122 ceph-mon[22147]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-20T21:29:13.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:12 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/809067164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c35e9ebb-ebc5-4547-9490-09c2944b3e9c"}]: dispatch 2024-04-20T21:29:13.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:12 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/809067164' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c35e9ebb-ebc5-4547-9490-09c2944b3e9c"}]': finished 2024-04-20T21:29:13.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:12 smithi122 ceph-mon[22147]: osdmap e67: 10 total, 9 up, 10 in 2024-04-20T21:29:13.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:12 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:13.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:12 smithi033 ceph-mon[21626]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-20T21:29:13.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:12 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/809067164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c35e9ebb-ebc5-4547-9490-09c2944b3e9c"}]: dispatch 2024-04-20T21:29:13.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:12 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/809067164' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c35e9ebb-ebc5-4547-9490-09c2944b3e9c"}]': finished 2024-04-20T21:29:13.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:12 smithi033 ceph-mon[21626]: osdmap e67: 10 total, 9 up, 10 in 2024-04-20T21:29:13.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:12 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:14.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:13 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1240059786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:29:14.026 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:13 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1240059786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:29:14.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:13 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1240059786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:29:15.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:14 smithi122 ceph-mon[22147]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:29:15.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:14 smithi033 ceph-mon[21626]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:29:15.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:14 smithi022 ceph-mon[16843]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:29:16.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:16 smithi022 ceph-mon[16843]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-20T21:29:17.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:16 smithi033 ceph-mon[21626]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-20T21:29:17.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:16 smithi122 ceph-mon[22147]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-20T21:29:18.822 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:18 smithi022 ceph-mon[16843]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-20T21:29:19.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:18 smithi033 ceph-mon[21626]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-20T21:29:19.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:18 smithi122 ceph-mon[22147]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-20T21:29:21.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:20 smithi033 ceph-mon[21626]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:29:21.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:20 smithi022 ceph-mon[16843]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:29:21.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:20 smithi122 ceph-mon[22147]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:29:21.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:21 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-20T21:29:21.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:21 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:22.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:21 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-20T21:29:22.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:21 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:22.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:21 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-20T21:29:22.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:21 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:23.011 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:22 smithi022 ceph-mon[16843]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:23.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:22 smithi022 ceph-mon[16843]: Deploying daemon osd.9 on smithi022 2024-04-20T21:29:23.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:22 smithi033 ceph-mon[21626]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:23.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:22 smithi033 ceph-mon[21626]: Deploying daemon osd.9 on smithi022 2024-04-20T21:29:23.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:22 smithi122 ceph-mon[22147]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:23.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:22 smithi122 ceph-mon[22147]: Deploying daemon osd.9 on smithi022 2024-04-20T21:29:25.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:24 smithi122 ceph-mon[22147]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:25.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:24 smithi033 ceph-mon[21626]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:25.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:24 smithi022 ceph-mon[16843]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:27.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:26 smithi033 ceph-mon[21626]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:27.092 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:26 smithi022 ceph-mon[16843]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:27.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:26 smithi122 ceph-mon[22147]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:29.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:28 smithi022 ceph-mon[16843]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:29.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:28 smithi122 ceph-mon[22147]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:29.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:28 smithi033 ceph-mon[21626]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:31.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:31 smithi022 ceph-mon[16843]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:31.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:31.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:31.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:31.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:31 smithi122 ceph-mon[22147]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:31.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:31.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:31.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:31.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:31 smithi033 ceph-mon[21626]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:31.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:31.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:31.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:33.402 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:33 smithi022 ceph-mon[16843]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:33.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:33 smithi122 ceph-mon[22147]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:33.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:33 smithi033 ceph-mon[21626]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:35.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:34 smithi022 ceph-mon[16843]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:35.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:34 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:35.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:34 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:35.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:34 smithi022 ceph-mon[16843]: from='osd.9 [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T21:29:35.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:34 smithi022 ceph-mon[16843]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T21:29:35.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:34 smithi122 ceph-mon[22147]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:35.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:34 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:35.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:34 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:35.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:34 smithi122 ceph-mon[22147]: from='osd.9 [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T21:29:35.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:34 smithi122 ceph-mon[22147]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T21:29:35.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:34 smithi033 ceph-mon[21626]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:35.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:34 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:35.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:34 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:35.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:34 smithi033 ceph-mon[21626]: from='osd.9 [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T21:29:35.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:34 smithi033 ceph-mon[21626]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T21:29:35.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.371+0000 7fb8c7400700 1 -- 172.21.15.22:0/3819431921 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8d0062e90 con 0x7fb8b40a3c40 2024-04-20T21:29:35.371 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 9 on host 'smithi022' 2024-04-20T21:29:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.376+0000 7fb8cfe00700 1 -- 172.21.15.22:0/3819431921 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb8b40a3c40 msgr2=0x7fb8b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.376+0000 7fb8cfe00700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb8b40a3c40 0x7fb8b40a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80038c0 tx=0x7fb8b8005fb0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.376+0000 7fb8cfe00700 1 -- 172.21.15.22:0/3819431921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d0104bf0 msgr2=0x7fb8d019adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.376+0000 7fb8cfe00700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d0104bf0 0x7fb8d019adb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000d770 tx=0x7fb8c000da80 comp rx=0 tx=0).stop 2024-04-20T21:29:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.376+0000 7fb8cfe00700 1 -- 172.21.15.22:0/3819431921 shutdown_connections 2024-04-20T21:29:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.376+0000 7fb8cfe00700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb8b40a3c40 0x7fb8b40a6100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.376+0000 7fb8cfe00700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8d0106790 0x7fb8d01a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.376+0000 7fb8cfe00700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d0104bf0 0x7fb8d019adb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.376+0000 7fb8cfe00700 1 --2- 172.21.15.22:0/3819431921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8d0105df0 0x7fb8d019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.376+0000 7fb8cfe00700 1 -- 172.21.15.22:0/3819431921 >> 172.21.15.22:0/3819431921 conn(0x7fb8d0100170 msgr2=0x7fb8d01090f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.377+0000 7fb8cfe00700 1 -- 172.21.15.22:0/3819431921 shutdown_connections 2024-04-20T21:29:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:29:35.377+0000 7fb8cfe00700 1 -- 172.21.15.22:0/3819431921 wait complete. 2024-04-20T21:29:36.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:35 smithi022 ceph-mon[16843]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-20T21:29:36.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:35 smithi022 ceph-mon[16843]: osdmap e68: 10 total, 9 up, 10 in 2024-04-20T21:29:36.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:36.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:35 smithi022 ceph-mon[16843]: from='osd.9 [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:29:36.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:35 smithi022 ceph-mon[16843]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:29:36.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:36.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:35 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:36.128 DEBUG:teuthology.orchestra.run.smithi022:osd.9> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.9.service 2024-04-20T21:29:36.132 INFO:tasks.cephadm:Deploying osd.10 on smithi033 with /dev/vg_nvme/lv_1... 2024-04-20T21:29:36.132 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-20T21:29:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:35 smithi122 ceph-mon[22147]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-20T21:29:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:35 smithi122 ceph-mon[22147]: osdmap e68: 10 total, 9 up, 10 in 2024-04-20T21:29:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:35 smithi122 ceph-mon[22147]: from='osd.9 [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:29:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:35 smithi122 ceph-mon[22147]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:29:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:36.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:35 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:36.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:35 smithi033 ceph-mon[21626]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-20T21:29:36.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:35 smithi033 ceph-mon[21626]: osdmap e68: 10 total, 9 up, 10 in 2024-04-20T21:29:36.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:36.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:35 smithi033 ceph-mon[21626]: from='osd.9 [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:29:36.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:35 smithi033 ceph-mon[21626]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-04-20T21:29:36.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:36.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:35 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:36.389 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:29:37.191 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:36 smithi022 ceph-mon[16843]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:37.191 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:36 smithi022 ceph-mon[16843]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-20T21:29:37.192 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:36 smithi022 ceph-mon[16843]: osdmap e69: 10 total, 9 up, 10 in 2024-04-20T21:29:37.192 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:36 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:37.192 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:36 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:37.192 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:36 smithi022 ceph-mon[16843]: from='osd.9 ' entity='osd.9' 2024-04-20T21:29:37.192 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:29:36 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:29:36.938+0000 7f0c7f600700 -1 osd.9 0 waiting for initial osdmap 2024-04-20T21:29:37.192 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:29:36 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:29:36.983+0000 7f0c75400700 -1 osd.9 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:29:37.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:36 smithi122 ceph-mon[22147]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:37.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:36 smithi122 ceph-mon[22147]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-20T21:29:37.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:36 smithi122 ceph-mon[22147]: osdmap e69: 10 total, 9 up, 10 in 2024-04-20T21:29:37.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:36 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:37.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:36 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:37.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:36 smithi122 ceph-mon[22147]: from='osd.9 ' entity='osd.9' 2024-04-20T21:29:37.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:36 smithi033 ceph-mon[21626]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:37.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:36 smithi033 ceph-mon[21626]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-04-20T21:29:37.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:36 smithi033 ceph-mon[21626]: osdmap e69: 10 total, 9 up, 10 in 2024-04-20T21:29:37.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:36 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:37.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:36 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:37.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:36 smithi033 ceph-mon[21626]: from='osd.9 ' entity='osd.9' 2024-04-20T21:29:38.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:37 smithi022 ceph-mon[16843]: purged_snaps scrub starts 2024-04-20T21:29:38.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:37 smithi022 ceph-mon[16843]: purged_snaps scrub ok 2024-04-20T21:29:38.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:37 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:38.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:37 smithi122 ceph-mon[22147]: purged_snaps scrub starts 2024-04-20T21:29:38.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:37 smithi122 ceph-mon[22147]: purged_snaps scrub ok 2024-04-20T21:29:38.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:37 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:38.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:37 smithi033 ceph-mon[21626]: purged_snaps scrub starts 2024-04-20T21:29:38.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:37 smithi033 ceph-mon[21626]: purged_snaps scrub ok 2024-04-20T21:29:38.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:37 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:38.407 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-20T21:29:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-20T21:29:38.408 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-20T21:29:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-20T21:29:38.408 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112508 s, 932 MB/s 2024-04-20T21:29:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-20T21:29:39.105 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_1 2024-04-20T21:29:39.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:38 smithi122 ceph-mon[22147]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:39.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:38 smithi122 ceph-mon[22147]: osd.9 [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711] boot 2024-04-20T21:29:39.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:38 smithi122 ceph-mon[22147]: osdmap e70: 10 total, 10 up, 10 in 2024-04-20T21:29:39.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:38 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:39.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:38 smithi033 ceph-mon[21626]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:39.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:38 smithi033 ceph-mon[21626]: osd.9 [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711] boot 2024-04-20T21:29:39.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:38 smithi033 ceph-mon[21626]: osdmap e70: 10 total, 10 up, 10 in 2024-04-20T21:29:39.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:38 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:39.352 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:29:39.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:38 smithi022 ceph-mon[16843]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:39.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:38 smithi022 ceph-mon[16843]: osd.9 [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711] boot 2024-04-20T21:29:39.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:38 smithi022 ceph-mon[16843]: osdmap e70: 10 total, 10 up, 10 in 2024-04-20T21:29:39.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:38 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:40.032 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:39 smithi033 ceph-mon[21626]: osdmap e71: 10 total, 10 up, 10 in 2024-04-20T21:29:40.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:39 smithi122 ceph-mon[22147]: osdmap e71: 10 total, 10 up, 10 in 2024-04-20T21:29:40.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:39 smithi022 ceph-mon[16843]: osdmap e71: 10 total, 10 up, 10 in 2024-04-20T21:29:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.852+0000 7f91191bf700 1 -- 172.21.15.33:0/35735524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9114104670 msgr2=0x7f911410af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.852+0000 7f91191bf700 1 --2- 172.21.15.33:0/35735524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9114104670 0x7f911410af20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f910c00b0d0 tx=0x7f910c00b3e0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.852+0000 7f91191bf700 1 -- 172.21.15.33:0/35735524 shutdown_connections 2024-04-20T21:29:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.852+0000 7f91191bf700 1 --2- 172.21.15.33:0/35735524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9114104670 0x7f911410af20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.852+0000 7f91191bf700 1 --2- 172.21.15.33:0/35735524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9114102ad0 0x7f9114102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.852+0000 7f91191bf700 1 --2- 172.21.15.33:0/35735524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9114103cd0 0x7f9114104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.852+0000 7f91191bf700 1 -- 172.21.15.33:0/35735524 >> 172.21.15.33:0/35735524 conn(0x7f91140fe070 msgr2=0x7f91141004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.853+0000 7f91191bf700 1 -- 172.21.15.33:0/35735524 shutdown_connections 2024-04-20T21:29:40.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.853+0000 7f91191bf700 1 -- 172.21.15.33:0/35735524 wait complete. 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.853+0000 7f91191bf700 1 Processor -- start 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f91191bf700 1 -- start start 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f91191bf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9114102ad0 0x7f91140767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f91191bf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9114103cd0 0x7f9114074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f91191bf700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9114104670 0x7f9114075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f91191bf700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f911410e730 con 0x7f9114103cd0 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f91191bf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f911410e5b0 con 0x7f9114102ad0 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f91191bf700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f911410e8b0 con 0x7f9114104670 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f9112d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9114102ad0 0x7f91140767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f9112d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9114102ad0 0x7f91140767f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57244/0 (socket says 172.21.15.33:57244) 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f9112d9d700 1 -- 172.21.15.33:0/2798083483 learned_addr learned my addr 172.21.15.33:0/2798083483 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f911359e700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9114104670 0x7f9114075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f9112d9d700 1 -- 172.21.15.33:0/2798083483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9114104670 msgr2=0x7f9114075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f9112d9d700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9114104670 0x7f9114075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f9112d9d700 1 -- 172.21.15.33:0/2798083483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9114103cd0 msgr2=0x7f9114074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f9112d9d700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9114103cd0 0x7f9114074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f9112d9d700 1 -- 172.21.15.33:0/2798083483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9114075c30 con 0x7f9114102ad0 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f911359e700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9114104670 0x7f9114075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.854+0000 7f9112d9d700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9114102ad0 0x7f91140767f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f910400b500 tx=0x7f910400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.855+0000 7f90fbfff700 1 -- 172.21.15.33:0/2798083483 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104015020 con 0x7f9114102ad0 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.855+0000 7f91191bf700 1 -- 172.21.15.33:0/2798083483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91141a2e50 con 0x7f9114102ad0 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.855+0000 7f90fbfff700 1 -- 172.21.15.33:0/2798083483 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9104011d10 con 0x7f9114102ad0 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.855+0000 7f90fbfff700 1 -- 172.21.15.33:0/2798083483 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910400f2e0 con 0x7f9114102ad0 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.855+0000 7f91191bf700 1 -- 172.21.15.33:0/2798083483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91141a3320 con 0x7f9114102ad0 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.856+0000 7f91191bf700 1 -- 172.21.15.33:0/2798083483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911402df90 con 0x7f9114102ad0 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.857+0000 7f90fbfff700 1 -- 172.21.15.33:0/2798083483 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9104018020 con 0x7f9114102ad0 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.857+0000 7f90fbfff700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f90fc0a3c40 0x7f90fc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.857+0000 7f90fbfff700 1 -- 172.21.15.33:0/2798083483 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6261+0+0 (secure 0 0 0) 0x7f91040cf830 con 0x7f9114102ad0 2024-04-20T21:29:40.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.858+0000 7f911259c700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f90fc0a3c40 0x7f90fc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.859+0000 7f911259c700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f90fc0a3c40 0x7f90fc0a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f910800a990 tx=0x7f9108005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:40.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:40.861+0000 7f90fbfff700 1 -- 172.21.15.33:0/2798083483 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f910401f020 con 0x7f9114102ad0 2024-04-20T21:29:41.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:29:41.080+0000 7f91191bf700 1 -- 172.21.15.33:0/2798083483 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9114062e90 con 0x7f90fc0a3c40 2024-04-20T21:29:41.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:40 smithi022 ceph-mon[16843]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:41.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:40 smithi122 ceph-mon[22147]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:41.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:40 smithi033 ceph-mon[21626]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:42.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: Detected new or changed devices on smithi022 2024-04-20T21:29:42.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:42.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi022 to 4273M 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:42.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:42.277 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: Detected new or changed devices on smithi022 2024-04-20T21:29:42.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:42.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi022 to 4273M 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:42.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: Detected new or changed devices on smithi022 2024-04-20T21:29:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi022 to 4273M 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:42.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:42.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:43.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:43 smithi022 ceph-mon[16843]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:43.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:43 smithi122 ceph-mon[22147]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:43.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:43 smithi033 ceph-mon[21626]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:44 smithi033 ceph-mon[21626]: from='client.? 172.21.15.33:0/570106911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c"}]: dispatch 2024-04-20T21:29:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:44 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c"}]: dispatch 2024-04-20T21:29:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:44 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c"}]': finished 2024-04-20T21:29:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:44 smithi033 ceph-mon[21626]: osdmap e72: 11 total, 10 up, 11 in 2024-04-20T21:29:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:44.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:44 smithi022 ceph-mon[16843]: from='client.? 172.21.15.33:0/570106911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c"}]: dispatch 2024-04-20T21:29:44.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:44 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c"}]: dispatch 2024-04-20T21:29:44.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:44 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c"}]': finished 2024-04-20T21:29:44.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:44 smithi022 ceph-mon[16843]: osdmap e72: 11 total, 10 up, 11 in 2024-04-20T21:29:44.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:44.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:44 smithi122 ceph-mon[22147]: from='client.? 172.21.15.33:0/570106911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c"}]: dispatch 2024-04-20T21:29:44.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:44 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c"}]: dispatch 2024-04-20T21:29:44.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:44 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c"}]': finished 2024-04-20T21:29:44.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:44 smithi122 ceph-mon[22147]: osdmap e72: 11 total, 10 up, 11 in 2024-04-20T21:29:44.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:45.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:45 smithi022 ceph-mon[16843]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:45.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:45 smithi022 ceph-mon[16843]: from='client.? 172.21.15.33:0/312488531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:29:45.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:45 smithi122 ceph-mon[22147]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:45.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:45 smithi122 ceph-mon[22147]: from='client.? 172.21.15.33:0/312488531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:29:45.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:45 smithi033 ceph-mon[21626]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:45.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:45 smithi033 ceph-mon[21626]: from='client.? 172.21.15.33:0/312488531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:29:47.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:47 smithi022 ceph-mon[16843]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:47.757 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:47 smithi033 ceph-mon[21626]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:47.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:47 smithi122 ceph-mon[22147]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:49.633 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:49 smithi033 ceph-mon[21626]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:49.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:49 smithi022 ceph-mon[16843]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:49.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:49 smithi122 ceph-mon[22147]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:51.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:51 smithi033 ceph-mon[21626]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:51.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:51 smithi122 ceph-mon[22147]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:51.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:51 smithi022 ceph-mon[16843]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:52 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-20T21:29:52.669 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:52 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:52.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:52 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-20T21:29:52.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:52 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:52.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:52 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-20T21:29:52.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:52 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:53.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:53 smithi033 ceph-mon[21626]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:53.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:53 smithi033 ceph-mon[21626]: Deploying daemon osd.10 on smithi033 2024-04-20T21:29:53.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:53 smithi122 ceph-mon[22147]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:53.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:53 smithi122 ceph-mon[22147]: Deploying daemon osd.10 on smithi033 2024-04-20T21:29:53.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:53 smithi022 ceph-mon[16843]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:53.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:53 smithi022 ceph-mon[16843]: Deploying daemon osd.10 on smithi033 2024-04-20T21:29:55.618 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:55 smithi033 ceph-mon[21626]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:55.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:55 smithi122 ceph-mon[22147]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:55.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:55 smithi022 ceph-mon[16843]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:57 smithi122 ceph-mon[22147]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:57.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:57 smithi033 ceph-mon[21626]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:57.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:57 smithi022 ceph-mon[16843]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:58 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:58 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:58 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:58.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:58 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:58.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:58 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:58.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:58 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:58.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:58 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:58.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:58 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:58.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:58 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:29:59.478 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:29:59 smithi033 ceph-mon[21626]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:59.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:29:59 smithi122 ceph-mon[22147]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:59.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:29:59 smithi022 ceph-mon[16843]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:30:00.705 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:00 smithi033 ceph-mon[21626]: overall HEALTH_OK 2024-04-20T21:30:00.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:00 smithi122 ceph-mon[22147]: overall HEALTH_OK 2024-04-20T21:30:00.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:00 smithi022 ceph-mon[16843]: overall HEALTH_OK 2024-04-20T21:30:01.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.468+0000 7f90fbfff700 1 -- 172.21.15.33:0/2798083483 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9114062e90 con 0x7f90fc0a3c40 2024-04-20T21:30:01.467 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 10 on host 'smithi033' 2024-04-20T21:30:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.470+0000 7f91191bf700 1 -- 172.21.15.33:0/2798083483 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f90fc0a3c40 msgr2=0x7f90fc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.470+0000 7f91191bf700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f90fc0a3c40 0x7f90fc0a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f910800a990 tx=0x7f9108005c80 comp rx=0 tx=0).stop 2024-04-20T21:30:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.470+0000 7f91191bf700 1 -- 172.21.15.33:0/2798083483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9114102ad0 msgr2=0x7f91140767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.470+0000 7f91191bf700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9114102ad0 0x7f91140767f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f910400b500 tx=0x7f910400b8c0 comp rx=0 tx=0).stop 2024-04-20T21:30:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.470+0000 7f91191bf700 1 -- 172.21.15.33:0/2798083483 shutdown_connections 2024-04-20T21:30:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.470+0000 7f91191bf700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f90fc0a3c40 0x7f90fc0a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:01.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.470+0000 7f91191bf700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9114104670 0x7f9114075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:01.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.470+0000 7f91191bf700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9114102ad0 0x7f91140767f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:01.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.470+0000 7f91191bf700 1 --2- 172.21.15.33:0/2798083483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9114103cd0 0x7f9114074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:01.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.470+0000 7f91191bf700 1 -- 172.21.15.33:0/2798083483 >> 172.21.15.33:0/2798083483 conn(0x7f91140fe070 msgr2=0x7f9114106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:01.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.470+0000 7f91191bf700 1 -- 172.21.15.33:0/2798083483 shutdown_connections 2024-04-20T21:30:01.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-20T21:30:01.470+0000 7f91191bf700 1 -- 172.21.15.33:0/2798083483 wait complete. 2024-04-20T21:30:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:01 smithi122 ceph-mon[22147]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:30:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:01 smithi122 ceph-mon[22147]: from='osd.10 [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T21:30:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:01 smithi122 ceph-mon[22147]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T21:30:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:01.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:01 smithi033 ceph-mon[21626]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:30:01.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:01.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:01.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:01 smithi033 ceph-mon[21626]: from='osd.10 [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T21:30:01.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:01 smithi033 ceph-mon[21626]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T21:30:01.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:01.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:01 smithi022 ceph-mon[16843]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:30:01.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:01.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:01.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:01 smithi022 ceph-mon[16843]: from='osd.10 [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T21:30:01.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:01 smithi022 ceph-mon[16843]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T21:30:01.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:01.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:02.132 DEBUG:teuthology.orchestra.run.smithi033:osd.10> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.10.service 2024-04-20T21:30:02.135 INFO:tasks.cephadm:Deploying osd.11 on smithi122 with /dev/vg_nvme/lv_1... 2024-04-20T21:30:02.135 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-20T21:30:02.383 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:30:02.676 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:02 smithi033 ceph-mon[21626]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-20T21:30:02.676 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:02 smithi033 ceph-mon[21626]: osdmap e73: 11 total, 10 up, 11 in 2024-04-20T21:30:02.676 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:02 smithi033 ceph-mon[21626]: from='osd.10 [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:30:02.676 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:02 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:30:02.676 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:02 smithi033 ceph-mon[21626]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:30:02.676 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:02 smithi033 ceph-mon[21626]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:30:02.677 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:30:02 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:30:02.536+0000 7f25d5927700 -1 osd.10 0 waiting for initial osdmap 2024-04-20T21:30:02.677 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:30:02 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:30:02.551+0000 7f25d171a700 -1 osd.10 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:30:02.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:02 smithi122 ceph-mon[22147]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-20T21:30:02.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:02 smithi122 ceph-mon[22147]: osdmap e73: 11 total, 10 up, 11 in 2024-04-20T21:30:02.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:02 smithi122 ceph-mon[22147]: from='osd.10 [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:30:02.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:02 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:30:02.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:02 smithi122 ceph-mon[22147]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:30:02.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:02 smithi122 ceph-mon[22147]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:30:02.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:02 smithi022 ceph-mon[16843]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-20T21:30:02.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:02 smithi022 ceph-mon[16843]: osdmap e73: 11 total, 10 up, 11 in 2024-04-20T21:30:02.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:02 smithi022 ceph-mon[16843]: from='osd.10 [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:30:02.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:02 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:30:02.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:02 smithi022 ceph-mon[16843]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-20T21:30:02.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:02 smithi022 ceph-mon[16843]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:30:03.752 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:03 smithi122 ceph-mon[22147]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-20T21:30:03.752 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:03 smithi122 ceph-mon[22147]: osdmap e74: 11 total, 10 up, 11 in 2024-04-20T21:30:03.752 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:03 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:30:03.752 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:03 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:30:03.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:03 smithi033 ceph-mon[21626]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-20T21:30:03.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:03 smithi033 ceph-mon[21626]: osdmap e74: 11 total, 10 up, 11 in 2024-04-20T21:30:03.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:03 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:30:03.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:03 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:30:03.841 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:03 smithi022 ceph-mon[16843]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-20T21:30:03.841 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:03 smithi022 ceph-mon[16843]: osdmap e74: 11 total, 10 up, 11 in 2024-04-20T21:30:03.841 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:30:03.841 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:03 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:30:04.336 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-20T21:30:04.336 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-20T21:30:04.336 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-20T21:30:04.336 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-20T21:30:04.336 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121472 s, 863 MB/s 2024-04-20T21:30:04.336 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-20T21:30:04.638 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:04 smithi122 ceph-mon[22147]: purged_snaps scrub starts 2024-04-20T21:30:04.638 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:04 smithi122 ceph-mon[22147]: purged_snaps scrub ok 2024-04-20T21:30:04.638 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:04 smithi122 ceph-mon[22147]: osd.10 [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244] boot 2024-04-20T21:30:04.638 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:04 smithi122 ceph-mon[22147]: osdmap e75: 11 total, 11 up, 11 in 2024-04-20T21:30:04.638 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:30:04.638 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:04 smithi122 ceph-mon[22147]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:04.638 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:04 smithi122 ceph-mon[22147]: osdmap e76: 11 total, 11 up, 11 in 2024-04-20T21:30:04.745 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:04 smithi033 ceph-mon[21626]: purged_snaps scrub starts 2024-04-20T21:30:04.745 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:04 smithi033 ceph-mon[21626]: purged_snaps scrub ok 2024-04-20T21:30:04.745 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:04 smithi033 ceph-mon[21626]: osd.10 [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244] boot 2024-04-20T21:30:04.745 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:04 smithi033 ceph-mon[21626]: osdmap e75: 11 total, 11 up, 11 in 2024-04-20T21:30:04.745 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:30:04.746 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:04 smithi033 ceph-mon[21626]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:04.746 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:04 smithi033 ceph-mon[21626]: osdmap e76: 11 total, 11 up, 11 in 2024-04-20T21:30:04.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:04 smithi022 ceph-mon[16843]: purged_snaps scrub starts 2024-04-20T21:30:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:04 smithi022 ceph-mon[16843]: purged_snaps scrub ok 2024-04-20T21:30:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:04 smithi022 ceph-mon[16843]: osd.10 [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244] boot 2024-04-20T21:30:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:04 smithi022 ceph-mon[16843]: osdmap e75: 11 total, 11 up, 11 in 2024-04-20T21:30:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:30:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:04 smithi022 ceph-mon[16843]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:04.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:04 smithi022 ceph-mon[16843]: osdmap e76: 11 total, 11 up, 11 in 2024-04-20T21:30:04.957 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_1 2024-04-20T21:30:05.199 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:30:06.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: osdmap e77: 11 total, 11 up, 11 in 2024-04-20T21:30:06.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: Detected new or changed devices on smithi033 2024-04-20T21:30:06.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:06.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:06.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:06.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:06.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:06.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:06.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi033 to 4273M 2024-04-20T21:30:06.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:06.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:06.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:06.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:06.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: osdmap e77: 11 total, 11 up, 11 in 2024-04-20T21:30:06.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: Detected new or changed devices on smithi033 2024-04-20T21:30:06.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:06.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:06.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:06.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:06.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:06.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:06.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi033 to 4273M 2024-04-20T21:30:06.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:06.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:06.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:06.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: osdmap e77: 11 total, 11 up, 11 in 2024-04-20T21:30:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: Detected new or changed devices on smithi033 2024-04-20T21:30:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi033 to 4273M 2024-04-20T21:30:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:06.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:06.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.844+0000 7f4ad5e97700 1 -- 172.21.15.122:0/148665313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0102bb0 msgr2=0x7f4ad0103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:06.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.844+0000 7f4ad5e97700 1 --2- 172.21.15.122:0/148665313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0102bb0 0x7f4ad0103010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009990 tx=0x7f4ab8009ca0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.844+0000 7f4ad5e97700 1 -- 172.21.15.122:0/148665313 shutdown_connections 2024-04-20T21:30:06.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.844+0000 7f4ad5e97700 1 --2- 172.21.15.122:0/148665313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad0103550 0x7f4ad0109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.844+0000 7f4ad5e97700 1 --2- 172.21.15.122:0/148665313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ad01019b0 0x7f4ad0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.844+0000 7f4ad5e97700 1 --2- 172.21.15.122:0/148665313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0102bb0 0x7f4ad0103010 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.844+0000 7f4ad5e97700 1 -- 172.21.15.122:0/148665313 >> 172.21.15.122:0/148665313 conn(0x7f4ad00fcf50 msgr2=0x7f4ad00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.845+0000 7f4ad5e97700 1 -- 172.21.15.122:0/148665313 shutdown_connections 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.845+0000 7f4ad5e97700 1 -- 172.21.15.122:0/148665313 wait complete. 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.845+0000 7f4ad5e97700 1 Processor -- start 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.845+0000 7f4ad5e97700 1 -- start start 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.845+0000 7f4ad5e97700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad01019b0 0x7f4ad006bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.845+0000 7f4ad5e97700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ad0102bb0 0x7f4ad019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.845+0000 7f4ad5e97700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0103550 0x7f4ad01a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.845+0000 7f4ad5e97700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad010d630 con 0x7f4ad0102bb0 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.845+0000 7f4ad5e97700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad010d4b0 con 0x7f4ad01019b0 2024-04-20T21:30:06.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.845+0000 7f4ad5e97700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad010d7b0 con 0x7f4ad0103550 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.845+0000 7f4acffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0103550 0x7f4ad01a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4acffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0103550 0x7f4ad01a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38586/0 (socket says 172.21.15.122:38586) 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4acffff700 1 -- 172.21.15.122:0/2469855200 learned_addr learned my addr 172.21.15.122:0/2469855200 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4acf7fe700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad01019b0 0x7f4ad006bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4acffff700 1 -- 172.21.15.122:0/2469855200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad01019b0 msgr2=0x7f4ad006bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4acffff700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad01019b0 0x7f4ad006bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4acffff700 1 -- 172.21.15.122:0/2469855200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ad0102bb0 msgr2=0x7f4ad019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4acffff700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ad0102bb0 0x7f4ad019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4acffff700 1 -- 172.21.15.122:0/2469855200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad01a0980 con 0x7f4ad0103550 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4acffff700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0103550 0x7f4ad01a00b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8007ee0 tx=0x7f4ac800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4acd7fa700 1 -- 172.21.15.122:0/2469855200 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac800dbd0 con 0x7f4ad0103550 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4ad5e97700 1 -- 172.21.15.122:0/2469855200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad01a0c70 con 0x7f4ad0103550 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4acd7fa700 1 -- 172.21.15.122:0/2469855200 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac800dd70 con 0x7f4ad0103550 2024-04-20T21:30:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4acd7fa700 1 -- 172.21.15.122:0/2469855200 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8005a60 con 0x7f4ad0103550 2024-04-20T21:30:06.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.846+0000 7f4ad5e97700 1 -- 172.21.15.122:0/2469855200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad01a1280 con 0x7f4ad0103550 2024-04-20T21:30:06.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.847+0000 7f4ad5e97700 1 -- 172.21.15.122:0/2469855200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad0069740 con 0x7f4ad0103550 2024-04-20T21:30:06.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.848+0000 7f4acd7fa700 1 -- 172.21.15.122:0/2469855200 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4ac801a020 con 0x7f4ad0103550 2024-04-20T21:30:06.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.849+0000 7f4acd7fa700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4abc0a3cc0 0x7f4abc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:06.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.849+0000 7f4acd7fa700 1 -- 172.21.15.122:0/2469855200 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6694+0+0 (secure 0 0 0) 0x7f4ac80cf420 con 0x7f4ad0103550 2024-04-20T21:30:06.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.849+0000 7f4acf7fe700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4abc0a3cc0 0x7f4abc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:06.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.850+0000 7f4acf7fe700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4abc0a3cc0 0x7f4abc0a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0102a10 tx=0x7f4ac0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:06.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:06.853+0000 7f4acd7fa700 1 -- 172.21.15.122:0/2469855200 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4ac8010070 con 0x7f4ad0103550 2024-04-20T21:30:07.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:07.065+0000 7f4ad5e97700 1 -- 172.21.15.122:0/2469855200 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4ad0062e90 con 0x7f4abc0a3cc0 2024-04-20T21:30:07.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:06 smithi122 ceph-mon[22147]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:07.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:06 smithi122 ceph-mon[22147]: osdmap e78: 11 total, 11 up, 11 in 2024-04-20T21:30:07.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:06 smithi033 ceph-mon[21626]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:07.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:06 smithi033 ceph-mon[21626]: osdmap e78: 11 total, 11 up, 11 in 2024-04-20T21:30:07.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:06 smithi022 ceph-mon[16843]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:07.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:06 smithi022 ceph-mon[16843]: osdmap e78: 11 total, 11 up, 11 in 2024-04-20T21:30:08.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:07 smithi122 ceph-mon[22147]: osdmap e79: 11 total, 11 up, 11 in 2024-04-20T21:30:08.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:07 smithi122 ceph-mon[22147]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:30:08.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:07 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:30:08.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:07 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:30:08.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:07 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:08.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:07 smithi033 ceph-mon[21626]: osdmap e79: 11 total, 11 up, 11 in 2024-04-20T21:30:08.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:07 smithi033 ceph-mon[21626]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:30:08.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:07 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:30:08.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:07 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:30:08.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:07 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:08.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:07 smithi022 ceph-mon[16843]: osdmap e79: 11 total, 11 up, 11 in 2024-04-20T21:30:08.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:07 smithi022 ceph-mon[16843]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:30:08.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:07 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:30:08.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:07 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:30:08.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:07 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:09.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:09 smithi122 ceph-mon[22147]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:09.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:09 smithi033 ceph-mon[21626]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:09.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:09 smithi022 ceph-mon[16843]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:10.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:10 smithi122 ceph-mon[22147]: from='client.? 172.21.15.122:0/649741967' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e35bd98e-f771-40fb-b428-3c182f1c59c1"}]: dispatch 2024-04-20T21:30:10.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:10 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e35bd98e-f771-40fb-b428-3c182f1c59c1"}]: dispatch 2024-04-20T21:30:10.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:10 smithi122 ceph-mon[22147]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e35bd98e-f771-40fb-b428-3c182f1c59c1"}]': finished 2024-04-20T21:30:10.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:10 smithi122 ceph-mon[22147]: osdmap e80: 12 total, 11 up, 12 in 2024-04-20T21:30:10.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:10 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:10.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:10 smithi033 ceph-mon[21626]: from='client.? 172.21.15.122:0/649741967' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e35bd98e-f771-40fb-b428-3c182f1c59c1"}]: dispatch 2024-04-20T21:30:10.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:10 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e35bd98e-f771-40fb-b428-3c182f1c59c1"}]: dispatch 2024-04-20T21:30:10.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:10 smithi033 ceph-mon[21626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e35bd98e-f771-40fb-b428-3c182f1c59c1"}]': finished 2024-04-20T21:30:10.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:10 smithi033 ceph-mon[21626]: osdmap e80: 12 total, 11 up, 12 in 2024-04-20T21:30:10.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:10 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:10.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:10 smithi022 ceph-mon[16843]: from='client.? 172.21.15.122:0/649741967' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e35bd98e-f771-40fb-b428-3c182f1c59c1"}]: dispatch 2024-04-20T21:30:10.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:10 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e35bd98e-f771-40fb-b428-3c182f1c59c1"}]: dispatch 2024-04-20T21:30:10.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:10 smithi022 ceph-mon[16843]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e35bd98e-f771-40fb-b428-3c182f1c59c1"}]': finished 2024-04-20T21:30:10.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:10 smithi022 ceph-mon[16843]: osdmap e80: 12 total, 11 up, 12 in 2024-04-20T21:30:10.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:10 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:11 smithi122 ceph-mon[22147]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:11.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:11 smithi122 ceph-mon[22147]: from='client.? 172.21.15.122:0/2348043736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:30:11.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:11 smithi033 ceph-mon[21626]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:11.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:11 smithi033 ceph-mon[21626]: from='client.? 172.21.15.122:0/2348043736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:30:11.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:11 smithi022 ceph-mon[16843]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:11.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:11 smithi022 ceph-mon[16843]: from='client.? 172.21.15.122:0/2348043736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:30:13.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:13 smithi122 ceph-mon[22147]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:30:13.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:13 smithi033 ceph-mon[21626]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:30:13.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:13 smithi022 ceph-mon[16843]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:30:15.158 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:15 smithi122 ceph-mon[22147]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-20T21:30:15.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:15 smithi033 ceph-mon[21626]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-20T21:30:15.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:15 smithi022 ceph-mon[16843]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-20T21:30:17.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:17 smithi122 ceph-mon[22147]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-20T21:30:17.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:17 smithi033 ceph-mon[21626]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-20T21:30:17.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:17 smithi022 ceph-mon[16843]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-20T21:30:18.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-20T21:30:18.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:18 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:18.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:18 smithi122 ceph-mon[22147]: Deploying daemon osd.11 on smithi122 2024-04-20T21:30:18.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-20T21:30:18.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:18 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:18.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:18 smithi033 ceph-mon[21626]: Deploying daemon osd.11 on smithi122 2024-04-20T21:30:18.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-20T21:30:18.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:18 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:18.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:18 smithi022 ceph-mon[16843]: Deploying daemon osd.11 on smithi122 2024-04-20T21:30:19.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:19 smithi122 ceph-mon[22147]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:30:19.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:19 smithi033 ceph-mon[21626]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:30:19.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:19 smithi022 ceph-mon[16843]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:30:21.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:21 smithi122 ceph-mon[22147]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:30:21.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:21 smithi033 ceph-mon[21626]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:30:21.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:21 smithi022 ceph-mon[16843]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:30:23.396 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:23 smithi033 ceph-mon[21626]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-20T21:30:23.396 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:23 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:23.396 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:23 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:23.396 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:23 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:23.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:23 smithi022 ceph-mon[16843]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-20T21:30:23.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:23 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:23.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:23 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:23.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:23 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:23.418 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:23 smithi122 ceph-mon[22147]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-20T21:30:23.418 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:23 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:23.418 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:23 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:23.418 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:23 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:25.270 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:25 smithi122 ceph-mon[22147]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:25.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:25 smithi022 ceph-mon[16843]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:25.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:25 smithi033 ceph-mon[21626]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.555+0000 7f4acd7fa700 1 -- 172.21.15.122:0/2469855200 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4ad0062e90 con 0x7f4abc0a3cc0 2024-04-20T21:30:26.555 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 11 on host 'smithi122' 2024-04-20T21:30:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.558+0000 7f4ad5e97700 1 -- 172.21.15.122:0/2469855200 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4abc0a3cc0 msgr2=0x7f4abc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.558+0000 7f4ad5e97700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4abc0a3cc0 0x7f4abc0a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0102a10 tx=0x7f4ac0009190 comp rx=0 tx=0).stop 2024-04-20T21:30:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.558+0000 7f4ad5e97700 1 -- 172.21.15.122:0/2469855200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0103550 msgr2=0x7f4ad01a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.558+0000 7f4ad5e97700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0103550 0x7f4ad01a00b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8007ee0 tx=0x7f4ac800d3b0 comp rx=0 tx=0).stop 2024-04-20T21:30:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.559+0000 7f4ad5e97700 1 -- 172.21.15.122:0/2469855200 shutdown_connections 2024-04-20T21:30:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.559+0000 7f4ad5e97700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4abc0a3cc0 0x7f4abc0a6180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.559+0000 7f4ad5e97700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0103550 0x7f4ad01a00b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.559+0000 7f4ad5e97700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad01019b0 0x7f4ad006bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.559+0000 7f4ad5e97700 1 --2- 172.21.15.122:0/2469855200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ad0102bb0 0x7f4ad019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.559+0000 7f4ad5e97700 1 -- 172.21.15.122:0/2469855200 >> 172.21.15.122:0/2469855200 conn(0x7f4ad00fcf50 msgr2=0x7f4ad0105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.559+0000 7f4ad5e97700 1 -- 172.21.15.122:0/2469855200 shutdown_connections 2024-04-20T21:30:26.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:26.559+0000 7f4ad5e97700 1 -- 172.21.15.122:0/2469855200 wait complete. 2024-04-20T21:30:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:26 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:26 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:26 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:26 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:26 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:26 smithi122 ceph-mon[22147]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:26 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:26 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:26 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:26 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:26 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:26 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:26 smithi033 ceph-mon[21626]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:26 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:26.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:26 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:26.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:26 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:26.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:26 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:26.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:26 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:26.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:26 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:26.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:26 smithi022 ceph-mon[16843]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:26.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:26 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:27.173 DEBUG:teuthology.orchestra.run.smithi122:osd.11> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.11.service 2024-04-20T21:30:27.175 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-20T21:30:27.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-20T21:30:27.655 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:30:27.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:27 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:27.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:27 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:27.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:27 smithi122 ceph-mon[22147]: from='osd.11 [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T21:30:27.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:27 smithi122 ceph-mon[22147]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T21:30:27.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:27 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:27.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:27 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:27.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:27 smithi022 ceph-mon[16843]: from='osd.11 [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T21:30:27.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:27 smithi022 ceph-mon[16843]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T21:30:27.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:27 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:27.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:27 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:27.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:27 smithi033 ceph-mon[21626]: from='osd.11 [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T21:30:27.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:27 smithi033 ceph-mon[21626]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T21:30:28.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:28 smithi022 ceph-mon[16843]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-20T21:30:28.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:28 smithi022 ceph-mon[16843]: from='osd.11 [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:30:28.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:28 smithi022 ceph-mon[16843]: osdmap e81: 12 total, 11 up, 12 in 2024-04-20T21:30:28.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:28.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:28 smithi022 ceph-mon[16843]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:30:28.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:28 smithi022 ceph-mon[16843]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:28 smithi122 ceph-mon[22147]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-20T21:30:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:28 smithi122 ceph-mon[22147]: from='osd.11 [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:30:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:28 smithi122 ceph-mon[22147]: osdmap e81: 12 total, 11 up, 12 in 2024-04-20T21:30:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:28 smithi122 ceph-mon[22147]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:30:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:28 smithi122 ceph-mon[22147]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:29.026 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:30:28 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:30:28.570+0000 7f19f0707700 -1 osd.11 0 waiting for initial osdmap 2024-04-20T21:30:29.026 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:30:28 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:30:28.590+0000 7f19ec4fa700 -1 osd.11 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:30:29.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:28 smithi033 ceph-mon[21626]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-20T21:30:29.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:28 smithi033 ceph-mon[21626]: from='osd.11 [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:30:29.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:28 smithi033 ceph-mon[21626]: osdmap e81: 12 total, 11 up, 12 in 2024-04-20T21:30:29.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:29.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:28 smithi033 ceph-mon[21626]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-20T21:30:29.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:28 smithi033 ceph-mon[21626]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:29.671 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:29 smithi122 ceph-mon[22147]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-20T21:30:29.672 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:29 smithi122 ceph-mon[22147]: osdmap e82: 12 total, 11 up, 12 in 2024-04-20T21:30:29.672 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:29 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:29.672 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:29 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:29.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:29 smithi022 ceph-mon[16843]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-20T21:30:29.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:29 smithi022 ceph-mon[16843]: osdmap e82: 12 total, 11 up, 12 in 2024-04-20T21:30:29.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:29.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:29 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:30.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:29 smithi033 ceph-mon[21626]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-20T21:30:30.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:29 smithi033 ceph-mon[21626]: osdmap e82: 12 total, 11 up, 12 in 2024-04-20T21:30:30.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:29 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:30.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:29 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:30.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.065+0000 7f4ace200700 1 -- 172.21.15.22:0/4051072982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ac8105dd0 msgr2=0x7f4ac8106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.065+0000 7f4ace200700 1 --2- 172.21.15.22:0/4051072982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ac8105dd0 0x7f4ac8106230 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009990 tx=0x7f4ab4009ca0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.065+0000 7f4ace200700 1 -- 172.21.15.22:0/4051072982 shutdown_connections 2024-04-20T21:30:30.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.065+0000 7f4ace200700 1 --2- 172.21.15.22:0/4051072982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ac8106770 0x7f4ac810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.065+0000 7f4ace200700 1 --2- 172.21.15.22:0/4051072982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac8104bd0 0x7f4ac8104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.065+0000 7f4ace200700 1 --2- 172.21.15.22:0/4051072982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ac8105dd0 0x7f4ac8106230 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.065+0000 7f4ace200700 1 -- 172.21.15.22:0/4051072982 >> 172.21.15.22:0/4051072982 conn(0x7f4ac8100190 msgr2=0x7f4ac81025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:30.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.065+0000 7f4ace200700 1 -- 172.21.15.22:0/4051072982 shutdown_connections 2024-04-20T21:30:30.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.065+0000 7f4ace200700 1 -- 172.21.15.22:0/4051072982 wait complete. 2024-04-20T21:30:30.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.067+0000 7f4ace200700 1 Processor -- start 2024-04-20T21:30:30.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.067+0000 7f4ace200700 1 -- start start 2024-04-20T21:30:30.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.067+0000 7f4ace200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ac8104bd0 0x7f4ac819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.067+0000 7f4ace200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ac8105dd0 0x7f4ac819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.067+0000 7f4ace200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac8106770 0x7f4ac81a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.067+0000 7f4ace200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81107b0 con 0x7f4ac8105dd0 2024-04-20T21:30:30.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.067+0000 7f4ace200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8110630 con 0x7f4ac8106770 2024-04-20T21:30:30.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.067+0000 7f4ace200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8110930 con 0x7f4ac8104bd0 2024-04-20T21:30:30.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.068+0000 7f4acce00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ac8104bd0 0x7f4ac819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.068+0000 7f4acce00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ac8104bd0 0x7f4ac819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:54344/0 (socket says 172.21.15.22:54344) 2024-04-20T21:30:30.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.068+0000 7f4acce00700 1 -- 172.21.15.22:0/1960825294 learned_addr learned my addr 172.21.15.22:0/1960825294 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:30:30.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.068+0000 7f4ac7e00700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ac8105dd0 0x7f4ac819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.068+0000 7f4acd800700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac8106770 0x7f4ac81a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.068+0000 7f4acce00700 1 -- 172.21.15.22:0/1960825294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac8106770 msgr2=0x7f4ac81a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.068+0000 7f4acce00700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac8106770 0x7f4ac81a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.068+0000 7f4acce00700 1 -- 172.21.15.22:0/1960825294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ac8105dd0 msgr2=0x7f4ac819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.068+0000 7f4acce00700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ac8105dd0 0x7f4ac819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.068+0000 7f4acce00700 1 -- 172.21.15.22:0/1960825294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac81a2a00 con 0x7f4ac8104bd0 2024-04-20T21:30:30.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.068+0000 7f4acce00700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ac8104bd0 0x7f4ac819adb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00e950 tx=0x7f4abc00ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:30.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.071+0000 7f4ac5600700 1 -- 172.21.15.22:0/1960825294 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc004230 con 0x7f4ac8104bd0 2024-04-20T21:30:30.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.071+0000 7f4ac5600700 1 -- 172.21.15.22:0/1960825294 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4abc0043d0 con 0x7f4ac8104bd0 2024-04-20T21:30:30.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.071+0000 7f4ac5600700 1 -- 172.21.15.22:0/1960825294 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc003e00 con 0x7f4ac8104bd0 2024-04-20T21:30:30.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.071+0000 7f4ace200700 1 -- 172.21.15.22:0/1960825294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac81a2cf0 con 0x7f4ac8104bd0 2024-04-20T21:30:30.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.071+0000 7f4ace200700 1 -- 172.21.15.22:0/1960825294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac81a3180 con 0x7f4ac8104bd0 2024-04-20T21:30:30.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.072+0000 7f4ace200700 1 -- 172.21.15.22:0/1960825294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac8069740 con 0x7f4ac8104bd0 2024-04-20T21:30:30.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.074+0000 7f4ac5600700 1 -- 172.21.15.22:0/1960825294 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4abc00c7c0 con 0x7f4ac8104bd0 2024-04-20T21:30:30.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.075+0000 7f4ac5600700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4ab00a3cc0 0x7f4ab00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.075+0000 7f4ac5600700 1 -- 172.21.15.22:0/1960825294 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f4abc0d03a0 con 0x7f4ac8104bd0 2024-04-20T21:30:30.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.076+0000 7f4ac7e00700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4ab00a3cc0 0x7f4ab00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.076+0000 7f4ac7e00700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4ab00a3cc0 0x7f4ab00a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400b2e0 tx=0x7f4ab4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:30.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.080+0000 7f4ac5600700 1 -- 172.21.15.22:0/1960825294 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4abc09d180 con 0x7f4ac8104bd0 2024-04-20T21:30:30.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.325+0000 7f4ace200700 1 -- 172.21.15.22:0/1960825294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4ac8063660 con 0x7f4ac8104bd0 2024-04-20T21:30:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.326+0000 7f4ac5600700 1 -- 172.21.15.22:0/1960825294 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7f4abc0bd0c0 con 0x7f4ac8104bd0 2024-04-20T21:30:30.325 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:30:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.329+0000 7f4ace200700 1 -- 172.21.15.22:0/1960825294 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4ab00a3cc0 msgr2=0x7f4ab00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.329+0000 7f4ace200700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4ab00a3cc0 0x7f4ab00a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400b2e0 tx=0x7f4ab4005fd0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.329+0000 7f4ace200700 1 -- 172.21.15.22:0/1960825294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ac8104bd0 msgr2=0x7f4ac819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.329+0000 7f4ace200700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ac8104bd0 0x7f4ac819adb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00e950 tx=0x7f4abc00ec60 comp rx=0 tx=0).stop 2024-04-20T21:30:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.330+0000 7f4ace200700 1 -- 172.21.15.22:0/1960825294 shutdown_connections 2024-04-20T21:30:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.330+0000 7f4ace200700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4ab00a3cc0 0x7f4ab00a6180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.330+0000 7f4ace200700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac8106770 0x7f4ac81a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.330+0000 7f4ace200700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ac8104bd0 0x7f4ac819adb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.330+0000 7f4ace200700 1 --2- 172.21.15.22:0/1960825294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ac8105dd0 0x7f4ac819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.330+0000 7f4ace200700 1 -- 172.21.15.22:0/1960825294 >> 172.21.15.22:0/1960825294 conn(0x7f4ac8100190 msgr2=0x7f4ac81090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.330+0000 7f4ace200700 1 -- 172.21.15.22:0/1960825294 shutdown_connections 2024-04-20T21:30:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:30.330+0000 7f4ace200700 1 -- 172.21.15.22:0/1960825294 wait complete. 2024-04-20T21:30:30.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: purged_snaps scrub starts 2024-04-20T21:30:30.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: purged_snaps scrub ok 2024-04-20T21:30:30.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:30.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: osd.11 [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204] boot 2024-04-20T21:30:30.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: osdmap e83: 12 total, 12 up, 12 in 2024-04-20T21:30:30.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:30.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:30.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: Detected new or changed devices on smithi122 2024-04-20T21:30:30.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:30.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:30.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:30.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:30.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:30.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:30.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi122 to 4272M 2024-04-20T21:30:30.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:30.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:30.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:30.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:30.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:30 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1960825294' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T21:30:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: purged_snaps scrub starts 2024-04-20T21:30:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: purged_snaps scrub ok 2024-04-20T21:30:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: osd.11 [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204] boot 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: osdmap e83: 12 total, 12 up, 12 in 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: Detected new or changed devices on smithi122 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi122 to 4272M 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:31.028 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:31.028 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:30 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1960825294' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T21:30:31.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: purged_snaps scrub starts 2024-04-20T21:30:31.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: purged_snaps scrub ok 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: osd.11 [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204] boot 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: osdmap e83: 12 total, 12 up, 12 in 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: Detected new or changed devices on smithi122 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:30:31.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi122 to 4272M 2024-04-20T21:30:31.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:31.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:31.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:31.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:31.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:30 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1960825294' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T21:30:31.200 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1713648629,"num_in_osds":12,"osd_in_since":1713648609,"num_remapped_pgs":0} 2024-04-20T21:30:31.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-20T21:30:31.659 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:30:31.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:31 smithi022 ceph-mon[16843]: osdmap e84: 12 total, 12 up, 12 in 2024-04-20T21:30:32.024 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:31 smithi122 ceph-mon[22147]: osdmap e84: 12 total, 12 up, 12 in 2024-04-20T21:30:32.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:31 smithi033 ceph-mon[21626]: osdmap e84: 12 total, 12 up, 12 in 2024-04-20T21:30:32.826 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:32 smithi022 ceph-mon[16843]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:33.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:32 smithi122 ceph-mon[22147]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:33.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:32 smithi033 ceph-mon[21626]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:33.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.826+0000 7f3e4b800700 1 -- 172.21.15.22:0/1031318304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e44104ed0 msgr2=0x7f3e441052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.826+0000 7f3e4b800700 1 --2- 172.21.15.22:0/1031318304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e44104ed0 0x7f3e441052b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3e340099c0 tx=0x7f3e34009cd0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.826+0000 7f3e4b800700 1 -- 172.21.15.22:0/1031318304 shutdown_connections 2024-04-20T21:30:33.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.826+0000 7f3e4b800700 1 --2- 172.21.15.22:0/1031318304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e44109fb0 0x7f3e44111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.826+0000 7f3e4b800700 1 --2- 172.21.15.22:0/1031318304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e44104ed0 0x7f3e441052b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.826+0000 7f3e4b800700 1 --2- 172.21.15.22:0/1031318304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e44105880 0x7f3e441098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.826+0000 7f3e4b800700 1 -- 172.21.15.22:0/1031318304 >> 172.21.15.22:0/1031318304 conn(0x7f3e44100760 msgr2=0x7f3e44102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:33.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.826+0000 7f3e4b800700 1 -- 172.21.15.22:0/1031318304 shutdown_connections 2024-04-20T21:30:33.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.827+0000 7f3e4b800700 1 -- 172.21.15.22:0/1031318304 wait complete. 2024-04-20T21:30:33.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.828+0000 7f3e4b800700 1 Processor -- start 2024-04-20T21:30:33.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.828+0000 7f3e4b800700 1 -- start start 2024-04-20T21:30:33.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.829+0000 7f3e4b800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e44104ed0 0x7f3e4419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.829+0000 7f3e4b800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e44105880 0x7f3e4419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.829+0000 7f3e4b800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e44109fb0 0x7f3e4419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.829+0000 7f3e4b800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44114ff0 con 0x7f3e44104ed0 2024-04-20T21:30:33.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.829+0000 7f3e4b800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44114e70 con 0x7f3e44109fb0 2024-04-20T21:30:33.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.829+0000 7f3e4b800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44115170 con 0x7f3e44105880 2024-04-20T21:30:33.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.829+0000 7f3e48e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e44104ed0 0x7f3e4419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.829+0000 7f3e48e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e44104ed0 0x7f3e4419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51510/0 (socket says 172.21.15.22:51510) 2024-04-20T21:30:33.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.829+0000 7f3e48e00700 1 -- 172.21.15.22:0/4012239441 learned_addr learned my addr 172.21.15.22:0/4012239441 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:30:33.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.829+0000 7f3e3be00700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e44105880 0x7f3e4419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.829+0000 7f3e49800700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e44109fb0 0x7f3e4419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.830+0000 7f3e3be00700 1 -- 172.21.15.22:0/4012239441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e44109fb0 msgr2=0x7f3e4419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.830+0000 7f3e3be00700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e44109fb0 0x7f3e4419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.830+0000 7f3e3be00700 1 -- 172.21.15.22:0/4012239441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e44104ed0 msgr2=0x7f3e4419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.830+0000 7f3e3be00700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e44104ed0 0x7f3e4419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.830+0000 7f3e3be00700 1 -- 172.21.15.22:0/4012239441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e441a0450 con 0x7f3e44105880 2024-04-20T21:30:33.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.830+0000 7f3e48e00700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e44104ed0 0x7f3e4419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:33.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.830+0000 7f3e3be00700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e44105880 0x7f3e4419b980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e9c0 tx=0x7f3e2c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:33.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.830+0000 7f3e42000700 1 -- 172.21.15.22:0/4012239441 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c00cb70 con 0x7f3e44105880 2024-04-20T21:30:33.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.830+0000 7f3e4b800700 1 -- 172.21.15.22:0/4012239441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e441a0740 con 0x7f3e44105880 2024-04-20T21:30:33.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.830+0000 7f3e42000700 1 -- 172.21.15.22:0/4012239441 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e2c004500 con 0x7f3e44105880 2024-04-20T21:30:33.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.830+0000 7f3e42000700 1 -- 172.21.15.22:0/4012239441 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c003de0 con 0x7f3e44105880 2024-04-20T21:30:33.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.831+0000 7f3e4b800700 1 -- 172.21.15.22:0/4012239441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e441a7fc0 con 0x7f3e44105880 2024-04-20T21:30:33.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.832+0000 7f3e4b800700 1 -- 172.21.15.22:0/4012239441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e44069740 con 0x7f3e44105880 2024-04-20T21:30:33.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.834+0000 7f3e42000700 1 -- 172.21.15.22:0/4012239441 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3e2c003fc0 con 0x7f3e44105880 2024-04-20T21:30:33.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.835+0000 7f3e42000700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3e280a3c40 0x7f3e280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.835+0000 7f3e42000700 1 -- 172.21.15.22:0/4012239441 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f3e2c0d0130 con 0x7f3e44105880 2024-04-20T21:30:33.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.835+0000 7f3e48e00700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3e280a3c40 0x7f3e280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.836+0000 7f3e48e00700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3e280a3c40 0x7f3e280a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009990 tx=0x7f3e34005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:33.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:33.840+0000 7f3e42000700 1 -- 172.21.15.22:0/4012239441 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3e2c00ee60 con 0x7f3e44105880 2024-04-20T21:30:34.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.071+0000 7f3e4b800700 1 -- 172.21.15.22:0/4012239441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e4402df90 con 0x7f3e44105880 2024-04-20T21:30:34.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.072+0000 7f3e42000700 1 -- 172.21.15.22:0/4012239441 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18433 (secure 0 0 0) 0x7f3e2c09cf10 con 0x7f3e44105880 2024-04-20T21:30:34.072 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:30:34.073 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":84,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:30:30.588294+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:34.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.075+0000 7f3e4b800700 1 -- 172.21.15.22:0/4012239441 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3e280a3c40 msgr2=0x7f3e280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:34.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.075+0000 7f3e4b800700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3e280a3c40 0x7f3e280a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009990 tx=0x7f3e34005ab0 comp rx=0 tx=0).stop 2024-04-20T21:30:34.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.075+0000 7f3e4b800700 1 -- 172.21.15.22:0/4012239441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e44105880 msgr2=0x7f3e4419b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:34.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.076+0000 7f3e4b800700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e44105880 0x7f3e4419b980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e9c0 tx=0x7f3e2c00ecd0 comp rx=0 tx=0).stop 2024-04-20T21:30:34.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.076+0000 7f3e4b800700 1 -- 172.21.15.22:0/4012239441 shutdown_connections 2024-04-20T21:30:34.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.076+0000 7f3e4b800700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3e280a3c40 0x7f3e280a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:34.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.076+0000 7f3e4b800700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e44109fb0 0x7f3e4419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:34.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.076+0000 7f3e4b800700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e44104ed0 0x7f3e4419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:34.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.076+0000 7f3e4b800700 1 --2- 172.21.15.22:0/4012239441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e44105880 0x7f3e4419b980 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:34.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.076+0000 7f3e4b800700 1 -- 172.21.15.22:0/4012239441 >> 172.21.15.22:0/4012239441 conn(0x7f3e44100760 msgr2=0x7f3e44102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:34.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.077+0000 7f3e4b800700 1 -- 172.21.15.22:0/4012239441 shutdown_connections 2024-04-20T21:30:34.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:34.077+0000 7f3e4b800700 1 -- 172.21.15.22:0/4012239441 wait complete. 2024-04-20T21:30:34.844 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:26:15.688818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:30:34.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-20T21:30:35.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:34 smithi122 ceph-mon[22147]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:35.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:34 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/4012239441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:35.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:34 smithi033 ceph-mon[21626]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:35.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:34 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/4012239441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:35.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:34 smithi022 ceph-mon[16843]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:35.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:34 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/4012239441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:35.315 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:30:37.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:36 smithi122 ceph-mon[22147]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:37.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:36 smithi033 ceph-mon[21626]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:37.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:36 smithi022 ceph-mon[16843]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.663+0000 7f7ac2000700 1 -- 172.21.15.22:0/3638520817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7abc10e3c0 msgr2=0x7f7abc115f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.663+0000 7f7ac2000700 1 --2- 172.21.15.22:0/3638520817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7abc10e3c0 0x7f7abc115f90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400b0d0 tx=0x7f7ab400b3e0 comp rx=0 tx=0).stop 2024-04-20T21:30:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.663+0000 7f7ac2000700 1 -- 172.21.15.22:0/3638520817 shutdown_connections 2024-04-20T21:30:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.663+0000 7f7ac2000700 1 --2- 172.21.15.22:0/3638520817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7abc10e3c0 0x7f7abc115f90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.663+0000 7f7ac2000700 1 --2- 172.21.15.22:0/3638520817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7abc1092e0 0x7f7abc1096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.663+0000 7f7ac2000700 1 --2- 172.21.15.22:0/3638520817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc109c90 0x7f7abc10dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.663+0000 7f7ac2000700 1 -- 172.21.15.22:0/3638520817 >> 172.21.15.22:0/3638520817 conn(0x7f7abc076220 msgr2=0x7f7abc107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.663+0000 7f7ac2000700 1 -- 172.21.15.22:0/3638520817 shutdown_connections 2024-04-20T21:30:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.663+0000 7f7ac2000700 1 -- 172.21.15.22:0/3638520817 wait complete. 2024-04-20T21:30:37.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.664+0000 7f7ac2000700 1 Processor -- start 2024-04-20T21:30:37.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.665+0000 7f7ac2000700 1 -- start start 2024-04-20T21:30:37.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.665+0000 7f7ac2000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc1092e0 0x7f7abc1a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:37.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.665+0000 7f7ac2000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7abc109c90 0x7f7abc1a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:37.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.665+0000 7f7ac2000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7abc10e3c0 0x7f7abc19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:37.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.665+0000 7f7ac2000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc1193b0 con 0x7f7abc109c90 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.665+0000 7f7ac2000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc119230 con 0x7f7abc1092e0 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.665+0000 7f7ac2000700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc119530 con 0x7f7abc10e3c0 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.665+0000 7f7ab3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7abc109c90 0x7f7abc1a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.665+0000 7f7ab3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7abc109c90 0x7f7abc1a59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51528/0 (socket says 172.21.15.22:51528) 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.665+0000 7f7ab3e00700 1 -- 172.21.15.22:0/1570690778 learned_addr learned my addr 172.21.15.22:0/1570690778 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.665+0000 7f7abbe00700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7abc10e3c0 0x7f7abc19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.665+0000 7f7abb400700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc1092e0 0x7f7abc1a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.666+0000 7f7ab3e00700 1 -- 172.21.15.22:0/1570690778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7abc10e3c0 msgr2=0x7f7abc19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.666+0000 7f7ab3e00700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7abc10e3c0 0x7f7abc19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.666+0000 7f7ab3e00700 1 -- 172.21.15.22:0/1570690778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc1092e0 msgr2=0x7f7abc1a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.666+0000 7f7ab3e00700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc1092e0 0x7f7abc1a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.666+0000 7f7ab3e00700 1 -- 172.21.15.22:0/1570690778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc19fdd0 con 0x7f7abc109c90 2024-04-20T21:30:37.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.666+0000 7f7ab3e00700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7abc109c90 0x7f7abc1a59a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400e970 tx=0x7f7aa400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:37.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.666+0000 7f7ab8c00700 1 -- 172.21.15.22:0/1570690778 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4004a30 con 0x7f7abc109c90 2024-04-20T21:30:37.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.667+0000 7f7ab8c00700 1 -- 172.21.15.22:0/1570690778 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa400ccc0 con 0x7f7abc109c90 2024-04-20T21:30:37.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.667+0000 7f7ac2000700 1 -- 172.21.15.22:0/1570690778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7abc1a00c0 con 0x7f7abc109c90 2024-04-20T21:30:37.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.667+0000 7f7ab8c00700 1 -- 172.21.15.22:0/1570690778 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4005650 con 0x7f7abc109c90 2024-04-20T21:30:37.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.667+0000 7f7ac2000700 1 -- 172.21.15.22:0/1570690778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7abc1ac330 con 0x7f7abc109c90 2024-04-20T21:30:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.668+0000 7f7ac2000700 1 -- 172.21.15.22:0/1570690778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7abc069740 con 0x7f7abc109c90 2024-04-20T21:30:37.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.670+0000 7f7ab8c00700 1 -- 172.21.15.22:0/1570690778 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7aa4004bd0 con 0x7f7abc109c90 2024-04-20T21:30:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.672+0000 7f7ab8c00700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7aa00a3cc0 0x7f7aa00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.672+0000 7f7ab8c00700 1 -- 172.21.15.22:0/1570690778 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f7aa40d0110 con 0x7f7abc109c90 2024-04-20T21:30:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.672+0000 7f7abb400700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7aa00a3cc0 0x7f7aa00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.673+0000 7f7abb400700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7aa00a3cc0 0x7f7aa00a6180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009c60 tx=0x7f7aac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:37.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.677+0000 7f7ab8c00700 1 -- 172.21.15.22:0/1570690778 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7aa409cef0 con 0x7f7abc109c90 2024-04-20T21:30:37.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.906+0000 7f7ac2000700 1 -- 172.21.15.22:0/1570690778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7abc1a0b50 con 0x7f7abc109c90 2024-04-20T21:30:37.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.906+0000 7f7ab8c00700 1 -- 172.21.15.22:0/1570690778 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f7aa409c7f0 con 0x7f7abc109c90 2024-04-20T21:30:37.906 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-20T21:30:37.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.909+0000 7f7ac2000700 1 -- 172.21.15.22:0/1570690778 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7aa00a3cc0 msgr2=0x7f7aa00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:37.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.909+0000 7f7ac2000700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7aa00a3cc0 0x7f7aa00a6180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009c60 tx=0x7f7aac009260 comp rx=0 tx=0).stop 2024-04-20T21:30:37.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.909+0000 7f7ac2000700 1 -- 172.21.15.22:0/1570690778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7abc109c90 msgr2=0x7f7abc1a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:37.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.909+0000 7f7ac2000700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7abc109c90 0x7f7abc1a59a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400e970 tx=0x7f7aa400ec80 comp rx=0 tx=0).stop 2024-04-20T21:30:37.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.910+0000 7f7ac2000700 1 -- 172.21.15.22:0/1570690778 shutdown_connections 2024-04-20T21:30:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.910+0000 7f7ac2000700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7aa00a3cc0 0x7f7aa00a6180 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.910+0000 7f7ac2000700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7abc10e3c0 0x7f7abc19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.910+0000 7f7ac2000700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc1092e0 0x7f7abc1a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.910+0000 7f7ac2000700 1 --2- 172.21.15.22:0/1570690778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7abc109c90 0x7f7abc1a59a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.910+0000 7f7ac2000700 1 -- 172.21.15.22:0/1570690778 >> 172.21.15.22:0/1570690778 conn(0x7f7abc076220 msgr2=0x7f7abc0767e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.910+0000 7f7ac2000700 1 -- 172.21.15.22:0/1570690778 shutdown_connections 2024-04-20T21:30:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:37.910+0000 7f7ac2000700 1 -- 172.21.15.22:0/1570690778 wait complete. 2024-04-20T21:30:38.857 INFO:tasks.cephadm:Adding mds.a on smithi022 2024-04-20T21:30:38.857 INFO:tasks.cephadm:Adding mds.d on smithi022 2024-04-20T21:30:38.857 INFO:tasks.cephadm:Adding mds.g on smithi022 2024-04-20T21:30:38.857 INFO:tasks.cephadm:Adding mds.j on smithi022 2024-04-20T21:30:38.857 INFO:tasks.cephadm:Adding mds.b on smithi033 2024-04-20T21:30:38.857 INFO:tasks.cephadm:Adding mds.e on smithi033 2024-04-20T21:30:38.857 INFO:tasks.cephadm:Adding mds.h on smithi033 2024-04-20T21:30:38.857 INFO:tasks.cephadm:Adding mds.k on smithi033 2024-04-20T21:30:38.857 INFO:tasks.cephadm:Adding mds.c on smithi122 2024-04-20T21:30:38.857 INFO:tasks.cephadm:Adding mds.f on smithi122 2024-04-20T21:30:38.857 INFO:tasks.cephadm:Adding mds.i on smithi122 2024-04-20T21:30:38.857 INFO:tasks.cephadm:Adding mds.l on smithi122 2024-04-20T21:30:38.857 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l' 2024-04-20T21:30:39.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:38 smithi122 ceph-mon[22147]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:39.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:38 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1570690778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:39.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:38 smithi033 ceph-mon[21626]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:39.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:38 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1570690778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:39.110 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:30:39.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:38 smithi022 ceph-mon[16843]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:39.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:38 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1570690778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:40.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.687+0000 7fe32215b700 1 -- 172.21.15.122:0/3483793369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe31c1019c0 msgr2=0x7fe31c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.687+0000 7fe32215b700 1 --2- 172.21.15.122:0/3483793369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe31c1019c0 0x7fe31c101de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00b0a0 tx=0x7fe30c00b3b0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.687+0000 7fe32215b700 1 -- 172.21.15.122:0/3483793369 shutdown_connections 2024-04-20T21:30:40.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.687+0000 7fe32215b700 1 --2- 172.21.15.122:0/3483793369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c103560 0x7fe31c109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.687+0000 7fe32215b700 1 --2- 172.21.15.122:0/3483793369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe31c1019c0 0x7fe31c101de0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.687+0000 7fe32215b700 1 --2- 172.21.15.122:0/3483793369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe31c102bc0 0x7fe31c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.687+0000 7fe32215b700 1 -- 172.21.15.122:0/3483793369 >> 172.21.15.122:0/3483793369 conn(0x7fe31c0fcf80 msgr2=0x7fe31c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:40.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.687+0000 7fe32215b700 1 -- 172.21.15.122:0/3483793369 shutdown_connections 2024-04-20T21:30:40.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.687+0000 7fe32215b700 1 -- 172.21.15.122:0/3483793369 wait complete. 2024-04-20T21:30:40.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.687+0000 7fe32215b700 1 Processor -- start 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.687+0000 7fe32215b700 1 -- start start 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe32215b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe31c1019c0 0x7fe31c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe32215b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe31c102bc0 0x7fe31c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe32215b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c103560 0x7fe31c1a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe32215b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c10e720 con 0x7fe31c103560 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe32215b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c10e5a0 con 0x7fe31c102bc0 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe32215b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c10e8a0 con 0x7fe31c1019c0 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe31b7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe31c1019c0 0x7fe31c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe31b7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe31c1019c0 0x7fe31c198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54888/0 (socket says 172.21.15.122:54888) 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe31b7fe700 1 -- 172.21.15.122:0/3291995026 learned_addr learned my addr 172.21.15.122:0/3291995026 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe31affd700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe31c102bc0 0x7fe31c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe31b7fe700 1 -- 172.21.15.122:0/3291995026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe31c102bc0 msgr2=0x7fe31c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe31b7fe700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe31c102bc0 0x7fe31c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe31b7fe700 1 -- 172.21.15.122:0/3291995026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c103560 msgr2=0x7fe31c1a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe31b7fe700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c103560 0x7fe31c1a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe31b7fe700 1 -- 172.21.15.122:0/3291995026 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31c1a0a00 con 0x7fe31c1019c0 2024-04-20T21:30:40.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe31b7fe700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe31c1019c0 0x7fe31c198db0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe30c006130 tx=0x7fe30c007d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:40.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe318ff9700 1 -- 172.21.15.122:0/3291995026 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c003e10 con 0x7fe31c1019c0 2024-04-20T21:30:40.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe32215b700 1 -- 172.21.15.122:0/3291995026 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31c1a0c90 con 0x7fe31c1019c0 2024-04-20T21:30:40.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe318ff9700 1 -- 172.21.15.122:0/3291995026 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe30c022070 con 0x7fe31c1019c0 2024-04-20T21:30:40.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe318ff9700 1 -- 172.21.15.122:0/3291995026 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c01b8a0 con 0x7fe31c1019c0 2024-04-20T21:30:40.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.688+0000 7fe32215b700 1 -- 172.21.15.122:0/3291995026 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe31c1a1140 con 0x7fe31c1019c0 2024-04-20T21:30:40.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.689+0000 7fe32215b700 1 -- 172.21.15.122:0/3291995026 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31c069740 con 0x7fe31c1019c0 2024-04-20T21:30:40.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.690+0000 7fe318ff9700 1 -- 172.21.15.122:0/3291995026 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe30c01db70 con 0x7fe31c1019c0 2024-04-20T21:30:40.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.691+0000 7fe318ff9700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe3040a3cc0 0x7fe3040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.691+0000 7fe318ff9700 1 -- 172.21.15.122:0/3291995026 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fe30c00f020 con 0x7fe31c1019c0 2024-04-20T21:30:40.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.691+0000 7fe31affd700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe3040a3cc0 0x7fe3040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.692+0000 7fe31affd700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe3040a3cc0 0x7fe3040a6180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe310005950 tx=0x7fe31000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:40.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.694+0000 7fe318ff9700 1 -- 172.21.15.122:0/3291995026 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe30c09e480 con 0x7fe31c1019c0 2024-04-20T21:30:40.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.904+0000 7fe32215b700 1 -- 172.21.15.122:0/3291995026 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}) -- 0x7fe31c062f90 con 0x7fe3040a3cc0 2024-04-20T21:30:40.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.921+0000 7fe318ff9700 1 -- 172.21.15.122:0/3291995026 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe31c062f90 con 0x7fe3040a3cc0 2024-04-20T21:30:40.920 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled mds.all update... 2024-04-20T21:30:40.920 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:40 smithi122 ceph-mon[22147]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:40.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.923+0000 7fe32215b700 1 -- 172.21.15.122:0/3291995026 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe3040a3cc0 msgr2=0x7fe3040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.923+0000 7fe32215b700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe3040a3cc0 0x7fe3040a6180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe310005950 tx=0x7fe31000b410 comp rx=0 tx=0).stop 2024-04-20T21:30:40.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.923+0000 7fe32215b700 1 -- 172.21.15.122:0/3291995026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe31c1019c0 msgr2=0x7fe31c198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.923+0000 7fe32215b700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe31c1019c0 0x7fe31c198db0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe30c006130 tx=0x7fe30c007d90 comp rx=0 tx=0).stop 2024-04-20T21:30:40.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.923+0000 7fe32215b700 1 -- 172.21.15.122:0/3291995026 shutdown_connections 2024-04-20T21:30:40.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.923+0000 7fe32215b700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe3040a3cc0 0x7fe3040a6180 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.923+0000 7fe32215b700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe31c103560 0x7fe31c1a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.923+0000 7fe32215b700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe31c1019c0 0x7fe31c198db0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.923+0000 7fe32215b700 1 --2- 172.21.15.122:0/3291995026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe31c102bc0 0x7fe31c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.923+0000 7fe32215b700 1 -- 172.21.15.122:0/3291995026 >> 172.21.15.122:0/3291995026 conn(0x7fe31c0fcf80 msgr2=0x7fe31c105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:40.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.923+0000 7fe32215b700 1 -- 172.21.15.122:0/3291995026 shutdown_connections 2024-04-20T21:30:40.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-20T21:30:40.923+0000 7fe32215b700 1 -- 172.21.15.122:0/3291995026 wait complete. 2024-04-20T21:30:41.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:40 smithi033 ceph-mon[21626]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:41.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:40 smithi022 ceph-mon[16843]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:41.551 DEBUG:teuthology.orchestra.run.smithi022:mds.a> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.a.service 2024-04-20T21:30:41.559 DEBUG:teuthology.orchestra.run.smithi022:mds.d> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.d.service 2024-04-20T21:30:41.605 DEBUG:teuthology.orchestra.run.smithi022:mds.g> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.g.service 2024-04-20T21:30:41.651 DEBUG:teuthology.orchestra.run.smithi022:mds.j> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.j.service 2024-04-20T21:30:41.696 DEBUG:teuthology.orchestra.run.smithi033:mds.b> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.b.service 2024-04-20T21:30:41.698 DEBUG:teuthology.orchestra.run.smithi033:mds.e> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.e.service 2024-04-20T21:30:41.742 DEBUG:teuthology.orchestra.run.smithi033:mds.h> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.h.service 2024-04-20T21:30:41.785 DEBUG:teuthology.orchestra.run.smithi033:mds.k> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.k.service 2024-04-20T21:30:41.828 DEBUG:teuthology.orchestra.run.smithi122:mds.c> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.c.service 2024-04-20T21:30:41.831 DEBUG:teuthology.orchestra.run.smithi122:mds.f> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.f.service 2024-04-20T21:30:41.874 DEBUG:teuthology.orchestra.run.smithi122:mds.i> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.i.service 2024-04-20T21:30:41.917 DEBUG:teuthology.orchestra.run.smithi122:mds.l> sudo journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.l.service 2024-04-20T21:30:41.959 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-20T21:30:41.960 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:30:42.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:41 smithi033 ceph-mon[21626]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:30:42.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:41 smithi033 ceph-mon[21626]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-04-20T21:30:42.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:42.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:41 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.190+0000 7fe2bec00640 1 -- 172.21.15.22:0/577898533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2b8107ac0 msgr2=0x7fe2b8109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.190+0000 7fe2bec00640 1 --2- 172.21.15.22:0/577898533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2b8107ac0 0x7fe2b8109f10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009860 tx=0x7fe2b002bb70 comp rx=0 tx=0).stop 2024-04-20T21:30:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.190+0000 7fe2bec00640 1 -- 172.21.15.22:0/577898533 shutdown_connections 2024-04-20T21:30:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.190+0000 7fe2bec00640 1 --2- 172.21.15.22:0/577898533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2b8107ac0 0x7fe2b8109f10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.190+0000 7fe2bec00640 1 --2- 172.21.15.22:0/577898533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2b8105190 0x7fe2b8107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.190+0000 7fe2bec00640 1 --2- 172.21.15.22:0/577898533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2b8102860 0x7fe2b8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.190+0000 7fe2bec00640 1 -- 172.21.15.22:0/577898533 >> 172.21.15.22:0/577898533 conn(0x7fe2b80fc230 msgr2=0x7fe2b80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.190+0000 7fe2bec00640 1 -- 172.21.15.22:0/577898533 shutdown_connections 2024-04-20T21:30:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.190+0000 7fe2bec00640 1 -- 172.21.15.22:0/577898533 wait complete. 2024-04-20T21:30:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.191+0000 7fe2bec00640 1 Processor -- start 2024-04-20T21:30:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.191+0000 7fe2bec00640 1 -- start start 2024-04-20T21:30:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.192+0000 7fe2bec00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2b8102860 0x7fe2b819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.192+0000 7fe2bec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2b8105190 0x7fe2b819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.192+0000 7fe2bec00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2b8107ac0 0x7fe2b81a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.192+0000 7fe2bec00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b810da60 con 0x7fe2b8107ac0 2024-04-20T21:30:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.192+0000 7fe2bec00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b810d8e0 con 0x7fe2b8102860 2024-04-20T21:30:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.192+0000 7fe2bec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b810dbe0 con 0x7fe2b8105190 2024-04-20T21:30:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.192+0000 7fe2be200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2b8107ac0 0x7fe2b81a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.192+0000 7fe2bd800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2b8102860 0x7fe2b819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.192+0000 7fe2bce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2b8105190 0x7fe2b819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.192+0000 7fe2bd800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2b8102860 0x7fe2b819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38518/0 (socket says 172.21.15.22:38518) 2024-04-20T21:30:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.192+0000 7fe2be200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2b8107ac0 0x7fe2b81a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51550/0 (socket says 172.21.15.22:51550) 2024-04-20T21:30:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.192+0000 7fe2bd800640 1 -- 172.21.15.22:0/1295540240 learned_addr learned my addr 172.21.15.22:0/1295540240 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:30:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.193+0000 7fe2be200640 1 -- 172.21.15.22:0/1295540240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2b8105190 msgr2=0x7fe2b819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.193+0000 7fe2be200640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2b8105190 0x7fe2b819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.193+0000 7fe2be200640 1 -- 172.21.15.22:0/1295540240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2b8102860 msgr2=0x7fe2b819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.193+0000 7fe2be200640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2b8102860 0x7fe2b819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.193+0000 7fe2be200640 1 -- 172.21.15.22:0/1295540240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b81a1ed0 con 0x7fe2b8107ac0 2024-04-20T21:30:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.193+0000 7fe2bce00640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2b8105190 0x7fe2b819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.193+0000 7fe2bd800640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2b8102860 0x7fe2b819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:42.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.193+0000 7fe2be200640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2b8107ac0 0x7fe2b81a1860 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009780 tx=0x7fe2b0032460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.195+0000 7fe2b6000640 1 -- 172.21.15.22:0/1295540240 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0041020 con 0x7fe2b8107ac0 2024-04-20T21:30:42.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.195+0000 7fe2b6000640 1 -- 172.21.15.22:0/1295540240 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b002c9f0 con 0x7fe2b8107ac0 2024-04-20T21:30:42.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.195+0000 7fe2bec00640 1 -- 172.21.15.22:0/1295540240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b81a2160 con 0x7fe2b8107ac0 2024-04-20T21:30:42.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.195+0000 7fe2b6000640 1 -- 172.21.15.22:0/1295540240 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0039040 con 0x7fe2b8107ac0 2024-04-20T21:30:42.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.196+0000 7fe2bec00640 1 -- 172.21.15.22:0/1295540240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b81a2670 con 0x7fe2b8107ac0 2024-04-20T21:30:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.197+0000 7fe2bec00640 1 -- 172.21.15.22:0/1295540240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b81095b0 con 0x7fe2b8107ac0 2024-04-20T21:30:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.199+0000 7fe2b6000640 1 -- 172.21.15.22:0/1295540240 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe2b0044020 con 0x7fe2b8107ac0 2024-04-20T21:30:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.200+0000 7fe2b6000640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe2a00a3c30 0x7fe2a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.200+0000 7fe2b6000640 1 -- 172.21.15.22:0/1295540240 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fe2b00f1860 con 0x7fe2b8107ac0 2024-04-20T21:30:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.200+0000 7fe2bd800640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe2a00a3c30 0x7fe2a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.201+0000 7fe2bd800640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe2a00a3c30 0x7fe2a00a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00e860 tx=0x7fe2ac010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.205+0000 7fe2b6000640 1 -- 172.21.15.22:0/1295540240 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe2b004b020 con 0x7fe2b8107ac0 2024-04-20T21:30:42.244 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:41 smithi122 ceph-mon[22147]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:30:42.244 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:41 smithi122 ceph-mon[22147]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-04-20T21:30:42.244 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:42.244 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:41 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:42.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:41 smithi022 ceph-mon[16843]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:30:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:41 smithi022 ceph-mon[16843]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-04-20T21:30:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:41 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:42.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.414+0000 7fe2bec00640 1 -- 172.21.15.22:0/1295540240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2b8062b60 con 0x7fe2b8107ac0 2024-04-20T21:30:42.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.416+0000 7fe2b6000640 1 -- 172.21.15.22:0/1295540240 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18433 (secure 0 0 0) 0x7fe2b00be640 con 0x7fe2b8107ac0 2024-04-20T21:30:42.415 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:30:42.416 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":84,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:30:30.588294+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.418+0000 7fe2bec00640 1 -- 172.21.15.22:0/1295540240 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe2a00a3c30 msgr2=0x7fe2a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.418+0000 7fe2bec00640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe2a00a3c30 0x7fe2a00a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00e860 tx=0x7fe2ac010040 comp rx=0 tx=0).stop 2024-04-20T21:30:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.418+0000 7fe2bec00640 1 -- 172.21.15.22:0/1295540240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2b8107ac0 msgr2=0x7fe2b81a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.418+0000 7fe2bec00640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2b8107ac0 0x7fe2b81a1860 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009780 tx=0x7fe2b0032460 comp rx=0 tx=0).stop 2024-04-20T21:30:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.418+0000 7fe2bec00640 1 -- 172.21.15.22:0/1295540240 shutdown_connections 2024-04-20T21:30:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.418+0000 7fe2bec00640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe2a00a3c30 0x7fe2a00a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.418+0000 7fe2bec00640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2b8107ac0 0x7fe2b81a1860 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.418+0000 7fe2bec00640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2b8105190 0x7fe2b819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.418+0000 7fe2bec00640 1 --2- 172.21.15.22:0/1295540240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2b8102860 0x7fe2b819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.418+0000 7fe2bec00640 1 -- 172.21.15.22:0/1295540240 >> 172.21.15.22:0/1295540240 conn(0x7fe2b80fc230 msgr2=0x7fe2b8101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.419+0000 7fe2bec00640 1 -- 172.21.15.22:0/1295540240 shutdown_connections 2024-04-20T21:30:42.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.419+0000 7fe2bec00640 1 -- 172.21.15.22:0/1295540240 wait complete. 2024-04-20T21:30:42.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:26:15.688818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:30:42.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:30:42.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.655+0000 7f3563e00640 1 -- 172.21.15.22:0/2622172723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35641005f0 msgr2=0x7f3564100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.655+0000 7f3563e00640 1 --2- 172.21.15.22:0/2622172723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35641005f0 0x7f3564100a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3554009900 tx=0x7f355402bc40 comp rx=0 tx=0).stop 2024-04-20T21:30:42.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.655+0000 7f3563e00640 1 -- 172.21.15.22:0/2622172723 shutdown_connections 2024-04-20T21:30:42.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.655+0000 7f3563e00640 1 --2- 172.21.15.22:0/2622172723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35641018f0 0x7f3564101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.656+0000 7f3563e00640 1 --2- 172.21.15.22:0/2622172723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3564100f50 0x7f35641013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.656+0000 7f3563e00640 1 --2- 172.21.15.22:0/2622172723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35641005f0 0x7f3564100a10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.656+0000 7f3563e00640 1 -- 172.21.15.22:0/2622172723 >> 172.21.15.22:0/2622172723 conn(0x7f35640fc230 msgr2=0x7f35640fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.656+0000 7f3563e00640 1 -- 172.21.15.22:0/2622172723 shutdown_connections 2024-04-20T21:30:42.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.656+0000 7f3563e00640 1 -- 172.21.15.22:0/2622172723 wait complete. 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.657+0000 7f3563e00640 1 Processor -- start 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.657+0000 7f3563e00640 1 -- start start 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.657+0000 7f3563e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35641005f0 0x7f356419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.657+0000 7f3563e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3564100f50 0x7f356419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.657+0000 7f3563e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35641018f0 0x7f35641a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.657+0000 7f3563e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f356410fc40 con 0x7f35641005f0 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.657+0000 7f3563e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f356410fac0 con 0x7f3564100f50 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.657+0000 7f3563e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f356410fdc0 con 0x7f35641018f0 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.658+0000 7f3562a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35641005f0 0x7f356419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.658+0000 7f3562a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35641005f0 0x7f356419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46288/0 (socket says 172.21.15.22:46288) 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.658+0000 7f3562a00640 1 -- 172.21.15.22:0/3642045927 learned_addr learned my addr 172.21.15.22:0/3642045927 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.658+0000 7f3563400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35641018f0 0x7f35641a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.658+0000 7f3562000640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3564100f50 0x7f356419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.658+0000 7f3563400640 1 -- 172.21.15.22:0/3642045927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3564100f50 msgr2=0x7f356419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.658+0000 7f3563400640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3564100f50 0x7f356419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.658+0000 7f3563400640 1 -- 172.21.15.22:0/3642045927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35641005f0 msgr2=0x7f356419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.658+0000 7f3563400640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35641005f0 0x7f356419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.658+0000 7f3563400640 1 -- 172.21.15.22:0/3642045927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35641a1e70 con 0x7f35641018f0 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.658+0000 7f3562a00640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35641005f0 0x7f356419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.658+0000 7f3563400640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35641018f0 0x7f35641a1800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f355c009fd0 tx=0x7f355c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.659+0000 7f355b400640 1 -- 172.21.15.22:0/3642045927 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c00ce20 con 0x7f35641018f0 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.659+0000 7f355b400640 1 -- 172.21.15.22:0/3642045927 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f355c005020 con 0x7f35641018f0 2024-04-20T21:30:42.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.659+0000 7f3563e00640 1 -- 172.21.15.22:0/3642045927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35641a2160 con 0x7f35641018f0 2024-04-20T21:30:42.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.659+0000 7f355b400640 1 -- 172.21.15.22:0/3642045927 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c010720 con 0x7f35641018f0 2024-04-20T21:30:42.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.659+0000 7f3563e00640 1 -- 172.21.15.22:0/3642045927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35641a2660 con 0x7f35641018f0 2024-04-20T21:30:42.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.660+0000 7f3563e00640 1 -- 172.21.15.22:0/3642045927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f356410b340 con 0x7f35641018f0 2024-04-20T21:30:42.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.662+0000 7f355b400640 1 -- 172.21.15.22:0/3642045927 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f355c005540 con 0x7f35641018f0 2024-04-20T21:30:42.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.663+0000 7f355b400640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f35480a3bb0 0x7f35480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.663+0000 7f355b400640 1 -- 172.21.15.22:0/3642045927 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f355c0d06b0 con 0x7f35641018f0 2024-04-20T21:30:42.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.663+0000 7f3562a00640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f35480a3bb0 0x7f35480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.664+0000 7f3562a00640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f35480a3bb0 0x7f35480a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3554006010 tx=0x7f355402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.668+0000 7f355b400640 1 -- 172.21.15.22:0/3642045927 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f355c09d490 con 0x7f35641018f0 2024-04-20T21:30:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.876+0000 7f3563e00640 1 -- 172.21.15.22:0/3642045927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3564062b60 con 0x7f35641018f0 2024-04-20T21:30:42.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.877+0000 7f355b400640 1 -- 172.21.15.22:0/3642045927 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f355c09cd90 con 0x7f35641018f0 2024-04-20T21:30:42.876 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.879+0000 7f3563e00640 1 -- 172.21.15.22:0/3642045927 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f35480a3bb0 msgr2=0x7f35480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.879+0000 7f3563e00640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f35480a3bb0 0x7f35480a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3554006010 tx=0x7f355402cf90 comp rx=0 tx=0).stop 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.879+0000 7f3563e00640 1 -- 172.21.15.22:0/3642045927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35641018f0 msgr2=0x7f35641a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.879+0000 7f3563e00640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35641018f0 0x7f35641a1800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f355c009fd0 tx=0x7f355c00ef10 comp rx=0 tx=0).stop 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.879+0000 7f3563e00640 1 -- 172.21.15.22:0/3642045927 shutdown_connections 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.879+0000 7f3563e00640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f35480a3bb0 0x7f35480a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.879+0000 7f3563e00640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35641018f0 0x7f35641a1800 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.879+0000 7f3563e00640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3564100f50 0x7f356419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.879+0000 7f3563e00640 1 --2- 172.21.15.22:0/3642045927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35641005f0 0x7f356419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.879+0000 7f3563e00640 1 -- 172.21.15.22:0/3642045927 >> 172.21.15.22:0/3642045927 conn(0x7f35640fc230 msgr2=0x7f35640fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.880+0000 7f3563e00640 1 -- 172.21.15.22:0/3642045927 shutdown_connections 2024-04-20T21:30:42.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:42.880+0000 7f3563e00640 1 -- 172.21.15.22:0/3642045927 wait complete. 2024-04-20T21:30:42.900 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.114+0000 7fdb69000640 1 -- 172.21.15.22:0/214033440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb641053b0 msgr2=0x7fdb64105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.114+0000 7fdb69000640 1 --2- 172.21.15.22:0/214033440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb641053b0 0x7fdb64105830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009900 tx=0x7fdb4c02bc40 comp rx=0 tx=0).stop 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.115+0000 7fdb69000640 1 -- 172.21.15.22:0/214033440 shutdown_connections 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.115+0000 7fdb69000640 1 --2- 172.21.15.22:0/214033440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb64105d70 0x7fdb6410c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.115+0000 7fdb69000640 1 --2- 172.21.15.22:0/214033440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb641053b0 0x7fdb64105830 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.115+0000 7fdb69000640 1 --2- 172.21.15.22:0/214033440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb641041c0 0x7fdb641045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.115+0000 7fdb69000640 1 -- 172.21.15.22:0/214033440 >> 172.21.15.22:0/214033440 conn(0x7fdb640ff750 msgr2=0x7fdb64101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.115+0000 7fdb69000640 1 -- 172.21.15.22:0/214033440 shutdown_connections 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.115+0000 7fdb69000640 1 -- 172.21.15.22:0/214033440 wait complete. 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.116+0000 7fdb69000640 1 Processor -- start 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.116+0000 7fdb69000640 1 -- start start 2024-04-20T21:30:43.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb69000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb641041c0 0x7fdb6419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb69000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb641053b0 0x7fdb6419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb69000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb64105d70 0x7fdb641a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb69000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6410fea0 con 0x7fdb64105d70 2024-04-20T21:30:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb69000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6410fd20 con 0x7fdb641053b0 2024-04-20T21:30:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb69000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdb64110020 con 0x7fdb641041c0 2024-04-20T21:30:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb62800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb64105d70 0x7fdb641a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb61e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb641041c0 0x7fdb6419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb62800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb64105d70 0x7fdb641a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46298/0 (socket says 172.21.15.22:46298) 2024-04-20T21:30:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb62800640 1 -- 172.21.15.22:0/1221579174 learned_addr learned my addr 172.21.15.22:0/1221579174 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:30:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb61400640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb641053b0 0x7fdb6419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb61e00640 1 -- 172.21.15.22:0/1221579174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb641053b0 msgr2=0x7fdb6419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb61e00640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb641053b0 0x7fdb6419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb61e00640 1 -- 172.21.15.22:0/1221579174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb64105d70 msgr2=0x7fdb641a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb61e00640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb64105d70 0x7fdb641a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb61e00640 1 -- 172.21.15.22:0/1221579174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb641a20c0 con 0x7fdb641041c0 2024-04-20T21:30:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.117+0000 7fdb62800640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb64105d70 0x7fdb641a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.118+0000 7fdb61e00640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb641041c0 0x7fdb6419a4a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb64105210 tx=0x7fdb5400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.118+0000 7fdb5aa00640 1 -- 172.21.15.22:0/1221579174 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54017070 con 0x7fdb641041c0 2024-04-20T21:30:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.118+0000 7fdb5aa00640 1 -- 172.21.15.22:0/1221579174 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb54009c30 con 0x7fdb641041c0 2024-04-20T21:30:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.118+0000 7fdb5aa00640 1 -- 172.21.15.22:0/1221579174 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5400f620 con 0x7fdb641041c0 2024-04-20T21:30:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.118+0000 7fdb69000640 1 -- 172.21.15.22:0/1221579174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb641a23b0 con 0x7fdb641041c0 2024-04-20T21:30:43.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.118+0000 7fdb69000640 1 -- 172.21.15.22:0/1221579174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb641a2920 con 0x7fdb641041c0 2024-04-20T21:30:43.120 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:42 smithi122 ceph-mon[22147]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:43.120 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:43.120 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:43.120 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:43.120 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:43.120 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:43.120 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:42 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:43.120 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:42 smithi122 ceph-mon[22147]: Deploying daemon mds.l on smithi122 2024-04-20T21:30:43.120 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:42 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1295540240' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:43.121 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:42 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3642045927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:43.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.119+0000 7fdb69000640 1 -- 172.21.15.22:0/1221579174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb6410b480 con 0x7fdb641041c0 2024-04-20T21:30:43.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.121+0000 7fdb5aa00640 1 -- 172.21.15.22:0/1221579174 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdb5401a070 con 0x7fdb641041c0 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.123+0000 7fdb5aa00640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdb480a3c30 0x7fdb480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.123+0000 7fdb5aa00640 1 -- 172.21.15.22:0/1221579174 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fdb540cf930 con 0x7fdb641041c0 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.123+0000 7fdb61400640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdb480a3c30 0x7fdb480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.124+0000 7fdb61400640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdb480a3c30 0x7fdb480a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c02c250 tx=0x7fdb4c02cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.127+0000 7fdb5aa00640 1 -- 172.21.15.22:0/1221579174 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdb54009de0 con 0x7fdb641041c0 2024-04-20T21:30:43.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:42 smithi022 ceph-mon[16843]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:43.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:43.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:43.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:43.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:43.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:43.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:42 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:43.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:42 smithi022 ceph-mon[16843]: Deploying daemon mds.l on smithi122 2024-04-20T21:30:43.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:42 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1295540240' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:43.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:42 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3642045927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:43.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:42 smithi033 ceph-mon[21626]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:43.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:43.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:43.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:43.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:43.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:43.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:42 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:43.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:42 smithi033 ceph-mon[21626]: Deploying daemon mds.l on smithi122 2024-04-20T21:30:43.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:42 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1295540240' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:43.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:42 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3642045927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:43.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.377+0000 7fdb69000640 1 -- 172.21.15.22:0/1221579174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb6402d5d0 con 0x7fdb641041c0 2024-04-20T21:30:43.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.378+0000 7fdb5aa00640 1 -- 172.21.15.22:0/1221579174 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fdb5409c0c0 con 0x7fdb641041c0 2024-04-20T21:30:43.377 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:30:43.377 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-20T21:30:43.378 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 1 2024-04-20T21:30:43.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.380+0000 7fdb69000640 1 -- 172.21.15.22:0/1221579174 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdb480a3c30 msgr2=0x7fdb480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.380+0000 7fdb69000640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdb480a3c30 0x7fdb480a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c02c250 tx=0x7fdb4c02cf90 comp rx=0 tx=0).stop 2024-04-20T21:30:43.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.380+0000 7fdb69000640 1 -- 172.21.15.22:0/1221579174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb641041c0 msgr2=0x7fdb6419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.380+0000 7fdb69000640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb641041c0 0x7fdb6419a4a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb64105210 tx=0x7fdb5400dde0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.381+0000 7fdb69000640 1 -- 172.21.15.22:0/1221579174 shutdown_connections 2024-04-20T21:30:43.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.381+0000 7fdb69000640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdb480a3c30 0x7fdb480a60f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.381+0000 7fdb69000640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb64105d70 0x7fdb641a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.381+0000 7fdb69000640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb641053b0 0x7fdb6419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.381+0000 7fdb69000640 1 --2- 172.21.15.22:0/1221579174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb641041c0 0x7fdb6419a4a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.381+0000 7fdb69000640 1 -- 172.21.15.22:0/1221579174 >> 172.21.15.22:0/1221579174 conn(0x7fdb640ff750 msgr2=0x7fdb64108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:43.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.381+0000 7fdb69000640 1 -- 172.21.15.22:0/1221579174 shutdown_connections 2024-04-20T21:30:43.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:43.381+0000 7fdb69000640 1 -- 172.21.15.22:0/1221579174 wait complete. 2024-04-20T21:30:44.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:43 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1221579174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:44.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:43 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1221579174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:44.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:43 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1221579174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:44.438 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:30:44 smithi122 systemd[1]: Starting Ceph mds.l for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:44.726 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:30:44 smithi122 podman[51892]: 2024-04-20 21:30:44.343269902 +0000 UTC m=+0.018106823 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:44.726 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:30:44 smithi122 podman[51892]: 2024-04-20 21:30:44.450018019 +0000 UTC m=+0.124854984 container create f62252f8aaf22fb2d3ae17ac5adb225afa849f3c0649d44be21cdbe3b0ebfce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-20T21:30:44.726 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:30:44 smithi122 podman[51892]: 2024-04-20 21:30:44.727577662 +0000 UTC m=+0.402414582 container init f62252f8aaf22fb2d3ae17ac5adb225afa849f3c0649d44be21cdbe3b0ebfce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-04-20T21:30:45.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:44 smithi122 ceph-mon[22147]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:45.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:44 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.122:6832/1883458757,v1:172.21.15.122:6833/1883458757] up:boot 2024-04-20T21:30:45.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:44 smithi122 ceph-mon[22147]: fsmap 1 up:standby 2024-04-20T21:30:45.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:30:45.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:30:45.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:45.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:45.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:45.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:45.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:44 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:45.026 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:30:44 smithi122 podman[51892]: 2024-04-20 21:30:44.730156689 +0000 UTC m=+0.404993612 container start f62252f8aaf22fb2d3ae17ac5adb225afa849f3c0649d44be21cdbe3b0ebfce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-20T21:30:45.026 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:30:44 smithi122 ceph-mds[51911]: -- 172.21.15.122:0/12160528 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a35e0ada40 con 0x55a35e081000 2024-04-20T21:30:45.026 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:30:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-l[51907]: starting mds.l at 2024-04-20T21:30:45.026 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:30:44 smithi122 bash[51892]: f62252f8aaf22fb2d3ae17ac5adb225afa849f3c0649d44be21cdbe3b0ebfce6 2024-04-20T21:30:45.026 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:30:44 smithi122 systemd[1]: Started Ceph mds.l for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:45.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:44 smithi033 ceph-mon[21626]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:45.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:44 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.122:6832/1883458757,v1:172.21.15.122:6833/1883458757] up:boot 2024-04-20T21:30:45.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:44 smithi033 ceph-mon[21626]: fsmap 1 up:standby 2024-04-20T21:30:45.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:30:45.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:30:45.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:45.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:45.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:45.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:45.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:44 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:45.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:45.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:44 smithi022 ceph-mon[16843]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:45.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:44 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.122:6832/1883458757,v1:172.21.15.122:6833/1883458757] up:boot 2024-04-20T21:30:45.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:44 smithi022 ceph-mon[16843]: fsmap 1 up:standby 2024-04-20T21:30:45.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:30:45.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:30:45.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:45.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:45.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:45.411 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:45.411 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:44 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:45.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.622+0000 7f1962a00640 1 -- 172.21.15.22:0/3792889929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f195c077b60 msgr2=0x7f195c077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:45.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.622+0000 7f1962a00640 1 --2- 172.21.15.22:0/3792889929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f195c077b60 0x7f195c077fc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1948009900 tx=0x7f194802bc40 comp rx=0 tx=0).stop 2024-04-20T21:30:45.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.622+0000 7f1962a00640 1 -- 172.21.15.22:0/3792889929 shutdown_connections 2024-04-20T21:30:45.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.622+0000 7f1962a00640 1 --2- 172.21.15.22:0/3792889929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f195c078500 0x7f195c078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.622+0000 7f1962a00640 1 --2- 172.21.15.22:0/3792889929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f195c077b60 0x7f195c077fc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.622+0000 7f1962a00640 1 --2- 172.21.15.22:0/3792889929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f195c076920 0x7f195c076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.622+0000 7f1962a00640 1 -- 172.21.15.22:0/3792889929 >> 172.21.15.22:0/3792889929 conn(0x7f195c0ff570 msgr2=0x7f195c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:45.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.622+0000 7f1962a00640 1 -- 172.21.15.22:0/3792889929 shutdown_connections 2024-04-20T21:30:45.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.622+0000 7f1962a00640 1 -- 172.21.15.22:0/3792889929 wait complete. 2024-04-20T21:30:45.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.623+0000 7f1962a00640 1 Processor -- start 2024-04-20T21:30:45.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.623+0000 7f1962a00640 1 -- start start 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1962a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f195c076920 0x7f195c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1962a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f195c077b60 0x7f195c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1962a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f195c078500 0x7f195c1a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1962a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f195c114060 con 0x7f195c078500 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1962a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f195c113ee0 con 0x7f195c076920 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1962a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1141e0 con 0x7f195c077b60 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1961600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f195c076920 0x7f195c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1960c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f195c077b60 0x7f195c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1962000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f195c078500 0x7f195c1a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1960c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f195c077b60 0x7f195c19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:41344/0 (socket says 172.21.15.22:41344) 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1961600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f195c076920 0x7f195c19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60120/0 (socket says 172.21.15.22:60120) 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1960c00640 1 -- 172.21.15.22:0/636274667 learned_addr learned my addr 172.21.15.22:0/636274667 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1961600640 1 -- 172.21.15.22:0/636274667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f195c077b60 msgr2=0x7f195c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:45.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1961600640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f195c077b60 0x7f195c19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1961600640 1 -- 172.21.15.22:0/636274667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f195c078500 msgr2=0x7f195c1a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:45.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1961600640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f195c078500 0x7f195c1a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1961600640 1 -- 172.21.15.22:0/636274667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c1a6340 con 0x7f195c076920 2024-04-20T21:30:45.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.624+0000 7f1962000640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f195c078500 0x7f195c1a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:45.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.625+0000 7f1961600640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f195c076920 0x7f195c19e720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f194c00e9f0 tx=0x7f194c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:45.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.625+0000 7f195a000640 1 -- 172.21.15.22:0/636274667 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c00ccc0 con 0x7f195c076920 2024-04-20T21:30:45.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.625+0000 7f195a000640 1 -- 172.21.15.22:0/636274667 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f194c005020 con 0x7f195c076920 2024-04-20T21:30:45.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.625+0000 7f195a000640 1 -- 172.21.15.22:0/636274667 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c004a00 con 0x7f195c076920 2024-04-20T21:30:45.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.625+0000 7f1962a00640 1 -- 172.21.15.22:0/636274667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195c1a6630 con 0x7f195c076920 2024-04-20T21:30:45.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.625+0000 7f1962a00640 1 -- 172.21.15.22:0/636274667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195c1a6b50 con 0x7f195c076920 2024-04-20T21:30:45.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.626+0000 7f1962a00640 1 -- 172.21.15.22:0/636274667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f195c110b90 con 0x7f195c076920 2024-04-20T21:30:45.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.628+0000 7f195a000640 1 -- 172.21.15.22:0/636274667 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f194c0051d0 con 0x7f195c076920 2024-04-20T21:30:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.629+0000 7f195a000640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f19400a3c30 0x7f19400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.630+0000 7f195a000640 1 -- 172.21.15.22:0/636274667 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f194c0cfec0 con 0x7f195c076920 2024-04-20T21:30:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.630+0000 7f1960c00640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f19400a3c30 0x7f19400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.631+0000 7f1960c00640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f19400a3c30 0x7f19400a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1948000c00 tx=0x7f194802c890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:45.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.634+0000 7f195a000640 1 -- 172.21.15.22:0/636274667 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f194c09cca0 con 0x7f195c076920 2024-04-20T21:30:45.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.881+0000 7f1962a00640 1 -- 172.21.15.22:0/636274667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f195c02d5d0 con 0x7f195c076920 2024-04-20T21:30:45.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.882+0000 7f195a000640 1 -- 172.21.15.22:0/636274667 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f194c09c5a0 con 0x7f195c076920 2024-04-20T21:30:45.881 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:30:45.881 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T21:30:45.881 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 2 2024-04-20T21:30:45.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.884+0000 7f1962a00640 1 -- 172.21.15.22:0/636274667 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f19400a3c30 msgr2=0x7f19400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:45.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.884+0000 7f1962a00640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f19400a3c30 0x7f19400a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1948000c00 tx=0x7f194802c890 comp rx=0 tx=0).stop 2024-04-20T21:30:45.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.884+0000 7f1962a00640 1 -- 172.21.15.22:0/636274667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f195c076920 msgr2=0x7f195c19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:45.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.884+0000 7f1962a00640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f195c076920 0x7f195c19e720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f194c00e9f0 tx=0x7f194c00edf0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.884+0000 7f1962a00640 1 -- 172.21.15.22:0/636274667 shutdown_connections 2024-04-20T21:30:45.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.884+0000 7f1962a00640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f19400a3c30 0x7f19400a60f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.884+0000 7f1962a00640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f195c078500 0x7f195c1a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.884+0000 7f1962a00640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f195c077b60 0x7f195c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.884+0000 7f1962a00640 1 --2- 172.21.15.22:0/636274667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f195c076920 0x7f195c19e720 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.884+0000 7f1962a00640 1 -- 172.21.15.22:0/636274667 >> 172.21.15.22:0/636274667 conn(0x7f195c0ff570 msgr2=0x7f195c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:45.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.884+0000 7f1962a00640 1 -- 172.21.15.22:0/636274667 shutdown_connections 2024-04-20T21:30:45.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:45.885+0000 7f1962a00640 1 -- 172.21.15.22:0/636274667 wait complete. 2024-04-20T21:30:46.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:45 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:46.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:45 smithi022 ceph-mon[16843]: Deploying daemon mds.k on smithi033 2024-04-20T21:30:46.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:45 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/636274667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:46.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:45 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:46.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:45 smithi122 ceph-mon[22147]: Deploying daemon mds.k on smithi033 2024-04-20T21:30:46.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:45 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/636274667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:46.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:45 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:46.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:45 smithi033 ceph-mon[21626]: Deploying daemon mds.k on smithi033 2024-04-20T21:30:46.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:45 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/636274667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:47.224 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:46 smithi033 ceph-mon[21626]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:47.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:46 smithi122 ceph-mon[22147]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:47.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:46 smithi022 ceph-mon[16843]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:47.553 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:30:47 smithi033 podman[50991]: 2024-04-20 21:30:47.454557261 +0000 UTC m=+0.124859708 container create 7f796241d1be80257b800de7d7dfdbe5811f16c5a66d93f271fd27f243c50251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-20T21:30:47.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:48.054 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:30:47 smithi033 podman[50991]: 2024-04-20 21:30:47.674012538 +0000 UTC m=+0.344314986 container init 7f796241d1be80257b800de7d7dfdbe5811f16c5a66d93f271fd27f243c50251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-20T21:30:48.054 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:30:47 smithi033 podman[50991]: 2024-04-20 21:30:47.676620595 +0000 UTC m=+0.346923048 container start 7f796241d1be80257b800de7d7dfdbe5811f16c5a66d93f271fd27f243c50251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-20T21:30:48.054 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:30:47 smithi033 ceph-mds[51010]: -- 172.21.15.33:0/644235743 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5563e65dba40 con 0x5563e65ae800 2024-04-20T21:30:48.054 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:30:47 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-k[51006]: starting mds.k at 2024-04-20T21:30:48.054 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:30:47 smithi033 bash[50991]: 7f796241d1be80257b800de7d7dfdbe5811f16c5a66d93f271fd27f243c50251 2024-04-20T21:30:48.054 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:30:47 smithi033 systemd[1]: Started Ceph mds.k for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.110+0000 7fe863200640 1 -- 172.21.15.22:0/1217084219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c1041e0 msgr2=0x7fe85c104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.110+0000 7fe863200640 1 --2- 172.21.15.22:0/1217084219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c1041e0 0x7fe85c104600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009900 tx=0x7fe84c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.110+0000 7fe863200640 1 -- 172.21.15.22:0/1217084219 shutdown_connections 2024-04-20T21:30:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.111+0000 7fe863200640 1 --2- 172.21.15.22:0/1217084219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85c105d90 0x7fe85c10c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.111+0000 7fe863200640 1 --2- 172.21.15.22:0/1217084219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe85c1053d0 0x7fe85c105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.111+0000 7fe863200640 1 --2- 172.21.15.22:0/1217084219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c1041e0 0x7fe85c104600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.111+0000 7fe863200640 1 -- 172.21.15.22:0/1217084219 >> 172.21.15.22:0/1217084219 conn(0x7fe85c0ff750 msgr2=0x7fe85c101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:48.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.111+0000 7fe863200640 1 -- 172.21.15.22:0/1217084219 shutdown_connections 2024-04-20T21:30:48.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.111+0000 7fe863200640 1 -- 172.21.15.22:0/1217084219 wait complete. 2024-04-20T21:30:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.112+0000 7fe863200640 1 Processor -- start 2024-04-20T21:30:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.112+0000 7fe863200640 1 -- start start 2024-04-20T21:30:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.112+0000 7fe863200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85c1041e0 0x7fe85c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.112+0000 7fe863200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe85c1053d0 0x7fe85c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.112+0000 7fe863200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c105d90 0x7fe85c1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.112+0000 7fe863200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c10fe40 con 0x7fe85c105d90 2024-04-20T21:30:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.112+0000 7fe863200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c10fcc0 con 0x7fe85c1041e0 2024-04-20T21:30:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.112+0000 7fe863200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c10ffc0 con 0x7fe85c1053d0 2024-04-20T21:30:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.112+0000 7fe861200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c105d90 0x7fe85c1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.113+0000 7fe853e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe85c1053d0 0x7fe85c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.113+0000 7fe85be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85c1041e0 0x7fe85c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.113+0000 7fe853e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe85c1053d0 0x7fe85c19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:41372/0 (socket says 172.21.15.22:41372) 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.113+0000 7fe853e00640 1 -- 172.21.15.22:0/1841490581 learned_addr learned my addr 172.21.15.22:0/1841490581 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.113+0000 7fe853e00640 1 -- 172.21.15.22:0/1841490581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85c1041e0 msgr2=0x7fe85c19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.113+0000 7fe853e00640 1 --2- 172.21.15.22:0/1841490581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85c1041e0 0x7fe85c19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.113+0000 7fe853e00640 1 -- 172.21.15.22:0/1841490581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c105d90 msgr2=0x7fe85c1a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.113+0000 7fe853e00640 1 --2- 172.21.15.22:0/1841490581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c105d90 0x7fe85c1a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.113+0000 7fe853e00640 1 -- 172.21.15.22:0/1841490581 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85c1a2140 con 0x7fe85c1053d0 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.113+0000 7fe853e00640 1 --2- 172.21.15.22:0/1841490581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe85c1053d0 0x7fe85c19aa60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe84400e9f0 tx=0x7fe84400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.113+0000 7fe859600640 1 -- 172.21.15.22:0/1841490581 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84400ccc0 con 0x7fe85c1053d0 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.114+0000 7fe859600640 1 -- 172.21.15.22:0/1841490581 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe844010910 con 0x7fe85c1053d0 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.114+0000 7fe863200640 1 -- 172.21.15.22:0/1841490581 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe85c1a2430 con 0x7fe85c1053d0 2024-04-20T21:30:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.114+0000 7fe859600640 1 -- 172.21.15.22:0/1841490581 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8440188a0 con 0x7fe85c1053d0 2024-04-20T21:30:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.114+0000 7fe863200640 1 -- 172.21.15.22:0/1841490581 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe85c1a2980 con 0x7fe85c1053d0 2024-04-20T21:30:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.115+0000 7fe863200640 1 -- 172.21.15.22:0/1841490581 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe85c10b520 con 0x7fe85c1053d0 2024-04-20T21:30:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.117+0000 7fe859600640 1 -- 172.21.15.22:0/1841490581 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe844010ac0 con 0x7fe85c1053d0 2024-04-20T21:30:48.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.118+0000 7fe859600640 1 --2- 172.21.15.22:0/1841490581 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe8400a3bb0 0x7fe8400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.118+0000 7fe859600640 1 -- 172.21.15.22:0/1841490581 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fe8440d0430 con 0x7fe85c1053d0 2024-04-20T21:30:48.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.118+0000 7fe85be00640 1 --2- 172.21.15.22:0/1841490581 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe8400a3bb0 0x7fe8400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.119+0000 7fe85be00640 1 --2- 172.21.15.22:0/1841490581 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe8400a3bb0 0x7fe8400a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe84c000c00 tx=0x7fe84c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:48.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.123+0000 7fe859600640 1 -- 172.21.15.22:0/1841490581 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe84409d210 con 0x7fe85c1053d0 2024-04-20T21:30:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.375+0000 7fe863200640 1 -- 172.21.15.22:0/1841490581 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe85c062b60 con 0x7fe85c1053d0 2024-04-20T21:30:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.376+0000 7fe859600640 1 -- 172.21.15.22:0/1841490581 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fe84409cb10 con 0x7fe85c1053d0 2024-04-20T21:30:48.376 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:30:48.376 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T21:30:48.376 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 3 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.379+0000 7fe863200640 1 -- 172.21.15.22:0/1841490581 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe8400a3bb0 msgr2=0x7fe8400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.379+0000 7fe863200640 1 --2- 172.21.15.22:0/1841490581 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe8400a3bb0 0x7fe8400a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe84c000c00 tx=0x7fe84c005fd0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.379+0000 7fe863200640 1 -- 172.21.15.22:0/1841490581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe85c1053d0 msgr2=0x7fe85c19aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.379+0000 7fe863200640 1 --2- 172.21.15.22:0/1841490581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe85c1053d0 0x7fe85c19aa60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe84400e9f0 tx=0x7fe84400edf0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.379+0000 7fe863200640 1 -- 172.21.15.22:0/1841490581 shutdown_connections 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.379+0000 7fe863200640 1 --2- 172.21.15.22:0/1841490581 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe8400a3bb0 0x7fe8400a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.379+0000 7fe863200640 1 --2- 172.21.15.22:0/1841490581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c105d90 0x7fe85c1a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.379+0000 7fe863200640 1 --2- 172.21.15.22:0/1841490581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe85c1053d0 0x7fe85c19aa60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.379+0000 7fe863200640 1 --2- 172.21.15.22:0/1841490581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85c1041e0 0x7fe85c19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.379+0000 7fe863200640 1 -- 172.21.15.22:0/1841490581 >> 172.21.15.22:0/1841490581 conn(0x7fe85c0ff750 msgr2=0x7fe85c1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.380+0000 7fe863200640 1 -- 172.21.15.22:0/1841490581 shutdown_connections 2024-04-20T21:30:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:48.380+0000 7fe863200640 1 -- 172.21.15.22:0/1841490581 wait complete. 2024-04-20T21:30:49.069 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:48 smithi033 ceph-mon[21626]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:49.069 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:49.069 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:49.069 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:49.070 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:49.070 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:49.070 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:49.070 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:48 smithi033 ceph-mon[21626]: Deploying daemon mds.j on smithi022 2024-04-20T21:30:49.070 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:48 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.33:6832/721989242,v1:172.21.15.33:6833/721989242] up:boot 2024-04-20T21:30:49.070 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:48 smithi033 ceph-mon[21626]: fsmap 2 up:standby 2024-04-20T21:30:49.070 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:48 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T21:30:49.070 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:48 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1841490581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:49.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:48 smithi022 ceph-mon[16843]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:49.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:49.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:49.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:49.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:49.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:49.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:49.080 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:48 smithi022 ceph-mon[16843]: Deploying daemon mds.j on smithi022 2024-04-20T21:30:49.080 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:48 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.33:6832/721989242,v1:172.21.15.33:6833/721989242] up:boot 2024-04-20T21:30:49.080 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:48 smithi022 ceph-mon[16843]: fsmap 2 up:standby 2024-04-20T21:30:49.080 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:48 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T21:30:49.080 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:48 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1841490581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:49.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:48 smithi122 ceph-mon[22147]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:49.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:49.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:49.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:49.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:49.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:49.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:49.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:48 smithi122 ceph-mon[22147]: Deploying daemon mds.j on smithi022 2024-04-20T21:30:49.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:48 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.33:6832/721989242,v1:172.21.15.33:6833/721989242] up:boot 2024-04-20T21:30:49.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:48 smithi122 ceph-mon[22147]: fsmap 2 up:standby 2024-04-20T21:30:49.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:48 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T21:30:49.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:48 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1841490581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:50.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:50.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.693+0000 7f417cc00640 1 -- 172.21.15.22:0/2396071442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4178102a20 msgr2=0x7f4178104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.693+0000 7f417cc00640 1 --2- 172.21.15.22:0/2396071442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4178102a20 0x7f4178104e10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4168009900 tx=0x7f416802bc40 comp rx=0 tx=0).stop 2024-04-20T21:30:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.693+0000 7f417cc00640 1 -- 172.21.15.22:0/2396071442 shutdown_connections 2024-04-20T21:30:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.693+0000 7f417cc00640 1 --2- 172.21.15.22:0/2396071442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4178107c80 0x7f417810a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.693+0000 7f417cc00640 1 --2- 172.21.15.22:0/2396071442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4178105350 0x7f4178107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.693+0000 7f417cc00640 1 --2- 172.21.15.22:0/2396071442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4178102a20 0x7f4178104e10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.693+0000 7f417cc00640 1 -- 172.21.15.22:0/2396071442 >> 172.21.15.22:0/2396071442 conn(0x7f41780fc410 msgr2=0x7f41780fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.693+0000 7f417cc00640 1 -- 172.21.15.22:0/2396071442 shutdown_connections 2024-04-20T21:30:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.693+0000 7f417cc00640 1 -- 172.21.15.22:0/2396071442 wait complete. 2024-04-20T21:30:50.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.694+0000 7f417cc00640 1 Processor -- start 2024-04-20T21:30:50.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.695+0000 7f417cc00640 1 -- start start 2024-04-20T21:30:50.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.695+0000 7f417cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4178102a20 0x7f41781023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:50.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.695+0000 7f417cc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4178105350 0x7f41781009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:50.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.695+0000 7f417cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4178107c80 0x7f4178100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:50.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.695+0000 7f417cc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f417810dc80 con 0x7f4178102a20 2024-04-20T21:30:50.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.695+0000 7f417cc00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f417810db00 con 0x7f4178105350 2024-04-20T21:30:50.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.695+0000 7f417cc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f417810de00 con 0x7f4178107c80 2024-04-20T21:30:50.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.696+0000 7f4176800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4178107c80 0x7f4178100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:50.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.696+0000 7f4175400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4178105350 0x7f41781009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:50.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.696+0000 7f4176800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4178107c80 0x7f4178100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:41390/0 (socket says 172.21.15.22:41390) 2024-04-20T21:30:50.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.696+0000 7f4175400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4178105350 0x7f41781009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60160/0 (socket says 172.21.15.22:60160) 2024-04-20T21:30:50.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.696+0000 7f4176800640 1 -- 172.21.15.22:0/1386180545 learned_addr learned my addr 172.21.15.22:0/1386180545 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:30:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.696+0000 7f4176800640 1 -- 172.21.15.22:0/1386180545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4178105350 msgr2=0x7f41781009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.696+0000 7f4176800640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4178105350 0x7f41781009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.696+0000 7f4176800640 1 -- 172.21.15.22:0/1386180545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4178102a20 msgr2=0x7f41781023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.696+0000 7f4176800640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4178102a20 0x7f41781023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.696+0000 7f4176800640 1 -- 172.21.15.22:0/1386180545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4178101470 con 0x7f4178107c80 2024-04-20T21:30:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.697+0000 7f4175400640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4178105350 0x7f41781009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:50.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.697+0000 7f4176800640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4178107c80 0x7f4178100f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f417000eb10 tx=0x7f417000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:50.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.697+0000 7f416ea00640 1 -- 172.21.15.22:0/1386180545 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170004b20 con 0x7f4178107c80 2024-04-20T21:30:50.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.698+0000 7f416ea00640 1 -- 172.21.15.22:0/1386180545 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4170004cc0 con 0x7f4178107c80 2024-04-20T21:30:50.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.698+0000 7f416ea00640 1 -- 172.21.15.22:0/1386180545 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170010760 con 0x7f4178107c80 2024-04-20T21:30:50.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.699+0000 7f417cc00640 1 -- 172.21.15.22:0/1386180545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4178101760 con 0x7f4178107c80 2024-04-20T21:30:50.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.699+0000 7f417cc00640 1 -- 172.21.15.22:0/1386180545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41781a6cc0 con 0x7f4178107c80 2024-04-20T21:30:50.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.701+0000 7f417cc00640 1 -- 172.21.15.22:0/1386180545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4178068d80 con 0x7f4178107c80 2024-04-20T21:30:50.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.703+0000 7f416ea00640 1 -- 172.21.15.22:0/1386180545 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4170005000 con 0x7f4178107c80 2024-04-20T21:30:50.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.706+0000 7f416ea00640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f415c0a3c30 0x7f415c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:50.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.706+0000 7f416ea00640 1 -- 172.21.15.22:0/1386180545 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f41700d0590 con 0x7f4178107c80 2024-04-20T21:30:50.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.706+0000 7f4175e00640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f415c0a3c30 0x7f415c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:50.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.707+0000 7f4175e00640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f415c0a3c30 0x7f415c0a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f41680057c0 tx=0x7f416802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:50.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.709+0000 7f416ea00640 1 -- 172.21.15.22:0/1386180545 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f417009d370 con 0x7f4178107c80 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.963+0000 7f417cc00640 1 -- 172.21.15.22:0/1386180545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41780fdf50 con 0x7f4178107c80 2024-04-20T21:30:50.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.964+0000 7f416ea00640 1 -- 172.21.15.22:0/1386180545 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f41780fdf50 con 0x7f4178107c80 2024-04-20T21:30:50.963 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:30:50.964 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T21:30:50.964 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 3 2024-04-20T21:30:50.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.967+0000 7f417cc00640 1 -- 172.21.15.22:0/1386180545 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f415c0a3c30 msgr2=0x7f415c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:50.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.967+0000 7f417cc00640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f415c0a3c30 0x7f415c0a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f41680057c0 tx=0x7f416802cf90 comp rx=0 tx=0).stop 2024-04-20T21:30:50.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.967+0000 7f417cc00640 1 -- 172.21.15.22:0/1386180545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4178107c80 msgr2=0x7f4178100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:50.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.967+0000 7f417cc00640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4178107c80 0x7f4178100f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f417000eb10 tx=0x7f417000c2d0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.967+0000 7f417cc00640 1 -- 172.21.15.22:0/1386180545 shutdown_connections 2024-04-20T21:30:50.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.967+0000 7f417cc00640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f415c0a3c30 0x7f415c0a60f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.967+0000 7f417cc00640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4178107c80 0x7f4178100f30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.967+0000 7f417cc00640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4178105350 0x7f41781009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.967+0000 7f417cc00640 1 --2- 172.21.15.22:0/1386180545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4178102a20 0x7f41781023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.967+0000 7f417cc00640 1 -- 172.21.15.22:0/1386180545 >> 172.21.15.22:0/1386180545 conn(0x7f41780fc410 msgr2=0x7f4178103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:50.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.967+0000 7f417cc00640 1 -- 172.21.15.22:0/1386180545 shutdown_connections 2024-04-20T21:30:50.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:50.968+0000 7f417cc00640 1 -- 172.21.15.22:0/1386180545 wait complete. 2024-04-20T21:30:51.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:50 smithi022 ceph-mon[16843]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:51.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:50 smithi122 ceph-mon[22147]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:51.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:50 smithi033 ceph-mon[21626]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:51.584 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:30:51 smithi022 podman[54787]: 2024-04-20 21:30:51.322110018 +0000 UTC m=+0.138106062 container create e1fde6631cfcba5dad119537dd3a1ed18ab49118a1c5db33810c414383c10fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-04-20T21:30:51.584 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:30:51 smithi022 podman[54787]: 2024-04-20 21:30:51.583811026 +0000 UTC m=+0.399807094 container init e1fde6631cfcba5dad119537dd3a1ed18ab49118a1c5db33810c414383c10fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-j, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:30:51.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:51 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1386180545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:51.907 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:30:51 smithi022 podman[54787]: 2024-04-20 21:30:51.596867368 +0000 UTC m=+0.412863363 container start e1fde6631cfcba5dad119537dd3a1ed18ab49118a1c5db33810c414383c10fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-20T21:30:51.907 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:30:51 smithi022 ceph-mds[54806]: -- 172.21.15.22:0/111735350 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c562fe1a40 con 0x55c562fb5000 2024-04-20T21:30:51.907 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:30:51 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-j[54802]: starting mds.j at 2024-04-20T21:30:51.907 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:30:51 smithi022 bash[54787]: e1fde6631cfcba5dad119537dd3a1ed18ab49118a1c5db33810c414383c10fb0 2024-04-20T21:30:51.907 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:30:51 smithi022 systemd[1]: Started Ceph mds.j for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:52.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:51 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1386180545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:52.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:51 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1386180545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:52.986 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:53.075 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:52 smithi022 ceph-mon[16843]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:53.075 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:52 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.22:6834/2822771951,v1:172.21.15.22:6835/2822771951] up:boot 2024-04-20T21:30:53.075 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:52 smithi022 ceph-mon[16843]: fsmap 3 up:standby 2024-04-20T21:30:53.075 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:52 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T21:30:53.075 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:52 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:53.075 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:52 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:53.076 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:52 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:53.076 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:52 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:53.076 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:52 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:53.076 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:52 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:53.076 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:52 smithi022 ceph-mon[16843]: Deploying daemon mds.d on smithi022 2024-04-20T21:30:53.205 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:52 smithi033 ceph-mon[21626]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:53.205 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:52 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.22:6834/2822771951,v1:172.21.15.22:6835/2822771951] up:boot 2024-04-20T21:30:53.205 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:52 smithi033 ceph-mon[21626]: fsmap 3 up:standby 2024-04-20T21:30:53.205 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:52 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T21:30:53.205 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:52 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:53.205 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:52 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:53.205 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:52 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:53.205 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:52 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:53.205 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:52 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:53.206 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:52 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:53.206 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:52 smithi033 ceph-mon[21626]: Deploying daemon mds.d on smithi022 2024-04-20T21:30:53.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.221+0000 7ff2f3e00640 1 -- 172.21.15.22:0/2874222772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4078500 msgr2=0x7ff2f4078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.221+0000 7ff2f3e00640 1 --2- 172.21.15.22:0/2874222772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4078500 0x7ff2f4078da0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009900 tx=0x7ff2ec02bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:53.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.221+0000 7ff2f3e00640 1 -- 172.21.15.22:0/2874222772 shutdown_connections 2024-04-20T21:30:53.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.221+0000 7ff2f3e00640 1 --2- 172.21.15.22:0/2874222772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4078500 0x7ff2f4078da0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.221+0000 7ff2f3e00640 1 --2- 172.21.15.22:0/2874222772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2f4077b60 0x7ff2f4077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.221+0000 7ff2f3e00640 1 --2- 172.21.15.22:0/2874222772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4076920 0x7ff2f4076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.221+0000 7ff2f3e00640 1 -- 172.21.15.22:0/2874222772 >> 172.21.15.22:0/2874222772 conn(0x7ff2f40ff570 msgr2=0x7ff2f41019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:53.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.222+0000 7ff2f3e00640 1 -- 172.21.15.22:0/2874222772 shutdown_connections 2024-04-20T21:30:53.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.222+0000 7ff2f3e00640 1 -- 172.21.15.22:0/2874222772 wait complete. 2024-04-20T21:30:53.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.222+0000 7ff2f3e00640 1 Processor -- start 2024-04-20T21:30:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.223+0000 7ff2f3e00640 1 -- start start 2024-04-20T21:30:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.223+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2f4076920 0x7ff2f419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.223+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4077b60 0x7ff2f419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.223+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4078500 0x7ff2f41a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.223+0000 7ff2f3e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41140d0 con 0x7ff2f4078500 2024-04-20T21:30:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.223+0000 7ff2f3e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4113f50 con 0x7ff2f4077b60 2024-04-20T21:30:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.223+0000 7ff2f3e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4114250 con 0x7ff2f4076920 2024-04-20T21:30:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.223+0000 7ff2f3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4078500 0x7ff2f41a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.223+0000 7ff2f3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4078500 0x7ff2f41a5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59518/0 (socket says 172.21.15.22:59518) 2024-04-20T21:30:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.223+0000 7ff2f3400640 1 -- 172.21.15.22:0/1353124424 learned_addr learned my addr 172.21.15.22:0/1353124424 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:30:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.223+0000 7ff2f2a00640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2f4076920 0x7ff2f419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.223+0000 7ff2ea000640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4077b60 0x7ff2f419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.224+0000 7ff2f3400640 1 -- 172.21.15.22:0/1353124424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2f4076920 msgr2=0x7ff2f419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.224+0000 7ff2f3400640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2f4076920 0x7ff2f419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.224+0000 7ff2f3400640 1 -- 172.21.15.22:0/1353124424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4077b60 msgr2=0x7ff2f419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.224+0000 7ff2f3400640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4077b60 0x7ff2f419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.224+0000 7ff2f3400640 1 -- 172.21.15.22:0/1353124424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f41a62c0 con 0x7ff2f4078500 2024-04-20T21:30:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.224+0000 7ff2f2a00640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2f4076920 0x7ff2f419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.224+0000 7ff2f3400640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4078500 0x7ff2f41a5c50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec02c8e0 tx=0x7ff2ec02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.224+0000 7ff2f8a00640 1 -- 172.21.15.22:0/1353124424 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec043020 con 0x7ff2f4078500 2024-04-20T21:30:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.225+0000 7ff2f8a00640 1 -- 172.21.15.22:0/1353124424 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ec0054e0 con 0x7ff2f4078500 2024-04-20T21:30:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.225+0000 7ff2f3e00640 1 -- 172.21.15.22:0/1353124424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f41a6550 con 0x7ff2f4078500 2024-04-20T21:30:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.225+0000 7ff2f8a00640 1 -- 172.21.15.22:0/1353124424 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec003d00 con 0x7ff2f4078500 2024-04-20T21:30:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.225+0000 7ff2f3e00640 1 -- 172.21.15.22:0/1353124424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f41a6af0 con 0x7ff2f4078500 2024-04-20T21:30:53.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.226+0000 7ff2f3e00640 1 -- 172.21.15.22:0/1353124424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f4110970 con 0x7ff2f4078500 2024-04-20T21:30:53.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.229+0000 7ff2f8a00640 1 -- 172.21.15.22:0/1353124424 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff2ec046020 con 0x7ff2f4078500 2024-04-20T21:30:53.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.231+0000 7ff2f8a00640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff2d80a3c30 0x7ff2d80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.231+0000 7ff2f8a00640 1 -- 172.21.15.22:0/1353124424 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff2ec0f23a0 con 0x7ff2f4078500 2024-04-20T21:30:53.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.231+0000 7ff2f2a00640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff2d80a3c30 0x7ff2d80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.232+0000 7ff2f2a00640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff2d80a3c30 0x7ff2d80a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005fd0 tx=0x7ff2e4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:53.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.234+0000 7ff2f8a00640 1 -- 172.21.15.22:0/1353124424 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff2ec04d020 con 0x7ff2f4078500 2024-04-20T21:30:53.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:52 smithi122 ceph-mon[22147]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:53.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:52 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.22:6834/2822771951,v1:172.21.15.22:6835/2822771951] up:boot 2024-04-20T21:30:53.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:52 smithi122 ceph-mon[22147]: fsmap 3 up:standby 2024-04-20T21:30:53.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:52 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T21:30:53.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:52 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:53.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:52 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:53.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:52 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:53.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:52 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:53.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:52 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:53.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:52 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:53.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:52 smithi122 ceph-mon[22147]: Deploying daemon mds.d on smithi022 2024-04-20T21:30:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.486+0000 7ff2f3e00640 1 -- 172.21.15.22:0/1353124424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2f402d5d0 con 0x7ff2f4078500 2024-04-20T21:30:53.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.487+0000 7ff2f8a00640 1 -- 172.21.15.22:0/1353124424 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7ff2ec0bf210 con 0x7ff2f4078500 2024-04-20T21:30:53.486 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:30:53.486 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T21:30:53.486 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 4 2024-04-20T21:30:53.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.489+0000 7ff2f3e00640 1 -- 172.21.15.22:0/1353124424 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff2d80a3c30 msgr2=0x7ff2d80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.489+0000 7ff2f3e00640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff2d80a3c30 0x7ff2d80a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005fd0 tx=0x7ff2e4009500 comp rx=0 tx=0).stop 2024-04-20T21:30:53.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.489+0000 7ff2f3e00640 1 -- 172.21.15.22:0/1353124424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4078500 msgr2=0x7ff2f41a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.489+0000 7ff2f3e00640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4078500 0x7ff2f41a5c50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec02c8e0 tx=0x7ff2ec02c640 comp rx=0 tx=0).stop 2024-04-20T21:30:53.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.490+0000 7ff2f3e00640 1 -- 172.21.15.22:0/1353124424 shutdown_connections 2024-04-20T21:30:53.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.490+0000 7ff2f3e00640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff2d80a3c30 0x7ff2d80a60f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.490+0000 7ff2f3e00640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2f4078500 0x7ff2f41a5c50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.490+0000 7ff2f3e00640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2f4077b60 0x7ff2f419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.490+0000 7ff2f3e00640 1 --2- 172.21.15.22:0/1353124424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2f4076920 0x7ff2f419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.490+0000 7ff2f3e00640 1 -- 172.21.15.22:0/1353124424 >> 172.21.15.22:0/1353124424 conn(0x7ff2f40ff570 msgr2=0x7ff2f4101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:53.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.490+0000 7ff2f3e00640 1 -- 172.21.15.22:0/1353124424 shutdown_connections 2024-04-20T21:30:53.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:53.490+0000 7ff2f3e00640 1 -- 172.21.15.22:0/1353124424 wait complete. 2024-04-20T21:30:54.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:53 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1353124424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:54.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:53 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1353124424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:54.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:53 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1353124424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:55.094 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:54 smithi022 ceph-mon[16843]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:55.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:54 smithi122 ceph-mon[22147]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:55.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:54 smithi033 ceph-mon[21626]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:55.407 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:30:55 smithi022 systemd[1]: Starting Ceph mds.d for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:55.508 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:55.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.736+0000 7fd8cdc00640 1 -- 172.21.15.22:0/668994725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c8102860 msgr2=0x7fd8c8104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.736+0000 7fd8cdc00640 1 --2- 172.21.15.22:0/668994725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c8102860 0x7fd8c8104c50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009900 tx=0x7fd8b802bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:55.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.736+0000 7fd8cdc00640 1 -- 172.21.15.22:0/668994725 shutdown_connections 2024-04-20T21:30:55.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.736+0000 7fd8cdc00640 1 --2- 172.21.15.22:0/668994725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8107ac0 0x7fd8c8109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.736+0000 7fd8cdc00640 1 --2- 172.21.15.22:0/668994725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8c8105190 0x7fd8c8107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.736+0000 7fd8cdc00640 1 --2- 172.21.15.22:0/668994725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c8102860 0x7fd8c8104c50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.736+0000 7fd8cdc00640 1 -- 172.21.15.22:0/668994725 >> 172.21.15.22:0/668994725 conn(0x7fd8c80fc270 msgr2=0x7fd8c80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:55.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.736+0000 7fd8cdc00640 1 -- 172.21.15.22:0/668994725 shutdown_connections 2024-04-20T21:30:55.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.736+0000 7fd8cdc00640 1 -- 172.21.15.22:0/668994725 wait complete. 2024-04-20T21:30:55.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.737+0000 7fd8cdc00640 1 Processor -- start 2024-04-20T21:30:55.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.737+0000 7fd8cdc00640 1 -- start start 2024-04-20T21:30:55.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.738+0000 7fd8cdc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8c8102860 0x7fd8c81021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.738+0000 7fd8cdc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c8105190 0x7fd8c8100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.738+0000 7fd8cdc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8107ac0 0x7fd8c8100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.738+0000 7fd8cdc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c810dab0 con 0x7fd8c8105190 2024-04-20T21:30:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.738+0000 7fd8cdc00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c810d930 con 0x7fd8c8107ac0 2024-04-20T21:30:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.738+0000 7fd8cdc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c810dc30 con 0x7fd8c8102860 2024-04-20T21:30:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.738+0000 7fd8c7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c8105190 0x7fd8c8100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.738+0000 7fd8c7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8c8102860 0x7fd8c81021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.738+0000 7fd8c7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c8105190 0x7fd8c8100810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59528/0 (socket says 172.21.15.22:59528) 2024-04-20T21:30:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.738+0000 7fd8c7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8c8102860 0x7fd8c81021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:54098/0 (socket says 172.21.15.22:54098) 2024-04-20T21:30:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.738+0000 7fd8c7400640 1 -- 172.21.15.22:0/4144546282 learned_addr learned my addr 172.21.15.22:0/4144546282 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:30:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.738+0000 7fd8cd200640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8107ac0 0x7fd8c8100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.739+0000 7fd8c7e00640 1 -- 172.21.15.22:0/4144546282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8107ac0 msgr2=0x7fd8c8100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.739+0000 7fd8c7e00640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8107ac0 0x7fd8c8100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.739+0000 7fd8c7e00640 1 -- 172.21.15.22:0/4144546282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c8105190 msgr2=0x7fd8c8100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.739+0000 7fd8c7e00640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c8105190 0x7fd8c8100810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.739+0000 7fd8c7e00640 1 -- 172.21.15.22:0/4144546282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c81015e0 con 0x7fd8c8102860 2024-04-20T21:30:55.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.739+0000 7fd8c7e00640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8c8102860 0x7fd8c81021d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8036040 tx=0x7fd8b802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.739+0000 7fd8c4c00640 1 -- 172.21.15.22:0/4144546282 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8043020 con 0x7fd8c8102860 2024-04-20T21:30:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.739+0000 7fd8c4c00640 1 -- 172.21.15.22:0/4144546282 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8b80054e0 con 0x7fd8c8102860 2024-04-20T21:30:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.739+0000 7fd8cdc00640 1 -- 172.21.15.22:0/4144546282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c81a22a0 con 0x7fd8c8102860 2024-04-20T21:30:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.739+0000 7fd8c4c00640 1 -- 172.21.15.22:0/4144546282 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8003ca0 con 0x7fd8c8102860 2024-04-20T21:30:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.739+0000 7fd8cdc00640 1 -- 172.21.15.22:0/4144546282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c81a27c0 con 0x7fd8c8102860 2024-04-20T21:30:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.741+0000 7fd8cdc00640 1 -- 172.21.15.22:0/4144546282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c80fdd60 con 0x7fd8c8102860 2024-04-20T21:30:55.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.743+0000 7fd8c4c00640 1 -- 172.21.15.22:0/4144546282 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd8b8046020 con 0x7fd8c8102860 2024-04-20T21:30:55.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.744+0000 7fd8c4c00640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd8ac0a3bb0 0x7fd8ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.744+0000 7fd8c4c00640 1 -- 172.21.15.22:0/4144546282 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fd8b80f2710 con 0x7fd8c8102860 2024-04-20T21:30:55.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.744+0000 7fd8c7400640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd8ac0a3bb0 0x7fd8ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:55.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.745+0000 7fd8c7400640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd8ac0a3bb0 0x7fd8ac0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8101f40 tx=0x7fd8b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:55.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:55.746+0000 7fd8c4c00640 1 -- 172.21.15.22:0/4144546282 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd8b804d020 con 0x7fd8c8102860 2024-04-20T21:30:55.752 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:30:55 smithi022 podman[55135]: 2024-04-20 21:30:55.402777086 +0000 UTC m=+0.047072502 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:55.753 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:30:55 smithi022 podman[55135]: 2024-04-20 21:30:55.509066537 +0000 UTC m=+0.153361960 container create 5952b92e908cb3154d274fd78226902eabf158424fc130b8bf41f92cef439eb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:30:55.753 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:30:55 smithi022 podman[55135]: 2024-04-20 21:30:55.75385532 +0000 UTC m=+0.398150731 container init 5952b92e908cb3154d274fd78226902eabf158424fc130b8bf41f92cef439eb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-20T21:30:56.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.009+0000 7fd8cdc00640 1 -- 172.21.15.22:0/4144546282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8c802d5d0 con 0x7fd8c8102860 2024-04-20T21:30:56.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.010+0000 7fd8c4c00640 1 -- 172.21.15.22:0/4144546282 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fd8b80bf4f0 con 0x7fd8c8102860 2024-04-20T21:30:56.009 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:30:56.010 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T21:30:56.010 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 5 2024-04-20T21:30:56.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.013+0000 7fd8cdc00640 1 -- 172.21.15.22:0/4144546282 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd8ac0a3bb0 msgr2=0x7fd8ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.013+0000 7fd8cdc00640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd8ac0a3bb0 0x7fd8ac0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8101f40 tx=0x7fd8b4005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:56.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.013+0000 7fd8cdc00640 1 -- 172.21.15.22:0/4144546282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8c8102860 msgr2=0x7fd8c81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.013+0000 7fd8cdc00640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8c8102860 0x7fd8c81021d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8036040 tx=0x7fd8b802c6c0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.013+0000 7fd8cdc00640 1 -- 172.21.15.22:0/4144546282 shutdown_connections 2024-04-20T21:30:56.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.013+0000 7fd8cdc00640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd8ac0a3bb0 0x7fd8ac0a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.013+0000 7fd8cdc00640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8107ac0 0x7fd8c8100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.013+0000 7fd8cdc00640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c8105190 0x7fd8c8100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.013+0000 7fd8cdc00640 1 --2- 172.21.15.22:0/4144546282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8c8102860 0x7fd8c81021d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.013+0000 7fd8cdc00640 1 -- 172.21.15.22:0/4144546282 >> 172.21.15.22:0/4144546282 conn(0x7fd8c80fc270 msgr2=0x7fd8c8103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:56.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.013+0000 7fd8cdc00640 1 -- 172.21.15.22:0/4144546282 shutdown_connections 2024-04-20T21:30:56.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:56.013+0000 7fd8cdc00640 1 -- 172.21.15.22:0/4144546282 wait complete. 2024-04-20T21:30:56.031 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:30:55 smithi022 podman[55135]: 2024-04-20 21:30:55.761787874 +0000 UTC m=+0.406083279 container start 5952b92e908cb3154d274fd78226902eabf158424fc130b8bf41f92cef439eb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-20T21:30:56.031 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:30:55 smithi022 ceph-mds[55193]: -- 172.21.15.22:0/1278658665 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56070ebdda40 con 0x56070ebb1000 2024-04-20T21:30:56.031 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:30:55 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-d[55170]: starting mds.d at 2024-04-20T21:30:56.031 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:30:55 smithi022 bash[55135]: 5952b92e908cb3154d274fd78226902eabf158424fc130b8bf41f92cef439eb6 2024-04-20T21:30:56.032 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:30:55 smithi022 systemd[1]: Started Ceph mds.d for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:56.940 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:56 smithi033 ceph-mon[21626]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:56.941 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:56 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.22:6836/2696291995,v1:172.21.15.22:6837/2696291995] up:boot 2024-04-20T21:30:56.941 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:56 smithi033 ceph-mon[21626]: fsmap 4 up:standby 2024-04-20T21:30:56.941 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:56 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T21:30:56.941 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:56 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:56.941 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:56 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:56.941 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:56 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/4144546282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:56.941 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:56 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:56.941 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:56 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:56.941 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:56 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:56.941 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:56 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:56.941 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:56 smithi033 ceph-mon[21626]: Deploying daemon mds.e on smithi033 2024-04-20T21:30:57.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:56 smithi022 ceph-mon[16843]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:57.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:56 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.22:6836/2696291995,v1:172.21.15.22:6837/2696291995] up:boot 2024-04-20T21:30:57.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:56 smithi022 ceph-mon[16843]: fsmap 4 up:standby 2024-04-20T21:30:57.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:56 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T21:30:57.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:56 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:57.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:56 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:57.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:56 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/4144546282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:57.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:56 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:57.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:56 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:57.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:56 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:57.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:56 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:57.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:56 smithi022 ceph-mon[16843]: Deploying daemon mds.e on smithi033 2024-04-20T21:30:57.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:56 smithi122 ceph-mon[22147]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:57.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:56 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.22:6836/2696291995,v1:172.21.15.22:6837/2696291995] up:boot 2024-04-20T21:30:57.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:56 smithi122 ceph-mon[22147]: fsmap 4 up:standby 2024-04-20T21:30:57.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:56 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T21:30:57.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:56 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:57.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:56 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:57.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:56 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/4144546282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:57.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:56 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:57.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:56 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:57.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:56 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:57.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:56 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:57.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:56 smithi122 ceph-mon[22147]: Deploying daemon mds.e on smithi033 2024-04-20T21:30:58.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:58.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.258+0000 7faa0ca00640 1 -- 172.21.15.22:0/3801144819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa08102fc0 msgr2=0x7faa08103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.258+0000 7faa0ca00640 1 --2- 172.21.15.22:0/3801144819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa08102fc0 0x7faa08103440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009900 tx=0x7fa9f002bc40 comp rx=0 tx=0).stop 2024-04-20T21:30:58.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.258+0000 7faa0ca00640 1 -- 172.21.15.22:0/3801144819 shutdown_connections 2024-04-20T21:30:58.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.258+0000 7faa0ca00640 1 --2- 172.21.15.22:0/3801144819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa08103980 0x7faa0810a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.258+0000 7faa0ca00640 1 --2- 172.21.15.22:0/3801144819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa08102fc0 0x7faa08103440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.258+0000 7faa0ca00640 1 --2- 172.21.15.22:0/3801144819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa08101dd0 0x7faa081021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.258+0000 7faa0ca00640 1 -- 172.21.15.22:0/3801144819 >> 172.21.15.22:0/3801144819 conn(0x7faa080fd340 msgr2=0x7faa080ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:58.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.259+0000 7faa0ca00640 1 -- 172.21.15.22:0/3801144819 shutdown_connections 2024-04-20T21:30:58.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.259+0000 7faa0ca00640 1 -- 172.21.15.22:0/3801144819 wait complete. 2024-04-20T21:30:58.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.260+0000 7faa0ca00640 1 Processor -- start 2024-04-20T21:30:58.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.260+0000 7faa0ca00640 1 -- start start 2024-04-20T21:30:58.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.260+0000 7faa0ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa08101dd0 0x7faa0819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.260+0000 7faa0ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa08102fc0 0x7faa0819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.260+0000 7faa0ca00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa08103980 0x7faa081a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.260+0000 7faa0ca00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faa0810da80 con 0x7faa08102fc0 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.260+0000 7faa0ca00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa0810d900 con 0x7faa08103980 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.260+0000 7faa0ca00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa0810dc00 con 0x7faa08101dd0 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.260+0000 7fa9fde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa08102fc0 0x7faa0819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.260+0000 7fa9fde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa08102fc0 0x7faa0819a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59568/0 (socket says 172.21.15.22:59568) 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.260+0000 7fa9fde00640 1 -- 172.21.15.22:0/1030508880 learned_addr learned my addr 172.21.15.22:0/1030508880 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.261+0000 7faa07200640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa08103980 0x7faa081a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.261+0000 7faa06800640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa08101dd0 0x7faa0819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.261+0000 7fa9fde00640 1 -- 172.21.15.22:0/1030508880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa08101dd0 msgr2=0x7faa0819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.261+0000 7fa9fde00640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa08101dd0 0x7faa0819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.261+0000 7fa9fde00640 1 -- 172.21.15.22:0/1030508880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa08103980 msgr2=0x7faa081a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.261+0000 7fa9fde00640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa08103980 0x7faa081a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.261+0000 7fa9fde00640 1 -- 172.21.15.22:0/1030508880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa081a1f00 con 0x7faa08102fc0 2024-04-20T21:30:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.261+0000 7fa9fde00640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa08102fc0 0x7faa0819a820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa9f002c8f0 tx=0x7fa9f002c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.261+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1030508880 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0041070 con 0x7faa08102fc0 2024-04-20T21:30:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.262+0000 7faa0ca00640 1 -- 172.21.15.22:0/1030508880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa081a2190 con 0x7faa08102fc0 2024-04-20T21:30:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.262+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1030508880 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f0004450 con 0x7faa08102fc0 2024-04-20T21:30:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.262+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1030508880 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0034440 con 0x7faa08102fc0 2024-04-20T21:30:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.262+0000 7faa0ca00640 1 -- 172.21.15.22:0/1030508880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa081a2680 con 0x7faa08102fc0 2024-04-20T21:30:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.263+0000 7faa0ca00640 1 -- 172.21.15.22:0/1030508880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa081a29a0 con 0x7faa08102fc0 2024-04-20T21:30:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.265+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1030508880 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa9f0044070 con 0x7faa08102fc0 2024-04-20T21:30:58.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.266+0000 7fa9ffe00640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.267+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1030508880 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fa9f00f1f10 con 0x7faa08102fc0 2024-04-20T21:30:58.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.267+0000 7faa06800640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.267+0000 7faa06800640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faa08102e20 tx=0x7fa9f8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:58.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.269+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1030508880 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa9f00be6a0 con 0x7faa08102fc0 2024-04-20T21:30:58.303 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:30:58 smithi033 systemd[1]: Starting Ceph mds.e for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:58.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.517+0000 7faa0ca00640 1 -- 172.21.15.22:0/1030508880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa081a2b70 con 0x7faa08102fc0 2024-04-20T21:30:58.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.518+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1030508880 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fa9f00c29a0 con 0x7faa08102fc0 2024-04-20T21:30:58.517 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:30:58.517 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T21:30:58.518 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 5 2024-04-20T21:30:58.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.520+0000 7faa0ca00640 1 -- 172.21.15.22:0/1030508880 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa9ec0a3bb0 msgr2=0x7fa9ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.520+0000 7faa0ca00640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faa08102e20 tx=0x7fa9f8009500 comp rx=0 tx=0).stop 2024-04-20T21:30:58.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.520+0000 7faa0ca00640 1 -- 172.21.15.22:0/1030508880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa08102fc0 msgr2=0x7faa0819a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.520+0000 7faa0ca00640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa08102fc0 0x7faa0819a820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa9f002c8f0 tx=0x7fa9f002c660 comp rx=0 tx=0).stop 2024-04-20T21:30:58.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.520+0000 7faa0ca00640 1 -- 172.21.15.22:0/1030508880 shutdown_connections 2024-04-20T21:30:58.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.520+0000 7faa0ca00640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.520+0000 7faa0ca00640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa08103980 0x7faa081a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.521+0000 7faa0ca00640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa08102fc0 0x7faa0819a820 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.521+0000 7faa0ca00640 1 --2- 172.21.15.22:0/1030508880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa08101dd0 0x7faa0819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.521+0000 7faa0ca00640 1 -- 172.21.15.22:0/1030508880 >> 172.21.15.22:0/1030508880 conn(0x7faa080fd340 msgr2=0x7faa08074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:58.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.521+0000 7faa0ca00640 1 -- 172.21.15.22:0/1030508880 shutdown_connections 2024-04-20T21:30:58.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:30:58.521+0000 7faa0ca00640 1 -- 172.21.15.22:0/1030508880 wait complete. 2024-04-20T21:30:58.667 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:30:58 smithi033 podman[51312]: 2024-04-20 21:30:58.312358236 +0000 UTC m=+0.029110907 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:58.667 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:30:58 smithi033 podman[51312]: 2024-04-20 21:30:58.424963521 +0000 UTC m=+0.141716236 container create bb6003d282922aa87bba475b98c50cde9aab9726f425872f38175705f2e76667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-20T21:30:59.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:58 smithi033 ceph-mon[21626]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:59.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:58 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1030508880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:59.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:58 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:59.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:58 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:59.054 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:30:58 smithi033 podman[51312]: 2024-04-20 21:30:58.669153562 +0000 UTC m=+0.385906221 container init bb6003d282922aa87bba475b98c50cde9aab9726f425872f38175705f2e76667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-20T21:30:59.054 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:30:58 smithi033 podman[51312]: 2024-04-20 21:30:58.672012287 +0000 UTC m=+0.388764943 container start bb6003d282922aa87bba475b98c50cde9aab9726f425872f38175705f2e76667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-20T21:30:59.054 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:30:58 smithi033 ceph-mds[51331]: -- 172.21.15.33:0/2749585737 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc650efa40 con 0x55cc650c2800 2024-04-20T21:30:59.054 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:30:58 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-e[51327]: starting mds.e at 2024-04-20T21:30:59.054 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:30:58 smithi033 bash[51312]: bb6003d282922aa87bba475b98c50cde9aab9726f425872f38175705f2e76667 2024-04-20T21:30:59.054 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:30:58 smithi033 systemd[1]: Started Ceph mds.e for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:59.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:58 smithi022 ceph-mon[16843]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:59.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:58 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1030508880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:59.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:58 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:59.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:58 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:59.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:58 smithi122 ceph-mon[22147]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:59.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:58 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1030508880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:59.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:58 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:30:59.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:58 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:00.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:59 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.33:6834/1440214109,v1:172.21.15.33:6835/1440214109] up:boot 2024-04-20T21:31:00.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:59 smithi033 ceph-mon[21626]: fsmap 5 up:standby 2024-04-20T21:31:00.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:59 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:31:00.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:59 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:00.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:59 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:00.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:59 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:00.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:59 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:00.055 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:30:59 smithi033 ceph-mon[21626]: Deploying daemon mds.f on smithi122 2024-04-20T21:31:00.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:59 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.33:6834/1440214109,v1:172.21.15.33:6835/1440214109] up:boot 2024-04-20T21:31:00.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:59 smithi022 ceph-mon[16843]: fsmap 5 up:standby 2024-04-20T21:31:00.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:59 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:31:00.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:59 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:00.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:59 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:00.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:59 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:00.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:59 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:00.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:30:59 smithi022 ceph-mon[16843]: Deploying daemon mds.f on smithi122 2024-04-20T21:31:00.230 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:59 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.33:6834/1440214109,v1:172.21.15.33:6835/1440214109] up:boot 2024-04-20T21:31:00.230 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:59 smithi122 ceph-mon[22147]: fsmap 5 up:standby 2024-04-20T21:31:00.230 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:59 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:31:00.230 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:59 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:00.230 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:59 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:00.230 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:59 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:00.230 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:59 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:00.230 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:30:59 smithi122 ceph-mon[22147]: Deploying daemon mds.f on smithi122 2024-04-20T21:31:00.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:00.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.762+0000 7f97b4c00640 1 -- 172.21.15.22:0/4106275783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b01051d0 msgr2=0x7f97b0105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.762+0000 7f97b4c00640 1 --2- 172.21.15.22:0/4106275783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b01051d0 0x7f97b0105650 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9798009900 tx=0x7f979802bc50 comp rx=0 tx=0).stop 2024-04-20T21:31:00.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.762+0000 7f97b4c00640 1 -- 172.21.15.22:0/4106275783 shutdown_connections 2024-04-20T21:31:00.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.762+0000 7f97b4c00640 1 --2- 172.21.15.22:0/4106275783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b0105b90 0x7f97b010c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.762+0000 7f97b4c00640 1 --2- 172.21.15.22:0/4106275783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b01051d0 0x7f97b0105650 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.762+0000 7f97b4c00640 1 --2- 172.21.15.22:0/4106275783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b0103fe0 0x7f97b0104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.762+0000 7f97b4c00640 1 -- 172.21.15.22:0/4106275783 >> 172.21.15.22:0/4106275783 conn(0x7f97b00ff570 msgr2=0x7f97b0101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:00.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.762+0000 7f97b4c00640 1 -- 172.21.15.22:0/4106275783 shutdown_connections 2024-04-20T21:31:00.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.762+0000 7f97b4c00640 1 -- 172.21.15.22:0/4106275783 wait complete. 2024-04-20T21:31:00.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.763+0000 7f97b4c00640 1 Processor -- start 2024-04-20T21:31:00.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.763+0000 7f97b4c00640 1 -- start start 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.763+0000 7f97b4c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b0103fe0 0x7f97b019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.763+0000 7f97b4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b01051d0 0x7f97b019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.763+0000 7f97b4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b0105b90 0x7f97b01a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97b4c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f97b010fc70 con 0x7f97b01051d0 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97b4c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97b010faf0 con 0x7f97b0103fe0 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97b4c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f97b010fdf0 con 0x7f97b0105b90 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b01051d0 0x7f97b019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97af400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b0103fe0 0x7f97b019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b01051d0 0x7f97b019a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59598/0 (socket says 172.21.15.22:59598) 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97a7e00640 1 -- 172.21.15.22:0/3649142009 learned_addr learned my addr 172.21.15.22:0/3649142009 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97afe00640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b0105b90 0x7f97b01a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97a7e00640 1 -- 172.21.15.22:0/3649142009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b0105b90 msgr2=0x7f97b01a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97a7e00640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b0105b90 0x7f97b01a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97a7e00640 1 -- 172.21.15.22:0/3649142009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b0103fe0 msgr2=0x7f97b019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97a7e00640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b0103fe0 0x7f97b019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97a7e00640 1 -- 172.21.15.22:0/3649142009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b01a1f60 con 0x7f97b01051d0 2024-04-20T21:31:00.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97afe00640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b0105b90 0x7f97b01a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:00.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.764+0000 7f97af400640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b0103fe0 0x7f97b019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:00.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.765+0000 7f97a7e00640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b01051d0 0x7f97b019a880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9798009230 tx=0x7f9798005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:00.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.765+0000 7f97acc00640 1 -- 172.21.15.22:0/3649142009 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798042020 con 0x7f97b01051d0 2024-04-20T21:31:00.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.765+0000 7f97b4c00640 1 -- 172.21.15.22:0/3649142009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b01a21f0 con 0x7f97b01051d0 2024-04-20T21:31:00.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.765+0000 7f97acc00640 1 -- 172.21.15.22:0/3649142009 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97980044d0 con 0x7f97b01051d0 2024-04-20T21:31:00.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.765+0000 7f97b4c00640 1 -- 172.21.15.22:0/3649142009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b01a2790 con 0x7f97b01051d0 2024-04-20T21:31:00.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.765+0000 7f97acc00640 1 -- 172.21.15.22:0/3649142009 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979803e460 con 0x7f97b01051d0 2024-04-20T21:31:00.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.766+0000 7f97b4c00640 1 -- 172.21.15.22:0/3649142009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b010b300 con 0x7f97b01051d0 2024-04-20T21:31:00.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.768+0000 7f97acc00640 1 -- 172.21.15.22:0/3649142009 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9798045020 con 0x7f97b01051d0 2024-04-20T21:31:00.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.769+0000 7f97acc00640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f97940a3bb0 0x7f97940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.770+0000 7f97acc00640 1 -- 172.21.15.22:0/3649142009 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f97980f1fe0 con 0x7f97b01051d0 2024-04-20T21:31:00.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.770+0000 7f97af400640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f97940a3bb0 0x7f97940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.771+0000 7f97af400640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f97940a3bb0 0x7f97940a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f97b0105030 tx=0x7f97a000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:00.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:00.774+0000 7f97acc00640 1 -- 172.21.15.22:0/3649142009 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f979804c020 con 0x7f97b01051d0 2024-04-20T21:31:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.028+0000 7f97b4c00640 1 -- 172.21.15.22:0/3649142009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97b0062b60 con 0x7f97b01051d0 2024-04-20T21:31:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.029+0000 7f97acc00640 1 -- 172.21.15.22:0/3649142009 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f97980bedc0 con 0x7f97b01051d0 2024-04-20T21:31:01.028 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:01.028 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-20T21:31:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 6 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.031+0000 7f97b4c00640 1 -- 172.21.15.22:0/3649142009 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f97940a3bb0 msgr2=0x7f97940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.031+0000 7f97b4c00640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f97940a3bb0 0x7f97940a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f97b0105030 tx=0x7f97a000b390 comp rx=0 tx=0).stop 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.031+0000 7f97b4c00640 1 -- 172.21.15.22:0/3649142009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b01051d0 msgr2=0x7f97b019a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.031+0000 7f97b4c00640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b01051d0 0x7f97b019a880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9798009230 tx=0x7f9798005d50 comp rx=0 tx=0).stop 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.032+0000 7f97b4c00640 1 -- 172.21.15.22:0/3649142009 shutdown_connections 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.032+0000 7f97b4c00640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f97940a3bb0 0x7f97940a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.032+0000 7f97b4c00640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b0105b90 0x7f97b01a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.032+0000 7f97b4c00640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97b01051d0 0x7f97b019a880 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.032+0000 7f97b4c00640 1 --2- 172.21.15.22:0/3649142009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b0103fe0 0x7f97b019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.032+0000 7f97b4c00640 1 -- 172.21.15.22:0/3649142009 >> 172.21.15.22:0/3649142009 conn(0x7f97b00ff570 msgr2=0x7f97b01084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.032+0000 7f97b4c00640 1 -- 172.21.15.22:0/3649142009 shutdown_connections 2024-04-20T21:31:01.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:01.032+0000 7f97b4c00640 1 -- 172.21.15.22:0/3649142009 wait complete. 2024-04-20T21:31:01.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:00 smithi022 ceph-mon[16843]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:01.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:00 smithi122 ceph-mon[22147]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:01.218 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:31:01 smithi122 systemd[1]: Starting Ceph mds.f for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:31:01.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:00 smithi033 ceph-mon[21626]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:01.520 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:31:01 smithi122 podman[52201]: 2024-04-20 21:31:01.219021336 +0000 UTC m=+0.101790735 container create 669031ef3cd064a93190a7a9511b281cf47d0ac7b059c00254c09e917113bd91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:31:01.520 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:31:01 smithi122 podman[52201]: 2024-04-20 21:31:01.13533312 +0000 UTC m=+0.018102529 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:31:01.520 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:31:01 smithi122 podman[52201]: 2024-04-20 21:31:01.438339621 +0000 UTC m=+0.321109013 container init 669031ef3cd064a93190a7a9511b281cf47d0ac7b059c00254c09e917113bd91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:31:01.520 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:31:01 smithi122 podman[52201]: 2024-04-20 21:31:01.440768999 +0000 UTC m=+0.323538394 container start 669031ef3cd064a93190a7a9511b281cf47d0ac7b059c00254c09e917113bd91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-20T21:31:01.520 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:31:01 smithi122 ceph-mds[52220]: -- 172.21.15.122:0/62479871 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5631ecc03a40 con 0x5631ecbd6800 2024-04-20T21:31:01.520 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:31:01 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-f[52216]: starting mds.f at 2024-04-20T21:31:01.520 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:31:01 smithi122 bash[52201]: 669031ef3cd064a93190a7a9511b281cf47d0ac7b059c00254c09e917113bd91 2024-04-20T21:31:01.774 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:31:01 smithi122 systemd[1]: Started Ceph mds.f for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:31:02.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:01 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3649142009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:02.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:02.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:02.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:02.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:02.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:02.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:01 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:02.265 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:01 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3649142009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:02.265 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:02.265 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:02.265 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:02.265 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:02.266 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:02.266 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:01 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:02.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:01 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3649142009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:02.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:02.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:02.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:02.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:02.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:02.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:01 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:03.052 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:03.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:02 smithi022 ceph-mon[16843]: Deploying daemon mds.c on smithi122 2024-04-20T21:31:03.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:02 smithi022 ceph-mon[16843]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:03.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:02 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.122:6834/3637338060,v1:172.21.15.122:6835/3637338060] up:boot 2024-04-20T21:31:03.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:02 smithi022 ceph-mon[16843]: fsmap 6 up:standby 2024-04-20T21:31:03.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:02 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:31:03.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.267+0000 7fc775200640 1 -- 172.21.15.22:0/2795742781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc770101ed0 msgr2=0x7fc770102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.267+0000 7fc775200640 1 --2- 172.21.15.22:0/2795742781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc770101ed0 0x7fc770102350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009900 tx=0x7fc75c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:31:03.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.267+0000 7fc775200640 1 -- 172.21.15.22:0/2795742781 shutdown_connections 2024-04-20T21:31:03.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.267+0000 7fc775200640 1 --2- 172.21.15.22:0/2795742781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc770102890 0x7fc770109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.267+0000 7fc775200640 1 --2- 172.21.15.22:0/2795742781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc770101ed0 0x7fc770102350 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.267+0000 7fc775200640 1 --2- 172.21.15.22:0/2795742781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc770100ce0 0x7fc770101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.267+0000 7fc775200640 1 -- 172.21.15.22:0/2795742781 >> 172.21.15.22:0/2795742781 conn(0x7fc7700fc230 msgr2=0x7fc7700fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:03.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.267+0000 7fc775200640 1 -- 172.21.15.22:0/2795742781 shutdown_connections 2024-04-20T21:31:03.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.267+0000 7fc775200640 1 -- 172.21.15.22:0/2795742781 wait complete. 2024-04-20T21:31:03.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.268+0000 7fc775200640 1 Processor -- start 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.269+0000 7fc775200640 1 -- start start 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.269+0000 7fc775200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc770100ce0 0x7fc770198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.269+0000 7fc775200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc770101ed0 0x7fc770198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.269+0000 7fc775200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc770102890 0x7fc77019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.269+0000 7fc775200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc77010c990 con 0x7fc770102890 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.269+0000 7fc775200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc77010c810 con 0x7fc770100ce0 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.269+0000 7fc775200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc77010cb10 con 0x7fc770101ed0 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.269+0000 7fc76fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc770102890 0x7fc77019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.269+0000 7fc76ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc770101ed0 0x7fc770198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.269+0000 7fc76fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc770102890 0x7fc77019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34420/0 (socket says 172.21.15.22:34420) 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.269+0000 7fc76fe00640 1 -- 172.21.15.22:0/1090793943 learned_addr learned my addr 172.21.15.22:0/1090793943 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.269+0000 7fc76f400640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc770100ce0 0x7fc770198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.270+0000 7fc76ea00640 1 -- 172.21.15.22:0/1090793943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc770100ce0 msgr2=0x7fc770198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.270+0000 7fc76ea00640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc770100ce0 0x7fc770198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.270+0000 7fc76ea00640 1 -- 172.21.15.22:0/1090793943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc770102890 msgr2=0x7fc77019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.270+0000 7fc76ea00640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc770102890 0x7fc77019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.270+0000 7fc76ea00640 1 -- 172.21.15.22:0/1090793943 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77019fd20 con 0x7fc770101ed0 2024-04-20T21:31:03.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.270+0000 7fc76ea00640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc770101ed0 0x7fc770198640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc75c036040 tx=0x7fc75c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:03.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.270+0000 7fc767e00640 1 -- 172.21.15.22:0/1090793943 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c043020 con 0x7fc770101ed0 2024-04-20T21:31:03.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.270+0000 7fc775200640 1 -- 172.21.15.22:0/1090793943 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77019ffb0 con 0x7fc770101ed0 2024-04-20T21:31:03.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.270+0000 7fc767e00640 1 -- 172.21.15.22:0/1090793943 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75c0054e0 con 0x7fc770101ed0 2024-04-20T21:31:03.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.270+0000 7fc767e00640 1 -- 172.21.15.22:0/1090793943 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c003c30 con 0x7fc770101ed0 2024-04-20T21:31:03.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.271+0000 7fc775200640 1 -- 172.21.15.22:0/1090793943 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7701a04c0 con 0x7fc770101ed0 2024-04-20T21:31:03.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.272+0000 7fc775200640 1 -- 172.21.15.22:0/1090793943 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7701a0690 con 0x7fc770101ed0 2024-04-20T21:31:03.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.274+0000 7fc767e00640 1 -- 172.21.15.22:0/1090793943 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc75c046020 con 0x7fc770101ed0 2024-04-20T21:31:03.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.275+0000 7fc767e00640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc7540a3bb0 0x7fc7540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.275+0000 7fc767e00640 1 -- 172.21.15.22:0/1090793943 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fc75c0f25b0 con 0x7fc770101ed0 2024-04-20T21:31:03.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:02 smithi122 ceph-mon[22147]: Deploying daemon mds.c on smithi122 2024-04-20T21:31:03.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:02 smithi122 ceph-mon[22147]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:03.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:02 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.122:6834/3637338060,v1:172.21.15.122:6835/3637338060] up:boot 2024-04-20T21:31:03.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:02 smithi122 ceph-mon[22147]: fsmap 6 up:standby 2024-04-20T21:31:03.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:02 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:31:03.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.275+0000 7fc76f400640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc7540a3bb0 0x7fc7540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.276+0000 7fc76f400640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc7540a3bb0 0x7fc7540a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc770101d30 tx=0x7fc760005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:03.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.278+0000 7fc767e00640 1 -- 172.21.15.22:0/1090793943 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc75c04d020 con 0x7fc770101ed0 2024-04-20T21:31:03.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:02 smithi033 ceph-mon[21626]: Deploying daemon mds.c on smithi122 2024-04-20T21:31:03.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:02 smithi033 ceph-mon[21626]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:03.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:02 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.122:6834/3637338060,v1:172.21.15.122:6835/3637338060] up:boot 2024-04-20T21:31:03.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:02 smithi033 ceph-mon[21626]: fsmap 6 up:standby 2024-04-20T21:31:03.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:02 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:31:03.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.525+0000 7fc775200640 1 -- 172.21.15.22:0/1090793943 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7701a0a20 con 0x7fc770101ed0 2024-04-20T21:31:03.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.526+0000 7fc767e00640 1 -- 172.21.15.22:0/1090793943 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7fc75c0bf390 con 0x7fc770101ed0 2024-04-20T21:31:03.525 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:03.526 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-20T21:31:03.526 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2024-04-20T21:31:03.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.529+0000 7fc775200640 1 -- 172.21.15.22:0/1090793943 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc7540a3bb0 msgr2=0x7fc7540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.529+0000 7fc775200640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc7540a3bb0 0x7fc7540a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc770101d30 tx=0x7fc760005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:03.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.529+0000 7fc775200640 1 -- 172.21.15.22:0/1090793943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc770101ed0 msgr2=0x7fc770198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.529+0000 7fc775200640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc770101ed0 0x7fc770198640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc75c036040 tx=0x7fc75c02c650 comp rx=0 tx=0).stop 2024-04-20T21:31:03.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.529+0000 7fc775200640 1 -- 172.21.15.22:0/1090793943 shutdown_connections 2024-04-20T21:31:03.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.529+0000 7fc775200640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc7540a3bb0 0x7fc7540a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.529+0000 7fc775200640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc770102890 0x7fc77019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.529+0000 7fc775200640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc770101ed0 0x7fc770198640 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.529+0000 7fc775200640 1 --2- 172.21.15.22:0/1090793943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc770100ce0 0x7fc770198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.529+0000 7fc775200640 1 -- 172.21.15.22:0/1090793943 >> 172.21.15.22:0/1090793943 conn(0x7fc7700fc230 msgr2=0x7fc7700fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:03.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.530+0000 7fc775200640 1 -- 172.21.15.22:0/1090793943 shutdown_connections 2024-04-20T21:31:03.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:03.530+0000 7fc775200640 1 -- 172.21.15.22:0/1090793943 wait complete. 2024-04-20T21:31:03.904 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:31:03 smithi122 systemd[1]: Starting Ceph mds.c for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:31:04.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:03 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1090793943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:04.222 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:03 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1090793943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:04.222 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:31:04 smithi122 podman[52515]: 2024-04-20 21:31:03.958146555 +0000 UTC m=+0.018265017 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:31:04.222 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:31:04 smithi122 podman[52515]: 2024-04-20 21:31:04.224113509 +0000 UTC m=+0.284231986 container create d6f9c5f0e01c9f6149716e80b514258067e1f6f87295451ec8ee4d82407a903b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-c, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:31:04.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:03 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1090793943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:04.516 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:31:04 smithi122 podman[52515]: 2024-04-20 21:31:04.434887932 +0000 UTC m=+0.495006371 container init d6f9c5f0e01c9f6149716e80b514258067e1f6f87295451ec8ee4d82407a903b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-04-20T21:31:04.516 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:31:04 smithi122 podman[52515]: 2024-04-20 21:31:04.437617195 +0000 UTC m=+0.497735638 container start d6f9c5f0e01c9f6149716e80b514258067e1f6f87295451ec8ee4d82407a903b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-20T21:31:04.516 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:31:04 smithi122 ceph-mds[52534]: -- 172.21.15.122:0/955288708 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a24ecd5a40 con 0x55a24eca8800 2024-04-20T21:31:04.517 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:31:04 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-c[52530]: starting mds.c at 2024-04-20T21:31:04.775 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:31:04 smithi122 bash[52515]: d6f9c5f0e01c9f6149716e80b514258067e1f6f87295451ec8ee4d82407a903b 2024-04-20T21:31:04.775 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:31:04 smithi122 systemd[1]: Started Ceph mds.c for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:31:05.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:04 smithi022 ceph-mon[16843]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:05.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:05.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:05.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:05.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:05.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:05.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:04 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:05.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:04 smithi122 ceph-mon[22147]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:05.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:05.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:05.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:05.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:05.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:05.276 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:04 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:05.298 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:04 smithi033 ceph-mon[21626]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:05.298 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:05.298 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:05.299 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:05.299 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:05.299 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:05.299 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:04 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:05.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.782+0000 7f5b60400640 1 -- 172.21.15.22:0/2796684089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b58077b60 msgr2=0x7f5b58077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.782+0000 7f5b60400640 1 --2- 172.21.15.22:0/2796684089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b58077b60 0x7f5b58077fc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009860 tx=0x7f5b4402bb70 comp rx=0 tx=0).stop 2024-04-20T21:31:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.782+0000 7f5b60400640 1 -- 172.21.15.22:0/2796684089 shutdown_connections 2024-04-20T21:31:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.782+0000 7f5b60400640 1 --2- 172.21.15.22:0/2796684089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b58078500 0x7f5b58078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.782+0000 7f5b60400640 1 --2- 172.21.15.22:0/2796684089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b58077b60 0x7f5b58077fc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.782+0000 7f5b60400640 1 --2- 172.21.15.22:0/2796684089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b58076920 0x7f5b58076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.782+0000 7f5b60400640 1 -- 172.21.15.22:0/2796684089 >> 172.21.15.22:0/2796684089 conn(0x7f5b580ff750 msgr2=0x7f5b58101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.782+0000 7f5b60400640 1 -- 172.21.15.22:0/2796684089 shutdown_connections 2024-04-20T21:31:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.782+0000 7f5b60400640 1 -- 172.21.15.22:0/2796684089 wait complete. 2024-04-20T21:31:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.784+0000 7f5b60400640 1 Processor -- start 2024-04-20T21:31:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.784+0000 7f5b60400640 1 -- start start 2024-04-20T21:31:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.784+0000 7f5b60400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b58076920 0x7f5b5819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.784+0000 7f5b60400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b58077b60 0x7f5b5819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.784+0000 7f5b60400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b58078500 0x7f5b581a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.784+0000 7f5b60400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5b58114230 con 0x7f5b58076920 2024-04-20T21:31:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.784+0000 7f5b60400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581140b0 con 0x7f5b58077b60 2024-04-20T21:31:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.784+0000 7f5b60400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581143b0 con 0x7f5b58078500 2024-04-20T21:31:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.785+0000 7f5b5d000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b58077b60 0x7f5b5819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.785+0000 7f5b5d000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b58077b60 0x7f5b5819edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:35208/0 (socket says 172.21.15.22:35208) 2024-04-20T21:31:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.785+0000 7f5b5d000640 1 -- 172.21.15.22:0/3940438025 learned_addr learned my addr 172.21.15.22:0/3940438025 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.785+0000 7f5b5da00640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b58076920 0x7f5b5819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:05.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.785+0000 7f5b5e400640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b58078500 0x7f5b581a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.785+0000 7f5b5d000640 1 -- 172.21.15.22:0/3940438025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b58078500 msgr2=0x7f5b581a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.785+0000 7f5b5d000640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b58078500 0x7f5b581a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.785+0000 7f5b5d000640 1 -- 172.21.15.22:0/3940438025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b58076920 msgr2=0x7f5b5819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.785+0000 7f5b5d000640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b58076920 0x7f5b5819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.785+0000 7f5b5d000640 1 -- 172.21.15.22:0/3940438025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b581a64d0 con 0x7f5b58077b60 2024-04-20T21:31:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.786+0000 7f5b5d000640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b58077b60 0x7f5b5819edf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5b440349e0 tx=0x7f5b44032450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.786+0000 7f5b56000640 1 -- 172.21.15.22:0/3940438025 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44041030 con 0x7f5b58077b60 2024-04-20T21:31:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.786+0000 7f5b56000640 1 -- 172.21.15.22:0/3940438025 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4402c510 con 0x7f5b58077b60 2024-04-20T21:31:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.786+0000 7f5b56000640 1 -- 172.21.15.22:0/3940438025 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44039040 con 0x7f5b58077b60 2024-04-20T21:31:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.786+0000 7f5b60400640 1 -- 172.21.15.22:0/3940438025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b581a6760 con 0x7f5b58077b60 2024-04-20T21:31:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.786+0000 7f5b60400640 1 -- 172.21.15.22:0/3940438025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b581a6c50 con 0x7f5b58077b60 2024-04-20T21:31:05.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.787+0000 7f5b60400640 1 -- 172.21.15.22:0/3940438025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b58110d00 con 0x7f5b58077b60 2024-04-20T21:31:05.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.790+0000 7f5b56000640 1 -- 172.21.15.22:0/3940438025 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b44044070 con 0x7f5b58077b60 2024-04-20T21:31:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.791+0000 7f5b56000640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f5b3c0a3bb0 0x7f5b3c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.792+0000 7f5b56000640 1 -- 172.21.15.22:0/3940438025 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f5b440f15d0 con 0x7f5b58077b60 2024-04-20T21:31:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.792+0000 7f5b5da00640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f5b3c0a3bb0 0x7f5b3c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.792+0000 7f5b5da00640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f5b3c0a3bb0 0x7f5b3c0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b48005fd0 tx=0x7f5b48009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:05.796+0000 7f5b56000640 1 -- 172.21.15.22:0/3940438025 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5b440be3b0 con 0x7f5b58077b60 2024-04-20T21:31:06.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.046+0000 7f5b60400640 1 -- 172.21.15.22:0/3940438025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b58062b60 con 0x7f5b58077b60 2024-04-20T21:31:06.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.047+0000 7f5b56000640 1 -- 172.21.15.22:0/3940438025 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f5b440bdcb0 con 0x7f5b58077b60 2024-04-20T21:31:06.046 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:06.046 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-20T21:31:06.046 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-04-20T21:31:06.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.049+0000 7f5b60400640 1 -- 172.21.15.22:0/3940438025 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f5b3c0a3bb0 msgr2=0x7f5b3c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:06.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.049+0000 7f5b60400640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f5b3c0a3bb0 0x7f5b3c0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b48005fd0 tx=0x7f5b48009500 comp rx=0 tx=0).stop 2024-04-20T21:31:06.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.049+0000 7f5b60400640 1 -- 172.21.15.22:0/3940438025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b58077b60 msgr2=0x7f5b5819edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:06.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.049+0000 7f5b60400640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b58077b60 0x7f5b5819edf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5b440349e0 tx=0x7f5b44032450 comp rx=0 tx=0).stop 2024-04-20T21:31:06.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.049+0000 7f5b60400640 1 -- 172.21.15.22:0/3940438025 shutdown_connections 2024-04-20T21:31:06.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.049+0000 7f5b60400640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f5b3c0a3bb0 0x7f5b3c0a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.049+0000 7f5b60400640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b58078500 0x7f5b581a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.049+0000 7f5b60400640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b58077b60 0x7f5b5819edf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.049+0000 7f5b60400640 1 --2- 172.21.15.22:0/3940438025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b58076920 0x7f5b5819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.049+0000 7f5b60400640 1 -- 172.21.15.22:0/3940438025 >> 172.21.15.22:0/3940438025 conn(0x7f5b580ff750 msgr2=0x7f5b58100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:06.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.050+0000 7f5b60400640 1 -- 172.21.15.22:0/3940438025 shutdown_connections 2024-04-20T21:31:06.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:06.050+0000 7f5b60400640 1 -- 172.21.15.22:0/3940438025 wait complete. 2024-04-20T21:31:06.265 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:05 smithi033 ceph-mon[21626]: Deploying daemon mds.h on smithi033 2024-04-20T21:31:06.265 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:05 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.122:6836/3716866957,v1:172.21.15.122:6837/3716866957] up:boot 2024-04-20T21:31:06.266 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:05 smithi033 ceph-mon[21626]: fsmap 7 up:standby 2024-04-20T21:31:06.266 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:05 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:31:06.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:05 smithi122 ceph-mon[22147]: Deploying daemon mds.h on smithi033 2024-04-20T21:31:06.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:05 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.122:6836/3716866957,v1:172.21.15.122:6837/3716866957] up:boot 2024-04-20T21:31:06.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:05 smithi122 ceph-mon[22147]: fsmap 7 up:standby 2024-04-20T21:31:06.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:05 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:31:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:05 smithi022 ceph-mon[16843]: Deploying daemon mds.h on smithi033 2024-04-20T21:31:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:05 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.122:6836/3716866957,v1:172.21.15.122:6837/3716866957] up:boot 2024-04-20T21:31:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:05 smithi022 ceph-mon[16843]: fsmap 7 up:standby 2024-04-20T21:31:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:05 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:31:06.928 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:31:06 smithi033 systemd[1]: Starting Ceph mds.h for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:31:07.220 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:06 smithi033 ceph-mon[21626]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:07.220 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:06 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3940438025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:07.221 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:31:06 smithi033 podman[51620]: 2024-04-20 21:31:06.944074127 +0000 UTC m=+0.093361009 container create fd7697cbf577a9405a3557b8cb29689be25e5516f7a089b958f15ba285b6a652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-20T21:31:07.221 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:31:06 smithi033 podman[51620]: 2024-04-20 21:31:06.868834341 +0000 UTC m=+0.018121243 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:31:07.221 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:31:07 smithi033 podman[51620]: 2024-04-20 21:31:07.138756442 +0000 UTC m=+0.288043325 container init fd7697cbf577a9405a3557b8cb29689be25e5516f7a089b958f15ba285b6a652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:31:07.221 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:31:07 smithi033 podman[51620]: 2024-04-20 21:31:07.141522375 +0000 UTC m=+0.290809258 container start fd7697cbf577a9405a3557b8cb29689be25e5516f7a089b958f15ba285b6a652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-20T21:31:07.221 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:31:07 smithi033 ceph-mds[51640]: -- 172.21.15.33:0/1608399080 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e30e55fa40 con 0x55e30e532800 2024-04-20T21:31:07.221 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:31:07 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-h[51636]: starting mds.h at 2024-04-20T21:31:07.221 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:31:07 smithi033 bash[51620]: fd7697cbf577a9405a3557b8cb29689be25e5516f7a089b958f15ba285b6a652 2024-04-20T21:31:07.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:06 smithi122 ceph-mon[22147]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:07.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:06 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3940438025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:07.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:06 smithi022 ceph-mon[16843]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:07.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:06 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3940438025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:07.553 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:31:07 smithi033 systemd[1]: Started Ceph mds.h for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:31:08.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:08.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.298+0000 7f6be5c00640 1 -- 172.21.15.22:0/1897563312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6be0101e00 msgr2=0x7f6be0102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:08.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.298+0000 7f6be5c00640 1 --2- 172.21.15.22:0/1897563312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6be0101e00 0x7f6be0102220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009900 tx=0x7f6bd002bc30 comp rx=0 tx=0).stop 2024-04-20T21:31:08.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.299+0000 7f6be5c00640 1 -- 172.21.15.22:0/1897563312 shutdown_connections 2024-04-20T21:31:08.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.299+0000 7f6be5c00640 1 --2- 172.21.15.22:0/1897563312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be01039b0 0x7f6be010a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:08.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.299+0000 7f6be5c00640 1 --2- 172.21.15.22:0/1897563312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6be0102ff0 0x7f6be0103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:08.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.299+0000 7f6be5c00640 1 --2- 172.21.15.22:0/1897563312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6be0101e00 0x7f6be0102220 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:08.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.299+0000 7f6be5c00640 1 -- 172.21.15.22:0/1897563312 >> 172.21.15.22:0/1897563312 conn(0x7f6be00fd370 msgr2=0x7f6be00ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:08.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.299+0000 7f6be5c00640 1 -- 172.21.15.22:0/1897563312 shutdown_connections 2024-04-20T21:31:08.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.299+0000 7f6be5c00640 1 -- 172.21.15.22:0/1897563312 wait complete. 2024-04-20T21:31:08.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.300+0000 7f6be5c00640 1 Processor -- start 2024-04-20T21:31:08.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.300+0000 7f6be5c00640 1 -- start start 2024-04-20T21:31:08.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6be5c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6be0101e00 0x7f6be0075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:08.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6be5c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be0102ff0 0x7f6be0074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:08.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6be5c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6be01039b0 0x7f6be0074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:08.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6be5c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6be010da80 con 0x7f6be0101e00 2024-04-20T21:31:08.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6be5c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6be010d900 con 0x7f6be0102ff0 2024-04-20T21:31:08.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6be5c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6be010dc00 con 0x7f6be01039b0 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6bdfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6be0101e00 0x7f6be0075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6be5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6be01039b0 0x7f6be0074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6bdfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6be0101e00 0x7f6be0075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34456/0 (socket says 172.21.15.22:34456) 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6bdfe00640 1 -- 172.21.15.22:0/987836262 learned_addr learned my addr 172.21.15.22:0/987836262 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6bdf400640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be0102ff0 0x7f6be0074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6be5200640 1 -- 172.21.15.22:0/987836262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be0102ff0 msgr2=0x7f6be0074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6be5200640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be0102ff0 0x7f6be0074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6be5200640 1 -- 172.21.15.22:0/987836262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6be0101e00 msgr2=0x7f6be0075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6be5200640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6be0101e00 0x7f6be0075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6be5200640 1 -- 172.21.15.22:0/987836262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be0075120 con 0x7f6be01039b0 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6bdf400640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be0102ff0 0x7f6be0074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.301+0000 7f6bdfe00640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6be0101e00 0x7f6be0075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.302+0000 7f6be5200640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6be01039b0 0x7f6be0074890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009fd0 tx=0x7f6bd800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:08.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.302+0000 7f6bdcc00640 1 -- 172.21.15.22:0/987836262 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800ce20 con 0x7f6be01039b0 2024-04-20T21:31:08.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.302+0000 7f6bdcc00640 1 -- 172.21.15.22:0/987836262 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bd8005020 con 0x7f6be01039b0 2024-04-20T21:31:08.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.302+0000 7f6be5c00640 1 -- 172.21.15.22:0/987836262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be01a2220 con 0x7f6be01039b0 2024-04-20T21:31:08.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.302+0000 7f6bdcc00640 1 -- 172.21.15.22:0/987836262 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8010720 con 0x7f6be01039b0 2024-04-20T21:31:08.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.302+0000 7f6be5c00640 1 -- 172.21.15.22:0/987836262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be01a2770 con 0x7f6be01039b0 2024-04-20T21:31:08.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.304+0000 7f6be5c00640 1 -- 172.21.15.22:0/987836262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be01091f0 con 0x7f6be01039b0 2024-04-20T21:31:08.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.305+0000 7f6bdcc00640 1 -- 172.21.15.22:0/987836262 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6bd8005540 con 0x7f6be01039b0 2024-04-20T21:31:08.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.307+0000 7f6bdcc00640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f6bc40a3bb0 0x7f6bc40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:08.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.307+0000 7f6bdcc00640 1 -- 172.21.15.22:0/987836262 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f6bd80d0700 con 0x7f6be01039b0 2024-04-20T21:31:08.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.307+0000 7f6bdfe00640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f6bc40a3bb0 0x7f6bc40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:08.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.308+0000 7f6bdfe00640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f6bc40a3bb0 0x7f6bc40a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6bd002c8e0 tx=0x7f6bd0005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:08.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.315+0000 7f6bdcc00640 1 -- 172.21.15.22:0/987836262 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6bd809d4e0 con 0x7f6be01039b0 2024-04-20T21:31:08.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:08.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:08.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:08.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:08.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:08.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:08.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:08 smithi022 ceph-mon[16843]: Deploying daemon mds.i on smithi122 2024-04-20T21:31:08.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:08 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.33:6836/1181941460,v1:172.21.15.33:6837/1181941460] up:boot 2024-04-20T21:31:08.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:08 smithi022 ceph-mon[16843]: fsmap 8 up:standby 2024-04-20T21:31:08.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:08 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:31:08.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:08.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:08 smithi033 ceph-mon[21626]: Deploying daemon mds.i on smithi122 2024-04-20T21:31:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:08 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.33:6836/1181941460,v1:172.21.15.33:6837/1181941460] up:boot 2024-04-20T21:31:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:08 smithi033 ceph-mon[21626]: fsmap 8 up:standby 2024-04-20T21:31:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:08 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:31:08.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.568+0000 7f6be5c00640 1 -- 172.21.15.22:0/987836262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6be0062b60 con 0x7f6be01039b0 2024-04-20T21:31:08.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.570+0000 7f6bdcc00640 1 -- 172.21.15.22:0/987836262 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7f6bd809cde0 con 0x7f6be01039b0 2024-04-20T21:31:08.568 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:08.569 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-20T21:31:08.569 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-04-20T21:31:08.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.572+0000 7f6be5c00640 1 -- 172.21.15.22:0/987836262 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f6bc40a3bb0 msgr2=0x7f6bc40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:08.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.572+0000 7f6be5c00640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f6bc40a3bb0 0x7f6bc40a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6bd002c8e0 tx=0x7f6bd0005b50 comp rx=0 tx=0).stop 2024-04-20T21:31:08.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.572+0000 7f6be5c00640 1 -- 172.21.15.22:0/987836262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6be01039b0 msgr2=0x7f6be0074890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:08.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.572+0000 7f6be5c00640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6be01039b0 0x7f6be0074890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009fd0 tx=0x7f6bd800ef10 comp rx=0 tx=0).stop 2024-04-20T21:31:08.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.572+0000 7f6be5c00640 1 -- 172.21.15.22:0/987836262 shutdown_connections 2024-04-20T21:31:08.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.572+0000 7f6be5c00640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f6bc40a3bb0 0x7f6bc40a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:08.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.572+0000 7f6be5c00640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6be01039b0 0x7f6be0074890 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:08.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.572+0000 7f6be5c00640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6be0102ff0 0x7f6be0074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:08.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.572+0000 7f6be5c00640 1 --2- 172.21.15.22:0/987836262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6be0101e00 0x7f6be0075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:08.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.572+0000 7f6be5c00640 1 -- 172.21.15.22:0/987836262 >> 172.21.15.22:0/987836262 conn(0x7f6be00fd370 msgr2=0x7f6be01063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:08.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.572+0000 7f6be5c00640 1 -- 172.21.15.22:0/987836262 shutdown_connections 2024-04-20T21:31:08.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:08.572+0000 7f6be5c00640 1 -- 172.21.15.22:0/987836262 wait complete. 2024-04-20T21:31:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:08 smithi122 ceph-mon[22147]: Deploying daemon mds.i on smithi122 2024-04-20T21:31:08.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:08 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.33:6836/1181941460,v1:172.21.15.33:6837/1181941460] up:boot 2024-04-20T21:31:08.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:08 smithi122 ceph-mon[22147]: fsmap 8 up:standby 2024-04-20T21:31:08.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:08 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:31:09.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:09 smithi022 ceph-mon[16843]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:09 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/987836262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:09 smithi122 ceph-mon[22147]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:09.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:09 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/987836262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:09.775 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:31:09 smithi122 systemd[1]: Starting Ceph mds.i for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:31:09.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:09 smithi033 ceph-mon[21626]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:09.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:09 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/987836262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:10.180 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:31:09 smithi122 podman[52822]: 2024-04-20 21:31:09.823424832 +0000 UTC m=+0.017729056 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:31:10.180 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:31:09 smithi122 podman[52822]: 2024-04-20 21:31:09.937910024 +0000 UTC m=+0.132214298 container create 18510cab3c10992bd574b17f5582c0cf1db691aa2c023ddaf921a72d8df8b94b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-20T21:31:10.180 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:31:10 smithi122 podman[52822]: 2024-04-20 21:31:10.182137804 +0000 UTC m=+0.376442013 container init 18510cab3c10992bd574b17f5582c0cf1db691aa2c023ddaf921a72d8df8b94b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-20T21:31:10.525 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:31:10 smithi122 podman[52822]: 2024-04-20 21:31:10.184753473 +0000 UTC m=+0.379057681 container start 18510cab3c10992bd574b17f5582c0cf1db691aa2c023ddaf921a72d8df8b94b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:31:10.525 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:31:10 smithi122 ceph-mds[52841]: -- 172.21.15.122:0/2029661026 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bc6708fa40 con 0x55bc67062c00 2024-04-20T21:31:10.525 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:31:10 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-i[52837]: starting mds.i at 2024-04-20T21:31:10.525 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:31:10 smithi122 bash[52822]: 18510cab3c10992bd574b17f5582c0cf1db691aa2c023ddaf921a72d8df8b94b 2024-04-20T21:31:10.525 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:31:10 smithi122 systemd[1]: Started Ceph mds.i for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:31:10.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:10.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.795+0000 7f36bfe00640 1 -- 172.21.15.22:0/1168093060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c0105190 msgr2=0x7f36c0107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.795+0000 7f36bfe00640 1 --2- 172.21.15.22:0/1168093060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c0105190 0x7f36c0107580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009900 tx=0x7f36a802bc50 comp rx=0 tx=0).stop 2024-04-20T21:31:10.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.795+0000 7f36bfe00640 1 -- 172.21.15.22:0/1168093060 shutdown_connections 2024-04-20T21:31:10.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.795+0000 7f36bfe00640 1 --2- 172.21.15.22:0/1168093060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0107ac0 0x7f36c0109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.795+0000 7f36bfe00640 1 --2- 172.21.15.22:0/1168093060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c0105190 0x7f36c0107580 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.795+0000 7f36bfe00640 1 --2- 172.21.15.22:0/1168093060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0102860 0x7f36c0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.795+0000 7f36bfe00640 1 -- 172.21.15.22:0/1168093060 >> 172.21.15.22:0/1168093060 conn(0x7f36c00fc230 msgr2=0x7f36c00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:10.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.795+0000 7f36bfe00640 1 -- 172.21.15.22:0/1168093060 shutdown_connections 2024-04-20T21:31:10.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.795+0000 7f36bfe00640 1 -- 172.21.15.22:0/1168093060 wait complete. 2024-04-20T21:31:10.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.796+0000 7f36bfe00640 1 Processor -- start 2024-04-20T21:31:10.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.797+0000 7f36bfe00640 1 -- start start 2024-04-20T21:31:10.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.797+0000 7f36bfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0102860 0x7f36c0102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.797+0000 7f36bfe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105190 0x7f36c0100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.797+0000 7f36bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c0107ac0 0x7f36c0100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.797+0000 7f36bfe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f36c010dab0 con 0x7f36c0107ac0 2024-04-20T21:31:10.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.797+0000 7f36bfe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f36c010d930 con 0x7f36c0105190 2024-04-20T21:31:10.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.797+0000 7f36bfe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36c010dc30 con 0x7f36c0102860 2024-04-20T21:31:10.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.797+0000 7f36bea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0102860 0x7f36c0102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.797+0000 7f36bf400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c0107ac0 0x7f36c0100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.797+0000 7f36be000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105190 0x7f36c0100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.797+0000 7f36bea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0102860 0x7f36c0102130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33998/0 (socket says 172.21.15.22:33998) 2024-04-20T21:31:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.797+0000 7f36bea00640 1 -- 172.21.15.22:0/905057731 learned_addr learned my addr 172.21.15.22:0/905057731 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.798+0000 7f36be000640 1 -- 172.21.15.22:0/905057731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0102860 msgr2=0x7f36c0102130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.798+0000 7f36be000640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0102860 0x7f36c0102130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.798+0000 7f36be000640 1 -- 172.21.15.22:0/905057731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c0107ac0 msgr2=0x7f36c0100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.798+0000 7f36be000640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c0107ac0 0x7f36c0100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.798+0000 7f36be000640 1 -- 172.21.15.22:0/905057731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c0101550 con 0x7f36c0105190 2024-04-20T21:31:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.798+0000 7f36bea00640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0102860 0x7f36c0102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.798+0000 7f36be000640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105190 0x7f36c0100810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009230 tx=0x7f36a802c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:10.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.798+0000 7f36b7400640 1 -- 172.21.15.22:0/905057731 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8041070 con 0x7f36c0105190 2024-04-20T21:31:10.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.798+0000 7f36b7400640 1 -- 172.21.15.22:0/905057731 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a80054e0 con 0x7f36c0105190 2024-04-20T21:31:10.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.798+0000 7f36bfe00640 1 -- 172.21.15.22:0/905057731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c01a2220 con 0x7f36c0105190 2024-04-20T21:31:10.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.799+0000 7f36b7400640 1 -- 172.21.15.22:0/905057731 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8003e60 con 0x7f36c0105190 2024-04-20T21:31:10.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.799+0000 7f36bfe00640 1 -- 172.21.15.22:0/905057731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c01a2790 con 0x7f36c0105190 2024-04-20T21:31:10.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.800+0000 7f36bfe00640 1 -- 172.21.15.22:0/905057731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c00fdd20 con 0x7f36c0105190 2024-04-20T21:31:10.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.802+0000 7f36b7400640 1 -- 172.21.15.22:0/905057731 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f36a8044070 con 0x7f36c0105190 2024-04-20T21:31:10.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.803+0000 7f36b7400640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f36a40a3c30 0x7f36a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.803+0000 7f36b7400640 1 -- 172.21.15.22:0/905057731 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f36a80f1900 con 0x7f36c0105190 2024-04-20T21:31:10.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.803+0000 7f36bea00640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f36a40a3c30 0x7f36a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.804+0000 7f36bea00640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f36a40a3c30 0x7f36a40a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f36b00095f0 tx=0x7f36b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:10.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:10.807+0000 7f36b7400640 1 -- 172.21.15.22:0/905057731 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f36a80be090 con 0x7f36c0105190 2024-04-20T21:31:11.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.054+0000 7f36bfe00640 1 -- 172.21.15.22:0/905057731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36c0062b60 con 0x7f36c0105190 2024-04-20T21:31:11.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.055+0000 7f36b7400640 1 -- 172.21.15.22:0/905057731 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7f36a80c2390 con 0x7f36c0105190 2024-04-20T21:31:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:11.054 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24568,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T21:31:11.055 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 10 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.057+0000 7f36bfe00640 1 -- 172.21.15.22:0/905057731 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f36a40a3c30 msgr2=0x7f36a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.057+0000 7f36bfe00640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f36a40a3c30 0x7f36a40a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f36b00095f0 tx=0x7f36b0005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.057+0000 7f36bfe00640 1 -- 172.21.15.22:0/905057731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105190 msgr2=0x7f36c0100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.057+0000 7f36bfe00640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105190 0x7f36c0100810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009230 tx=0x7f36a802c670 comp rx=0 tx=0).stop 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.058+0000 7f36bfe00640 1 -- 172.21.15.22:0/905057731 shutdown_connections 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.058+0000 7f36bfe00640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f36a40a3c30 0x7f36a40a60f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.058+0000 7f36bfe00640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c0107ac0 0x7f36c0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.058+0000 7f36bfe00640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105190 0x7f36c0100810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.058+0000 7f36bfe00640 1 --2- 172.21.15.22:0/905057731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0102860 0x7f36c0102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.058+0000 7f36bfe00640 1 -- 172.21.15.22:0/905057731 >> 172.21.15.22:0/905057731 conn(0x7f36c00fc230 msgr2=0x7f36c01031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.058+0000 7f36bfe00640 1 -- 172.21.15.22:0/905057731 shutdown_connections 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:11.058+0000 7f36bfe00640 1 -- 172.21.15.22:0/905057731 wait complete. 2024-04-20T21:31:11.591 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:11.591 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:11.591 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:11.591 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.122:6838/2642727200,v1:172.21.15.122:6839/2642727200] up:boot 2024-04-20T21:31:11.591 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: fsmap 9 up:standby 2024-04-20T21:31:11.591 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:31:11.591 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:31:11.591 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:11.591 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:11.591 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:11.591 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:11.592 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: Deploying daemon mds.g on smithi022 2024-04-20T21:31:11.592 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:11 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/905057731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:11.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:11.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:11.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:11.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.122:6838/2642727200,v1:172.21.15.122:6839/2642727200] up:boot 2024-04-20T21:31:11.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: fsmap 9 up:standby 2024-04-20T21:31:11.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:31:11.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:31:11.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:11.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:11.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:11.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:11.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: Deploying daemon mds.g on smithi022 2024-04-20T21:31:11.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:11 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/905057731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:11.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:11.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:11.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:11.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.122:6838/2642727200,v1:172.21.15.122:6839/2642727200] up:boot 2024-04-20T21:31:11.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: fsmap 9 up:standby 2024-04-20T21:31:11.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:31:11.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:31:11.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:11.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:11.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:11.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:11.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: Deploying daemon mds.g on smithi022 2024-04-20T21:31:11.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:11 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/905057731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:13.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:13.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.313+0000 7fa6cf200640 1 -- 172.21.15.22:0/1060940855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d01039b0 msgr2=0x7fa6d010a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.313+0000 7fa6cf200640 1 --2- 172.21.15.22:0/1060940855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d01039b0 0x7fa6d010a240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009be0 tx=0x7fa6c802bef0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.313+0000 7fa6cf200640 1 -- 172.21.15.22:0/1060940855 shutdown_connections 2024-04-20T21:31:13.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.313+0000 7fa6cf200640 1 --2- 172.21.15.22:0/1060940855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d01039b0 0x7fa6d010a240 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.313+0000 7fa6cf200640 1 --2- 172.21.15.22:0/1060940855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0102ff0 0x7fa6d0103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.313+0000 7fa6cf200640 1 --2- 172.21.15.22:0/1060940855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d0101e00 0x7fa6d0102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.313+0000 7fa6cf200640 1 -- 172.21.15.22:0/1060940855 >> 172.21.15.22:0/1060940855 conn(0x7fa6d00fd370 msgr2=0x7fa6d00ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:13.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.313+0000 7fa6cf200640 1 -- 172.21.15.22:0/1060940855 shutdown_connections 2024-04-20T21:31:13.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.313+0000 7fa6cf200640 1 -- 172.21.15.22:0/1060940855 wait complete. 2024-04-20T21:31:13.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.314+0000 7fa6cf200640 1 Processor -- start 2024-04-20T21:31:13.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.314+0000 7fa6cf200640 1 -- start start 2024-04-20T21:31:13.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.314+0000 7fa6cf200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0101e00 0x7fa6d019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.314+0000 7fa6cf200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d0102ff0 0x7fa6d019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.314+0000 7fa6cf200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d01039b0 0x7fa6d01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.314+0000 7fa6cf200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d010dad0 con 0x7fa6d01039b0 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.314+0000 7fa6cf200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d010d950 con 0x7fa6d0101e00 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.314+0000 7fa6cf200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d010dc50 con 0x7fa6d0102ff0 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.315+0000 7fa6cde00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0101e00 0x7fa6d019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.315+0000 7fa6cde00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0101e00 0x7fa6d019a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:58702/0 (socket says 172.21.15.22:58702) 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.315+0000 7fa6cde00640 1 -- 172.21.15.22:0/1609066210 learned_addr learned my addr 172.21.15.22:0/1609066210 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.315+0000 7fa6c7e00640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d0102ff0 0x7fa6d019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.315+0000 7fa6ce800640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d01039b0 0x7fa6d01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.315+0000 7fa6cde00640 1 -- 172.21.15.22:0/1609066210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d0102ff0 msgr2=0x7fa6d019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.315+0000 7fa6cde00640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d0102ff0 0x7fa6d019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.315+0000 7fa6cde00640 1 -- 172.21.15.22:0/1609066210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d01039b0 msgr2=0x7fa6d01a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.315+0000 7fa6cde00640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d01039b0 0x7fa6d01a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.315+0000 7fa6cde00640 1 -- 172.21.15.22:0/1609066210 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d01a1ef0 con 0x7fa6d0101e00 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.316+0000 7fa6cde00640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0101e00 0x7fa6d019a2d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000e9f0 tx=0x7fa6c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:13.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.316+0000 7fa6c6a00640 1 -- 172.21.15.22:0/1609066210 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c000ccc0 con 0x7fa6d0101e00 2024-04-20T21:31:13.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.316+0000 7fa6c6a00640 1 -- 172.21.15.22:0/1609066210 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c00035f0 con 0x7fa6d0101e00 2024-04-20T21:31:13.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.316+0000 7fa6c6a00640 1 -- 172.21.15.22:0/1609066210 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c000ce60 con 0x7fa6d0101e00 2024-04-20T21:31:13.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.316+0000 7fa6cf200640 1 -- 172.21.15.22:0/1609066210 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d01a21e0 con 0x7fa6d0101e00 2024-04-20T21:31:13.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.316+0000 7fa6cf200640 1 -- 172.21.15.22:0/1609066210 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d01a26e0 con 0x7fa6d0101e00 2024-04-20T21:31:13.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.317+0000 7fa6cf200640 1 -- 172.21.15.22:0/1609066210 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d01a29a0 con 0x7fa6d0101e00 2024-04-20T21:31:13.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.319+0000 7fa6c6a00640 1 -- 172.21.15.22:0/1609066210 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa6c00037a0 con 0x7fa6d0101e00 2024-04-20T21:31:13.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.321+0000 7fa6c6a00640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa6b40a3bb0 0x7fa6b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.321+0000 7fa6c6a00640 1 -- 172.21.15.22:0/1609066210 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fa6c00d02f0 con 0x7fa6d0101e00 2024-04-20T21:31:13.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.321+0000 7fa6c7e00640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa6b40a3bb0 0x7fa6b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.322+0000 7fa6c7e00640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa6b40a3bb0 0x7fa6b40a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa6d019b7e0 tx=0x7fa6b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:13.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.323+0000 7fa6c6a00640 1 -- 172.21.15.22:0/1609066210 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa6c009d0d0 con 0x7fa6d0101e00 2024-04-20T21:31:13.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:13 smithi022 ceph-mon[16843]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:13.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.578+0000 7fa6cf200640 1 -- 172.21.15.22:0/1609066210 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6d01a2b70 con 0x7fa6d0101e00 2024-04-20T21:31:13.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.578+0000 7fa6c6a00640 1 -- 172.21.15.22:0/1609066210 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7fa6c009c9d0 con 0x7fa6d0101e00 2024-04-20T21:31:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:13.578 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24568,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T21:31:13.578 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 10 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.582+0000 7fa6cf200640 1 -- 172.21.15.22:0/1609066210 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa6b40a3bb0 msgr2=0x7fa6b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.582+0000 7fa6cf200640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa6b40a3bb0 0x7fa6b40a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa6d019b7e0 tx=0x7fa6b8005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.582+0000 7fa6cf200640 1 -- 172.21.15.22:0/1609066210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0101e00 msgr2=0x7fa6d019a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.582+0000 7fa6cf200640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0101e00 0x7fa6d019a2d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000e9f0 tx=0x7fa6c000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.582+0000 7fa6cf200640 1 -- 172.21.15.22:0/1609066210 shutdown_connections 2024-04-20T21:31:13.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.582+0000 7fa6cf200640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fa6b40a3bb0 0x7fa6b40a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.582+0000 7fa6cf200640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6d01039b0 0x7fa6d01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.582+0000 7fa6cf200640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d0102ff0 0x7fa6d019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.582+0000 7fa6cf200640 1 --2- 172.21.15.22:0/1609066210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0101e00 0x7fa6d019a2d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.582+0000 7fa6cf200640 1 -- 172.21.15.22:0/1609066210 >> 172.21.15.22:0/1609066210 conn(0x7fa6d00fd370 msgr2=0x7fa6d0074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:13.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.582+0000 7fa6cf200640 1 -- 172.21.15.22:0/1609066210 shutdown_connections 2024-04-20T21:31:13.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:13.582+0000 7fa6cf200640 1 -- 172.21.15.22:0/1609066210 wait complete. 2024-04-20T21:31:13.676 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:13 smithi122 ceph-mon[22147]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:13.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:13 smithi033 ceph-mon[21626]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:14.157 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:31:13 smithi022 podman[55779]: 2024-04-20 21:31:13.803154088 +0000 UTC m=+0.035113106 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:31:14.157 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:31:14 smithi022 podman[55779]: 2024-04-20 21:31:14.157794202 +0000 UTC m=+0.389753263 container init 84dd5dfa8f035ae5915f92dc7288fed0b8334d5e0242f98f8a669367f16704c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-20T21:31:14.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:14 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1609066210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:14.657 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:31:14 smithi022 podman[55779]: 2024-04-20 21:31:14.168006896 +0000 UTC m=+0.399965914 container start 84dd5dfa8f035ae5915f92dc7288fed0b8334d5e0242f98f8a669367f16704c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-20T21:31:14.657 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:31:14 smithi022 ceph-mds[55798]: -- 172.21.15.22:0/2552507154 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b7281b1a40 con 0x55b728184c00 2024-04-20T21:31:14.657 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:31:14 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-g[55794]: starting mds.g at 2024-04-20T21:31:14.657 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:31:14 smithi022 bash[55779]: 84dd5dfa8f035ae5915f92dc7288fed0b8334d5e0242f98f8a669367f16704c2 2024-04-20T21:31:14.657 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:31:14 smithi022 systemd[1]: Started Ceph mds.g for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:31:14.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:14 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1609066210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:14 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1609066210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:15.607 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:15 smithi022 ceph-mon[16843]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:15 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.22:6838/4122889060,v1:172.21.15.22:6839/4122889060] up:boot 2024-04-20T21:31:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:15 smithi022 ceph-mon[16843]: fsmap 10 up:standby 2024-04-20T21:31:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T21:31:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:15 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:15.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:15 smithi022 ceph-mon[16843]: Deploying daemon mds.a on smithi022 2024-04-20T21:31:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:15 smithi122 ceph-mon[22147]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:15 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.22:6838/4122889060,v1:172.21.15.22:6839/4122889060] up:boot 2024-04-20T21:31:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:15 smithi122 ceph-mon[22147]: fsmap 10 up:standby 2024-04-20T21:31:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T21:31:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:15.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:15 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:15.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:15 smithi122 ceph-mon[22147]: Deploying daemon mds.a on smithi022 2024-04-20T21:31:15.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:15 smithi033 ceph-mon[21626]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:15.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:15 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.22:6838/4122889060,v1:172.21.15.22:6839/4122889060] up:boot 2024-04-20T21:31:15.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:15 smithi033 ceph-mon[21626]: fsmap 10 up:standby 2024-04-20T21:31:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T21:31:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:15 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:15.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:15 smithi033 ceph-mon[21626]: Deploying daemon mds.a on smithi022 2024-04-20T21:31:15.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.835+0000 7fcafc800640 1 -- 172.21.15.22:0/2319000895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf41041e0 msgr2=0x7fcaf4104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:15.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.835+0000 7fcafc800640 1 --2- 172.21.15.22:0/2319000895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf41041e0 0x7fcaf4104600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009860 tx=0x7fcae802bb70 comp rx=0 tx=0).stop 2024-04-20T21:31:15.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.835+0000 7fcafc800640 1 -- 172.21.15.22:0/2319000895 shutdown_connections 2024-04-20T21:31:15.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.835+0000 7fcafc800640 1 --2- 172.21.15.22:0/2319000895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf4105d90 0x7fcaf410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.835+0000 7fcafc800640 1 --2- 172.21.15.22:0/2319000895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf41053d0 0x7fcaf4105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.835+0000 7fcafc800640 1 --2- 172.21.15.22:0/2319000895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf41041e0 0x7fcaf4104600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.835+0000 7fcafc800640 1 -- 172.21.15.22:0/2319000895 >> 172.21.15.22:0/2319000895 conn(0x7fcaf40ff750 msgr2=0x7fcaf4101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:15.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.836+0000 7fcafc800640 1 -- 172.21.15.22:0/2319000895 shutdown_connections 2024-04-20T21:31:15.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.836+0000 7fcafc800640 1 -- 172.21.15.22:0/2319000895 wait complete. 2024-04-20T21:31:15.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.837+0000 7fcafc800640 1 Processor -- start 2024-04-20T21:31:15.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.838+0000 7fcafc800640 1 -- start start 2024-04-20T21:31:15.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.838+0000 7fcafc800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf41041e0 0x7fcaf419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:15.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.838+0000 7fcafc800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf41053d0 0x7fcaf419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:15.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.838+0000 7fcafc800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf4105d90 0x7fcaf41a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.838+0000 7fcafc800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf410fe40 con 0x7fcaf41041e0 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.838+0000 7fcafc800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf410fcc0 con 0x7fcaf41053d0 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.838+0000 7fcafc800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf410ffc0 con 0x7fcaf4105d90 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.838+0000 7fcaf9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf41041e0 0x7fcaf419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.838+0000 7fcaf9400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf41053d0 0x7fcaf419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.838+0000 7fcaf9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf41041e0 0x7fcaf419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58882/0 (socket says 172.21.15.22:58882) 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.838+0000 7fcaf9e00640 1 -- 172.21.15.22:0/1988215044 learned_addr learned my addr 172.21.15.22:0/1988215044 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.838+0000 7fcafa800640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf4105d90 0x7fcaf41a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.839+0000 7fcaf9400640 1 -- 172.21.15.22:0/1988215044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf4105d90 msgr2=0x7fcaf41a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.839+0000 7fcaf9400640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf4105d90 0x7fcaf41a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.839+0000 7fcaf9400640 1 -- 172.21.15.22:0/1988215044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf41041e0 msgr2=0x7fcaf419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.839+0000 7fcaf9400640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf41041e0 0x7fcaf419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.839+0000 7fcaf9400640 1 -- 172.21.15.22:0/1988215044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf41a2100 con 0x7fcaf41053d0 2024-04-20T21:31:15.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.839+0000 7fcaf9400640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf41053d0 0x7fcaf419aa20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcae400e790 tx=0x7fcae400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:15.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.839+0000 7fcaeea00640 1 -- 172.21.15.22:0/1988215044 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae400eea0 con 0x7fcaf41053d0 2024-04-20T21:31:15.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.839+0000 7fcaeea00640 1 -- 172.21.15.22:0/1988215044 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae400ccc0 con 0x7fcaf41053d0 2024-04-20T21:31:15.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.839+0000 7fcafc800640 1 -- 172.21.15.22:0/1988215044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf41a23f0 con 0x7fcaf41053d0 2024-04-20T21:31:15.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.839+0000 7fcaeea00640 1 -- 172.21.15.22:0/1988215044 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4018780 con 0x7fcaf41053d0 2024-04-20T21:31:15.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.840+0000 7fcafc800640 1 -- 172.21.15.22:0/1988215044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf41a2940 con 0x7fcaf41053d0 2024-04-20T21:31:15.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.841+0000 7fcafc800640 1 -- 172.21.15.22:0/1988215044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf410b580 con 0x7fcaf41053d0 2024-04-20T21:31:15.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.844+0000 7fcaeea00640 1 -- 172.21.15.22:0/1988215044 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcae4010430 con 0x7fcaf41053d0 2024-04-20T21:31:15.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.845+0000 7fcaeea00640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fcadc0a3bb0 0x7fcadc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:15.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.845+0000 7fcaeea00640 1 -- 172.21.15.22:0/1988215044 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fcae40d0150 con 0x7fcaf41053d0 2024-04-20T21:31:15.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.845+0000 7fcaf9e00640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fcadc0a3bb0 0x7fcadc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:15.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.846+0000 7fcaf9e00640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fcadc0a3bb0 0x7fcadc0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcae80095f0 tx=0x7fcae8034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:15.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:15.849+0000 7fcaeea00640 1 -- 172.21.15.22:0/1988215044 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcae409c8e0 con 0x7fcaf41053d0 2024-04-20T21:31:16.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.100+0000 7fcafc800640 1 -- 172.21.15.22:0/1988215044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaf402d5d0 con 0x7fcaf41053d0 2024-04-20T21:31:16.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.101+0000 7fcaeea00640 1 -- 172.21.15.22:0/1988215044 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7fcae40a0be0 con 0x7fcaf41053d0 2024-04-20T21:31:16.100 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:16.100 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24568,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T21:31:16.100 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 11 2024-04-20T21:31:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.104+0000 7fcafc800640 1 -- 172.21.15.22:0/1988215044 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fcadc0a3bb0 msgr2=0x7fcadc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.104+0000 7fcafc800640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fcadc0a3bb0 0x7fcadc0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcae80095f0 tx=0x7fcae8034ad0 comp rx=0 tx=0).stop 2024-04-20T21:31:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.104+0000 7fcafc800640 1 -- 172.21.15.22:0/1988215044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf41053d0 msgr2=0x7fcaf419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.104+0000 7fcafc800640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf41053d0 0x7fcaf419aa20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcae400e790 tx=0x7fcae400ec50 comp rx=0 tx=0).stop 2024-04-20T21:31:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.104+0000 7fcafc800640 1 -- 172.21.15.22:0/1988215044 shutdown_connections 2024-04-20T21:31:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.104+0000 7fcafc800640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fcadc0a3bb0 0x7fcadc0a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.104+0000 7fcafc800640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf4105d90 0x7fcaf41a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.104+0000 7fcafc800640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf41053d0 0x7fcaf419aa20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.104+0000 7fcafc800640 1 --2- 172.21.15.22:0/1988215044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf41041e0 0x7fcaf419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.104+0000 7fcafc800640 1 -- 172.21.15.22:0/1988215044 >> 172.21.15.22:0/1988215044 conn(0x7fcaf40ff750 msgr2=0x7fcaf4108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.104+0000 7fcafc800640 1 -- 172.21.15.22:0/1988215044 shutdown_connections 2024-04-20T21:31:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:16.105+0000 7fcafc800640 1 -- 172.21.15.22:0/1988215044 wait complete. 2024-04-20T21:31:16.632 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:16 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1988215044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:16.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:16 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1988215044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:16 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1988215044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:17.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:17 smithi022 ceph-mon[16843]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:17 smithi122 ceph-mon[22147]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:17.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:17 smithi033 ceph-mon[21626]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:17.903 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:31:17 smithi022 systemd[1]: Starting Ceph mds.a for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:31:18.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:18.156 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:31:17 smithi022 podman[56127]: 2024-04-20 21:31:17.809420271 +0000 UTC m=+0.039772546 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:31:18.156 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:31:17 smithi022 podman[56127]: 2024-04-20 21:31:17.93340818 +0000 UTC m=+0.163760493 container create 9c4294888507995bd1473a6fbaa9e55ea8ee00a749262e6e84fb1de21ae1ad3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-20T21:31:18.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.344+0000 7fc3eaa00640 1 -- 172.21.15.22:0/2434982239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3e4103b60 msgr2=0x7fc3e410a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:18.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.344+0000 7fc3eaa00640 1 --2- 172.21.15.22:0/2434982239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3e4103b60 0x7fc3e410a3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009ca0 tx=0x7fc3dc02c100 comp rx=0 tx=0).stop 2024-04-20T21:31:18.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.344+0000 7fc3eaa00640 1 -- 172.21.15.22:0/2434982239 shutdown_connections 2024-04-20T21:31:18.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.344+0000 7fc3eaa00640 1 --2- 172.21.15.22:0/2434982239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3e4103b60 0x7fc3e410a3f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.344+0000 7fc3eaa00640 1 --2- 172.21.15.22:0/2434982239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e41031a0 0x7fc3e4103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.344+0000 7fc3eaa00640 1 --2- 172.21.15.22:0/2434982239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3e4101fb0 0x7fc3e41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.344+0000 7fc3eaa00640 1 -- 172.21.15.22:0/2434982239 >> 172.21.15.22:0/2434982239 conn(0x7fc3e40fd540 msgr2=0x7fc3e40ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:18.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.344+0000 7fc3eaa00640 1 -- 172.21.15.22:0/2434982239 shutdown_connections 2024-04-20T21:31:18.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.344+0000 7fc3eaa00640 1 -- 172.21.15.22:0/2434982239 wait complete. 2024-04-20T21:31:18.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.345+0000 7fc3eaa00640 1 Processor -- start 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.345+0000 7fc3eaa00640 1 -- start start 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.346+0000 7fc3eaa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3e4101fb0 0x7fc3e4075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.346+0000 7fc3eaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e41031a0 0x7fc3e4074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.346+0000 7fc3eaa00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3e4103b60 0x7fc3e4074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.346+0000 7fc3eaa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e410dc00 con 0x7fc3e4101fb0 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.346+0000 7fc3eaa00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e410da80 con 0x7fc3e4103b60 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.346+0000 7fc3eaa00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e410dd80 con 0x7fc3e41031a0 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.346+0000 7fc3e3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3e4101fb0 0x7fc3e4075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.346+0000 7fc3e3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3e4101fb0 0x7fc3e4075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58888/0 (socket says 172.21.15.22:58888) 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.346+0000 7fc3e3e00640 1 -- 172.21.15.22:0/3778043860 learned_addr learned my addr 172.21.15.22:0/3778043860 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.346+0000 7fc3e3400640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e41031a0 0x7fc3e4074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.346+0000 7fc3e8a00640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3e4103b60 0x7fc3e4074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.347+0000 7fc3e3e00640 1 -- 172.21.15.22:0/3778043860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e41031a0 msgr2=0x7fc3e4074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:18.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.347+0000 7fc3e3e00640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e41031a0 0x7fc3e4074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.347+0000 7fc3e3e00640 1 -- 172.21.15.22:0/3778043860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3e4103b60 msgr2=0x7fc3e4074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:18.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.347+0000 7fc3e3e00640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3e4103b60 0x7fc3e4074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.347+0000 7fc3e3e00640 1 -- 172.21.15.22:0/3778043860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e4075120 con 0x7fc3e4101fb0 2024-04-20T21:31:18.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.347+0000 7fc3e3e00640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3e4101fb0 0x7fc3e4075d10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400d950 tx=0x7fc3d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:18.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.347+0000 7fc3e0c00640 1 -- 172.21.15.22:0/3778043860 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4017020 con 0x7fc3e4101fb0 2024-04-20T21:31:18.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.347+0000 7fc3eaa00640 1 -- 172.21.15.22:0/3778043860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e41a2390 con 0x7fc3e4101fb0 2024-04-20T21:31:18.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.347+0000 7fc3e0c00640 1 -- 172.21.15.22:0/3778043860 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d4009c30 con 0x7fc3e4101fb0 2024-04-20T21:31:18.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.347+0000 7fc3e0c00640 1 -- 172.21.15.22:0/3778043860 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d400f6b0 con 0x7fc3e4101fb0 2024-04-20T21:31:18.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.347+0000 7fc3eaa00640 1 -- 172.21.15.22:0/3778043860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e41a2930 con 0x7fc3e4101fb0 2024-04-20T21:31:18.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.348+0000 7fc3eaa00640 1 -- 172.21.15.22:0/3778043860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e41092d0 con 0x7fc3e4101fb0 2024-04-20T21:31:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.351+0000 7fc3e0c00640 1 -- 172.21.15.22:0/3778043860 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc3d401a020 con 0x7fc3e4101fb0 2024-04-20T21:31:18.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.352+0000 7fc3e0c00640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc3c80a3bb0 0x7fc3c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:18.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.353+0000 7fc3e0c00640 1 -- 172.21.15.22:0/3778043860 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fc3d40cff90 con 0x7fc3e4101fb0 2024-04-20T21:31:18.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.353+0000 7fc3e3400640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc3c80a3bb0 0x7fc3c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:18.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.354+0000 7fc3e3400640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc3c80a3bb0 0x7fc3c80a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4075a80 tx=0x7fc3d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.358+0000 7fc3e0c00640 1 -- 172.21.15.22:0/3778043860 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc3d4021020 con 0x7fc3e4101fb0 2024-04-20T21:31:18.451 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:31:18 smithi022 podman[56127]: 2024-04-20 21:31:18.370167445 +0000 UTC m=+0.600519738 container init 9c4294888507995bd1473a6fbaa9e55ea8ee00a749262e6e84fb1de21ae1ad3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-04-20T21:31:18.451 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:31:18 smithi022 podman[56127]: 2024-04-20 21:31:18.379531952 +0000 UTC m=+0.609884228 container start 9c4294888507995bd1473a6fbaa9e55ea8ee00a749262e6e84fb1de21ae1ad3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:31:18.451 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:31:18 smithi022 ceph-mds[56184]: -- 172.21.15.22:0/3162996869 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e2d2a7a40 con 0x563e2d27b000 2024-04-20T21:31:18.452 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:31:18 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-a[56141]: starting mds.a at 2024-04-20T21:31:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.631+0000 7fc3eaa00640 1 -- 172.21.15.22:0/3778043860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3e402d5d0 con 0x7fc3e4101fb0 2024-04-20T21:31:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.632+0000 7fc3e0c00640 1 -- 172.21.15.22:0/3778043860 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7fc3d409cd70 con 0x7fc3e4101fb0 2024-04-20T21:31:18.631 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:18.631 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24568,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T21:31:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 11 2024-04-20T21:31:18.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.635+0000 7fc3eaa00640 1 -- 172.21.15.22:0/3778043860 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc3c80a3bb0 msgr2=0x7fc3c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:18.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.635+0000 7fc3eaa00640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc3c80a3bb0 0x7fc3c80a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4075a80 tx=0x7fc3d0005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:18.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.635+0000 7fc3eaa00640 1 -- 172.21.15.22:0/3778043860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3e4101fb0 msgr2=0x7fc3e4075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:18.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.635+0000 7fc3eaa00640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3e4101fb0 0x7fc3e4075d10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400d950 tx=0x7fc3d400de10 comp rx=0 tx=0).stop 2024-04-20T21:31:18.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.635+0000 7fc3eaa00640 1 -- 172.21.15.22:0/3778043860 shutdown_connections 2024-04-20T21:31:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.635+0000 7fc3eaa00640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc3c80a3bb0 0x7fc3c80a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.635+0000 7fc3eaa00640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3e4103b60 0x7fc3e4074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.635+0000 7fc3eaa00640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e41031a0 0x7fc3e4074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.635+0000 7fc3eaa00640 1 --2- 172.21.15.22:0/3778043860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3e4101fb0 0x7fc3e4075d10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.635+0000 7fc3eaa00640 1 -- 172.21.15.22:0/3778043860 >> 172.21.15.22:0/3778043860 conn(0x7fc3e40fd540 msgr2=0x7fc3e41064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.635+0000 7fc3eaa00640 1 -- 172.21.15.22:0/3778043860 shutdown_connections 2024-04-20T21:31:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:18.635+0000 7fc3eaa00640 1 -- 172.21.15.22:0/3778043860 wait complete. 2024-04-20T21:31:18.906 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:31:18 smithi022 bash[56127]: 9c4294888507995bd1473a6fbaa9e55ea8ee00a749262e6e84fb1de21ae1ad3b 2024-04-20T21:31:18.907 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:31:18 smithi022 systemd[1]: Started Ceph mds.a for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:31:19.581 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:19 smithi033 ceph-mon[21626]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:19.581 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:19 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:19.581 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:19 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:19.581 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:19 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3778043860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:19.581 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:19 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:19.582 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:19 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:19.582 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:19 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:19.582 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:19 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:19 smithi022 ceph-mon[16843]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:19 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3778043860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:19.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:19.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:19.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:19 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:19.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:19 smithi122 ceph-mon[22147]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:19.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:19 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:19.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:19 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:19.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:19 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3778043860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:19.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:19 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:19.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:19 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:31:19.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:19 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:31:19.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:19 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:20.477 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:20 smithi033 ceph-mon[21626]: Deploying daemon mds.b on smithi033 2024-04-20T21:31:20.478 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:20 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.22:6840/2604771291,v1:172.21.15.22:6841/2604771291] up:boot 2024-04-20T21:31:20.478 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:20 smithi033 ceph-mon[21626]: fsmap 11 up:standby 2024-04-20T21:31:20.478 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:20 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T21:31:20.656 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:20.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:20 smithi022 ceph-mon[16843]: Deploying daemon mds.b on smithi033 2024-04-20T21:31:20.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:20 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.22:6840/2604771291,v1:172.21.15.22:6841/2604771291] up:boot 2024-04-20T21:31:20.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:20 smithi022 ceph-mon[16843]: fsmap 11 up:standby 2024-04-20T21:31:20.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:20 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T21:31:20.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:20 smithi122 ceph-mon[22147]: Deploying daemon mds.b on smithi033 2024-04-20T21:31:20.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:20 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.22:6840/2604771291,v1:172.21.15.22:6841/2604771291] up:boot 2024-04-20T21:31:20.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:20 smithi122 ceph-mon[22147]: fsmap 11 up:standby 2024-04-20T21:31:20.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:20 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T21:31:20.803 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:31:20 smithi033 systemd[1]: Starting Ceph mds.b for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:31:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.887+0000 7f58d6200640 1 -- 172.21.15.22:0/1428259720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d0101e00 msgr2=0x7f58d0102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.887+0000 7f58d6200640 1 --2- 172.21.15.22:0/1428259720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d0101e00 0x7f58d0102220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f58c0009900 tx=0x7f58c002bc30 comp rx=0 tx=0).stop 2024-04-20T21:31:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.887+0000 7f58d6200640 1 -- 172.21.15.22:0/1428259720 shutdown_connections 2024-04-20T21:31:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.887+0000 7f58d6200640 1 --2- 172.21.15.22:0/1428259720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d01039b0 0x7f58d010a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.887+0000 7f58d6200640 1 --2- 172.21.15.22:0/1428259720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d0102ff0 0x7f58d0103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.887+0000 7f58d6200640 1 --2- 172.21.15.22:0/1428259720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d0101e00 0x7f58d0102220 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.887+0000 7f58d6200640 1 -- 172.21.15.22:0/1428259720 >> 172.21.15.22:0/1428259720 conn(0x7f58d00fd370 msgr2=0x7f58d00ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:20.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.887+0000 7f58d6200640 1 -- 172.21.15.22:0/1428259720 shutdown_connections 2024-04-20T21:31:20.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.887+0000 7f58d6200640 1 -- 172.21.15.22:0/1428259720 wait complete. 2024-04-20T21:31:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.888+0000 7f58d6200640 1 Processor -- start 2024-04-20T21:31:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.888+0000 7f58d6200640 1 -- start start 2024-04-20T21:31:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.888+0000 7f58d6200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d0101e00 0x7f58d019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.888+0000 7f58d6200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0102ff0 0x7f58d019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.888+0000 7f58d6200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d01039b0 0x7f58d01a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.888+0000 7f58d6200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010daa0 con 0x7f58d0102ff0 2024-04-20T21:31:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.888+0000 7f58d6200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010d920 con 0x7f58d0101e00 2024-04-20T21:31:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.888+0000 7f58d6200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010dc20 con 0x7f58d01039b0 2024-04-20T21:31:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.889+0000 7f58d4e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d0101e00 0x7f58d019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.889+0000 7f58d4e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d0101e00 0x7f58d019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:58822/0 (socket says 172.21.15.22:58822) 2024-04-20T21:31:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.889+0000 7f58d4e00640 1 -- 172.21.15.22:0/1282325624 learned_addr learned my addr 172.21.15.22:0/1282325624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.889+0000 7f58c7e00640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0102ff0 0x7f58d019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.889+0000 7f58d5800640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d01039b0 0x7f58d01a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.889+0000 7f58d4e00640 1 -- 172.21.15.22:0/1282325624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d01039b0 msgr2=0x7f58d01a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.889+0000 7f58d4e00640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d01039b0 0x7f58d01a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.889+0000 7f58d4e00640 1 -- 172.21.15.22:0/1282325624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0102ff0 msgr2=0x7f58d019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.889+0000 7f58d4e00640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0102ff0 0x7f58d019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.889+0000 7f58d4e00640 1 -- 172.21.15.22:0/1282325624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d01a1f80 con 0x7f58d0101e00 2024-04-20T21:31:20.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.890+0000 7f58d4e00640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d0101e00 0x7f58d019a400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f58c0000c00 tx=0x7f58c002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:20.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.890+0000 7f58ce000640 1 -- 172.21.15.22:0/1282325624 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0043020 con 0x7f58d0101e00 2024-04-20T21:31:20.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.890+0000 7f58ce000640 1 -- 172.21.15.22:0/1282325624 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c00054e0 con 0x7f58d0101e00 2024-04-20T21:31:20.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.890+0000 7f58ce000640 1 -- 172.21.15.22:0/1282325624 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0003c40 con 0x7f58d0101e00 2024-04-20T21:31:20.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.890+0000 7f58d6200640 1 -- 172.21.15.22:0/1282325624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d01a2210 con 0x7f58d0101e00 2024-04-20T21:31:20.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.890+0000 7f58d6200640 1 -- 172.21.15.22:0/1282325624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d01a2710 con 0x7f58d0101e00 2024-04-20T21:31:20.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.891+0000 7f58d6200640 1 -- 172.21.15.22:0/1282325624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d01a2910 con 0x7f58d0101e00 2024-04-20T21:31:20.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.893+0000 7f58ce000640 1 -- 172.21.15.22:0/1282325624 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f58c0046020 con 0x7f58d0101e00 2024-04-20T21:31:20.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.895+0000 7f58ce000640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f58b40a3c30 0x7f58b40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:20.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.895+0000 7f58ce000640 1 -- 172.21.15.22:0/1282325624 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f58c00f2630 con 0x7f58d0101e00 2024-04-20T21:31:20.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.895+0000 7f58c7e00640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f58b40a3c30 0x7f58b40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:20.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.896+0000 7f58c7e00640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f58b40a3c30 0x7f58b40a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f58b80095f0 tx=0x7f58b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:20.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:20.897+0000 7f58ce000640 1 -- 172.21.15.22:0/1282325624 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f58c004d020 con 0x7f58d0101e00 2024-04-20T21:31:21.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.144+0000 7f58d6200640 1 -- 172.21.15.22:0/1282325624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58d01a2ae0 con 0x7f58d0101e00 2024-04-20T21:31:21.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.145+0000 7f58ce000640 1 -- 172.21.15.22:0/1282325624 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f58c00bf410 con 0x7f58d0101e00 2024-04-20T21:31:21.144 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:21.144 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24568,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T21:31:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 12 2024-04-20T21:31:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.148+0000 7f58d6200640 1 -- 172.21.15.22:0/1282325624 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f58b40a3c30 msgr2=0x7f58b40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:21.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.148+0000 7f58d6200640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f58b40a3c30 0x7f58b40a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f58b80095f0 tx=0x7f58b8005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:21.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.148+0000 7f58d6200640 1 -- 172.21.15.22:0/1282325624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d0101e00 msgr2=0x7f58d019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:21.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.148+0000 7f58d6200640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d0101e00 0x7f58d019a400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f58c0000c00 tx=0x7f58c002c650 comp rx=0 tx=0).stop 2024-04-20T21:31:21.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.148+0000 7f58d6200640 1 -- 172.21.15.22:0/1282325624 shutdown_connections 2024-04-20T21:31:21.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.148+0000 7f58d6200640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f58b40a3c30 0x7f58b40a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:21.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.148+0000 7f58d6200640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d01039b0 0x7f58d01a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:21.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.148+0000 7f58d6200640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58d0102ff0 0x7f58d019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:21.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.148+0000 7f58d6200640 1 --2- 172.21.15.22:0/1282325624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d0101e00 0x7f58d019a400 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:21.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.148+0000 7f58d6200640 1 -- 172.21.15.22:0/1282325624 >> 172.21.15.22:0/1282325624 conn(0x7f58d00fd370 msgr2=0x7f58d0074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:21.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.149+0000 7f58d6200640 1 -- 172.21.15.22:0/1282325624 shutdown_connections 2024-04-20T21:31:21.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:21.149+0000 7f58d6200640 1 -- 172.21.15.22:0/1282325624 wait complete. 2024-04-20T21:31:21.199 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:31:20 smithi033 podman[51927]: 2024-04-20 21:31:20.840761472 +0000 UTC m=+0.018501270 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:31:21.200 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:31:20 smithi033 podman[51927]: 2024-04-20 21:31:20.938417005 +0000 UTC m=+0.116156790 container create ac595150a4206b95bd3afe73304d384f80202452b2a59474322011e3fb6f013e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-20T21:31:21.200 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:31:21 smithi033 podman[51927]: 2024-04-20 21:31:21.174312517 +0000 UTC m=+0.352052303 container init ac595150a4206b95bd3afe73304d384f80202452b2a59474322011e3fb6f013e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-20T21:31:21.200 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:31:21 smithi033 podman[51927]: 2024-04-20 21:31:21.177070159 +0000 UTC m=+0.354809945 container start ac595150a4206b95bd3afe73304d384f80202452b2a59474322011e3fb6f013e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:31:21.200 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:31:21 smithi033 ceph-mds[51946]: -- 172.21.15.33:0/657509998 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6ce02da40 con 0x55b6ce000c00 2024-04-20T21:31:21.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:21 smithi033 ceph-mon[21626]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:21.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:21 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1282325624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:21.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:21 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:21.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:21 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:21.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:21 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:21.554 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:31:21 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b[51942]: starting mds.b at 2024-04-20T21:31:21.554 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:31:21 smithi033 bash[51927]: ac595150a4206b95bd3afe73304d384f80202452b2a59474322011e3fb6f013e 2024-04-20T21:31:21.554 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:31:21 smithi033 systemd[1]: Started Ceph mds.b for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:31:21.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:21 smithi022 ceph-mon[16843]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:21.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:21 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1282325624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:21.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:21 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:21.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:21 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:21.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:21 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:21.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:21 smithi122 ceph-mon[22147]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:21.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:21 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1282325624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:21.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:21 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:21.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:21 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:21.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:21 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:22.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:22 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:22.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:22 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] up:boot 2024-04-20T21:31:22.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:22 smithi122 ceph-mon[22147]: fsmap 12 up:standby 2024-04-20T21:31:22.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:22 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:31:22.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:22 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:22.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:22 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:31:22.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:22 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:22.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:22 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] up:boot 2024-04-20T21:31:22.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:22 smithi033 ceph-mon[21626]: fsmap 12 up:standby 2024-04-20T21:31:22.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:22 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:31:22.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:22 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:22.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:22 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:31:22.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:22 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:22.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:22 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] up:boot 2024-04-20T21:31:22.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:22 smithi022 ceph-mon[16843]: fsmap 12 up:standby 2024-04-20T21:31:22.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:22 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:31:22.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:22 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:22.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:22 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:31:23.167 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:23.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.371+0000 7f385d800640 1 -- 172.21.15.22:0/3676167475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3858105190 msgr2=0x7f3858107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:23.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.371+0000 7f385d800640 1 --2- 172.21.15.22:0/3676167475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3858105190 0x7f3858107580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3840009900 tx=0x7f384002bc40 comp rx=0 tx=0).stop 2024-04-20T21:31:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.371+0000 7f385d800640 1 -- 172.21.15.22:0/3676167475 shutdown_connections 2024-04-20T21:31:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.371+0000 7f385d800640 1 --2- 172.21.15.22:0/3676167475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3858107ac0 0x7f3858109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.371+0000 7f385d800640 1 --2- 172.21.15.22:0/3676167475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3858105190 0x7f3858107580 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.371+0000 7f385d800640 1 --2- 172.21.15.22:0/3676167475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3858102860 0x7f3858104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.371+0000 7f385d800640 1 -- 172.21.15.22:0/3676167475 >> 172.21.15.22:0/3676167475 conn(0x7f38580fc230 msgr2=0x7f38580fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.371+0000 7f385d800640 1 -- 172.21.15.22:0/3676167475 shutdown_connections 2024-04-20T21:31:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.371+0000 7f385d800640 1 -- 172.21.15.22:0/3676167475 wait complete. 2024-04-20T21:31:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.372+0000 7f385d800640 1 Processor -- start 2024-04-20T21:31:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.372+0000 7f385d800640 1 -- start start 2024-04-20T21:31:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f385d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3858102860 0x7f385819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f385d800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3858105190 0x7f385819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f385d800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3858107ac0 0x7f38581a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f385d800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f385810daa0 con 0x7f3858105190 2024-04-20T21:31:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f385d800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f385810d920 con 0x7f3858107ac0 2024-04-20T21:31:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f385d800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f385810dc20 con 0x7f3858102860 2024-04-20T21:31:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f3857e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3858102860 0x7f385819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f384f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3858105190 0x7f385819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f3857e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3858102860 0x7f385819a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:39012/0 (socket says 172.21.15.22:39012) 2024-04-20T21:31:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f3857e00640 1 -- 172.21.15.22:0/789963615 learned_addr learned my addr 172.21.15.22:0/789963615 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f385ce00640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3858107ac0 0x7f38581a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f3857e00640 1 -- 172.21.15.22:0/789963615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3858107ac0 msgr2=0x7f38581a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f3857e00640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3858107ac0 0x7f38581a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f3857e00640 1 -- 172.21.15.22:0/789963615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3858105190 msgr2=0x7f385819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f3857e00640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3858105190 0x7f385819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.373+0000 7f3857e00640 1 -- 172.21.15.22:0/789963615 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38581a1f30 con 0x7f3858102860 2024-04-20T21:31:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.374+0000 7f3857e00640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3858102860 0x7f385819a310 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f384800eab0 tx=0x7f384800ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.374+0000 7f3855600640 1 -- 172.21.15.22:0/789963615 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384800ce40 con 0x7f3858102860 2024-04-20T21:31:23.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.374+0000 7f385d800640 1 -- 172.21.15.22:0/789963615 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38581a2220 con 0x7f3858102860 2024-04-20T21:31:23.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.374+0000 7f3855600640 1 -- 172.21.15.22:0/789963615 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3848011950 con 0x7f3858102860 2024-04-20T21:31:23.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.374+0000 7f3855600640 1 -- 172.21.15.22:0/789963615 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848010ae0 con 0x7f3858102860 2024-04-20T21:31:23.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.374+0000 7f385d800640 1 -- 172.21.15.22:0/789963615 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38581a2740 con 0x7f3858102860 2024-04-20T21:31:23.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.375+0000 7f385d800640 1 -- 172.21.15.22:0/789963615 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3858109610 con 0x7f3858102860 2024-04-20T21:31:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.378+0000 7f3855600640 1 -- 172.21.15.22:0/789963615 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3848011b00 con 0x7f3858102860 2024-04-20T21:31:23.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.379+0000 7f3855600640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f383c0a3c00 0x7f383c0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:23.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.379+0000 7f3855600640 1 -- 172.21.15.22:0/789963615 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f38480d0850 con 0x7f3858102860 2024-04-20T21:31:23.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.379+0000 7f384f400640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f383c0a3c00 0x7f383c0a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:23.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.380+0000 7f384f400640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f383c0a3c00 0x7f383c0a60c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3840009850 tx=0x7f3840000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:23.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.381+0000 7f3855600640 1 -- 172.21.15.22:0/789963615 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3848016030 con 0x7f3858102860 2024-04-20T21:31:23.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.664+0000 7f385d800640 1 -- 172.21.15.22:0/789963615 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f385802d5d0 con 0x7f3858102860 2024-04-20T21:31:23.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.665+0000 7f3855600640 1 -- 172.21.15.22:0/789963615 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7f384809d630 con 0x7f3858102860 2024-04-20T21:31:23.665 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:23.665 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24568,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-20T21:31:23.665 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 13 2024-04-20T21:31:23.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.668+0000 7f385d800640 1 -- 172.21.15.22:0/789963615 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f383c0a3c00 msgr2=0x7f383c0a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:23.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.668+0000 7f385d800640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f383c0a3c00 0x7f383c0a60c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3840009850 tx=0x7f3840000bc0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.668+0000 7f385d800640 1 -- 172.21.15.22:0/789963615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3858102860 msgr2=0x7f385819a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:23.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.668+0000 7f385d800640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3858102860 0x7f385819a310 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f384800eab0 tx=0x7f384800ef70 comp rx=0 tx=0).stop 2024-04-20T21:31:23.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.668+0000 7f385d800640 1 -- 172.21.15.22:0/789963615 shutdown_connections 2024-04-20T21:31:23.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.668+0000 7f385d800640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f383c0a3c00 0x7f383c0a60c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.668+0000 7f385d800640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3858107ac0 0x7f38581a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.668+0000 7f385d800640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3858105190 0x7f385819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.668+0000 7f385d800640 1 --2- 172.21.15.22:0/789963615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3858102860 0x7f385819a310 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.668+0000 7f385d800640 1 -- 172.21.15.22:0/789963615 >> 172.21.15.22:0/789963615 conn(0x7f38580fc230 msgr2=0x7f3858101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:23.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.668+0000 7f385d800640 1 -- 172.21.15.22:0/789963615 shutdown_connections 2024-04-20T21:31:23.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.668+0000 7f385d800640 1 -- 172.21.15.22:0/789963615 wait complete. 2024-04-20T21:31:23.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:23 smithi022 ceph-mon[16843]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:23.690 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:31:23.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:23 smithi122 ceph-mon[22147]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:23.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:23 smithi033 ceph-mon[21626]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.876+0000 7f8163200640 1 -- 172.21.15.22:0/2585492467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f815c078500 msgr2=0x7f815c078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.876+0000 7f8163200640 1 --2- 172.21.15.22:0/2585492467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f815c078500 0x7f815c078da0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8154009ca0 tx=0x7f815402c0f0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.876+0000 7f8163200640 1 -- 172.21.15.22:0/2585492467 shutdown_connections 2024-04-20T21:31:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.876+0000 7f8163200640 1 --2- 172.21.15.22:0/2585492467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f815c078500 0x7f815c078da0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.876+0000 7f8163200640 1 --2- 172.21.15.22:0/2585492467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f815c077b60 0x7f815c077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.876+0000 7f8163200640 1 --2- 172.21.15.22:0/2585492467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f815c076920 0x7f815c076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.876+0000 7f8163200640 1 -- 172.21.15.22:0/2585492467 >> 172.21.15.22:0/2585492467 conn(0x7f815c0ff750 msgr2=0x7f815c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.876+0000 7f8163200640 1 -- 172.21.15.22:0/2585492467 shutdown_connections 2024-04-20T21:31:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.876+0000 7f8163200640 1 -- 172.21.15.22:0/2585492467 wait complete. 2024-04-20T21:31:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.877+0000 7f8163200640 1 Processor -- start 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.877+0000 7f8163200640 1 -- start start 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f8163200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f815c076920 0x7f815c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f8163200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f815c077b60 0x7f815c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f8163200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f815c078500 0x7f815c1a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f8163200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f815c114250 con 0x7f815c077b60 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f8163200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1140d0 con 0x7f815c076920 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f8163200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1143d0 con 0x7f815c078500 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f815be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f815c076920 0x7f815c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f815b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f815c077b60 0x7f815c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f815be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f815c076920 0x7f815c19e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39232/0 (socket says 172.21.15.22:39232) 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f815be00640 1 -- 172.21.15.22:0/1155228145 learned_addr learned my addr 172.21.15.22:0/1155228145 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f8161200640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f815c078500 0x7f815c1a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f815be00640 1 -- 172.21.15.22:0/1155228145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f815c078500 msgr2=0x7f815c1a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:23.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f815be00640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f815c078500 0x7f815c1a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f815be00640 1 -- 172.21.15.22:0/1155228145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f815c077b60 msgr2=0x7f815c19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:23.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f815be00640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f815c077b60 0x7f815c19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:23.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.878+0000 7f815be00640 1 -- 172.21.15.22:0/1155228145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c1a64c0 con 0x7f815c076920 2024-04-20T21:31:23.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.879+0000 7f815be00640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f815c076920 0x7f815c19e8a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f814c00e9f0 tx=0x7f814c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:23.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.879+0000 7f8158c00640 1 -- 172.21.15.22:0/1155228145 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c00cd80 con 0x7f815c076920 2024-04-20T21:31:23.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.879+0000 7f8158c00640 1 -- 172.21.15.22:0/1155228145 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f814c010910 con 0x7f815c076920 2024-04-20T21:31:23.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.879+0000 7f8158c00640 1 -- 172.21.15.22:0/1155228145 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c018990 con 0x7f815c076920 2024-04-20T21:31:23.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.879+0000 7f8163200640 1 -- 172.21.15.22:0/1155228145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c1a67b0 con 0x7f815c076920 2024-04-20T21:31:23.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.879+0000 7f8163200640 1 -- 172.21.15.22:0/1155228145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c1a6cb0 con 0x7f815c076920 2024-04-20T21:31:23.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.880+0000 7f8163200640 1 -- 172.21.15.22:0/1155228145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815c110cc0 con 0x7f815c076920 2024-04-20T21:31:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.882+0000 7f8158c00640 1 -- 172.21.15.22:0/1155228145 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f814c010ac0 con 0x7f815c076920 2024-04-20T21:31:23.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.884+0000 7f8158c00640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f81400a3c30 0x7f81400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:23.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.884+0000 7f8158c00640 1 -- 172.21.15.22:0/1155228145 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f814c0d0270 con 0x7f815c076920 2024-04-20T21:31:23.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.884+0000 7f815b400640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f81400a3c30 0x7f81400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:23.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.885+0000 7f815b400640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f81400a3c30 0x7f81400a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f815c19fdb0 tx=0x7f814800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:23.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:23.888+0000 7f8158c00640 1 -- 172.21.15.22:0/1155228145 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f814c09ca00 con 0x7f815c076920 2024-04-20T21:31:24.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.101+0000 7f8163200640 1 -- 172.21.15.22:0/1155228145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f815c062b90 con 0x7f815c076920 2024-04-20T21:31:24.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.103+0000 7f8158c00640 1 -- 172.21.15.22:0/1155228145 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18433 (secure 0 0 0) 0x7f814c0a0d00 con 0x7f815c076920 2024-04-20T21:31:24.102 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:24.102 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":84,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:30:30.588294+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:31:24.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.105+0000 7f8163200640 1 -- 172.21.15.22:0/1155228145 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f81400a3c30 msgr2=0x7f81400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:24.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.105+0000 7f8163200640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f81400a3c30 0x7f81400a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f815c19fdb0 tx=0x7f814800b390 comp rx=0 tx=0).stop 2024-04-20T21:31:24.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.105+0000 7f8163200640 1 -- 172.21.15.22:0/1155228145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f815c076920 msgr2=0x7f815c19e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:24.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.105+0000 7f8163200640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f815c076920 0x7f815c19e8a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f814c00e9f0 tx=0x7f814c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.105+0000 7f8163200640 1 -- 172.21.15.22:0/1155228145 shutdown_connections 2024-04-20T21:31:24.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.105+0000 7f8163200640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f81400a3c30 0x7f81400a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.105+0000 7f8163200640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f815c078500 0x7f815c1a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.105+0000 7f8163200640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f815c077b60 0x7f815c19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.105+0000 7f8163200640 1 --2- 172.21.15.22:0/1155228145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f815c076920 0x7f815c19e8a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.105+0000 7f8163200640 1 -- 172.21.15.22:0/1155228145 >> 172.21.15.22:0/1155228145 conn(0x7f815c0ff750 msgr2=0x7f815c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:24.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.105+0000 7f8163200640 1 -- 172.21.15.22:0/1155228145 shutdown_connections 2024-04-20T21:31:24.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.105+0000 7f8163200640 1 -- 172.21.15.22:0/1155228145 wait complete. 2024-04-20T21:31:24.128 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:26:15.688818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:31:24.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:31:24.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.325+0000 7fdb44e00640 1 -- 172.21.15.22:0/760493684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb40103180 msgr2=0x7fdb401055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.325+0000 7fdb44e00640 1 --2- 172.21.15.22:0/760493684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb40103180 0x7fdb401055f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009860 tx=0x7fdb2c02bb70 comp rx=0 tx=0).stop 2024-04-20T21:31:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.325+0000 7fdb44e00640 1 -- 172.21.15.22:0/760493684 shutdown_connections 2024-04-20T21:31:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.325+0000 7fdb44e00640 1 --2- 172.21.15.22:0/760493684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb40105b30 0x7fdb40105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.325+0000 7fdb44e00640 1 --2- 172.21.15.22:0/760493684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb40103180 0x7fdb401055f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.325+0000 7fdb44e00640 1 --2- 172.21.15.22:0/760493684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb40100810 0x7fdb40102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.325+0000 7fdb44e00640 1 -- 172.21.15.22:0/760493684 >> 172.21.15.22:0/760493684 conn(0x7fdb400fc410 msgr2=0x7fdb400fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.325+0000 7fdb44e00640 1 -- 172.21.15.22:0/760493684 shutdown_connections 2024-04-20T21:31:24.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.325+0000 7fdb44e00640 1 -- 172.21.15.22:0/760493684 wait complete. 2024-04-20T21:31:24.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.326+0000 7fdb44e00640 1 Processor -- start 2024-04-20T21:31:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.326+0000 7fdb44e00640 1 -- start start 2024-04-20T21:31:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb44e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb40100810 0x7fdb4019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb44e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb40103180 0x7fdb4019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb44e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb40105b30 0x7fdb401a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb44e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdb401141f0 con 0x7fdb40100810 2024-04-20T21:31:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb44e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb40114070 con 0x7fdb40103180 2024-04-20T21:31:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb44e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdb40114370 con 0x7fdb40105b30 2024-04-20T21:31:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb3d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb40103180 0x7fdb4019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb3d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb40103180 0x7fdb4019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39238/0 (socket says 172.21.15.22:39238) 2024-04-20T21:31:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb3d400640 1 -- 172.21.15.22:0/3896500078 learned_addr learned my addr 172.21.15.22:0/3896500078 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:24.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb3de00640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb40100810 0x7fdb4019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb3e800640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb40105b30 0x7fdb401a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb3d400640 1 -- 172.21.15.22:0/3896500078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb40105b30 msgr2=0x7fdb401a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb3d400640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb40105b30 0x7fdb401a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb3d400640 1 -- 172.21.15.22:0/3896500078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb40100810 msgr2=0x7fdb4019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb3d400640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb40100810 0x7fdb4019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.327+0000 7fdb3d400640 1 -- 172.21.15.22:0/3896500078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb401a64c0 con 0x7fdb40103180 2024-04-20T21:31:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.328+0000 7fdb3d400640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb40103180 0x7fdb4019ede0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0349e0 tx=0x7fdb2c032450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.328+0000 7fdb36a00640 1 -- 172.21.15.22:0/3896500078 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c041030 con 0x7fdb40103180 2024-04-20T21:31:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.328+0000 7fdb36a00640 1 -- 172.21.15.22:0/3896500078 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb2c02c510 con 0x7fdb40103180 2024-04-20T21:31:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.328+0000 7fdb44e00640 1 -- 172.21.15.22:0/3896500078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb401a6750 con 0x7fdb40103180 2024-04-20T21:31:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.328+0000 7fdb36a00640 1 -- 172.21.15.22:0/3896500078 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c039040 con 0x7fdb40103180 2024-04-20T21:31:24.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.328+0000 7fdb44e00640 1 -- 172.21.15.22:0/3896500078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb401a6c40 con 0x7fdb40103180 2024-04-20T21:31:24.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.330+0000 7fdb44e00640 1 -- 172.21.15.22:0/3896500078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb400fdf00 con 0x7fdb40103180 2024-04-20T21:31:24.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.331+0000 7fdb36a00640 1 -- 172.21.15.22:0/3896500078 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdb2c044070 con 0x7fdb40103180 2024-04-20T21:31:24.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.333+0000 7fdb36a00640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdb240a3bb0 0x7fdb240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:24.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.333+0000 7fdb36a00640 1 -- 172.21.15.22:0/3896500078 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fdb2c0f1520 con 0x7fdb40103180 2024-04-20T21:31:24.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.333+0000 7fdb3de00640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdb240a3bb0 0x7fdb240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:24.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.334+0000 7fdb3de00640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdb240a3bb0 0x7fdb240a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009bd0 tx=0x7fdb30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:24.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.337+0000 7fdb36a00640 1 -- 172.21.15.22:0/3896500078 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdb2c0be300 con 0x7fdb40103180 2024-04-20T21:31:24.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.561+0000 7fdb44e00640 1 -- 172.21.15.22:0/3896500078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdb40062b60 con 0x7fdb40103180 2024-04-20T21:31:24.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.562+0000 7fdb36a00640 1 -- 172.21.15.22:0/3896500078 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fdb2c0bdc00 con 0x7fdb40103180 2024-04-20T21:31:24.561 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-20T21:31:24.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.564+0000 7fdb44e00640 1 -- 172.21.15.22:0/3896500078 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdb240a3bb0 msgr2=0x7fdb240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:24.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.564+0000 7fdb44e00640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdb240a3bb0 0x7fdb240a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009bd0 tx=0x7fdb30005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:24.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.564+0000 7fdb44e00640 1 -- 172.21.15.22:0/3896500078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb40103180 msgr2=0x7fdb4019ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:24.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.564+0000 7fdb44e00640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb40103180 0x7fdb4019ede0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0349e0 tx=0x7fdb2c032450 comp rx=0 tx=0).stop 2024-04-20T21:31:24.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.564+0000 7fdb44e00640 1 -- 172.21.15.22:0/3896500078 shutdown_connections 2024-04-20T21:31:24.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.564+0000 7fdb44e00640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fdb240a3bb0 0x7fdb240a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.564+0000 7fdb44e00640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb40105b30 0x7fdb401a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.564+0000 7fdb44e00640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb40103180 0x7fdb4019ede0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.564+0000 7fdb44e00640 1 --2- 172.21.15.22:0/3896500078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb40100810 0x7fdb4019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.564+0000 7fdb44e00640 1 -- 172.21.15.22:0/3896500078 >> 172.21.15.22:0/3896500078 conn(0x7fdb400fc410 msgr2=0x7fdb401011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:24.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.564+0000 7fdb44e00640 1 -- 172.21.15.22:0/3896500078 shutdown_connections 2024-04-20T21:31:24.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.564+0000 7fdb44e00640 1 -- 172.21.15.22:0/3896500078 wait complete. 2024-04-20T21:31:24.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T21:31:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.771+0000 7f52e1200640 1 -- 172.21.15.22:0/2494387138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52dc076920 msgr2=0x7f52dc076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.771+0000 7f52e1200640 1 --2- 172.21.15.22:0/2494387138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52dc076920 0x7f52dc076d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009ca0 tx=0x7f52cc02c0f0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.772+0000 7f52e1200640 1 -- 172.21.15.22:0/2494387138 shutdown_connections 2024-04-20T21:31:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.772+0000 7f52e1200640 1 --2- 172.21.15.22:0/2494387138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc078500 0x7f52dc078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.772+0000 7f52e1200640 1 --2- 172.21.15.22:0/2494387138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52dc077b60 0x7f52dc077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.772+0000 7f52e1200640 1 --2- 172.21.15.22:0/2494387138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52dc076920 0x7f52dc076d40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.772+0000 7f52e1200640 1 -- 172.21.15.22:0/2494387138 >> 172.21.15.22:0/2494387138 conn(0x7f52dc0ff570 msgr2=0x7f52dc101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:24.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.772+0000 7f52e1200640 1 -- 172.21.15.22:0/2494387138 shutdown_connections 2024-04-20T21:31:24.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.772+0000 7f52e1200640 1 -- 172.21.15.22:0/2494387138 wait complete. 2024-04-20T21:31:24.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.773+0000 7f52e1200640 1 Processor -- start 2024-04-20T21:31:24.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.773+0000 7f52e1200640 1 -- start start 2024-04-20T21:31:24.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.773+0000 7f52e1200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc076920 0x7f52dc19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:24.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.773+0000 7f52e1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52dc077b60 0x7f52dc19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:24.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.773+0000 7f52e1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52dc078500 0x7f52dc1a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:24.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.773+0000 7f52e1200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc114030 con 0x7f52dc078500 2024-04-20T21:31:24.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.773+0000 7f52e1200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc113eb0 con 0x7f52dc076920 2024-04-20T21:31:24.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.773+0000 7f52e1200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1141b0 con 0x7f52dc077b60 2024-04-20T21:31:24.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.774+0000 7f52db400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc076920 0x7f52dc19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:24.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.774+0000 7f52dbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52dc078500 0x7f52dc1a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:24.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.774+0000 7f52d2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52dc077b60 0x7f52dc19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:24.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.774+0000 7f52db400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc076920 0x7f52dc19e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39264/0 (socket says 172.21.15.22:39264) 2024-04-20T21:31:24.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.774+0000 7f52db400640 1 -- 172.21.15.22:0/3910183000 learned_addr learned my addr 172.21.15.22:0/3910183000 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:24.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.774+0000 7f52db400640 1 -- 172.21.15.22:0/3910183000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52dc077b60 msgr2=0x7f52dc19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:24.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.774+0000 7f52db400640 1 --2- 172.21.15.22:0/3910183000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52dc077b60 0x7f52dc19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.774+0000 7f52db400640 1 -- 172.21.15.22:0/3910183000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52dc078500 msgr2=0x7f52dc1a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:24.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.774+0000 7f52db400640 1 --2- 172.21.15.22:0/3910183000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52dc078500 0x7f52dc1a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:24.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.774+0000 7f52db400640 1 -- 172.21.15.22:0/3910183000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc1a6300 con 0x7f52dc076920 2024-04-20T21:31:24.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:24 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:24.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:24 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:24.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:24 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:24.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:24 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:24.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:24 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/789963615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:24.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:24 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1155228145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.774+0000 7f52db400640 1 --2- 172.21.15.22:0/3910183000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc076920 0x7f52dc19e6e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0325f0 tx=0x7f52cc032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.775+0000 7f52d8c00640 1 -- 172.21.15.22:0/3910183000 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc041020 con 0x7f52dc076920 2024-04-20T21:31:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.775+0000 7f52d8c00640 1 -- 172.21.15.22:0/3910183000 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52cc034cd0 con 0x7f52dc076920 2024-04-20T21:31:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.775+0000 7f52e1200640 1 -- 172.21.15.22:0/3910183000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52dc1a6590 con 0x7f52dc076920 2024-04-20T21:31:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.775+0000 7f52d8c00640 1 -- 172.21.15.22:0/3910183000 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc039040 con 0x7f52dc076920 2024-04-20T21:31:24.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.775+0000 7f52e1200640 1 -- 172.21.15.22:0/3910183000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52dc1a6a50 con 0x7f52dc076920 2024-04-20T21:31:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.776+0000 7f52e1200640 1 -- 172.21.15.22:0/3910183000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52dc110ad0 con 0x7f52dc076920 2024-04-20T21:31:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.778+0000 7f52d8c00640 1 -- 172.21.15.22:0/3910183000 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f52cc044020 con 0x7f52dc076920 2024-04-20T21:31:24.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.779+0000 7f52d8c00640 1 --2- 172.21.15.22:0/3910183000 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f52c00a3bb0 0x7f52c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:24.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.779+0000 7f52d8c00640 1 -- 172.21.15.22:0/3910183000 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f52cc0f1ed0 con 0x7f52dc076920 2024-04-20T21:31:24.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.780+0000 7f52d2a00640 1 --2- 172.21.15.22:0/3910183000 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f52c00a3bb0 0x7f52c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:24.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.780+0000 7f52d2a00640 1 --2- 172.21.15.22:0/3910183000 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f52c00a3bb0 0x7f52c00a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f52dc19fbf0 tx=0x7f52c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:24.783+0000 7f52d8c00640 1 -- 172.21.15.22:0/3910183000 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f52cc04b020 con 0x7f52dc076920 2024-04-20T21:31:24.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:24 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:24.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:24 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:24.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:24 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:24.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:24 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:24.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:24 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/789963615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:24.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:24 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1155228145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:24.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:24 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:24.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:24 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:24.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:24 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:24.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:24 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:24.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:24 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/789963615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:24.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:24 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1155228145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:25.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.035+0000 7f52e1200640 1 -- 172.21.15.22:0/3910183000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f52dc062b60 con 0x7f52dc076920 2024-04-20T21:31:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.036+0000 7f52d8c00640 1 -- 172.21.15.22:0/3910183000 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) ==== 84+0+65 (secure 0 0 0) 0x7f52cc0becb0 con 0x7f52dc076920 2024-04-20T21:31:25.035 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:25.035 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-04-20T21:31:25.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:31:25.035 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-04-20T21:31:25.035 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-04-20T21:31:25.035 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:31:25.035 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-04-20T21:31:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.038+0000 7f52e1200640 1 -- 172.21.15.22:0/3910183000 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f52c00a3bb0 msgr2=0x7f52c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.038+0000 7f52e1200640 1 --2- 172.21.15.22:0/3910183000 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f52c00a3bb0 0x7f52c00a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f52dc19fbf0 tx=0x7f52c400b410 comp rx=0 tx=0).stop 2024-04-20T21:31:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.038+0000 7f52e1200640 1 -- 172.21.15.22:0/3910183000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc076920 msgr2=0x7f52dc19e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.038+0000 7f52e1200640 1 --2- 172.21.15.22:0/3910183000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc076920 0x7f52dc19e6e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0325f0 tx=0x7f52cc032ec0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.038+0000 7f52e1200640 1 -- 172.21.15.22:0/3910183000 shutdown_connections 2024-04-20T21:31:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.038+0000 7f52e1200640 1 --2- 172.21.15.22:0/3910183000 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f52c00a3bb0 0x7f52c00a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.038+0000 7f52e1200640 1 --2- 172.21.15.22:0/3910183000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52dc078500 0x7f52dc1a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.038+0000 7f52e1200640 1 --2- 172.21.15.22:0/3910183000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52dc077b60 0x7f52dc19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.038+0000 7f52e1200640 1 --2- 172.21.15.22:0/3910183000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc076920 0x7f52dc19e6e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.038+0000 7f52e1200640 1 -- 172.21.15.22:0/3910183000 >> 172.21.15.22:0/3910183000 conn(0x7f52dc0ff570 msgr2=0x7f52dc100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.038+0000 7f52e1200640 1 -- 172.21.15.22:0/3910183000 shutdown_connections 2024-04-20T21:31:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.039+0000 7f52e1200640 1 -- 172.21.15.22:0/3910183000 wait complete. 2024-04-20T21:31:25.056 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T21:31:25.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T21:31:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.270+0000 7fef77c00640 1 -- 172.21.15.22:0/47920845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef701041e0 msgr2=0x7fef70104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.270+0000 7fef77c00640 1 --2- 172.21.15.22:0/47920845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef701041e0 0x7fef70104600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fef60009ca0 tx=0x7fef6002c100 comp rx=0 tx=0).stop 2024-04-20T21:31:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.270+0000 7fef77c00640 1 -- 172.21.15.22:0/47920845 shutdown_connections 2024-04-20T21:31:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.270+0000 7fef77c00640 1 --2- 172.21.15.22:0/47920845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef70105d90 0x7fef7010c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.270+0000 7fef77c00640 1 --2- 172.21.15.22:0/47920845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef701053d0 0x7fef70105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.270+0000 7fef77c00640 1 --2- 172.21.15.22:0/47920845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef701041e0 0x7fef70104600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.270+0000 7fef77c00640 1 -- 172.21.15.22:0/47920845 >> 172.21.15.22:0/47920845 conn(0x7fef700ff750 msgr2=0x7fef70101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.270+0000 7fef77c00640 1 -- 172.21.15.22:0/47920845 shutdown_connections 2024-04-20T21:31:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.270+0000 7fef77c00640 1 -- 172.21.15.22:0/47920845 wait complete. 2024-04-20T21:31:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.271+0000 7fef77c00640 1 Processor -- start 2024-04-20T21:31:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.272+0000 7fef77c00640 1 -- start start 2024-04-20T21:31:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.272+0000 7fef77c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef701041e0 0x7fef7019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.272+0000 7fef77c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef701053d0 0x7fef7019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.272+0000 7fef77c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef70105d90 0x7fef701a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.272+0000 7fef77c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fef7010fe70 con 0x7fef70105d90 2024-04-20T21:31:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.272+0000 7fef77c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fef7010fcf0 con 0x7fef701041e0 2024-04-20T21:31:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.272+0000 7fef77c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fef7010fff0 con 0x7fef701053d0 2024-04-20T21:31:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.272+0000 7fef75200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef701041e0 0x7fef7019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.272+0000 7fef6fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef701053d0 0x7fef7019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.272+0000 7fef75200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef701041e0 0x7fef7019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39282/0 (socket says 172.21.15.22:39282) 2024-04-20T21:31:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.272+0000 7fef75200640 1 -- 172.21.15.22:0/4182457915 learned_addr learned my addr 172.21.15.22:0/4182457915 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:25.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.272+0000 7fef75c00640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef70105d90 0x7fef701a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.273+0000 7fef6fe00640 1 -- 172.21.15.22:0/4182457915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef701041e0 msgr2=0x7fef7019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.273+0000 7fef6fe00640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef701041e0 0x7fef7019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.273+0000 7fef6fe00640 1 -- 172.21.15.22:0/4182457915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef70105d90 msgr2=0x7fef701a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.273+0000 7fef6fe00640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef70105d90 0x7fef701a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.273+0000 7fef6fe00640 1 -- 172.21.15.22:0/4182457915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef701a2110 con 0x7fef701053d0 2024-04-20T21:31:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.273+0000 7fef75200640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef701041e0 0x7fef7019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.273+0000 7fef6fe00640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef701053d0 0x7fef7019aa30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fef58005fd0 tx=0x7fef5800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.273+0000 7fef6d600640 1 -- 172.21.15.22:0/4182457915 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58009c30 con 0x7fef701053d0 2024-04-20T21:31:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.273+0000 7fef6d600640 1 -- 172.21.15.22:0/4182457915 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef58009dd0 con 0x7fef701053d0 2024-04-20T21:31:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.273+0000 7fef6d600640 1 -- 172.21.15.22:0/4182457915 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58010460 con 0x7fef701053d0 2024-04-20T21:31:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.273+0000 7fef77c00640 1 -- 172.21.15.22:0/4182457915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef701a2400 con 0x7fef701053d0 2024-04-20T21:31:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.273+0000 7fef77c00640 1 -- 172.21.15.22:0/4182457915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef701a2950 con 0x7fef701053d0 2024-04-20T21:31:25.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.275+0000 7fef77c00640 1 -- 172.21.15.22:0/4182457915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7010b580 con 0x7fef701053d0 2024-04-20T21:31:25.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.276+0000 7fef6d600640 1 -- 172.21.15.22:0/4182457915 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fef58003f50 con 0x7fef701053d0 2024-04-20T21:31:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.278+0000 7fef6d600640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fef540a3bb0 0x7fef540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.278+0000 7fef6d600640 1 -- 172.21.15.22:0/4182457915 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fef580cff30 con 0x7fef701053d0 2024-04-20T21:31:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.278+0000 7fef75200640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fef540a3bb0 0x7fef540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.279+0000 7fef75200640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fef540a3bb0 0x7fef540a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef60005e30 tx=0x7fef60004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:25.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.283+0000 7fef6d600640 1 -- 172.21.15.22:0/4182457915 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fef58017020 con 0x7fef701053d0 2024-04-20T21:31:25.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.494+0000 7fef77c00640 1 -- 172.21.15.22:0/4182457915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fef70062b60 con 0x7fef701053d0 2024-04-20T21:31:25.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.657+0000 7fef6d600640 1 -- 172.21.15.22:0/4182457915 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) ==== 138+0+0 (secure 0 0 0) 0x7fef5809c6c0 con 0x7fef701053d0 2024-04-20T21:31:25.657 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-04-20T21:31:25.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.663+0000 7fef77c00640 1 -- 172.21.15.22:0/4182457915 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fef540a3bb0 msgr2=0x7fef540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.663+0000 7fef77c00640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fef540a3bb0 0x7fef540a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef60005e30 tx=0x7fef60004a40 comp rx=0 tx=0).stop 2024-04-20T21:31:25.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.663+0000 7fef77c00640 1 -- 172.21.15.22:0/4182457915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef701053d0 msgr2=0x7fef7019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.663+0000 7fef77c00640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef701053d0 0x7fef7019aa30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fef58005fd0 tx=0x7fef5800dde0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.664+0000 7fef77c00640 1 -- 172.21.15.22:0/4182457915 shutdown_connections 2024-04-20T21:31:25.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.664+0000 7fef77c00640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fef540a3bb0 0x7fef540a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.664+0000 7fef77c00640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef70105d90 0x7fef701a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.664+0000 7fef77c00640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef701053d0 0x7fef7019aa30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.664+0000 7fef77c00640 1 --2- 172.21.15.22:0/4182457915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef701041e0 0x7fef7019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.664+0000 7fef77c00640 1 -- 172.21.15.22:0/4182457915 >> 172.21.15.22:0/4182457915 conn(0x7fef700ff750 msgr2=0x7fef70108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:25.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.664+0000 7fef77c00640 1 -- 172.21.15.22:0/4182457915 shutdown_connections 2024-04-20T21:31:25.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.664+0000 7fef77c00640 1 -- 172.21.15.22:0/4182457915 wait complete. 2024-04-20T21:31:25.685 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T21:31:25.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:25 smithi122 ceph-mon[22147]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:25.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:25 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:25.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:25 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:25.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:25 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:25.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:25 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:31:25.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:25 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:25.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:25 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3896500078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:31:25.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:25 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:25.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:25 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:31:25.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:25 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3910183000' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-20T21:31:25.782 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:25 smithi033 ceph-mon[21626]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:25.782 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:25 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:25.782 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:25 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:25.782 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:25 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:25.782 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:25 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:31:25.782 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:25 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:25.782 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:25 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3896500078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:31:25.782 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:25 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:25.782 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:25 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:31:25.782 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:25 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3910183000' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-20T21:31:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.888+0000 7fda07000640 1 -- 172.21.15.22:0/2728799376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00102a20 msgr2=0x7fda00104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.888+0000 7fda07000640 1 --2- 172.21.15.22:0/2728799376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00102a20 0x7fda00104e10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009900 tx=0x7fd9f002bc30 comp rx=0 tx=0).stop 2024-04-20T21:31:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.889+0000 7fda07000640 1 -- 172.21.15.22:0/2728799376 shutdown_connections 2024-04-20T21:31:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.889+0000 7fda07000640 1 --2- 172.21.15.22:0/2728799376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda00107c80 0x7fda0010a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.889+0000 7fda07000640 1 --2- 172.21.15.22:0/2728799376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda00105350 0x7fda00107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.889+0000 7fda07000640 1 --2- 172.21.15.22:0/2728799376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00102a20 0x7fda00104e10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.889+0000 7fda07000640 1 -- 172.21.15.22:0/2728799376 >> 172.21.15.22:0/2728799376 conn(0x7fda000fc410 msgr2=0x7fda000fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.889+0000 7fda07000640 1 -- 172.21.15.22:0/2728799376 shutdown_connections 2024-04-20T21:31:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.889+0000 7fda07000640 1 -- 172.21.15.22:0/2728799376 wait complete. 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fda07000640 1 Processor -- start 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fda07000640 1 -- start start 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fda07000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda00102a20 0x7fda00102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fda07000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda00105350 0x7fda001009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fda07000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00107c80 0x7fda00100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fda07000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fda0010dc80 con 0x7fda00105350 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fda07000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fda0010db00 con 0x7fda00107c80 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fda07000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fda0010de00 con 0x7fda00102a20 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fd9ffe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda00102a20 0x7fda00102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fd9ffe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda00102a20 0x7fda00102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:39106/0 (socket says 172.21.15.22:39106) 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fd9ffe00640 1 -- 172.21.15.22:0/1029279093 learned_addr learned my addr 172.21.15.22:0/1029279093 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fd9ff400640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda00105350 0x7fda001009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.890+0000 7fda05000640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00107c80 0x7fda00100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.891+0000 7fd9ffe00640 1 -- 172.21.15.22:0/1029279093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00107c80 msgr2=0x7fda00100f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.891+0000 7fd9ffe00640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00107c80 0x7fda00100f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.891+0000 7fd9ffe00640 1 -- 172.21.15.22:0/1029279093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda00105350 msgr2=0x7fda001009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.891+0000 7fd9ffe00640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda00105350 0x7fda001009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.891+0000 7fd9ffe00640 1 -- 172.21.15.22:0/1029279093 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda001017a0 con 0x7fda00102a20 2024-04-20T21:31:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.891+0000 7fd9ffe00640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda00102a20 0x7fda00102390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0000c00 tx=0x7fd9f002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.891+0000 7fd9fcc00640 1 -- 172.21.15.22:0/1029279093 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0043020 con 0x7fda00102a20 2024-04-20T21:31:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.891+0000 7fda07000640 1 -- 172.21.15.22:0/1029279093 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda001a2420 con 0x7fda00102a20 2024-04-20T21:31:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.891+0000 7fd9fcc00640 1 -- 172.21.15.22:0/1029279093 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9f00054e0 con 0x7fda00102a20 2024-04-20T21:31:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.892+0000 7fd9fcc00640 1 -- 172.21.15.22:0/1029279093 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0003c40 con 0x7fda00102a20 2024-04-20T21:31:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.892+0000 7fda07000640 1 -- 172.21.15.22:0/1029279093 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda001a29c0 con 0x7fda00102a20 2024-04-20T21:31:25.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.893+0000 7fda07000640 1 -- 172.21.15.22:0/1029279093 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda000fdf00 con 0x7fda00102a20 2024-04-20T21:31:25.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.895+0000 7fd9fcc00640 1 -- 172.21.15.22:0/1029279093 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd9f0046020 con 0x7fda00102a20 2024-04-20T21:31:25.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.896+0000 7fd9fcc00640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd9e40a3bb0 0x7fd9e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:25.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.896+0000 7fd9fcc00640 1 -- 172.21.15.22:0/1029279093 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7fd9f00f2140 con 0x7fda00102a20 2024-04-20T21:31:25.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.896+0000 7fd9ff400640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd9e40a3bb0 0x7fd9e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:25.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.897+0000 7fd9ff400640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd9e40a3bb0 0x7fd9e40a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fda00102100 tx=0x7fd9ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:25.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:25.900+0000 7fd9fcc00640 1 -- 172.21.15.22:0/1029279093 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd9f004d020 con 0x7fda00102a20 2024-04-20T21:31:25.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:25 smithi022 ceph-mon[16843]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:25.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:25 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:25.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:25 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:25.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:25 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:25.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:25 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:31:25.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:25 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:25.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:25 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3896500078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:31:25.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:25 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:25.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:25 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:31:25.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:25 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3910183000' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-20T21:31:26.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.112+0000 7fda07000640 1 -- 172.21.15.22:0/1029279093 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fda0002d5d0 con 0x7fda00102a20 2024-04-20T21:31:26.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.675+0000 7fd9fcc00640 1 -- 172.21.15.22:0/1029279093 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7fd9f00bedb0 con 0x7fda00102a20 2024-04-20T21:31:26.679 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-04-20T21:31:26.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.692+0000 7fda07000640 1 -- 172.21.15.22:0/1029279093 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd9e40a3bb0 msgr2=0x7fd9e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:26.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.692+0000 7fda07000640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd9e40a3bb0 0x7fd9e40a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fda00102100 tx=0x7fd9ec005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.692+0000 7fda07000640 1 -- 172.21.15.22:0/1029279093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda00102a20 msgr2=0x7fda00102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.692+0000 7fda07000640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda00102a20 0x7fda00102390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0000c00 tx=0x7fd9f002c650 comp rx=0 tx=0).stop 2024-04-20T21:31:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.693+0000 7fda07000640 1 -- 172.21.15.22:0/1029279093 shutdown_connections 2024-04-20T21:31:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.693+0000 7fda07000640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd9e40a3bb0 0x7fd9e40a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.693+0000 7fda07000640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00107c80 0x7fda00100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.693+0000 7fda07000640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda00105350 0x7fda001009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.693+0000 7fda07000640 1 --2- 172.21.15.22:0/1029279093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda00102a20 0x7fda00102390 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0000c00 tx=0x7fd9f002c650 comp rx=0 tx=0).stop 2024-04-20T21:31:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.693+0000 7fda07000640 1 -- 172.21.15.22:0/1029279093 >> 172.21.15.22:0/1029279093 conn(0x7fda000fc410 msgr2=0x7fda00103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.694+0000 7fda07000640 1 -- 172.21.15.22:0/1029279093 shutdown_connections 2024-04-20T21:31:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.695+0000 7fda07000640 1 -- 172.21.15.22:0/1029279093 wait complete. 2024-04-20T21:31:26.718 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T21:31:26.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:26 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/4182457915' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T21:31:26.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:26 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T21:31:26.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:26 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-20T21:31:26.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:26 smithi022 ceph-mon[16843]: osdmap e85: 12 total, 12 up, 12 in 2024-04-20T21:31:26.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:26 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1029279093' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T21:31:26.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:26 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T21:31:26.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:26 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/4182457915' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T21:31:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:26 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T21:31:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:26 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-20T21:31:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:26 smithi122 ceph-mon[22147]: osdmap e85: 12 total, 12 up, 12 in 2024-04-20T21:31:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:26 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1029279093' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T21:31:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:26 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T21:31:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:26 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/4182457915' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T21:31:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:26 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T21:31:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:26 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-20T21:31:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:26 smithi033 ceph-mon[21626]: osdmap e85: 12 total, 12 up, 12 in 2024-04-20T21:31:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:26 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1029279093' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T21:31:26.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:26 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T21:31:26.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.900+0000 7f710a400640 1 -- 172.21.15.22:0/3913893559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71041041c0 msgr2=0x7f71041045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:26.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.900+0000 7f710a400640 1 --2- 172.21.15.22:0/3913893559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71041041c0 0x7f71041045e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009860 tx=0x7f70f402bb70 comp rx=0 tx=0).stop 2024-04-20T21:31:26.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.900+0000 7f710a400640 1 -- 172.21.15.22:0/3913893559 shutdown_connections 2024-04-20T21:31:26.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.900+0000 7f710a400640 1 --2- 172.21.15.22:0/3913893559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7104105d70 0x7f710410c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:26.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.900+0000 7f710a400640 1 --2- 172.21.15.22:0/3913893559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71041053b0 0x7f7104105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:26.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.900+0000 7f710a400640 1 --2- 172.21.15.22:0/3913893559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71041041c0 0x7f71041045e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:26.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.900+0000 7f710a400640 1 -- 172.21.15.22:0/3913893559 >> 172.21.15.22:0/3913893559 conn(0x7f71040ff710 msgr2=0x7f7104101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:26.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.900+0000 7f710a400640 1 -- 172.21.15.22:0/3913893559 shutdown_connections 2024-04-20T21:31:26.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.900+0000 7f710a400640 1 -- 172.21.15.22:0/3913893559 wait complete. 2024-04-20T21:31:26.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.901+0000 7f710a400640 1 Processor -- start 2024-04-20T21:31:26.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.901+0000 7f710a400640 1 -- start start 2024-04-20T21:31:26.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f710a400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71041041c0 0x7f710419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:26.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f710a400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71041053b0 0x7f710419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f710a400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7104105d70 0x7f71041a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f710a400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f710410fe20 con 0x7f71041041c0 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f710a400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f710410fca0 con 0x7f7104105d70 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f710a400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f710410ffa0 con 0x7f71041053b0 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f7103400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71041041c0 0x7f710419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f7103e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7104105d70 0x7f71041a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f7103400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71041041c0 0x7f710419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57440/0 (socket says 172.21.15.22:57440) 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f7103400640 1 -- 172.21.15.22:0/3426471520 learned_addr learned my addr 172.21.15.22:0/3426471520 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f7102a00640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71041053b0 0x7f710419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f7103e00640 1 -- 172.21.15.22:0/3426471520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71041053b0 msgr2=0x7f710419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f7103e00640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71041053b0 0x7f710419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f7103e00640 1 -- 172.21.15.22:0/3426471520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71041041c0 msgr2=0x7f710419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f7103e00640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71041041c0 0x7f710419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:26.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.902+0000 7f7103e00640 1 -- 172.21.15.22:0/3426471520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71041a20e0 con 0x7f7104105d70 2024-04-20T21:31:26.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.903+0000 7f7103e00640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7104105d70 0x7f71041a1a70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00ef00 tx=0x7f70fc00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:26.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.903+0000 7f70fbe00640 1 -- 172.21.15.22:0/3426471520 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc004b20 con 0x7f7104105d70 2024-04-20T21:31:26.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.903+0000 7f70fbe00640 1 -- 172.21.15.22:0/3426471520 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70fc003e50 con 0x7f7104105d70 2024-04-20T21:31:26.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.903+0000 7f70fbe00640 1 -- 172.21.15.22:0/3426471520 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00b570 con 0x7f7104105d70 2024-04-20T21:31:26.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.903+0000 7f710a400640 1 -- 172.21.15.22:0/3426471520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71041a23d0 con 0x7f7104105d70 2024-04-20T21:31:26.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.903+0000 7f710a400640 1 -- 172.21.15.22:0/3426471520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71041a28d0 con 0x7f7104105d70 2024-04-20T21:31:26.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.904+0000 7f710a400640 1 -- 172.21.15.22:0/3426471520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710410b560 con 0x7f7104105d70 2024-04-20T21:31:26.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.907+0000 7f70fbe00640 1 -- 172.21.15.22:0/3426471520 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f70fc01d020 con 0x7f7104105d70 2024-04-20T21:31:26.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.908+0000 7f70fbe00640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f70e80a3c30 0x7f70e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:26.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.908+0000 7f70fbe00640 1 -- 172.21.15.22:0/3426471520 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7879+0+0 (secure 0 0 0) 0x7f70fc0d00d0 con 0x7f7104105d70 2024-04-20T21:31:26.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.909+0000 7f7103400640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f70e80a3c30 0x7f70e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:26.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.909+0000 7f7103400640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f70e80a3c30 0x7f70e80a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f70f40057c0 tx=0x7f70f4034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:26.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:26.913+0000 7f70fbe00640 1 -- 172.21.15.22:0/3426471520 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f70fc09cbc0 con 0x7f7104105d70 2024-04-20T21:31:27.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.174+0000 7f710a400640 1 -- 172.21.15.22:0/3426471520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7104062bd0 con 0x7f7104105d70 2024-04-20T21:31:27.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.673+0000 7f70fbe00640 1 -- 172.21.15.22:0/3426471520 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f70fc09c4c0 con 0x7f7104105d70 2024-04-20T21:31:27.673 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 2 and data pool 3 2024-04-20T21:31:27.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.681+0000 7f710a400640 1 -- 172.21.15.22:0/3426471520 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f70e80a3c30 msgr2=0x7f70e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:27.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.681+0000 7f710a400640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f70e80a3c30 0x7f70e80a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f70f40057c0 tx=0x7f70f4034ab0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.681+0000 7f710a400640 1 -- 172.21.15.22:0/3426471520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7104105d70 msgr2=0x7f71041a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.681+0000 7f710a400640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7104105d70 0x7f71041a1a70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00ef00 tx=0x7f70fc00c740 comp rx=0 tx=0).stop 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.682+0000 7f710a400640 1 -- 172.21.15.22:0/3426471520 shutdown_connections 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.682+0000 7f710a400640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f70e80a3c30 0x7f70e80a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.682+0000 7f710a400640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7104105d70 0x7f71041a1a70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.682+0000 7f710a400640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71041053b0 0x7f710419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.682+0000 7f710a400640 1 --2- 172.21.15.22:0/3426471520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71041041c0 0x7f710419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.682+0000 7f710a400640 1 -- 172.21.15.22:0/3426471520 >> 172.21.15.22:0/3426471520 conn(0x7f71040ff710 msgr2=0x7f7104108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:27.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.682+0000 7f710a400640 1 -- 172.21.15.22:0/3426471520 shutdown_connections 2024-04-20T21:31:27.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.684+0000 7f710a400640 1 -- 172.21.15.22:0/3426471520 wait complete. 2024-04-20T21:31:27.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:31:27.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:27 smithi033 ceph-mon[21626]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:27.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:27 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:27.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:27 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:27.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:27 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:27.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:27 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:27.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:27 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-20T21:31:27.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:27 smithi033 ceph-mon[21626]: osdmap e86: 12 total, 12 up, 12 in 2024-04-20T21:31:27.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:27 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3426471520' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T21:31:27.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:27 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T21:31:27.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:27 smithi033 ceph-mon[21626]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-20T21:31:27.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:27 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:27.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:27 smithi033 ceph-mon[21626]: osdmap e87: 12 total, 12 up, 12 in 2024-04-20T21:31:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.900+0000 7fcd29000640 1 -- 172.21.15.22:0/2491838940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2410a170 msgr2=0x7fcd24110a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.900+0000 7fcd29000640 1 --2- 172.21.15.22:0/2491838940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2410a170 0x7fcd24110a00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009ca0 tx=0x7fcd1c02c100 comp rx=0 tx=0).stop 2024-04-20T21:31:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.900+0000 7fcd29000640 1 -- 172.21.15.22:0/2491838940 shutdown_connections 2024-04-20T21:31:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.900+0000 7fcd29000640 1 --2- 172.21.15.22:0/2491838940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2410a170 0x7fcd24110a00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.900+0000 7fcd29000640 1 --2- 172.21.15.22:0/2491838940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd241097b0 0x7fcd24109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.900+0000 7fcd29000640 1 --2- 172.21.15.22:0/2491838940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd241085c0 0x7fcd241089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.900+0000 7fcd29000640 1 -- 172.21.15.22:0/2491838940 >> 172.21.15.22:0/2491838940 conn(0x7fcd24077580 msgr2=0x7fcd240799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.900+0000 7fcd29000640 1 -- 172.21.15.22:0/2491838940 shutdown_connections 2024-04-20T21:31:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.900+0000 7fcd29000640 1 -- 172.21.15.22:0/2491838940 wait complete. 2024-04-20T21:31:27.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.901+0000 7fcd29000640 1 Processor -- start 2024-04-20T21:31:27.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.901+0000 7fcd29000640 1 -- start start 2024-04-20T21:31:27.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.902+0000 7fcd29000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd241085c0 0x7fcd2419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.902+0000 7fcd29000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd241097b0 0x7fcd2419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.902+0000 7fcd29000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2410a170 0x7fcd241a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.902+0000 7fcd29000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcd24114240 con 0x7fcd241097b0 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.902+0000 7fcd29000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241140c0 con 0x7fcd2410a170 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.902+0000 7fcd29000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241143c0 con 0x7fcd241085c0 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.902+0000 7fcd21e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd241085c0 0x7fcd2419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.902+0000 7fcd21e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd241085c0 0x7fcd2419e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:39134/0 (socket says 172.21.15.22:39134) 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.902+0000 7fcd21e00640 1 -- 172.21.15.22:0/1680943344 learned_addr learned my addr 172.21.15.22:0/1680943344 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.902+0000 7fcd21400640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd241097b0 0x7fcd2419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.902+0000 7fcd22800640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2410a170 0x7fcd241a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.903+0000 7fcd21e00640 1 -- 172.21.15.22:0/1680943344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2410a170 msgr2=0x7fcd241a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.903+0000 7fcd21e00640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2410a170 0x7fcd241a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.903+0000 7fcd21e00640 1 -- 172.21.15.22:0/1680943344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd241097b0 msgr2=0x7fcd2419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:27.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.903+0000 7fcd21e00640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd241097b0 0x7fcd2419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:27.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.903+0000 7fcd21e00640 1 -- 172.21.15.22:0/1680943344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd241a6510 con 0x7fcd241085c0 2024-04-20T21:31:27.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.903+0000 7fcd21e00640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd241085c0 0x7fcd2419e8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400e9c0 tx=0x7fcd1400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:27.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.903+0000 7fcd1aa00640 1 -- 172.21.15.22:0/1680943344 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1400cd80 con 0x7fcd241085c0 2024-04-20T21:31:27.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.903+0000 7fcd1aa00640 1 -- 172.21.15.22:0/1680943344 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd140035f0 con 0x7fcd241085c0 2024-04-20T21:31:27.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.903+0000 7fcd29000640 1 -- 172.21.15.22:0/1680943344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd241a6800 con 0x7fcd241085c0 2024-04-20T21:31:27.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.903+0000 7fcd1aa00640 1 -- 172.21.15.22:0/1680943344 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14010690 con 0x7fcd241085c0 2024-04-20T21:31:27.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.903+0000 7fcd29000640 1 -- 172.21.15.22:0/1680943344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd241a6d50 con 0x7fcd241085c0 2024-04-20T21:31:27.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.905+0000 7fcd29000640 1 -- 172.21.15.22:0/1680943344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2410f880 con 0x7fcd241085c0 2024-04-20T21:31:27.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.907+0000 7fcd1aa00640 1 -- 172.21.15.22:0/1680943344 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcd140037a0 con 0x7fcd241085c0 2024-04-20T21:31:27.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:27 smithi022 ceph-mon[16843]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:27.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:27 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:27.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:27 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:27.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:27 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:27.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:27 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:27.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:27 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-20T21:31:27.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:27 smithi022 ceph-mon[16843]: osdmap e86: 12 total, 12 up, 12 in 2024-04-20T21:31:27.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:27 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3426471520' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T21:31:27.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:27 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T21:31:27.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:27 smithi022 ceph-mon[16843]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-20T21:31:27.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:27 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:27.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:27 smithi022 ceph-mon[16843]: osdmap e87: 12 total, 12 up, 12 in 2024-04-20T21:31:27.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.908+0000 7fcd1aa00640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fcd080a3bb0 0x7fcd080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:27.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.908+0000 7fcd1aa00640 1 -- 172.21.15.22:0/1680943344 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fcd140d0890 con 0x7fcd241085c0 2024-04-20T21:31:27.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.908+0000 7fcd21400640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fcd080a3bb0 0x7fcd080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:27.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.909+0000 7fcd21400640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fcd080a3bb0 0x7fcd080a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcd2419fe00 tx=0x7fcd10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:27.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:27.910+0000 7fcd1aa00640 1 -- 172.21.15.22:0/1680943344 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcd1409d320 con 0x7fcd241085c0 2024-04-20T21:31:28.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:27 smithi122 ceph-mon[22147]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:28.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:27 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:28.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:27 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:28.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:27 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:28.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:27 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:28.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:27 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-20T21:31:28.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:27 smithi122 ceph-mon[22147]: osdmap e86: 12 total, 12 up, 12 in 2024-04-20T21:31:28.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:27 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3426471520' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T21:31:28.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:27 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T21:31:28.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:27 smithi122 ceph-mon[22147]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-20T21:31:28.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:27 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:28.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:27 smithi122 ceph-mon[22147]: osdmap e87: 12 total, 12 up, 12 in 2024-04-20T21:31:28.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.116+0000 7fcd29000640 1 -- 172.21.15.22:0/1680943344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd2402d7c0 con 0x7fcd241085c0 2024-04-20T21:31:28.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.118+0000 7fcd1aa00640 1 -- 172.21.15.22:0/1680943344 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22209 (secure 0 0 0) 0x7fcd14087090 con 0x7fcd241085c0 2024-04-20T21:31:28.118 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:28.118 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":87,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:31:27.176361+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:31:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.121+0000 7fcd29000640 1 -- 172.21.15.22:0/1680943344 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fcd080a3bb0 msgr2=0x7fcd080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.121+0000 7fcd29000640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fcd080a3bb0 0x7fcd080a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcd2419fe00 tx=0x7fcd10005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.121+0000 7fcd29000640 1 -- 172.21.15.22:0/1680943344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd241085c0 msgr2=0x7fcd2419e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.121+0000 7fcd29000640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd241085c0 0x7fcd2419e8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400e9c0 tx=0x7fcd1400ee80 comp rx=0 tx=0).stop 2024-04-20T21:31:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.121+0000 7fcd29000640 1 -- 172.21.15.22:0/1680943344 shutdown_connections 2024-04-20T21:31:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.121+0000 7fcd29000640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fcd080a3bb0 0x7fcd080a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.121+0000 7fcd29000640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2410a170 0x7fcd241a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.121+0000 7fcd29000640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd241097b0 0x7fcd2419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.121+0000 7fcd29000640 1 --2- 172.21.15.22:0/1680943344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd241085c0 0x7fcd2419e8f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.121+0000 7fcd29000640 1 -- 172.21.15.22:0/1680943344 >> 172.21.15.22:0/1680943344 conn(0x7fcd24077580 msgr2=0x7fcd2410ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.122+0000 7fcd29000640 1 -- 172.21.15.22:0/1680943344 shutdown_connections 2024-04-20T21:31:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.122+0000 7fcd29000640 1 -- 172.21.15.22:0/1680943344 wait complete. 2024-04-20T21:31:28.139 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:31:28.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.343+0000 7f493d600640 1 -- 172.21.15.22:0/2305881035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4938100fc0 msgr2=0x7f49381013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.343+0000 7f493d600640 1 --2- 172.21.15.22:0/2305881035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4938100fc0 0x7f49381013e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4928009860 tx=0x7f492802bb70 comp rx=0 tx=0).stop 2024-04-20T21:31:28.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.343+0000 7f493d600640 1 -- 172.21.15.22:0/2305881035 shutdown_connections 2024-04-20T21:31:28.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.343+0000 7f493d600640 1 --2- 172.21.15.22:0/2305881035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4938100450 0x7f4938102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.343+0000 7f493d600640 1 --2- 172.21.15.22:0/2305881035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49380ffa90 0x7f49380fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.343+0000 7f493d600640 1 --2- 172.21.15.22:0/2305881035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4938100fc0 0x7f49381013e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.343+0000 7f493d600640 1 -- 172.21.15.22:0/2305881035 >> 172.21.15.22:0/2305881035 conn(0x7f49380fb590 msgr2=0x7f49380fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:28.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.343+0000 7f493d600640 1 -- 172.21.15.22:0/2305881035 shutdown_connections 2024-04-20T21:31:28.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.343+0000 7f493d600640 1 -- 172.21.15.22:0/2305881035 wait complete. 2024-04-20T21:31:28.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.344+0000 7f493d600640 1 Processor -- start 2024-04-20T21:31:28.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.344+0000 7f493d600640 1 -- start start 2024-04-20T21:31:28.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.345+0000 7f493d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49380ffa90 0x7f493819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.345+0000 7f493d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4938100450 0x7f493819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.345+0000 7f493d600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4938100fc0 0x7f49381a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.345+0000 7f493d600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f493810fc70 con 0x7f4938100450 2024-04-20T21:31:28.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.345+0000 7f493d600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f493810faf0 con 0x7f4938100fc0 2024-04-20T21:31:28.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.345+0000 7f493d600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f493810fdf0 con 0x7f49380ffa90 2024-04-20T21:31:28.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.345+0000 7f4937e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49380ffa90 0x7f493819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.345+0000 7f4937e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49380ffa90 0x7f493819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:39166/0 (socket says 172.21.15.22:39166) 2024-04-20T21:31:28.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.345+0000 7f4937e00640 1 -- 172.21.15.22:0/3922496842 learned_addr learned my addr 172.21.15.22:0/3922496842 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:28.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.345+0000 7f4937400640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4938100450 0x7f493819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.345+0000 7f493cc00640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4938100fc0 0x7f49381a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.346+0000 7f4937e00640 1 -- 172.21.15.22:0/3922496842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4938100fc0 msgr2=0x7f49381a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.346+0000 7f4937e00640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4938100fc0 0x7f49381a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.346+0000 7f4937e00640 1 -- 172.21.15.22:0/3922496842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4938100450 msgr2=0x7f493819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.346+0000 7f4937e00640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4938100450 0x7f493819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.346+0000 7f4937e00640 1 -- 172.21.15.22:0/3922496842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49381a1f20 con 0x7f49380ffa90 2024-04-20T21:31:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.346+0000 7f4937e00640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49380ffa90 0x7f493819a300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4928005940 tx=0x7f4928032470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.346+0000 7f4934c00640 1 -- 172.21.15.22:0/3922496842 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928041020 con 0x7f49380ffa90 2024-04-20T21:31:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.346+0000 7f4934c00640 1 -- 172.21.15.22:0/3922496842 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f492802c9f0 con 0x7f49380ffa90 2024-04-20T21:31:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.346+0000 7f493d600640 1 -- 172.21.15.22:0/3922496842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49381a21b0 con 0x7f49380ffa90 2024-04-20T21:31:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.346+0000 7f4934c00640 1 -- 172.21.15.22:0/3922496842 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928039040 con 0x7f49380ffa90 2024-04-20T21:31:28.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.347+0000 7f493d600640 1 -- 172.21.15.22:0/3922496842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49381a26a0 con 0x7f49380ffa90 2024-04-20T21:31:28.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.348+0000 7f493d600640 1 -- 172.21.15.22:0/3922496842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f493810c220 con 0x7f49380ffa90 2024-04-20T21:31:28.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.350+0000 7f4934c00640 1 -- 172.21.15.22:0/3922496842 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4928044020 con 0x7f49380ffa90 2024-04-20T21:31:28.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.352+0000 7f4934c00640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f491c0a3bb0 0x7f491c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.352+0000 7f4934c00640 1 -- 172.21.15.22:0/3922496842 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f49280f20f0 con 0x7f49380ffa90 2024-04-20T21:31:28.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.352+0000 7f4937400640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f491c0a3bb0 0x7f491c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.353+0000 7f4937400640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f491c0a3bb0 0x7f491c0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f493819b810 tx=0x7f4920009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:28.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.353+0000 7f4934c00640 1 -- 172.21.15.22:0/3922496842 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f492804b020 con 0x7f49380ffa90 2024-04-20T21:31:28.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.569+0000 7f493d600640 1 -- 172.21.15.22:0/3922496842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4938062b60 con 0x7f49380ffa90 2024-04-20T21:31:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.571+0000 7f4934c00640 1 -- 172.21.15.22:0/3922496842 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22196 (secure 0 0 0) 0x7f49280beb80 con 0x7f49380ffa90 2024-04-20T21:31:28.571 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:28.571 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":88,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:31:28.510347+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:31:28.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.574+0000 7f493d600640 1 -- 172.21.15.22:0/3922496842 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f491c0a3bb0 msgr2=0x7f491c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.574+0000 7f493d600640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f491c0a3bb0 0x7f491c0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f493819b810 tx=0x7f4920009500 comp rx=0 tx=0).stop 2024-04-20T21:31:28.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.574+0000 7f493d600640 1 -- 172.21.15.22:0/3922496842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49380ffa90 msgr2=0x7f493819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.574+0000 7f493d600640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49380ffa90 0x7f493819a300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4928005940 tx=0x7f4928032470 comp rx=0 tx=0).stop 2024-04-20T21:31:28.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.575+0000 7f493d600640 1 -- 172.21.15.22:0/3922496842 shutdown_connections 2024-04-20T21:31:28.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.575+0000 7f493d600640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f491c0a3bb0 0x7f491c0a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.575+0000 7f493d600640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4938100fc0 0x7f49381a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.575+0000 7f493d600640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4938100450 0x7f493819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.575+0000 7f493d600640 1 --2- 172.21.15.22:0/3922496842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49380ffa90 0x7f493819a300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.575+0000 7f493d600640 1 -- 172.21.15.22:0/3922496842 >> 172.21.15.22:0/3922496842 conn(0x7f49380fb590 msgr2=0x7f49380fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:28.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.575+0000 7f493d600640 1 -- 172.21.15.22:0/3922496842 shutdown_connections 2024-04-20T21:31:28.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.575+0000 7f493d600640 1 -- 172.21.15.22:0/3922496842 wait complete. 2024-04-20T21:31:28.594 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T21:31:28.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:28.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:28.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:31:28.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:28.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:28 smithi033 ceph-mon[21626]: pgmap v316: 97 pgs: 8 creating+peering, 2 active+clean, 87 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:28.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:28 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-20T21:31:28.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:28 smithi033 ceph-mon[21626]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-20T21:31:28.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:28 smithi033 ceph-mon[21626]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-20T21:31:28.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:28 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1680943344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:28.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:28 smithi033 ceph-mon[21626]: osdmap e88: 12 total, 12 up, 12 in 2024-04-20T21:31:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.815+0000 7f7cde600640 1 -- 172.21.15.22:0/323647733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cd81097b0 msgr2=0x7f7cd8109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.815+0000 7f7cde600640 1 --2- 172.21.15.22:0/323647733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cd81097b0 0x7f7cd8109c30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009900 tx=0x7f7cc402bc30 comp rx=0 tx=0).stop 2024-04-20T21:31:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.815+0000 7f7cde600640 1 -- 172.21.15.22:0/323647733 shutdown_connections 2024-04-20T21:31:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.815+0000 7f7cde600640 1 --2- 172.21.15.22:0/323647733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd810a170 0x7f7cd8110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.815+0000 7f7cde600640 1 --2- 172.21.15.22:0/323647733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cd81097b0 0x7f7cd8109c30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.815+0000 7f7cde600640 1 --2- 172.21.15.22:0/323647733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd81085c0 0x7f7cd81089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.815+0000 7f7cde600640 1 -- 172.21.15.22:0/323647733 >> 172.21.15.22:0/323647733 conn(0x7f7cd8077560 msgr2=0x7f7cd80799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.815+0000 7f7cde600640 1 -- 172.21.15.22:0/323647733 shutdown_connections 2024-04-20T21:31:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.815+0000 7f7cde600640 1 -- 172.21.15.22:0/323647733 wait complete. 2024-04-20T21:31:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.816+0000 7f7cde600640 1 Processor -- start 2024-04-20T21:31:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.816+0000 7f7cde600640 1 -- start start 2024-04-20T21:31:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.816+0000 7f7cde600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd81085c0 0x7f7cd819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.816+0000 7f7cde600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd81097b0 0x7f7cd819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.816+0000 7f7cde600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cd810a170 0x7f7cd81a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cde600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd8114220 con 0x7f7cd810a170 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cde600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd81140a0 con 0x7f7cd81097b0 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cde600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd81143a0 con 0x7f7cd81085c0 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cd7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd81085c0 0x7f7cd819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cd7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd81085c0 0x7f7cd819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:39182/0 (socket says 172.21.15.22:39182) 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cd7400640 1 -- 172.21.15.22:0/1565620492 learned_addr learned my addr 172.21.15.22:0/1565620492 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cd7e00640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cd810a170 0x7f7cd81a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cd6a00640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd81097b0 0x7f7cd819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cd7400640 1 -- 172.21.15.22:0/1565620492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd81097b0 msgr2=0x7f7cd819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cd7400640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd81097b0 0x7f7cd819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cd7400640 1 -- 172.21.15.22:0/1565620492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cd810a170 msgr2=0x7f7cd81a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cd7400640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cd810a170 0x7f7cd81a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cd7400640 1 -- 172.21.15.22:0/1565620492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd81a6540 con 0x7f7cd81085c0 2024-04-20T21:31:28.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cd6a00640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd81097b0 0x7f7cd819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:28.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.817+0000 7f7cd7e00640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cd810a170 0x7f7cd81a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:28.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.818+0000 7f7cd7400640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd81085c0 0x7f7cd819e920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800e9c0 tx=0x7f7cc800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:28.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.818+0000 7f7ccfe00640 1 -- 172.21.15.22:0/1565620492 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc800ccc0 con 0x7f7cd81085c0 2024-04-20T21:31:28.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.818+0000 7f7ccfe00640 1 -- 172.21.15.22:0/1565620492 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc80035f0 con 0x7f7cd81085c0 2024-04-20T21:31:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.818+0000 7f7cde600640 1 -- 172.21.15.22:0/1565620492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd81a6830 con 0x7f7cd81085c0 2024-04-20T21:31:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.818+0000 7f7ccfe00640 1 -- 172.21.15.22:0/1565620492 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc800ce60 con 0x7f7cd81085c0 2024-04-20T21:31:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.818+0000 7f7cde600640 1 -- 172.21.15.22:0/1565620492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd81a6d50 con 0x7f7cd81085c0 2024-04-20T21:31:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.819+0000 7f7cde600640 1 -- 172.21.15.22:0/1565620492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd810f900 con 0x7f7cd81085c0 2024-04-20T21:31:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.821+0000 7f7ccfe00640 1 -- 172.21.15.22:0/1565620492 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7cc80037a0 con 0x7f7cd81085c0 2024-04-20T21:31:28.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.822+0000 7f7ccfe00640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7cbc0a3bb0 0x7f7cbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:28.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.823+0000 7f7ccfe00640 1 -- 172.21.15.22:0/1565620492 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f7cc80d06e0 con 0x7f7cd81085c0 2024-04-20T21:31:28.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.823+0000 7f7cd6a00640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7cbc0a3bb0 0x7f7cbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:28.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.823+0000 7f7cd6a00640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7cbc0a3bb0 0x7f7cbc0a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7cc402c7b0 tx=0x7f7cc4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:28.826+0000 7f7ccfe00640 1 -- 172.21.15.22:0/1565620492 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7cc809d170 con 0x7f7cd81085c0 2024-04-20T21:31:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:31:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:28 smithi022 ceph-mon[16843]: pgmap v316: 97 pgs: 8 creating+peering, 2 active+clean, 87 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:28 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-20T21:31:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:28 smithi022 ceph-mon[16843]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-20T21:31:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:28 smithi022 ceph-mon[16843]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-20T21:31:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:28 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1680943344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:28.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:28 smithi022 ceph-mon[16843]: osdmap e88: 12 total, 12 up, 12 in 2024-04-20T21:31:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:31:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:31:29.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:31:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:28 smithi122 ceph-mon[22147]: pgmap v316: 97 pgs: 8 creating+peering, 2 active+clean, 87 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:31:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:28 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-20T21:31:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:28 smithi122 ceph-mon[22147]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-20T21:31:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:28 smithi122 ceph-mon[22147]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-20T21:31:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:28 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1680943344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:29.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:28 smithi122 ceph-mon[22147]: osdmap e88: 12 total, 12 up, 12 in 2024-04-20T21:31:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:29.077+0000 7f7cde600640 1 -- 172.21.15.22:0/1565620492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7cd8062b60 con 0x7f7cd81085c0 2024-04-20T21:31:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:29 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3922496842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:29 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] up:active 2024-04-20T21:31:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:29 smithi022 ceph-mon[16843]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:31:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:29 smithi022 ceph-mon[16843]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:31:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:29 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1565620492' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T21:31:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:29 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T21:31:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.011+0000 7f7ccfe00640 1 -- 172.21.15.22:0/1565620492 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f7cc809ca70 con 0x7f7cd81085c0 2024-04-20T21:31:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.014+0000 7f7cde600640 1 -- 172.21.15.22:0/1565620492 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7cbc0a3bb0 msgr2=0x7f7cbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.014+0000 7f7cde600640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7cbc0a3bb0 0x7f7cbc0a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7cc402c7b0 tx=0x7f7cc4005fd0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.014+0000 7f7cde600640 1 -- 172.21.15.22:0/1565620492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd81085c0 msgr2=0x7f7cd819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.014+0000 7f7cde600640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd81085c0 0x7f7cd819e920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800e9c0 tx=0x7f7cc800edc0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.015+0000 7f7cde600640 1 -- 172.21.15.22:0/1565620492 shutdown_connections 2024-04-20T21:31:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.015+0000 7f7cde600640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f7cbc0a3bb0 0x7f7cbc0a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.015+0000 7f7cde600640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cd810a170 0x7f7cd81a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.015+0000 7f7cde600640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd81097b0 0x7f7cd819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.015+0000 7f7cde600640 1 --2- 172.21.15.22:0/1565620492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd81085c0 0x7f7cd819e920 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.015+0000 7f7cde600640 1 -- 172.21.15.22:0/1565620492 >> 172.21.15.22:0/1565620492 conn(0x7f7cd8077560 msgr2=0x7f7cd810cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.015+0000 7f7cde600640 1 -- 172.21.15.22:0/1565620492 shutdown_connections 2024-04-20T21:31:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.015+0000 7f7cde600640 1 -- 172.21.15.22:0/1565620492 wait complete. 2024-04-20T21:31:30.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:29 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3922496842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:30.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:29 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] up:active 2024-04-20T21:31:30.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:29 smithi122 ceph-mon[22147]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:31:30.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:29 smithi122 ceph-mon[22147]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:31:30.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:29 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1565620492' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T21:31:30.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:29 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T21:31:30.035 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-20T21:31:30.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-20T21:31:30.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:29 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3922496842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:30.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:29 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] up:active 2024-04-20T21:31:30.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:29 smithi033 ceph-mon[21626]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:31:30.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:29 smithi033 ceph-mon[21626]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:31:30.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:29 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1565620492' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T21:31:30.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:29 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T21:31:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.251+0000 7fd214a00640 1 -- 172.21.15.22:0/3383157985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd210109690 msgr2=0x7fd210109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.251+0000 7fd214a00640 1 --2- 172.21.15.22:0/3383157985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd210109690 0x7fd210109b10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009860 tx=0x7fd1f802bb70 comp rx=0 tx=0).stop 2024-04-20T21:31:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.252+0000 7fd214a00640 1 -- 172.21.15.22:0/3383157985 shutdown_connections 2024-04-20T21:31:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.252+0000 7fd214a00640 1 --2- 172.21.15.22:0/3383157985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd21010a050 0x7fd2101108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.252+0000 7fd214a00640 1 --2- 172.21.15.22:0/3383157985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd210109690 0x7fd210109b10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.252+0000 7fd214a00640 1 --2- 172.21.15.22:0/3383157985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2101084a0 0x7fd2101088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.252+0000 7fd214a00640 1 -- 172.21.15.22:0/3383157985 >> 172.21.15.22:0/3383157985 conn(0x7fd210077430 msgr2=0x7fd210079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.252+0000 7fd214a00640 1 -- 172.21.15.22:0/3383157985 shutdown_connections 2024-04-20T21:31:30.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.252+0000 7fd214a00640 1 -- 172.21.15.22:0/3383157985 wait complete. 2024-04-20T21:31:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.253+0000 7fd214a00640 1 Processor -- start 2024-04-20T21:31:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.253+0000 7fd214a00640 1 -- start start 2024-04-20T21:31:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.253+0000 7fd214a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2101084a0 0x7fd21019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.253+0000 7fd214a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd210109690 0x7fd21019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.253+0000 7fd214a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd21010a050 0x7fd2101a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.253+0000 7fd214a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd210114130 con 0x7fd21010a050 2024-04-20T21:31:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.253+0000 7fd214a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd210113fb0 con 0x7fd210109690 2024-04-20T21:31:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.253+0000 7fd214a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101142b0 con 0x7fd2101084a0 2024-04-20T21:31:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.254+0000 7fd20e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2101084a0 0x7fd21019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.254+0000 7fd20e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2101084a0 0x7fd21019e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:39194/0 (socket says 172.21.15.22:39194) 2024-04-20T21:31:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.254+0000 7fd20e800640 1 -- 172.21.15.22:0/2583292345 learned_addr learned my addr 172.21.15.22:0/2583292345 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.254+0000 7fd20de00640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd210109690 0x7fd21019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.254+0000 7fd20f200640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd21010a050 0x7fd2101a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.254+0000 7fd20e800640 1 -- 172.21.15.22:0/2583292345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd210109690 msgr2=0x7fd21019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.254+0000 7fd20e800640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd210109690 0x7fd21019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.254+0000 7fd20e800640 1 -- 172.21.15.22:0/2583292345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd21010a050 msgr2=0x7fd2101a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.254+0000 7fd20e800640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd21010a050 0x7fd2101a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.254+0000 7fd20e800640 1 -- 172.21.15.22:0/2583292345 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2101a6350 con 0x7fd2101084a0 2024-04-20T21:31:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.254+0000 7fd20e800640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2101084a0 0x7fd21019e730 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd20000c3a0 tx=0x7fd20000c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.255+0000 7fd207400640 1 -- 172.21.15.22:0/2583292345 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200003970 con 0x7fd2101084a0 2024-04-20T21:31:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.255+0000 7fd214a00640 1 -- 172.21.15.22:0/2583292345 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2101a6640 con 0x7fd2101084a0 2024-04-20T21:31:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.255+0000 7fd207400640 1 -- 172.21.15.22:0/2583292345 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd200004020 con 0x7fd2101084a0 2024-04-20T21:31:30.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.255+0000 7fd207400640 1 -- 172.21.15.22:0/2583292345 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20000b9f0 con 0x7fd2101084a0 2024-04-20T21:31:30.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.255+0000 7fd214a00640 1 -- 172.21.15.22:0/2583292345 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2101a6bb0 con 0x7fd2101084a0 2024-04-20T21:31:30.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.256+0000 7fd214a00640 1 -- 172.21.15.22:0/2583292345 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd21010f7c0 con 0x7fd2101084a0 2024-04-20T21:31:30.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.258+0000 7fd207400640 1 -- 172.21.15.22:0/2583292345 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd20001d020 con 0x7fd2101084a0 2024-04-20T21:31:30.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.259+0000 7fd207400640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd1f40a3bb0 0x7fd1f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:30.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.259+0000 7fd207400640 1 -- 172.21.15.22:0/2583292345 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd2000d0b00 con 0x7fd2101084a0 2024-04-20T21:31:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.260+0000 7fd20de00640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd1f40a3bb0 0x7fd1f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.260+0000 7fd20de00640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd1f40a3bb0 0x7fd1f40a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009830 tx=0x7fd1f8034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:30.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.262+0000 7fd207400640 1 -- 172.21.15.22:0/2583292345 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd200024070 con 0x7fd2101084a0 2024-04-20T21:31:30.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:30.510+0000 7fd214a00640 1 -- 172.21.15.22:0/2583292345 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fd21002d860 con 0x7fd2101084a0 2024-04-20T21:31:31.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.014+0000 7fd207400640 1 -- 172.21.15.22:0/2583292345 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fd20009d590 con 0x7fd2101084a0 2024-04-20T21:31:31.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.018+0000 7fd214a00640 1 -- 172.21.15.22:0/2583292345 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd1f40a3bb0 msgr2=0x7fd1f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:31.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.018+0000 7fd214a00640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd1f40a3bb0 0x7fd1f40a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009830 tx=0x7fd1f8034ab0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.018+0000 7fd214a00640 1 -- 172.21.15.22:0/2583292345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2101084a0 msgr2=0x7fd21019e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:31.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.018+0000 7fd214a00640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2101084a0 0x7fd21019e730 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd20000c3a0 tx=0x7fd20000c860 comp rx=0 tx=0).stop 2024-04-20T21:31:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.018+0000 7fd214a00640 1 -- 172.21.15.22:0/2583292345 shutdown_connections 2024-04-20T21:31:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.018+0000 7fd214a00640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd1f40a3bb0 0x7fd1f40a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.018+0000 7fd214a00640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd21010a050 0x7fd2101a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.018+0000 7fd214a00640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd210109690 0x7fd21019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.018+0000 7fd214a00640 1 --2- 172.21.15.22:0/2583292345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2101084a0 0x7fd21019e730 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.018+0000 7fd214a00640 1 -- 172.21.15.22:0/2583292345 >> 172.21.15.22:0/2583292345 conn(0x7fd210077430 msgr2=0x7fd21010c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.018+0000 7fd214a00640 1 -- 172.21.15.22:0/2583292345 shutdown_connections 2024-04-20T21:31:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.018+0000 7fd214a00640 1 -- 172.21.15.22:0/2583292345 wait complete. 2024-04-20T21:31:31.042 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-20T21:31:31.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.263+0000 7f4bb5000640 1 -- 172.21.15.22:0/3259562135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb01040a0 msgr2=0x7f4bb01044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:31.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.263+0000 7f4bb5000640 1 --2- 172.21.15.22:0/3259562135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb01040a0 0x7f4bb01044c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009900 tx=0x7f4ba002bc50 comp rx=0 tx=0).stop 2024-04-20T21:31:31.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.263+0000 7f4bb5000640 1 -- 172.21.15.22:0/3259562135 shutdown_connections 2024-04-20T21:31:31.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.263+0000 7f4bb5000640 1 --2- 172.21.15.22:0/3259562135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb0105c50 0x7f4bb010c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.263+0000 7f4bb5000640 1 --2- 172.21.15.22:0/3259562135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb0105290 0x7f4bb0105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.263+0000 7f4bb5000640 1 --2- 172.21.15.22:0/3259562135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb01040a0 0x7f4bb01044c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.263+0000 7f4bb5000640 1 -- 172.21.15.22:0/3259562135 >> 172.21.15.22:0/3259562135 conn(0x7f4bb00ff610 msgr2=0x7f4bb0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:31.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.264+0000 7f4bb5000640 1 -- 172.21.15.22:0/3259562135 shutdown_connections 2024-04-20T21:31:31.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.264+0000 7f4bb5000640 1 -- 172.21.15.22:0/3259562135 wait complete. 2024-04-20T21:31:31.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.264+0000 7f4bb5000640 1 Processor -- start 2024-04-20T21:31:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.265+0000 7f4bb5000640 1 -- start start 2024-04-20T21:31:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.265+0000 7f4bb5000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb01040a0 0x7f4bb019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.265+0000 7f4bb5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0105290 0x7f4bb019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.265+0000 7f4bb5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb0105c50 0x7f4bb01a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.265+0000 7f4bb5000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb010fd30 con 0x7f4bb0105290 2024-04-20T21:31:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.265+0000 7f4bb5000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb010fbb0 con 0x7f4bb01040a0 2024-04-20T21:31:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.265+0000 7f4bb5000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb010feb0 con 0x7f4bb0105c50 2024-04-20T21:31:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.265+0000 7f4ba7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0105290 0x7f4bb019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.265+0000 7f4ba7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0105290 0x7f4bb019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57518/0 (socket says 172.21.15.22:57518) 2024-04-20T21:31:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.265+0000 7f4ba7e00640 1 -- 172.21.15.22:0/4170774321 learned_addr learned my addr 172.21.15.22:0/4170774321 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.265+0000 7f4bafe00640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb0105c50 0x7f4bb01a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.266+0000 7f4baf400640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb01040a0 0x7f4bb019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.266+0000 7f4ba7e00640 1 -- 172.21.15.22:0/4170774321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb0105c50 msgr2=0x7f4bb01a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.266+0000 7f4ba7e00640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb0105c50 0x7f4bb01a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.266+0000 7f4ba7e00640 1 -- 172.21.15.22:0/4170774321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb01040a0 msgr2=0x7f4bb019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.266+0000 7f4ba7e00640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb01040a0 0x7f4bb019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.266+0000 7f4ba7e00640 1 -- 172.21.15.22:0/4170774321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb01a2070 con 0x7f4bb0105290 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.266+0000 7f4baf400640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb01040a0 0x7f4bb019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.266+0000 7f4bafe00640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb0105c50 0x7f4bb01a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.266+0000 7f4ba7e00640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0105290 0x7f4bb019a990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800e9f0 tx=0x7f4b9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.266+0000 7f4bacc00640 1 -- 172.21.15.22:0/4170774321 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800cd80 con 0x7f4bb0105290 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.266+0000 7f4bacc00640 1 -- 172.21.15.22:0/4170774321 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b98010910 con 0x7f4bb0105290 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.266+0000 7f4bb5000640 1 -- 172.21.15.22:0/4170774321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb01a2360 con 0x7f4bb0105290 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.267+0000 7f4bacc00640 1 -- 172.21.15.22:0/4170774321 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98018990 con 0x7f4bb0105290 2024-04-20T21:31:31.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.267+0000 7f4bb5000640 1 -- 172.21.15.22:0/4170774321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb01a28b0 con 0x7f4bb0105290 2024-04-20T21:31:31.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.268+0000 7f4bb5000640 1 -- 172.21.15.22:0/4170774321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb010b3c0 con 0x7f4bb0105290 2024-04-20T21:31:31.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.270+0000 7f4bacc00640 1 -- 172.21.15.22:0/4170774321 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4b98010ac0 con 0x7f4bb0105290 2024-04-20T21:31:31.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.271+0000 7f4bacc00640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4b940a3bb0 0x7f4b940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:31.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.272+0000 7f4bacc00640 1 -- 172.21.15.22:0/4170774321 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f4b980d0b30 con 0x7f4bb0105290 2024-04-20T21:31:31.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.272+0000 7f4baf400640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4b940a3bb0 0x7f4b940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:31.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.272+0000 7f4baf400640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4b940a3bb0 0x7f4b940a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4ba002c7d0 tx=0x7f4ba0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.274+0000 7f4bacc00640 1 -- 172.21.15.22:0/4170774321 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4b9809d5c0 con 0x7f4bb0105290 2024-04-20T21:31:31.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:31 smithi122 ceph-mon[22147]: pgmap v318: 97 pgs: 8 creating+peering, 23 active+clean, 66 unknown; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-04-20T21:31:31.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:31 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-20T21:31:31.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:31 smithi122 ceph-mon[22147]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:31:31.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:31 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2583292345' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T21:31:31.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:31 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T21:31:31.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:31 smithi033 ceph-mon[21626]: pgmap v318: 97 pgs: 8 creating+peering, 23 active+clean, 66 unknown; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-04-20T21:31:31.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:31 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-20T21:31:31.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:31 smithi033 ceph-mon[21626]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:31:31.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:31 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2583292345' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T21:31:31.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:31 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T21:31:31.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:31 smithi022 ceph-mon[16843]: pgmap v318: 97 pgs: 8 creating+peering, 23 active+clean, 66 unknown; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-04-20T21:31:31.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:31 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-20T21:31:31.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:31 smithi022 ceph-mon[16843]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:31:31.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:31 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2583292345' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T21:31:31.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:31 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T21:31:31.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:31.523+0000 7f4bb5000640 1 -- 172.21.15.22:0/4170774321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f4bb002d820 con 0x7f4bb0105290 2024-04-20T21:31:32.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.030+0000 7f4bacc00640 1 -- 172.21.15.22:0/4170774321 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f4b9809cec0 con 0x7f4bb0105290 2024-04-20T21:31:32.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.034+0000 7f4bb5000640 1 -- 172.21.15.22:0/4170774321 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4b940a3bb0 msgr2=0x7f4b940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:32.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.034+0000 7f4bb5000640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4b940a3bb0 0x7f4b940a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4ba002c7d0 tx=0x7f4ba0005fd0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.035+0000 7f4bb5000640 1 -- 172.21.15.22:0/4170774321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0105290 msgr2=0x7f4bb019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:32.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.035+0000 7f4bb5000640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0105290 0x7f4bb019a990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800e9f0 tx=0x7f4b9800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.035+0000 7f4bb5000640 1 -- 172.21.15.22:0/4170774321 shutdown_connections 2024-04-20T21:31:32.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.035+0000 7f4bb5000640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f4b940a3bb0 0x7f4b940a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.035+0000 7f4bb5000640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb0105c50 0x7f4bb01a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.035+0000 7f4bb5000640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bb0105290 0x7f4bb019a990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.035+0000 7f4bb5000640 1 --2- 172.21.15.22:0/4170774321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb01040a0 0x7f4bb019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.035+0000 7f4bb5000640 1 -- 172.21.15.22:0/4170774321 >> 172.21.15.22:0/4170774321 conn(0x7f4bb00ff610 msgr2=0x7f4bb0108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:32.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.035+0000 7f4bb5000640 1 -- 172.21.15.22:0/4170774321 shutdown_connections 2024-04-20T21:31:32.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.035+0000 7f4bb5000640 1 -- 172.21.15.22:0/4170774321 wait complete. 2024-04-20T21:31:32.060 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-20T21:31:32.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-20T21:31:32.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.278+0000 7f752ea00640 1 -- 172.21.15.22:0/3302785419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7528102fc0 msgr2=0x7f7528103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:32.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.278+0000 7f752ea00640 1 --2- 172.21.15.22:0/3302785419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7528102fc0 0x7f7528103440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7518009860 tx=0x7f751802bb70 comp rx=0 tx=0).stop 2024-04-20T21:31:32.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.278+0000 7f752ea00640 1 -- 172.21.15.22:0/3302785419 shutdown_connections 2024-04-20T21:31:32.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.278+0000 7f752ea00640 1 --2- 172.21.15.22:0/3302785419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528103980 0x7f752810a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.279+0000 7f752ea00640 1 --2- 172.21.15.22:0/3302785419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7528102fc0 0x7f7528103440 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.279+0000 7f752ea00640 1 --2- 172.21.15.22:0/3302785419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7528101e70 0x7f7528102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.279+0000 7f752ea00640 1 -- 172.21.15.22:0/3302785419 >> 172.21.15.22:0/3302785419 conn(0x7f75280fd400 msgr2=0x7f75280ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:32.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.279+0000 7f752ea00640 1 -- 172.21.15.22:0/3302785419 shutdown_connections 2024-04-20T21:31:32.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.279+0000 7f752ea00640 1 -- 172.21.15.22:0/3302785419 wait complete. 2024-04-20T21:31:32.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.280+0000 7f752ea00640 1 Processor -- start 2024-04-20T21:31:32.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.280+0000 7f752ea00640 1 -- start start 2024-04-20T21:31:32.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752ea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528101e70 0x7f752819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:32.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7528102fc0 0x7f752819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:32.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7528103980 0x7f75281a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:32.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752ea00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f752810db10 con 0x7f7528103980 2024-04-20T21:31:32.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752ea00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f752810d990 con 0x7f7528101e70 2024-04-20T21:31:32.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752ea00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f752810dc90 con 0x7f7528102fc0 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752d600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528101e70 0x7f752819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752d600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528101e70 0x7f752819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39394/0 (socket says 172.21.15.22:39394) 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752d600640 1 -- 172.21.15.22:0/2857168990 learned_addr learned my addr 172.21.15.22:0/2857168990 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752cc00640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7528102fc0 0x7f752819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752e000640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7528103980 0x7f75281a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752d600640 1 -- 172.21.15.22:0/2857168990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7528102fc0 msgr2=0x7f752819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752d600640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7528102fc0 0x7f752819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752d600640 1 -- 172.21.15.22:0/2857168990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7528103980 msgr2=0x7f75281a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752d600640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7528103980 0x7f75281a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.281+0000 7f752d600640 1 -- 172.21.15.22:0/2857168990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75281a2020 con 0x7f7528101e70 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.282+0000 7f752d600640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528101e70 0x7f752819a400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f751c00e9f0 tx=0x7f751c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.282+0000 7f7526000640 1 -- 172.21.15.22:0/2857168990 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c00ccc0 con 0x7f7528101e70 2024-04-20T21:31:32.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.282+0000 7f752ea00640 1 -- 172.21.15.22:0/2857168990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75281a2310 con 0x7f7528101e70 2024-04-20T21:31:32.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.282+0000 7f7526000640 1 -- 172.21.15.22:0/2857168990 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f751c005500 con 0x7f7528101e70 2024-04-20T21:31:32.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.282+0000 7f7526000640 1 -- 172.21.15.22:0/2857168990 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c010690 con 0x7f7528101e70 2024-04-20T21:31:32.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.282+0000 7f752ea00640 1 -- 172.21.15.22:0/2857168990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75281a27e0 con 0x7f7528101e70 2024-04-20T21:31:32.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.283+0000 7f752ea00640 1 -- 172.21.15.22:0/2857168990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75281a2aa0 con 0x7f7528101e70 2024-04-20T21:31:32.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.285+0000 7f7526000640 1 -- 172.21.15.22:0/2857168990 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f751c0046c0 con 0x7f7528101e70 2024-04-20T21:31:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.287+0000 7f7526000640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f75100a3c30 0x7f75100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.287+0000 7f7526000640 1 -- 172.21.15.22:0/2857168990 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f751c0d05d0 con 0x7f7528101e70 2024-04-20T21:31:32.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.287+0000 7f752cc00640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f75100a3c30 0x7f75100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:32.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.288+0000 7f752cc00640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f75100a3c30 0x7f75100a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f75180349e0 tx=0x7f7518034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:32.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.291+0000 7f7526000640 1 -- 172.21.15.22:0/2857168990 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f751c09d060 con 0x7f7528101e70 2024-04-20T21:31:32.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:32 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-20T21:31:32.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:32 smithi033 ceph-mon[21626]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:31:32.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:32 smithi033 ceph-mon[21626]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-20T21:31:32.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:32 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/4170774321' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T21:31:32.307 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:32 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-20T21:31:32.308 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:32 smithi022 ceph-mon[16843]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:31:32.308 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:32 smithi022 ceph-mon[16843]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-20T21:31:32.308 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:32 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/4170774321' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T21:31:32.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:32 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-20T21:31:32.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:32 smithi122 ceph-mon[22147]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:31:32.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:32 smithi122 ceph-mon[22147]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-20T21:31:32.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:32 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/4170774321' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T21:31:32.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.530+0000 7f752ea00640 1 -- 172.21.15.22:0/2857168990 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f75280758b0 con 0x7f75100a3c30 2024-04-20T21:31:32.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.561+0000 7f7526000640 1 -- 172.21.15.22:0/2857168990 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f75280758b0 con 0x7f75100a3c30 2024-04-20T21:31:32.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.563+0000 7f752ea00640 1 -- 172.21.15.22:0/2857168990 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f75100a3c30 msgr2=0x7f75100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:32.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.563+0000 7f752ea00640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f75100a3c30 0x7f75100a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f75180349e0 tx=0x7f7518034ab0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.563+0000 7f752ea00640 1 -- 172.21.15.22:0/2857168990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528101e70 msgr2=0x7f752819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:32.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.563+0000 7f752ea00640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528101e70 0x7f752819a400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f751c00e9f0 tx=0x7f751c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.564+0000 7f752ea00640 1 -- 172.21.15.22:0/2857168990 shutdown_connections 2024-04-20T21:31:32.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.564+0000 7f752ea00640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f75100a3c30 0x7f75100a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.564+0000 7f752ea00640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7528103980 0x7f75281a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.564+0000 7f752ea00640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7528102fc0 0x7f752819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.564+0000 7f752ea00640 1 --2- 172.21.15.22:0/2857168990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528101e70 0x7f752819a400 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.564+0000 7f752ea00640 1 -- 172.21.15.22:0/2857168990 >> 172.21.15.22:0/2857168990 conn(0x7f75280fd400 msgr2=0x7f7528074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:32.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.564+0000 7f752ea00640 1 -- 172.21.15.22:0/2857168990 shutdown_connections 2024-04-20T21:31:32.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.564+0000 7f752ea00640 1 -- 172.21.15.22:0/2857168990 wait complete. 2024-04-20T21:31:32.579 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:31:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:31:32.542+0000 7f06eae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:31:32.579 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:31:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:31:32.542+0000 7f06eae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:31:32.580 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:31:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:31:32.542+0000 7f06eae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:31:32.580 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:31:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:31:32.542+0000 7f06eae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:31:32.580 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:31:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:31:32.542+0000 7f06eae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:31:32.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-20T21:31:32.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.778+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1357282702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf641041e0 msgr2=0x7fbf64104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:32.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.778+0000 7fbf6ba00640 1 --2- 172.21.15.22:0/1357282702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf641041e0 0x7fbf64104600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009900 tx=0x7fbf5802bc30 comp rx=0 tx=0).stop 2024-04-20T21:31:32.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.778+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1357282702 shutdown_connections 2024-04-20T21:31:32.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.778+0000 7fbf6ba00640 1 --2- 172.21.15.22:0/1357282702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf64105d90 0x7fbf6410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.778+0000 7fbf6ba00640 1 --2- 172.21.15.22:0/1357282702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf641053d0 0x7fbf64105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.778+0000 7fbf6ba00640 1 --2- 172.21.15.22:0/1357282702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf641041e0 0x7fbf64104600 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.778+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1357282702 >> 172.21.15.22:0/1357282702 conn(0x7fbf640ff750 msgr2=0x7fbf64101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:32.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.779+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1357282702 shutdown_connections 2024-04-20T21:31:32.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.779+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1357282702 wait complete. 2024-04-20T21:31:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.780+0000 7fbf6ba00640 1 Processor -- start 2024-04-20T21:31:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.780+0000 7fbf6ba00640 1 -- start start 2024-04-20T21:31:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.780+0000 7fbf6ba00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf641041e0 0x7fbf6419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.780+0000 7fbf6ba00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf641053d0 0x7fbf6419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.780+0000 7fbf6ba00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf64105d90 0x7fbf641a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.780+0000 7fbf6ba00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6410fe70 con 0x7fbf641053d0 2024-04-20T21:31:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.780+0000 7fbf6ba00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6410fcf0 con 0x7fbf641041e0 2024-04-20T21:31:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.780+0000 7fbf6ba00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6410fff0 con 0x7fbf64105d90 2024-04-20T21:31:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.780+0000 7fbf63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf641053d0 0x7fbf6419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.781+0000 7fbf63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf641053d0 0x7fbf6419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35548/0 (socket says 172.21.15.22:35548) 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.781+0000 7fbf63e00640 1 -- 172.21.15.22:0/1714555351 learned_addr learned my addr 172.21.15.22:0/1714555351 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.781+0000 7fbf69000640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf641041e0 0x7fbf6419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.781+0000 7fbf69a00640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf64105d90 0x7fbf641a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.781+0000 7fbf63e00640 1 -- 172.21.15.22:0/1714555351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf64105d90 msgr2=0x7fbf641a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.781+0000 7fbf63e00640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf64105d90 0x7fbf641a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.781+0000 7fbf63e00640 1 -- 172.21.15.22:0/1714555351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf641041e0 msgr2=0x7fbf6419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.781+0000 7fbf63e00640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf641041e0 0x7fbf6419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.781+0000 7fbf63e00640 1 -- 172.21.15.22:0/1714555351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf641a2140 con 0x7fbf641053d0 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.781+0000 7fbf69000640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf641041e0 0x7fbf6419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.781+0000 7fbf63e00640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf641053d0 0x7fbf6419aa60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400e9f0 tx=0x7fbf5400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.782+0000 7fbf61600640 1 -- 172.21.15.22:0/1714555351 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400cd80 con 0x7fbf641053d0 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.782+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1714555351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf641a2430 con 0x7fbf641053d0 2024-04-20T21:31:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.782+0000 7fbf61600640 1 -- 172.21.15.22:0/1714555351 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf54005500 con 0x7fbf641053d0 2024-04-20T21:31:32.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.782+0000 7fbf61600640 1 -- 172.21.15.22:0/1714555351 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54010690 con 0x7fbf641053d0 2024-04-20T21:31:32.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.782+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1714555351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf641a2980 con 0x7fbf641053d0 2024-04-20T21:31:32.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.783+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1714555351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6410b550 con 0x7fbf641053d0 2024-04-20T21:31:32.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.787+0000 7fbf61600640 1 -- 172.21.15.22:0/1714555351 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbf540046c0 con 0x7fbf641053d0 2024-04-20T21:31:32.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.788+0000 7fbf61600640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fbf4c0a3bb0 0x7fbf4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:32.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.789+0000 7fbf61600640 1 -- 172.21.15.22:0/1714555351 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fbf540d08b0 con 0x7fbf641053d0 2024-04-20T21:31:32.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.789+0000 7fbf69000640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fbf4c0a3bb0 0x7fbf4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:32.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.789+0000 7fbf61600640 1 -- 172.21.15.22:0/1714555351 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbf54053ff0 con 0x7fbf641053d0 2024-04-20T21:31:32.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:32.790+0000 7fbf69000640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fbf4c0a3bb0 0x7fbf4c0a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf58000c00 tx=0x7fbf58005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:32.803 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:31:32 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b[51942]: 2024-04-20T21:31:32.406+0000 7f24c586a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:31:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.031+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1714555351 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbf640623a0 con 0x7fbf4c0a3bb0 2024-04-20T21:31:33.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.068+0000 7fbf61600640 1 -- 172.21.15.22:0/1714555351 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbf640623a0 con 0x7fbf4c0a3bb0 2024-04-20T21:31:33.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.071+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1714555351 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fbf4c0a3bb0 msgr2=0x7fbf4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.071+0000 7fbf6ba00640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fbf4c0a3bb0 0x7fbf4c0a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf58000c00 tx=0x7fbf58005fd0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.071+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1714555351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf641053d0 msgr2=0x7fbf6419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.071+0000 7fbf6ba00640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf641053d0 0x7fbf6419aa60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400e9f0 tx=0x7fbf5400edf0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.071+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1714555351 shutdown_connections 2024-04-20T21:31:33.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.071+0000 7fbf6ba00640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fbf4c0a3bb0 0x7fbf4c0a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.071+0000 7fbf6ba00640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf64105d90 0x7fbf641a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.071+0000 7fbf6ba00640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf641053d0 0x7fbf6419aa60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.071+0000 7fbf6ba00640 1 --2- 172.21.15.22:0/1714555351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf641041e0 0x7fbf6419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.072+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1714555351 >> 172.21.15.22:0/1714555351 conn(0x7fbf640ff750 msgr2=0x7fbf64108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:33.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.072+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1714555351 shutdown_connections 2024-04-20T21:31:33.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.072+0000 7fbf6ba00640 1 -- 172.21.15.22:0/1714555351 wait complete. 2024-04-20T21:31:33.090 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-20T21:31:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.303+0000 7f8316a00640 1 -- 172.21.15.22:0/2693698017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8310105d90 msgr2=0x7f831010c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.303+0000 7f8316a00640 1 --2- 172.21.15.22:0/2693698017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8310105d90 0x7f831010c620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8308009ca0 tx=0x7f830802c100 comp rx=0 tx=0).stop 2024-04-20T21:31:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.303+0000 7f8316a00640 1 -- 172.21.15.22:0/2693698017 shutdown_connections 2024-04-20T21:31:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.303+0000 7f8316a00640 1 --2- 172.21.15.22:0/2693698017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8310105d90 0x7f831010c620 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.303+0000 7f8316a00640 1 --2- 172.21.15.22:0/2693698017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83101053d0 0x7f8310105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.303+0000 7f8316a00640 1 --2- 172.21.15.22:0/2693698017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83101041e0 0x7f8310104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.303+0000 7f8316a00640 1 -- 172.21.15.22:0/2693698017 >> 172.21.15.22:0/2693698017 conn(0x7f83100ff750 msgr2=0x7f8310101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.303+0000 7f8316a00640 1 -- 172.21.15.22:0/2693698017 shutdown_connections 2024-04-20T21:31:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.303+0000 7f8316a00640 1 -- 172.21.15.22:0/2693698017 wait complete. 2024-04-20T21:31:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.304+0000 7f8316a00640 1 Processor -- start 2024-04-20T21:31:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f8316a00640 1 -- start start 2024-04-20T21:31:33.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:33 smithi033 ceph-mon[21626]: pgmap v319: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-20T21:31:33.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:33 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/4170774321' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-20T21:31:33.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:33 smithi033 ceph-mon[21626]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-20T21:31:33.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f8316a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83101041e0 0x7f831019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f8316a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83101053d0 0x7f831019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f8316a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8310105d90 0x7f83101a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f8316a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f831010fe70 con 0x7f83101053d0 2024-04-20T21:31:33.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f8316a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f831010fcf0 con 0x7f8310105d90 2024-04-20T21:31:33.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f8316a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f831010fff0 con 0x7f83101041e0 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f830f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83101053d0 0x7f831019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f830fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83101041e0 0x7f831019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f8314a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8310105d90 0x7f83101a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f830fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83101041e0 0x7f831019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42812/0 (socket says 172.21.15.22:42812) 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f830fe00640 1 -- 172.21.15.22:0/3789430595 learned_addr learned my addr 172.21.15.22:0/3789430595 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f8314a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8310105d90 0x7f83101a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:49028/0 (socket says 172.21.15.22:49028) 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.305+0000 7f830f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83101053d0 0x7f831019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35570/0 (socket says 172.21.15.22:35570) 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.306+0000 7f830fe00640 1 -- 172.21.15.22:0/3789430595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8310105d90 msgr2=0x7f83101a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.306+0000 7f830fe00640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8310105d90 0x7f83101a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.306+0000 7f830fe00640 1 -- 172.21.15.22:0/3789430595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83101053d0 msgr2=0x7f831019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.306+0000 7f830fe00640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83101053d0 0x7f831019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.306+0000 7f830fe00640 1 -- 172.21.15.22:0/3789430595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83101a2090 con 0x7f83101041e0 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.306+0000 7f830f400640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83101053d0 0x7f831019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:33.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.306+0000 7f830fe00640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83101041e0 0x7f831019a470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f830000e9c0 tx=0x7f830000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.306+0000 7f830cc00640 1 -- 172.21.15.22:0/3789430595 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830000cd80 con 0x7f83101041e0 2024-04-20T21:31:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.306+0000 7f830cc00640 1 -- 172.21.15.22:0/3789430595 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83000035f0 con 0x7f83101041e0 2024-04-20T21:31:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.306+0000 7f8316a00640 1 -- 172.21.15.22:0/3789430595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83101a2380 con 0x7f83101041e0 2024-04-20T21:31:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.306+0000 7f830cc00640 1 -- 172.21.15.22:0/3789430595 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300010690 con 0x7f83101041e0 2024-04-20T21:31:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.307+0000 7f8316a00640 1 -- 172.21.15.22:0/3789430595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83101a2880 con 0x7f83101041e0 2024-04-20T21:31:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.308+0000 7f8316a00640 1 -- 172.21.15.22:0/3789430595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831010b4a0 con 0x7f83101041e0 2024-04-20T21:31:33.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.311+0000 7f830cc00640 1 -- 172.21.15.22:0/3789430595 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f83000037a0 con 0x7f83101041e0 2024-04-20T21:31:33.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.313+0000 7f830cc00640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f82f40a3bb0 0x7f82f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.313+0000 7f830cc00640 1 -- 172.21.15.22:0/3789430595 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f83000d0690 con 0x7f83101041e0 2024-04-20T21:31:33.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.313+0000 7f830f400640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f82f40a3bb0 0x7f82f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.313+0000 7f830cc00640 1 -- 172.21.15.22:0/3789430595 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f830009d120 con 0x7f83101041e0 2024-04-20T21:31:33.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.314+0000 7f830f400640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f82f40a3bb0 0x7f82f40a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f831019b980 tx=0x7f82fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:33.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:33 smithi022 ceph-mon[16843]: pgmap v319: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-20T21:31:33.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:33 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/4170774321' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-20T21:31:33.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:33 smithi022 ceph-mon[16843]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-20T21:31:33.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:33 smithi122 ceph-mon[22147]: pgmap v319: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-20T21:31:33.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:33 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/4170774321' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-20T21:31:33.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:33 smithi122 ceph-mon[22147]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-20T21:31:33.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.551+0000 7f8316a00640 1 -- 172.21.15.22:0/3789430595 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f83100623a0 con 0x7f82f40a3bb0 2024-04-20T21:31:33.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.562+0000 7f830cc00640 1 -- 172.21.15.22:0/3789430595 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f83100623a0 con 0x7f82f40a3bb0 2024-04-20T21:31:33.562 INFO:teuthology.orchestra.run.smithi022.stdout:/volumes/qa/sv_0/502b341a-9a97-4379-b335-7a9593cb2d34 2024-04-20T21:31:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.565+0000 7f8316a00640 1 -- 172.21.15.22:0/3789430595 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f82f40a3bb0 msgr2=0x7f82f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.565+0000 7f8316a00640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f82f40a3bb0 0x7f82f40a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f831019b980 tx=0x7f82fc005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.565+0000 7f8316a00640 1 -- 172.21.15.22:0/3789430595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83101041e0 msgr2=0x7f831019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.565+0000 7f8316a00640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83101041e0 0x7f831019a470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f830000e9c0 tx=0x7f830000ee80 comp rx=0 tx=0).stop 2024-04-20T21:31:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.566+0000 7f8316a00640 1 -- 172.21.15.22:0/3789430595 shutdown_connections 2024-04-20T21:31:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.566+0000 7f8316a00640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f82f40a3bb0 0x7f82f40a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.566+0000 7f8316a00640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8310105d90 0x7f83101a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.566+0000 7f8316a00640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83101053d0 0x7f831019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.566+0000 7f8316a00640 1 --2- 172.21.15.22:0/3789430595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83101041e0 0x7f831019a470 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.566+0000 7f8316a00640 1 -- 172.21.15.22:0/3789430595 >> 172.21.15.22:0/3789430595 conn(0x7f83100ff750 msgr2=0x7f8310108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.566+0000 7f8316a00640 1 -- 172.21.15.22:0/3789430595 shutdown_connections 2024-04-20T21:31:33.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.566+0000 7f8316a00640 1 -- 172.21.15.22:0/3789430595 wait complete. 2024-04-20T21:31:33.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-20T21:31:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.781+0000 7fd838a00640 1 -- 172.21.15.22:0/150350540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd834109d20 msgr2=0x7fd83410c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.781+0000 7fd838a00640 1 --2- 172.21.15.22:0/150350540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd834109d20 0x7fd83410c1c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009be0 tx=0x7fd82c02bf80 comp rx=0 tx=0).stop 2024-04-20T21:31:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.781+0000 7fd838a00640 1 -- 172.21.15.22:0/150350540 shutdown_connections 2024-04-20T21:31:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.781+0000 7fd838a00640 1 --2- 172.21.15.22:0/150350540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd834109d20 0x7fd83410c1c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.781+0000 7fd838a00640 1 --2- 172.21.15.22:0/150350540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd834074960 0x7fd834074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.781+0000 7fd838a00640 1 --2- 172.21.15.22:0/150350540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd834075f40 0x7fd834074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.781+0000 7fd838a00640 1 -- 172.21.15.22:0/150350540 >> 172.21.15.22:0/150350540 conn(0x7fd8340fd400 msgr2=0x7fd8340ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.781+0000 7fd838a00640 1 -- 172.21.15.22:0/150350540 shutdown_connections 2024-04-20T21:31:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.781+0000 7fd838a00640 1 -- 172.21.15.22:0/150350540 wait complete. 2024-04-20T21:31:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.783+0000 7fd838a00640 1 Processor -- start 2024-04-20T21:31:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.783+0000 7fd838a00640 1 -- start start 2024-04-20T21:31:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.783+0000 7fd838a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd834074960 0x7fd83419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.783+0000 7fd838a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd834075f40 0x7fd83419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.783+0000 7fd838a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd834109d20 0x7fd8341a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.783+0000 7fd838a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410fd70 con 0x7fd834074960 2024-04-20T21:31:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.783+0000 7fd838a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410fbf0 con 0x7fd834109d20 2024-04-20T21:31:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.783+0000 7fd838a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410fef0 con 0x7fd834075f40 2024-04-20T21:31:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.783+0000 7fd833400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd834074960 0x7fd83419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.784+0000 7fd833400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd834074960 0x7fd83419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35580/0 (socket says 172.21.15.22:35580) 2024-04-20T21:31:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.784+0000 7fd833400640 1 -- 172.21.15.22:0/3046096750 learned_addr learned my addr 172.21.15.22:0/3046096750 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.784+0000 7fd833e00640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd834109d20 0x7fd8341a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.784+0000 7fd832a00640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd834075f40 0x7fd83419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.784+0000 7fd833400640 1 -- 172.21.15.22:0/3046096750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd834075f40 msgr2=0x7fd83419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.784+0000 7fd833400640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd834075f40 0x7fd83419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.784+0000 7fd833400640 1 -- 172.21.15.22:0/3046096750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd834109d20 msgr2=0x7fd8341a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.784+0000 7fd833400640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd834109d20 0x7fd8341a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.784+0000 7fd833400640 1 -- 172.21.15.22:0/3046096750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8341a1f50 con 0x7fd834074960 2024-04-20T21:31:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.784+0000 7fd833400640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd834074960 0x7fd83419a330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd82400e9c0 tx=0x7fd82400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.785+0000 7fd82be00640 1 -- 172.21.15.22:0/3046096750 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82400ccc0 con 0x7fd834074960 2024-04-20T21:31:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.785+0000 7fd82be00640 1 -- 172.21.15.22:0/3046096750 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd824005500 con 0x7fd834074960 2024-04-20T21:31:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.785+0000 7fd838a00640 1 -- 172.21.15.22:0/3046096750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8341a2240 con 0x7fd834074960 2024-04-20T21:31:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.785+0000 7fd82be00640 1 -- 172.21.15.22:0/3046096750 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824010430 con 0x7fd834074960 2024-04-20T21:31:33.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.785+0000 7fd838a00640 1 -- 172.21.15.22:0/3046096750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8341a27e0 con 0x7fd834074960 2024-04-20T21:31:33.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.786+0000 7fd838a00640 1 -- 172.21.15.22:0/3046096750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd834108ae0 con 0x7fd834074960 2024-04-20T21:31:33.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.788+0000 7fd82be00640 1 -- 172.21.15.22:0/3046096750 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd8240046c0 con 0x7fd834074960 2024-04-20T21:31:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.789+0000 7fd82be00640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd8180a3c30 0x7fd8180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.790+0000 7fd82be00640 1 -- 172.21.15.22:0/3046096750 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd8240d07b0 con 0x7fd834074960 2024-04-20T21:31:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.790+0000 7fd832a00640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd8180a3c30 0x7fd8180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.790+0000 7fd832a00640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd8180a3c30 0x7fd8180a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd83419b840 tx=0x7fd820005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:33.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:33.796+0000 7fd82be00640 1 -- 172.21.15.22:0/3046096750 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd824053ff0 con 0x7fd834074960 2024-04-20T21:31:34.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.033+0000 7fd838a00640 1 -- 172.21.15.22:0/3046096750 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd8340623a0 con 0x7fd8180a3c30 2024-04-20T21:31:34.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.056+0000 7fd82be00640 1 -- 172.21.15.22:0/3046096750 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd8240991b0 con 0x7fd834074960 2024-04-20T21:31:34.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.078+0000 7fd82be00640 1 -- 172.21.15.22:0/3046096750 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd8340623a0 con 0x7fd8180a3c30 2024-04-20T21:31:34.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.081+0000 7fd838a00640 1 -- 172.21.15.22:0/3046096750 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd8180a3c30 msgr2=0x7fd8180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:34.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.081+0000 7fd838a00640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd8180a3c30 0x7fd8180a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd83419b840 tx=0x7fd820005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:34.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.081+0000 7fd838a00640 1 -- 172.21.15.22:0/3046096750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd834074960 msgr2=0x7fd83419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:34.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.081+0000 7fd838a00640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd834074960 0x7fd83419a330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd82400e9c0 tx=0x7fd82400ee80 comp rx=0 tx=0).stop 2024-04-20T21:31:34.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.081+0000 7fd838a00640 1 -- 172.21.15.22:0/3046096750 shutdown_connections 2024-04-20T21:31:34.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.081+0000 7fd838a00640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd8180a3c30 0x7fd8180a60f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.081+0000 7fd838a00640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd834109d20 0x7fd8341a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.081+0000 7fd838a00640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd834075f40 0x7fd83419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.081+0000 7fd838a00640 1 --2- 172.21.15.22:0/3046096750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd834074960 0x7fd83419a330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.081+0000 7fd838a00640 1 -- 172.21.15.22:0/3046096750 >> 172.21.15.22:0/3046096750 conn(0x7fd8340fd400 msgr2=0x7fd834105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:34.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.081+0000 7fd838a00640 1 -- 172.21.15.22:0/3046096750 shutdown_connections 2024-04-20T21:31:34.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.081+0000 7fd838a00640 1 -- 172.21.15.22:0/3046096750 wait complete. 2024-04-20T21:31:34.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-20T21:31:34.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:34 smithi033 ceph-mon[21626]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:34.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:34 smithi033 ceph-mon[21626]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:34.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.311+0000 7f17f3200640 1 -- 172.21.15.22:0/169501627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17f4102fe0 msgr2=0x7f17f4103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:34.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.311+0000 7f17f3200640 1 --2- 172.21.15.22:0/169501627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17f4102fe0 0x7f17f4103460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009900 tx=0x7f17e002bc30 comp rx=0 tx=0).stop 2024-04-20T21:31:34.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.311+0000 7f17f3200640 1 -- 172.21.15.22:0/169501627 shutdown_connections 2024-04-20T21:31:34.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.311+0000 7f17f3200640 1 --2- 172.21.15.22:0/169501627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17f41039a0 0x7f17f410a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.311+0000 7f17f3200640 1 --2- 172.21.15.22:0/169501627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17f4102fe0 0x7f17f4103460 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.311+0000 7f17f3200640 1 --2- 172.21.15.22:0/169501627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17f4101e90 0x7f17f41022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.311+0000 7f17f3200640 1 -- 172.21.15.22:0/169501627 >> 172.21.15.22:0/169501627 conn(0x7f17f40fd400 msgr2=0x7f17f40ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:34.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.312+0000 7f17f3200640 1 -- 172.21.15.22:0/169501627 shutdown_connections 2024-04-20T21:31:34.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.312+0000 7f17f3200640 1 -- 172.21.15.22:0/169501627 wait complete. 2024-04-20T21:31:34.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.313+0000 7f17f3200640 1 Processor -- start 2024-04-20T21:31:34.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.313+0000 7f17f3200640 1 -- start start 2024-04-20T21:31:34.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.313+0000 7f17f3200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17f4101e90 0x7f17f4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:34.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.313+0000 7f17f3200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17f4102fe0 0x7f17f41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:34.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.313+0000 7f17f3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17f41039a0 0x7f17f419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:34.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.313+0000 7f17f3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17f410db20 con 0x7f17f41039a0 2024-04-20T21:31:34.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.313+0000 7f17f3200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17f410d9a0 con 0x7f17f4101e90 2024-04-20T21:31:34.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.313+0000 7f17f3200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17f410dca0 con 0x7f17f4102fe0 2024-04-20T21:31:34.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.314+0000 7f17f1e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17f4101e90 0x7f17f4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:34.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.314+0000 7f17f1e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17f4101e90 0x7f17f4198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:49056/0 (socket says 172.21.15.22:49056) 2024-04-20T21:31:34.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.314+0000 7f17f1e00640 1 -- 172.21.15.22:0/1415295145 learned_addr learned my addr 172.21.15.22:0/1415295145 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:34.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.314+0000 7f17f2800640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17f41039a0 0x7f17f419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:34.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.314+0000 7f17f1400640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17f4102fe0 0x7f17f41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:34.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.314+0000 7f17f1e00640 1 -- 172.21.15.22:0/1415295145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17f4102fe0 msgr2=0x7f17f41986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:34.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.314+0000 7f17f1e00640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17f4102fe0 0x7f17f41986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.314+0000 7f17f1e00640 1 -- 172.21.15.22:0/1415295145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17f41039a0 msgr2=0x7f17f419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:34.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.314+0000 7f17f1e00640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17f41039a0 0x7f17f419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.314+0000 7f17f1e00640 1 -- 172.21.15.22:0/1415295145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f419fdb0 con 0x7f17f4101e90 2024-04-20T21:31:34.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.314+0000 7f17f1e00640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17f4101e90 0x7f17f4198190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17e400e9f0 tx=0x7f17e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:34.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.315+0000 7f17eaa00640 1 -- 172.21.15.22:0/1415295145 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e400cd80 con 0x7f17f4101e90 2024-04-20T21:31:34.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.315+0000 7f17eaa00640 1 -- 172.21.15.22:0/1415295145 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e4010910 con 0x7f17f4101e90 2024-04-20T21:31:34.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.315+0000 7f17f3200640 1 -- 172.21.15.22:0/1415295145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f41a00a0 con 0x7f17f4101e90 2024-04-20T21:31:34.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.315+0000 7f17eaa00640 1 -- 172.21.15.22:0/1415295145 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4018990 con 0x7f17f4101e90 2024-04-20T21:31:34.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.315+0000 7f17f3200640 1 -- 172.21.15.22:0/1415295145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f41a0610 con 0x7f17f4101e90 2024-04-20T21:31:34.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.316+0000 7f17f3200640 1 -- 172.21.15.22:0/1415295145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f4109220 con 0x7f17f4101e90 2024-04-20T21:31:34.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.318+0000 7f17eaa00640 1 -- 172.21.15.22:0/1415295145 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f17e4010ac0 con 0x7f17f4101e90 2024-04-20T21:31:34.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.319+0000 7f17eaa00640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f17d80a3cc0 0x7f17d80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:34.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.320+0000 7f17eaa00640 1 -- 172.21.15.22:0/1415295145 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f17e40d0960 con 0x7f17f4101e90 2024-04-20T21:31:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.320+0000 7f17f1400640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f17d80a3cc0 0x7f17d80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.320+0000 7f17f1400640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f17d80a3cc0 0x7f17d80a6180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17e0000c00 tx=0x7f17e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:34.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.322+0000 7f17eaa00640 1 -- 172.21.15.22:0/1415295145 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f17e409cc40 con 0x7f17f4101e90 2024-04-20T21:31:34.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:34 smithi022 ceph-mon[16843]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:34.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:34 smithi022 ceph-mon[16843]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:34.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:34 smithi122 ceph-mon[22147]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:34.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:34 smithi122 ceph-mon[22147]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:34.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.563+0000 7f17f3200640 1 -- 172.21.15.22:0/1415295145 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f17f4062430 con 0x7f17d80a3cc0 2024-04-20T21:31:34.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.571+0000 7f17eaa00640 1 -- 172.21.15.22:0/1415295145 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f17f4062430 con 0x7f17d80a3cc0 2024-04-20T21:31:34.570 INFO:teuthology.orchestra.run.smithi022.stdout:/volumes/qa/sv_1/0bd71eb4-0894-49b6-ab83-fa3698d7b690 2024-04-20T21:31:34.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.573+0000 7f17f3200640 1 -- 172.21.15.22:0/1415295145 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f17d80a3cc0 msgr2=0x7f17d80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:34.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.573+0000 7f17f3200640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f17d80a3cc0 0x7f17d80a6180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17e0000c00 tx=0x7f17e0005fd0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.573+0000 7f17f3200640 1 -- 172.21.15.22:0/1415295145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17f4101e90 msgr2=0x7f17f4198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:34.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.573+0000 7f17f3200640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17f4101e90 0x7f17f4198190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17e400e9f0 tx=0x7f17e400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.574+0000 7f17f3200640 1 -- 172.21.15.22:0/1415295145 shutdown_connections 2024-04-20T21:31:34.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.574+0000 7f17f3200640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f17d80a3cc0 0x7f17d80a6180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.574+0000 7f17f3200640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17f41039a0 0x7f17f419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.574+0000 7f17f3200640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17f4102fe0 0x7f17f41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.574+0000 7f17f3200640 1 --2- 172.21.15.22:0/1415295145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17f4101e90 0x7f17f4198190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.574+0000 7f17f3200640 1 -- 172.21.15.22:0/1415295145 >> 172.21.15.22:0/1415295145 conn(0x7f17f40fd400 msgr2=0x7f17f41063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:34.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.574+0000 7f17f3200640 1 -- 172.21.15.22:0/1415295145 shutdown_connections 2024-04-20T21:31:34.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.574+0000 7f17f3200640 1 -- 172.21.15.22:0/1415295145 wait complete. 2024-04-20T21:31:34.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.804+0000 7f3e95800640 1 -- 172.21.15.22:0/1498608031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e901083e0 msgr2=0x7f3e90108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.804+0000 7f3e95800640 1 --2- 172.21.15.22:0/1498608031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e901083e0 0x7f3e90108800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009860 tx=0x7f3e8002bb70 comp rx=0 tx=0).stop 2024-04-20T21:31:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.804+0000 7f3e95800640 1 -- 172.21.15.22:0/1498608031 shutdown_connections 2024-04-20T21:31:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.804+0000 7f3e95800640 1 --2- 172.21.15.22:0/1498608031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e90109f90 0x7f3e90110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.804+0000 7f3e95800640 1 --2- 172.21.15.22:0/1498608031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e901095d0 0x7f3e90109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.804+0000 7f3e95800640 1 --2- 172.21.15.22:0/1498608031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e901083e0 0x7f3e90108800 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.804+0000 7f3e95800640 1 -- 172.21.15.22:0/1498608031 >> 172.21.15.22:0/1498608031 conn(0x7f3e900773a0 msgr2=0x7f3e900797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.804+0000 7f3e95800640 1 -- 172.21.15.22:0/1498608031 shutdown_connections 2024-04-20T21:31:34.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.804+0000 7f3e95800640 1 -- 172.21.15.22:0/1498608031 wait complete. 2024-04-20T21:31:34.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.805+0000 7f3e95800640 1 Processor -- start 2024-04-20T21:31:34.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.805+0000 7f3e95800640 1 -- start start 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.806+0000 7f3e95800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e901083e0 0x7f3e9019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.806+0000 7f3e95800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e901095d0 0x7f3e9019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.806+0000 7f3e95800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e90109f90 0x7f3e901a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.806+0000 7f3e95800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e90114040 con 0x7f3e90109f90 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.806+0000 7f3e95800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e90113ec0 con 0x7f3e901095d0 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.806+0000 7f3e95800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e901141c0 con 0x7f3e901083e0 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.806+0000 7f3e8fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e901083e0 0x7f3e9019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.806+0000 7f3e87e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e901095d0 0x7f3e9019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.806+0000 7f3e8fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e901083e0 0x7f3e9019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42850/0 (socket says 172.21.15.22:42850) 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.806+0000 7f3e8fe00640 1 -- 172.21.15.22:0/3905375300 learned_addr learned my addr 172.21.15.22:0/3905375300 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.806+0000 7f3e94e00640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e90109f90 0x7f3e901a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.807+0000 7f3e87e00640 1 -- 172.21.15.22:0/3905375300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e901083e0 msgr2=0x7f3e9019e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.807+0000 7f3e87e00640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e901083e0 0x7f3e9019e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.807+0000 7f3e87e00640 1 -- 172.21.15.22:0/3905375300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e90109f90 msgr2=0x7f3e901a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:34.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.807+0000 7f3e87e00640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e90109f90 0x7f3e901a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:34.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.807+0000 7f3e87e00640 1 -- 172.21.15.22:0/3905375300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e901a6310 con 0x7f3e901095d0 2024-04-20T21:31:34.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.807+0000 7f3e94e00640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e90109f90 0x7f3e901a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:34.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.807+0000 7f3e87e00640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e901095d0 0x7f3e9019ec30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e78005fd0 tx=0x7f3e7800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.807+0000 7f3e8d600640 1 -- 172.21.15.22:0/3905375300 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7800cd00 con 0x7f3e901095d0 2024-04-20T21:31:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.807+0000 7f3e95800640 1 -- 172.21.15.22:0/3905375300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e901a6600 con 0x7f3e901095d0 2024-04-20T21:31:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.807+0000 7f3e8d600640 1 -- 172.21.15.22:0/3905375300 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e780035f0 con 0x7f3e901095d0 2024-04-20T21:31:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.807+0000 7f3e8d600640 1 -- 172.21.15.22:0/3905375300 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78010690 con 0x7f3e901095d0 2024-04-20T21:31:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.807+0000 7f3e95800640 1 -- 172.21.15.22:0/3905375300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e901a6ba0 con 0x7f3e901095d0 2024-04-20T21:31:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.809+0000 7f3e95800640 1 -- 172.21.15.22:0/3905375300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e9010f780 con 0x7f3e901095d0 2024-04-20T21:31:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.811+0000 7f3e8d600640 1 -- 172.21.15.22:0/3905375300 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3e780037a0 con 0x7f3e901095d0 2024-04-20T21:31:34.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.812+0000 7f3e8d600640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3e740a3c40 0x7f3e740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:34.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.812+0000 7f3e8d600640 1 -- 172.21.15.22:0/3905375300 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f3e780d0760 con 0x7f3e901095d0 2024-04-20T21:31:34.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.812+0000 7f3e8fe00640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3e740a3c40 0x7f3e740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.813+0000 7f3e8fe00640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3e740a3c40 0x7f3e740a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3e800095f0 tx=0x7f3e80034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:34.816+0000 7f3e8d600640 1 -- 172.21.15.22:0/3905375300 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3e78009510 con 0x7f3e901095d0 2024-04-20T21:31:35.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.066+0000 7f3e95800640 1 -- 172.21.15.22:0/3905375300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e9002d7a0 con 0x7f3e901095d0 2024-04-20T21:31:35.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.067+0000 7f3e8d600640 1 -- 172.21.15.22:0/3905375300 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f3e7809d090 con 0x7f3e901095d0 2024-04-20T21:31:35.067 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:35.067 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:31:35.068 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:31:35.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.070+0000 7f3e95800640 1 -- 172.21.15.22:0/3905375300 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3e740a3c40 msgr2=0x7f3e740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.070+0000 7f3e95800640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3e740a3c40 0x7f3e740a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3e800095f0 tx=0x7f3e80034ad0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.070+0000 7f3e95800640 1 -- 172.21.15.22:0/3905375300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e901095d0 msgr2=0x7f3e9019ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.070+0000 7f3e95800640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e901095d0 0x7f3e9019ec30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e78005fd0 tx=0x7f3e7800ee50 comp rx=0 tx=0).stop 2024-04-20T21:31:35.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.070+0000 7f3e95800640 1 -- 172.21.15.22:0/3905375300 shutdown_connections 2024-04-20T21:31:35.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.070+0000 7f3e95800640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3e740a3c40 0x7f3e740a6100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.070+0000 7f3e95800640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e90109f90 0x7f3e901a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.070+0000 7f3e95800640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e901095d0 0x7f3e9019ec30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.070+0000 7f3e95800640 1 --2- 172.21.15.22:0/3905375300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e901083e0 0x7f3e9019e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.070+0000 7f3e95800640 1 -- 172.21.15.22:0/3905375300 >> 172.21.15.22:0/3905375300 conn(0x7f3e900773a0 msgr2=0x7f3e9010c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:35.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.071+0000 7f3e95800640 1 -- 172.21.15.22:0/3905375300 shutdown_connections 2024-04-20T21:31:35.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.071+0000 7f3e95800640 1 -- 172.21.15.22:0/3905375300 wait complete. 2024-04-20T21:31:35.092 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:31:35.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.299+0000 7f17c7200640 1 -- 172.21.15.22:0/3575486515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c81095d0 msgr2=0x7f17c8109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.299+0000 7f17c7200640 1 --2- 172.21.15.22:0/3575486515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c81095d0 0x7f17c8109a50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009900 tx=0x7f17b802bc50 comp rx=0 tx=0).stop 2024-04-20T21:31:35.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.299+0000 7f17c7200640 1 -- 172.21.15.22:0/3575486515 shutdown_connections 2024-04-20T21:31:35.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.299+0000 7f17c7200640 1 --2- 172.21.15.22:0/3575486515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17c8109f90 0x7f17c8110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.299+0000 7f17c7200640 1 --2- 172.21.15.22:0/3575486515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c81095d0 0x7f17c8109a50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.299+0000 7f17c7200640 1 --2- 172.21.15.22:0/3575486515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c81083e0 0x7f17c8108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.299+0000 7f17c7200640 1 -- 172.21.15.22:0/3575486515 >> 172.21.15.22:0/3575486515 conn(0x7f17c80773a0 msgr2=0x7f17c80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:35.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.299+0000 7f17c7200640 1 -- 172.21.15.22:0/3575486515 shutdown_connections 2024-04-20T21:31:35.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.299+0000 7f17c7200640 1 -- 172.21.15.22:0/3575486515 wait complete. 2024-04-20T21:31:35.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.300+0000 7f17c7200640 1 Processor -- start 2024-04-20T21:31:35.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.300+0000 7f17c7200640 1 -- start start 2024-04-20T21:31:35.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.301+0000 7f17c7200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c81083e0 0x7f17c819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.301+0000 7f17c7200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c81095d0 0x7f17c819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.301+0000 7f17c7200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17c8109f90 0x7f17c81a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.301+0000 7f17c7200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81140c0 con 0x7f17c81083e0 2024-04-20T21:31:35.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.301+0000 7f17c7200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8113f40 con 0x7f17c81095d0 2024-04-20T21:31:35.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.301+0000 7f17c7200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8114240 con 0x7f17c8109f90 2024-04-20T21:31:35.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.301+0000 7f17c5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c81083e0 0x7f17c819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.301+0000 7f17c5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c81083e0 0x7f17c819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35636/0 (socket says 172.21.15.22:35636) 2024-04-20T21:31:35.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.301+0000 7f17c5e00640 1 -- 172.21.15.22:0/4040134540 learned_addr learned my addr 172.21.15.22:0/4040134540 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:35.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.301+0000 7f17c6800640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17c8109f90 0x7f17c81a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.301+0000 7f17c5400640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c81095d0 0x7f17c819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.302+0000 7f17c5e00640 1 -- 172.21.15.22:0/4040134540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17c8109f90 msgr2=0x7f17c81a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.302+0000 7f17c5e00640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17c8109f90 0x7f17c81a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.302+0000 7f17c5e00640 1 -- 172.21.15.22:0/4040134540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c81095d0 msgr2=0x7f17c819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.302+0000 7f17c5e00640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c81095d0 0x7f17c819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.302+0000 7f17c5e00640 1 -- 172.21.15.22:0/4040134540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c81a62f0 con 0x7f17c81083e0 2024-04-20T21:31:35.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.302+0000 7f17c5e00640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c81083e0 0x7f17c819e6d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f17b400e9c0 tx=0x7f17b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.302+0000 7f17bea00640 1 -- 172.21.15.22:0/4040134540 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b400cde0 con 0x7f17c81083e0 2024-04-20T21:31:35.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.302+0000 7f17c7200640 1 -- 172.21.15.22:0/4040134540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c81a65e0 con 0x7f17c81083e0 2024-04-20T21:31:35.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.302+0000 7f17c7200640 1 -- 172.21.15.22:0/4040134540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c81a6b50 con 0x7f17c81083e0 2024-04-20T21:31:35.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:35 smithi033 ceph-mon[21626]: from='client.24709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:35.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:35 smithi033 ceph-mon[21626]: pgmap v320: 97 pgs: 97 active+clean; 598 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-04-20T21:31:35.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:35 smithi033 ceph-mon[21626]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:35.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:35 smithi033 ceph-mon[21626]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-20T21:31:35.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.302+0000 7f17bea00640 1 -- 172.21.15.22:0/4040134540 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b4010910 con 0x7f17c81083e0 2024-04-20T21:31:35.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.302+0000 7f17bea00640 1 -- 172.21.15.22:0/4040134540 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4018980 con 0x7f17c81083e0 2024-04-20T21:31:35.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.304+0000 7f17c7200640 1 -- 172.21.15.22:0/4040134540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c80690f0 con 0x7f17c81083e0 2024-04-20T21:31:35.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.305+0000 7f17bea00640 1 -- 172.21.15.22:0/4040134540 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f17b401f020 con 0x7f17c81083e0 2024-04-20T21:31:35.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.307+0000 7f17bea00640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f17ac0a3c40 0x7f17ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.307+0000 7f17bea00640 1 -- 172.21.15.22:0/4040134540 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f17b40d0860 con 0x7f17c81083e0 2024-04-20T21:31:35.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.307+0000 7f17c5400640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f17ac0a3c40 0x7f17ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.308+0000 7f17c5400640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f17ac0a3c40 0x7f17ac0a6100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f17b802c260 tx=0x7f17b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.311+0000 7f17bea00640 1 -- 172.21.15.22:0/4040134540 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f17b409d190 con 0x7f17c81083e0 2024-04-20T21:31:35.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:35 smithi022 ceph-mon[16843]: from='client.24709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:35.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:35 smithi022 ceph-mon[16843]: pgmap v320: 97 pgs: 97 active+clean; 598 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-04-20T21:31:35.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:35 smithi022 ceph-mon[16843]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:35.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:35 smithi022 ceph-mon[16843]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-20T21:31:35.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.516+0000 7f17c7200640 1 -- 172.21.15.22:0/4040134540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17c8062b60 con 0x7f17c81083e0 2024-04-20T21:31:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.521+0000 7f17bea00640 1 -- 172.21.15.22:0/4040134540 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22196 (secure 0 0 0) 0x7f17b409ca90 con 0x7f17c81083e0 2024-04-20T21:31:35.521 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:35.521 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":88,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:31:28.510347+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:31:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.524+0000 7f17c7200640 1 -- 172.21.15.22:0/4040134540 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f17ac0a3c40 msgr2=0x7f17ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.524+0000 7f17c7200640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f17ac0a3c40 0x7f17ac0a6100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f17b802c260 tx=0x7f17b8005fd0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.524+0000 7f17c7200640 1 -- 172.21.15.22:0/4040134540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c81083e0 msgr2=0x7f17c819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.524+0000 7f17c7200640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c81083e0 0x7f17c819e6d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f17b400e9c0 tx=0x7f17b400ee80 comp rx=0 tx=0).stop 2024-04-20T21:31:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.524+0000 7f17c7200640 1 -- 172.21.15.22:0/4040134540 shutdown_connections 2024-04-20T21:31:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.524+0000 7f17c7200640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f17ac0a3c40 0x7f17ac0a6100 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.524+0000 7f17c7200640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17c8109f90 0x7f17c81a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.524+0000 7f17c7200640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c81095d0 0x7f17c819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.524+0000 7f17c7200640 1 --2- 172.21.15.22:0/4040134540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c81083e0 0x7f17c819e6d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.524+0000 7f17c7200640 1 -- 172.21.15.22:0/4040134540 >> 172.21.15.22:0/4040134540 conn(0x7f17c80773a0 msgr2=0x7f17c810c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:35 smithi122 ceph-mon[22147]: from='client.24709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:35 smithi122 ceph-mon[22147]: pgmap v320: 97 pgs: 97 active+clean; 598 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-04-20T21:31:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:35 smithi122 ceph-mon[22147]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:35 smithi122 ceph-mon[22147]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-20T21:31:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.525+0000 7f17c7200640 1 -- 172.21.15.22:0/4040134540 shutdown_connections 2024-04-20T21:31:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.525+0000 7f17c7200640 1 -- 172.21.15.22:0/4040134540 wait complete. 2024-04-20T21:31:35.544 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T21:31:35.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:31:35.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.741+0000 7f64f5800640 1 -- 172.21.15.22:0/3615888462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f0074940 msgr2=0x7f64f0074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.741+0000 7f64f5800640 1 --2- 172.21.15.22:0/3615888462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f0074940 0x7f64f0074da0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009900 tx=0x7f64d802bcc0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.742+0000 7f64f5800640 1 -- 172.21.15.22:0/3615888462 shutdown_connections 2024-04-20T21:31:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.742+0000 7f64f5800640 1 --2- 172.21.15.22:0/3615888462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64f0109cb0 0x7f64f010c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.742+0000 7f64f5800640 1 --2- 172.21.15.22:0/3615888462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f0074940 0x7f64f0074da0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.742+0000 7f64f5800640 1 --2- 172.21.15.22:0/3615888462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f0075f20 0x7f64f0074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.742+0000 7f64f5800640 1 -- 172.21.15.22:0/3615888462 >> 172.21.15.22:0/3615888462 conn(0x7f64f00fd370 msgr2=0x7f64f00ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.742+0000 7f64f5800640 1 -- 172.21.15.22:0/3615888462 shutdown_connections 2024-04-20T21:31:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.742+0000 7f64f5800640 1 -- 172.21.15.22:0/3615888462 wait complete. 2024-04-20T21:31:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.743+0000 7f64f5800640 1 Processor -- start 2024-04-20T21:31:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.743+0000 7f64f5800640 1 -- start start 2024-04-20T21:31:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.743+0000 7f64f5800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f0074940 0x7f64f019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.743+0000 7f64f5800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64f0075f20 0x7f64f019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.743+0000 7f64f5800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f0109cb0 0x7f64f01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.743+0000 7f64f5800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f64f010fca0 con 0x7f64f0075f20 2024-04-20T21:31:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.743+0000 7f64f5800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f64f010fb20 con 0x7f64f0109cb0 2024-04-20T21:31:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.743+0000 7f64f5800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f64f010fe20 con 0x7f64f0074940 2024-04-20T21:31:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.743+0000 7f64ef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64f0075f20 0x7f64f019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.743+0000 7f64ef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64f0075f20 0x7f64f019a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35654/0 (socket says 172.21.15.22:35654) 2024-04-20T21:31:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.743+0000 7f64ef400640 1 -- 172.21.15.22:0/2327866256 learned_addr learned my addr 172.21.15.22:0/2327866256 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.743+0000 7f64f4e00640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f0109cb0 0x7f64f01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.744+0000 7f64efe00640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f0074940 0x7f64f019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.744+0000 7f64ef400640 1 -- 172.21.15.22:0/2327866256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f0074940 msgr2=0x7f64f019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.744+0000 7f64ef400640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f0074940 0x7f64f019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.744+0000 7f64ef400640 1 -- 172.21.15.22:0/2327866256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f0109cb0 msgr2=0x7f64f01a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.744+0000 7f64ef400640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f0109cb0 0x7f64f01a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.744+0000 7f64ef400640 1 -- 172.21.15.22:0/2327866256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f01a1ef0 con 0x7f64f0075f20 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.744+0000 7f64efe00640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f0074940 0x7f64f019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.744+0000 7f64ef400640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64f0075f20 0x7f64f019a810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009230 tx=0x7f64d802c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.744+0000 7f64ecc00640 1 -- 172.21.15.22:0/2327866256 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8043020 con 0x7f64f0075f20 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.744+0000 7f64ecc00640 1 -- 172.21.15.22:0/2327866256 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d80054e0 con 0x7f64f0075f20 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.744+0000 7f64ecc00640 1 -- 172.21.15.22:0/2327866256 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8003bc0 con 0x7f64f0075f20 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.744+0000 7f64f5800640 1 -- 172.21.15.22:0/2327866256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f01a2180 con 0x7f64f0075f20 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.745+0000 7f64f5800640 1 -- 172.21.15.22:0/2327866256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f01a2680 con 0x7f64f0075f20 2024-04-20T21:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.746+0000 7f64f5800640 1 -- 172.21.15.22:0/2327866256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f0108a70 con 0x7f64f0075f20 2024-04-20T21:31:35.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.748+0000 7f64ecc00640 1 -- 172.21.15.22:0/2327866256 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f64d8046020 con 0x7f64f0075f20 2024-04-20T21:31:35.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.750+0000 7f64ecc00640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f64d40a3c40 0x7f64d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.751+0000 7f64ecc00640 1 -- 172.21.15.22:0/2327866256 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f64d80f28b0 con 0x7f64f0075f20 2024-04-20T21:31:35.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.751+0000 7f64efe00640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f64d40a3c40 0x7f64d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.752+0000 7f64efe00640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f64d40a3c40 0x7f64d40a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009c20 tx=0x7f64e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.753+0000 7f64ecc00640 1 -- 172.21.15.22:0/2327866256 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f64d804d020 con 0x7f64f0075f20 2024-04-20T21:31:35.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.956+0000 7f64f5800640 1 -- 172.21.15.22:0/2327866256 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64f00623a0 con 0x7f64d40a3c40 2024-04-20T21:31:35.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.967+0000 7f64ecc00640 1 -- 172.21.15.22:0/2327866256 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258685 (secure 0 0 0) 0x7f64f00623a0 con 0x7f64d40a3c40 2024-04-20T21:31:35.970 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:35.971 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-20T21:31:35.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.973+0000 7f64f5800640 1 -- 172.21.15.22:0/2327866256 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f64d40a3c40 msgr2=0x7f64d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.973+0000 7f64f5800640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f64d40a3c40 0x7f64d40a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009c20 tx=0x7f64e0005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:35.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.973+0000 7f64f5800640 1 -- 172.21.15.22:0/2327866256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64f0075f20 msgr2=0x7f64f019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.973+0000 7f64f5800640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64f0075f20 0x7f64f019a810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009230 tx=0x7f64d802c6e0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.973+0000 7f64f5800640 1 -- 172.21.15.22:0/2327866256 shutdown_connections 2024-04-20T21:31:35.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.973+0000 7f64f5800640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f64d40a3c40 0x7f64d40a6100 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.973+0000 7f64f5800640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f0109cb0 0x7f64f01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.973+0000 7f64f5800640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64f0075f20 0x7f64f019a810 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.973+0000 7f64f5800640 1 --2- 172.21.15.22:0/2327866256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f0074940 0x7f64f019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.973+0000 7f64f5800640 1 -- 172.21.15.22:0/2327866256 >> 172.21.15.22:0/2327866256 conn(0x7f64f00fd370 msgr2=0x7f64f0105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:35.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.974+0000 7f64f5800640 1 -- 172.21.15.22:0/2327866256 shutdown_connections 2024-04-20T21:31:35.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:35.974+0000 7f64f5800640 1 -- 172.21.15.22:0/2327866256 wait complete. 2024-04-20T21:31:35.992 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-20T21:31:35.653227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":612295,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":151,"num_read_kb":137,"num_write":237,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378824,"kb_used_data":12988,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563000,"statfs":{"total":1151940427776,"available":1151552512000,"internally_reserved":0,"allocated":13299712,"data_stored":7256821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21927,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":15,"num_read_kb":18,"num_write":56,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.163438"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516987+0000","last_change":"2024-04-20T21:31:27.527151+0000","last_active":"2024-04-20T21:31:28.516987+0000","last_peered":"2024-04-20T21:31:28.516987+0000","last_clean":"2024-04-20T21:31:28.516987+0000","last_became_active":"2024-04-20T21:31:27.526739+0000","last_became_peered":"2024-04-20T21:31:27.526739+0000","last_unstale":"2024-04-20T21:31:28.516987+0000","last_undegraded":"2024-04-20T21:31:28.516987+0000","last_fullsized":"2024-04-20T21:31:28.516987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:16:39.204523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.194016+0000","last_change":"2024-04-20T21:31:27.527437+0000","last_active":"2024-04-20T21:31:29.194016+0000","last_peered":"2024-04-20T21:31:29.194016+0000","last_clean":"2024-04-20T21:31:29.194016+0000","last_became_active":"2024-04-20T21:31:27.527040+0000","last_became_peered":"2024-04-20T21:31:27.527040+0000","last_unstale":"2024-04-20T21:31:29.194016+0000","last_undegraded":"2024-04-20T21:31:29.194016+0000","last_fullsized":"2024-04-20T21:31:29.194016+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:53:39.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.193528+0000","last_change":"2024-04-20T21:31:27.531671+0000","last_active":"2024-04-20T21:31:29.193528+0000","last_peered":"2024-04-20T21:31:29.193528+0000","last_clean":"2024-04-20T21:31:29.193528+0000","last_became_active":"2024-04-20T21:31:27.531267+0000","last_became_peered":"2024-04-20T21:31:27.531267+0000","last_unstale":"2024-04-20T21:31:29.193528+0000","last_undegraded":"2024-04-20T21:31:29.193528+0000","last_fullsized":"2024-04-20T21:31:29.193528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:47.002481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516388+0000","last_change":"2024-04-20T21:31:27.512033+0000","last_active":"2024-04-20T21:31:28.516388+0000","last_peered":"2024-04-20T21:31:28.516388+0000","last_clean":"2024-04-20T21:31:28.516388+0000","last_became_active":"2024-04-20T21:31:27.510801+0000","last_became_peered":"2024-04-20T21:31:27.510801+0000","last_unstale":"2024-04-20T21:31:28.516388+0000","last_undegraded":"2024-04-20T21:31:28.516388+0000","last_fullsized":"2024-04-20T21:31:28.516388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:43:25.407173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.683813+0000","last_change":"2024-04-20T21:31:27.545862+0000","last_active":"2024-04-20T21:31:28.683813+0000","last_peered":"2024-04-20T21:31:28.683813+0000","last_clean":"2024-04-20T21:31:28.683813+0000","last_became_active":"2024-04-20T21:31:27.545500+0000","last_became_peered":"2024-04-20T21:31:27.545500+0000","last_unstale":"2024-04-20T21:31:28.683813+0000","last_undegraded":"2024-04-20T21:31:28.683813+0000","last_fullsized":"2024-04-20T21:31:28.683813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:37.640777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191640+0000","last_change":"2024-04-20T21:31:27.526408+0000","last_active":"2024-04-20T21:31:29.191640+0000","last_peered":"2024-04-20T21:31:29.191640+0000","last_clean":"2024-04-20T21:31:29.191640+0000","last_became_active":"2024-04-20T21:31:27.526015+0000","last_became_peered":"2024-04-20T21:31:27.526015+0000","last_unstale":"2024-04-20T21:31:29.191640+0000","last_undegraded":"2024-04-20T21:31:29.191640+0000","last_fullsized":"2024-04-20T21:31:29.191640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:56.904696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192281+0000","last_change":"2024-04-20T21:31:27.526272+0000","last_active":"2024-04-20T21:31:29.192281+0000","last_peered":"2024-04-20T21:31:29.192281+0000","last_clean":"2024-04-20T21:31:29.192281+0000","last_became_active":"2024-04-20T21:31:27.525852+0000","last_became_peered":"2024-04-20T21:31:27.525852+0000","last_unstale":"2024-04-20T21:31:29.192281+0000","last_undegraded":"2024-04-20T21:31:29.192281+0000","last_fullsized":"2024-04-20T21:31:29.192281+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:33:57.811339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.684874+0000","last_change":"2024-04-20T21:31:27.543548+0000","last_active":"2024-04-20T21:31:28.684874+0000","last_peered":"2024-04-20T21:31:28.684874+0000","last_clean":"2024-04-20T21:31:28.684874+0000","last_became_active":"2024-04-20T21:31:27.541487+0000","last_became_peered":"2024-04-20T21:31:27.541487+0000","last_unstale":"2024-04-20T21:31:28.684874+0000","last_undegraded":"2024-04-20T21:31:28.684874+0000","last_fullsized":"2024-04-20T21:31:28.684874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:23:21.009261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191372+0000","last_change":"2024-04-20T21:31:27.514810+0000","last_active":"2024-04-20T21:31:29.191372+0000","last_peered":"2024-04-20T21:31:29.191372+0000","last_clean":"2024-04-20T21:31:29.191372+0000","last_became_active":"2024-04-20T21:31:27.512548+0000","last_became_peered":"2024-04-20T21:31:27.512548+0000","last_unstale":"2024-04-20T21:31:29.191372+0000","last_undegraded":"2024-04-20T21:31:29.191372+0000","last_fullsized":"2024-04-20T21:31:29.191372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:11:30.375294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192277+0000","last_change":"2024-04-20T21:31:27.510303+0000","last_active":"2024-04-20T21:31:29.192277+0000","last_peered":"2024-04-20T21:31:29.192277+0000","last_clean":"2024-04-20T21:31:29.192277+0000","last_became_active":"2024-04-20T21:31:27.509907+0000","last_became_peered":"2024-04-20T21:31:27.509907+0000","last_unstale":"2024-04-20T21:31:29.192277+0000","last_undegraded":"2024-04-20T21:31:29.192277+0000","last_fullsized":"2024-04-20T21:31:29.192277+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:31.160785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191604+0000","last_change":"2024-04-20T21:31:27.513749+0000","last_active":"2024-04-20T21:31:29.191604+0000","last_peered":"2024-04-20T21:31:29.191604+0000","last_clean":"2024-04-20T21:31:29.191604+0000","last_became_active":"2024-04-20T21:31:27.513404+0000","last_became_peered":"2024-04-20T21:31:27.513404+0000","last_unstale":"2024-04-20T21:31:29.191604+0000","last_undegraded":"2024-04-20T21:31:29.191604+0000","last_fullsized":"2024-04-20T21:31:29.191604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:26.279697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516215+0000","last_change":"2024-04-20T21:31:27.507727+0000","last_active":"2024-04-20T21:31:28.516215+0000","last_peered":"2024-04-20T21:31:28.516215+0000","last_clean":"2024-04-20T21:31:28.516215+0000","last_became_active":"2024-04-20T21:31:27.507363+0000","last_became_peered":"2024-04-20T21:31:27.507363+0000","last_unstale":"2024-04-20T21:31:28.516215+0000","last_undegraded":"2024-04-20T21:31:28.516215+0000","last_fullsized":"2024-04-20T21:31:28.516215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:01.139963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.518273+0000","last_change":"2024-04-20T21:31:27.512941+0000","last_active":"2024-04-20T21:31:28.518273+0000","last_peered":"2024-04-20T21:31:28.518273+0000","last_clean":"2024-04-20T21:31:28.518273+0000","last_became_active":"2024-04-20T21:31:27.512571+0000","last_became_peered":"2024-04-20T21:31:27.512571+0000","last_unstale":"2024-04-20T21:31:28.518273+0000","last_undegraded":"2024-04-20T21:31:28.518273+0000","last_fullsized":"2024-04-20T21:31:28.518273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:30.977624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.193494+0000","last_change":"2024-04-20T21:31:27.521439+0000","last_active":"2024-04-20T21:31:29.193494+0000","last_peered":"2024-04-20T21:31:29.193494+0000","last_clean":"2024-04-20T21:31:29.193494+0000","last_became_active":"2024-04-20T21:31:27.520245+0000","last_became_peered":"2024-04-20T21:31:27.520245+0000","last_unstale":"2024-04-20T21:31:29.193494+0000","last_undegraded":"2024-04-20T21:31:29.193494+0000","last_fullsized":"2024-04-20T21:31:29.193494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:53:26.510845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.193165+0000","last_change":"2024-04-20T21:31:27.530544+0000","last_active":"2024-04-20T21:31:29.193165+0000","last_peered":"2024-04-20T21:31:29.193165+0000","last_clean":"2024-04-20T21:31:29.193165+0000","last_became_active":"2024-04-20T21:31:27.530195+0000","last_became_peered":"2024-04-20T21:31:27.530195+0000","last_unstale":"2024-04-20T21:31:29.193165+0000","last_undegraded":"2024-04-20T21:31:29.193165+0000","last_fullsized":"2024-04-20T21:31:29.193165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:28:23.472948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190775+0000","last_change":"2024-04-20T21:31:27.521533+0000","last_active":"2024-04-20T21:31:29.190775+0000","last_peered":"2024-04-20T21:31:29.190775+0000","last_clean":"2024-04-20T21:31:29.190775+0000","last_became_active":"2024-04-20T21:31:27.520920+0000","last_became_peered":"2024-04-20T21:31:27.520920+0000","last_unstale":"2024-04-20T21:31:29.190775+0000","last_undegraded":"2024-04-20T21:31:29.190775+0000","last_fullsized":"2024-04-20T21:31:29.190775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:28:13.605238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516788+0000","last_change":"2024-04-20T21:31:27.527484+0000","last_active":"2024-04-20T21:31:28.516788+0000","last_peered":"2024-04-20T21:31:28.516788+0000","last_clean":"2024-04-20T21:31:28.516788+0000","last_became_active":"2024-04-20T21:31:27.527048+0000","last_became_peered":"2024-04-20T21:31:27.527048+0000","last_unstale":"2024-04-20T21:31:28.516788+0000","last_undegraded":"2024-04-20T21:31:28.516788+0000","last_fullsized":"2024-04-20T21:31:28.516788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:35:56.467661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.680864+0000","last_change":"2024-04-20T21:31:27.527870+0000","last_active":"2024-04-20T21:31:28.680864+0000","last_peered":"2024-04-20T21:31:28.680864+0000","last_clean":"2024-04-20T21:31:28.680864+0000","last_became_active":"2024-04-20T21:31:27.525918+0000","last_became_peered":"2024-04-20T21:31:27.525918+0000","last_unstale":"2024-04-20T21:31:28.680864+0000","last_undegraded":"2024-04-20T21:31:28.680864+0000","last_fullsized":"2024-04-20T21:31:28.680864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:34.425370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191210+0000","last_change":"2024-04-20T21:31:27.544987+0000","last_active":"2024-04-20T21:31:29.191210+0000","last_peered":"2024-04-20T21:31:29.191210+0000","last_clean":"2024-04-20T21:31:29.191210+0000","last_became_active":"2024-04-20T21:31:27.544628+0000","last_became_peered":"2024-04-20T21:31:27.544628+0000","last_unstale":"2024-04-20T21:31:29.191210+0000","last_undegraded":"2024-04-20T21:31:29.191210+0000","last_fullsized":"2024-04-20T21:31:29.191210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:05:42.396775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.681074+0000","last_change":"2024-04-20T21:31:26.702803+0000","last_active":"2024-04-20T21:31:28.681074+0000","last_peered":"2024-04-20T21:31:28.681074+0000","last_clean":"2024-04-20T21:31:28.681074+0000","last_became_active":"2024-04-20T21:31:26.695781+0000","last_became_peered":"2024-04-20T21:31:26.695781+0000","last_unstale":"2024-04-20T21:31:28.681074+0000","last_undegraded":"2024-04-20T21:31:28.681074+0000","last_fullsized":"2024-04-20T21:31:28.681074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:49:17.131832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.686168+0000","last_change":"2024-04-20T21:31:27.535264+0000","last_active":"2024-04-20T21:31:28.686168+0000","last_peered":"2024-04-20T21:31:28.686168+0000","last_clean":"2024-04-20T21:31:28.686168+0000","last_became_active":"2024-04-20T21:31:27.534888+0000","last_became_peered":"2024-04-20T21:31:27.534888+0000","last_unstale":"2024-04-20T21:31:28.686168+0000","last_undegraded":"2024-04-20T21:31:28.686168+0000","last_fullsized":"2024-04-20T21:31:28.686168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:04:20.951916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.686277+0000","last_change":"2024-04-20T21:31:26.700378+0000","last_active":"2024-04-20T21:31:28.686277+0000","last_peered":"2024-04-20T21:31:28.686277+0000","last_clean":"2024-04-20T21:31:28.686277+0000","last_became_active":"2024-04-20T21:31:26.700077+0000","last_became_peered":"2024-04-20T21:31:26.700077+0000","last_unstale":"2024-04-20T21:31:28.686277+0000","last_undegraded":"2024-04-20T21:31:28.686277+0000","last_fullsized":"2024-04-20T21:31:28.686277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:34:21.646402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.194041+0000","last_change":"2024-04-20T21:31:27.531069+0000","last_active":"2024-04-20T21:31:29.194041+0000","last_peered":"2024-04-20T21:31:29.194041+0000","last_clean":"2024-04-20T21:31:29.194041+0000","last_became_active":"2024-04-20T21:31:27.530457+0000","last_became_peered":"2024-04-20T21:31:27.530457+0000","last_unstale":"2024-04-20T21:31:29.194041+0000","last_undegraded":"2024-04-20T21:31:29.194041+0000","last_fullsized":"2024-04-20T21:31:29.194041+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:46:59.829623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.684155+0000","last_change":"2024-04-20T21:31:26.684150+0000","last_active":"2024-04-20T21:31:28.684155+0000","last_peered":"2024-04-20T21:31:28.684155+0000","last_clean":"2024-04-20T21:31:28.684155+0000","last_became_active":"2024-04-20T21:31:26.683823+0000","last_became_peered":"2024-04-20T21:31:26.683823+0000","last_unstale":"2024-04-20T21:31:28.684155+0000","last_undegraded":"2024-04-20T21:31:28.684155+0000","last_fullsized":"2024-04-20T21:31:28.684155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:14.413606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191606+0000","last_change":"2024-04-20T21:31:27.507499+0000","last_active":"2024-04-20T21:31:29.191606+0000","last_peered":"2024-04-20T21:31:29.191606+0000","last_clean":"2024-04-20T21:31:29.191606+0000","last_became_active":"2024-04-20T21:31:27.506922+0000","last_became_peered":"2024-04-20T21:31:27.506922+0000","last_unstale":"2024-04-20T21:31:29.191606+0000","last_undegraded":"2024-04-20T21:31:29.191606+0000","last_fullsized":"2024-04-20T21:31:29.191606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:40.865232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.683982+0000","last_change":"2024-04-20T21:31:26.683100+0000","last_active":"2024-04-20T21:31:28.683982+0000","last_peered":"2024-04-20T21:31:28.683982+0000","last_clean":"2024-04-20T21:31:28.683982+0000","last_became_active":"2024-04-20T21:31:26.682086+0000","last_became_peered":"2024-04-20T21:31:26.682086+0000","last_unstale":"2024-04-20T21:31:28.683982+0000","last_undegraded":"2024-04-20T21:31:28.683982+0000","last_fullsized":"2024-04-20T21:31:28.683982+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:26.215174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191591+0000","last_change":"2024-04-20T21:31:27.517839+0000","last_active":"2024-04-20T21:31:29.191591+0000","last_peered":"2024-04-20T21:31:29.191591+0000","last_clean":"2024-04-20T21:31:29.191591+0000","last_became_active":"2024-04-20T21:31:27.517044+0000","last_became_peered":"2024-04-20T21:31:27.517044+0000","last_unstale":"2024-04-20T21:31:29.191591+0000","last_undegraded":"2024-04-20T21:31:29.191591+0000","last_fullsized":"2024-04-20T21:31:29.191591+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:50.042665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190859+0000","last_change":"2024-04-20T21:31:26.686151+0000","last_active":"2024-04-20T21:31:29.190859+0000","last_peered":"2024-04-20T21:31:29.190859+0000","last_clean":"2024-04-20T21:31:29.190859+0000","last_became_active":"2024-04-20T21:31:26.685827+0000","last_became_peered":"2024-04-20T21:31:26.685827+0000","last_unstale":"2024-04-20T21:31:29.190859+0000","last_undegraded":"2024-04-20T21:31:29.190859+0000","last_fullsized":"2024-04-20T21:31:29.190859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:41:17.263420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.683760+0000","last_change":"2024-04-20T21:31:27.526387+0000","last_active":"2024-04-20T21:31:28.683760+0000","last_peered":"2024-04-20T21:31:28.683760+0000","last_clean":"2024-04-20T21:31:28.683760+0000","last_became_active":"2024-04-20T21:31:27.525912+0000","last_became_peered":"2024-04-20T21:31:27.525912+0000","last_unstale":"2024-04-20T21:31:28.683760+0000","last_undegraded":"2024-04-20T21:31:28.683760+0000","last_fullsized":"2024-04-20T21:31:28.683760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:34:24.471806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191443+0000","last_change":"2024-04-20T21:31:26.684573+0000","last_active":"2024-04-20T21:31:29.191443+0000","last_peered":"2024-04-20T21:31:29.191443+0000","last_clean":"2024-04-20T21:31:29.191443+0000","last_became_active":"2024-04-20T21:31:26.684163+0000","last_became_peered":"2024-04-20T21:31:26.684163+0000","last_unstale":"2024-04-20T21:31:29.191443+0000","last_undegraded":"2024-04-20T21:31:29.191443+0000","last_fullsized":"2024-04-20T21:31:29.191443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:39.445068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191067+0000","last_change":"2024-04-20T21:31:27.543680+0000","last_active":"2024-04-20T21:31:29.191067+0000","last_peered":"2024-04-20T21:31:29.191067+0000","last_clean":"2024-04-20T21:31:29.191067+0000","last_became_active":"2024-04-20T21:31:27.543379+0000","last_became_peered":"2024-04-20T21:31:27.543379+0000","last_unstale":"2024-04-20T21:31:29.191067+0000","last_undegraded":"2024-04-20T21:31:29.191067+0000","last_fullsized":"2024-04-20T21:31:29.191067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:58:17.168813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.517404+0000","last_change":"2024-04-20T21:31:26.689665+0000","last_active":"2024-04-20T21:31:28.517404+0000","last_peered":"2024-04-20T21:31:28.517404+0000","last_clean":"2024-04-20T21:31:28.517404+0000","last_became_active":"2024-04-20T21:31:26.689175+0000","last_became_peered":"2024-04-20T21:31:26.689175+0000","last_unstale":"2024-04-20T21:31:28.517404+0000","last_undegraded":"2024-04-20T21:31:28.517404+0000","last_fullsized":"2024-04-20T21:31:28.517404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:21.342700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.685136+0000","last_change":"2024-04-20T21:31:27.544137+0000","last_active":"2024-04-20T21:31:28.685136+0000","last_peered":"2024-04-20T21:31:28.685136+0000","last_clean":"2024-04-20T21:31:28.685136+0000","last_became_active":"2024-04-20T21:31:27.542841+0000","last_became_peered":"2024-04-20T21:31:27.542841+0000","last_unstale":"2024-04-20T21:31:28.685136+0000","last_undegraded":"2024-04-20T21:31:28.685136+0000","last_fullsized":"2024-04-20T21:31:28.685136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:04.610144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190846+0000","last_change":"2024-04-20T21:31:26.681937+0000","last_active":"2024-04-20T21:31:29.190846+0000","last_peered":"2024-04-20T21:31:29.190846+0000","last_clean":"2024-04-20T21:31:29.190846+0000","last_became_active":"2024-04-20T21:31:26.681621+0000","last_became_peered":"2024-04-20T21:31:26.681621+0000","last_unstale":"2024-04-20T21:31:29.190846+0000","last_undegraded":"2024-04-20T21:31:29.190846+0000","last_fullsized":"2024-04-20T21:31:29.190846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:06.641608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.679993+0000","last_change":"2024-04-20T21:31:27.540225+0000","last_active":"2024-04-20T21:31:28.679993+0000","last_peered":"2024-04-20T21:31:28.679993+0000","last_clean":"2024-04-20T21:31:28.679993+0000","last_became_active":"2024-04-20T21:31:27.539860+0000","last_became_peered":"2024-04-20T21:31:27.539860+0000","last_unstale":"2024-04-20T21:31:28.679993+0000","last_undegraded":"2024-04-20T21:31:28.679993+0000","last_fullsized":"2024-04-20T21:31:28.679993+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:50.146761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192405+0000","last_change":"2024-04-20T21:31:26.701816+0000","last_active":"2024-04-20T21:31:29.192405+0000","last_peered":"2024-04-20T21:31:29.192405+0000","last_clean":"2024-04-20T21:31:29.192405+0000","last_became_active":"2024-04-20T21:31:26.701530+0000","last_became_peered":"2024-04-20T21:31:26.701530+0000","last_unstale":"2024-04-20T21:31:29.192405+0000","last_undegraded":"2024-04-20T21:31:29.192405+0000","last_fullsized":"2024-04-20T21:31:29.192405+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:31.175330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191307+0000","last_change":"2024-04-20T21:31:27.527222+0000","last_active":"2024-04-20T21:31:29.191307+0000","last_peered":"2024-04-20T21:31:29.191307+0000","last_clean":"2024-04-20T21:31:29.191307+0000","last_became_active":"2024-04-20T21:31:27.526779+0000","last_became_peered":"2024-04-20T21:31:27.526779+0000","last_unstale":"2024-04-20T21:31:29.191307+0000","last_undegraded":"2024-04-20T21:31:29.191307+0000","last_fullsized":"2024-04-20T21:31:29.191307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:13:54.500516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191164+0000","last_change":"2024-04-20T21:31:26.685969+0000","last_active":"2024-04-20T21:31:29.191164+0000","last_peered":"2024-04-20T21:31:29.191164+0000","last_clean":"2024-04-20T21:31:29.191164+0000","last_became_active":"2024-04-20T21:31:26.685580+0000","last_became_peered":"2024-04-20T21:31:26.685580+0000","last_unstale":"2024-04-20T21:31:29.191164+0000","last_undegraded":"2024-04-20T21:31:29.191164+0000","last_fullsized":"2024-04-20T21:31:29.191164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:22:50.558526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.194009+0000","last_change":"2024-04-20T21:31:27.525862+0000","last_active":"2024-04-20T21:31:29.194009+0000","last_peered":"2024-04-20T21:31:29.194009+0000","last_clean":"2024-04-20T21:31:29.194009+0000","last_became_active":"2024-04-20T21:31:27.525372+0000","last_became_peered":"2024-04-20T21:31:27.525372+0000","last_unstale":"2024-04-20T21:31:29.194009+0000","last_undegraded":"2024-04-20T21:31:29.194009+0000","last_fullsized":"2024-04-20T21:31:29.194009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:44:15.527137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191413+0000","last_change":"2024-04-20T21:31:26.691091+0000","last_active":"2024-04-20T21:31:29.191413+0000","last_peered":"2024-04-20T21:31:29.191413+0000","last_clean":"2024-04-20T21:31:29.191413+0000","last_became_active":"2024-04-20T21:31:26.690608+0000","last_became_peered":"2024-04-20T21:31:26.690608+0000","last_unstale":"2024-04-20T21:31:29.191413+0000","last_undegraded":"2024-04-20T21:31:29.191413+0000","last_fullsized":"2024-04-20T21:31:29.191413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:44:25.336193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191871+0000","last_change":"2024-04-20T21:31:27.507900+0000","last_active":"2024-04-20T21:31:29.191871+0000","last_peered":"2024-04-20T21:31:29.191871+0000","last_clean":"2024-04-20T21:31:29.191871+0000","last_became_active":"2024-04-20T21:31:27.507360+0000","last_became_peered":"2024-04-20T21:31:27.507360+0000","last_unstale":"2024-04-20T21:31:29.191871+0000","last_undegraded":"2024-04-20T21:31:29.191871+0000","last_fullsized":"2024-04-20T21:31:29.191871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:27.110877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'13","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:33.053729+0000","last_change":"2024-04-20T21:31:26.685736+0000","last_active":"2024-04-20T21:31:33.053729+0000","last_peered":"2024-04-20T21:31:33.053729+0000","last_clean":"2024-04-20T21:31:33.053729+0000","last_became_active":"2024-04-20T21:31:26.685309+0000","last_became_peered":"2024-04-20T21:31:26.685309+0000","last_unstale":"2024-04-20T21:31:33.053729+0000","last_undegraded":"2024-04-20T21:31:33.053729+0000","last_fullsized":"2024-04-20T21:31:33.053729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:06.501753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":12,"num_write":13,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.684010+0000","last_change":"2024-04-20T21:31:27.507532+0000","last_active":"2024-04-20T21:31:28.684010+0000","last_peered":"2024-04-20T21:31:28.684010+0000","last_clean":"2024-04-20T21:31:28.684010+0000","last_became_active":"2024-04-20T21:31:27.507188+0000","last_became_peered":"2024-04-20T21:31:27.507188+0000","last_unstale":"2024-04-20T21:31:28.684010+0000","last_undegraded":"2024-04-20T21:31:28.684010+0000","last_fullsized":"2024-04-20T21:31:28.684010+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:58.814116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:30.478395+0000","last_change":"2024-04-20T21:31:26.701361+0000","last_active":"2024-04-20T21:31:30.478395+0000","last_peered":"2024-04-20T21:31:30.478395+0000","last_clean":"2024-04-20T21:31:30.478395+0000","last_became_active":"2024-04-20T21:31:26.701068+0000","last_became_peered":"2024-04-20T21:31:26.701068+0000","last_unstale":"2024-04-20T21:31:30.478395+0000","last_undegraded":"2024-04-20T21:31:30.478395+0000","last_fullsized":"2024-04-20T21:31:30.478395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:22:51.745294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.517816+0000","last_change":"2024-04-20T21:31:27.542189+0000","last_active":"2024-04-20T21:31:28.517816+0000","last_peered":"2024-04-20T21:31:28.517816+0000","last_clean":"2024-04-20T21:31:28.517816+0000","last_became_active":"2024-04-20T21:31:27.541800+0000","last_became_peered":"2024-04-20T21:31:27.541800+0000","last_unstale":"2024-04-20T21:31:28.517816+0000","last_undegraded":"2024-04-20T21:31:28.517816+0000","last_fullsized":"2024-04-20T21:31:28.517816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:31.898443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191008+0000","last_change":"2024-04-20T21:31:26.687652+0000","last_active":"2024-04-20T21:31:29.191008+0000","last_peered":"2024-04-20T21:31:29.191008+0000","last_clean":"2024-04-20T21:31:29.191008+0000","last_became_active":"2024-04-20T21:31:26.687333+0000","last_became_peered":"2024-04-20T21:31:26.687333+0000","last_unstale":"2024-04-20T21:31:29.191008+0000","last_undegraded":"2024-04-20T21:31:29.191008+0000","last_fullsized":"2024-04-20T21:31:29.191008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:12.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192615+0000","last_change":"2024-04-20T21:31:27.522535+0000","last_active":"2024-04-20T21:31:29.192615+0000","last_peered":"2024-04-20T21:31:29.192615+0000","last_clean":"2024-04-20T21:31:29.192615+0000","last_became_active":"2024-04-20T21:31:27.521821+0000","last_became_peered":"2024-04-20T21:31:27.521821+0000","last_unstale":"2024-04-20T21:31:29.192615+0000","last_undegraded":"2024-04-20T21:31:29.192615+0000","last_fullsized":"2024-04-20T21:31:29.192615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:47:53.401129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516185+0000","last_change":"2024-04-20T21:31:26.681758+0000","last_active":"2024-04-20T21:31:28.516185+0000","last_peered":"2024-04-20T21:31:28.516185+0000","last_clean":"2024-04-20T21:31:28.516185+0000","last_became_active":"2024-04-20T21:31:26.681395+0000","last_became_peered":"2024-04-20T21:31:26.681395+0000","last_unstale":"2024-04-20T21:31:28.516185+0000","last_undegraded":"2024-04-20T21:31:28.516185+0000","last_fullsized":"2024-04-20T21:31:28.516185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:33:00.924879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191875+0000","last_change":"2024-04-20T21:31:27.512873+0000","last_active":"2024-04-20T21:31:29.191875+0000","last_peered":"2024-04-20T21:31:29.191875+0000","last_clean":"2024-04-20T21:31:29.191875+0000","last_became_active":"2024-04-20T21:31:27.512554+0000","last_became_peered":"2024-04-20T21:31:27.512554+0000","last_unstale":"2024-04-20T21:31:29.191875+0000","last_undegraded":"2024-04-20T21:31:29.191875+0000","last_fullsized":"2024-04-20T21:31:29.191875+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:26:06.670726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.683579+0000","last_change":"2024-04-20T21:31:26.691210+0000","last_active":"2024-04-20T21:31:28.683579+0000","last_peered":"2024-04-20T21:31:28.683579+0000","last_clean":"2024-04-20T21:31:28.683579+0000","last_became_active":"2024-04-20T21:31:26.690859+0000","last_became_peered":"2024-04-20T21:31:26.690859+0000","last_unstale":"2024-04-20T21:31:28.683579+0000","last_undegraded":"2024-04-20T21:31:28.683579+0000","last_fullsized":"2024-04-20T21:31:28.683579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:36.789429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190955+0000","last_change":"2024-04-20T21:31:27.517593+0000","last_active":"2024-04-20T21:31:29.190955+0000","last_peered":"2024-04-20T21:31:29.190955+0000","last_clean":"2024-04-20T21:31:29.190955+0000","last_became_active":"2024-04-20T21:31:27.516580+0000","last_became_peered":"2024-04-20T21:31:27.516580+0000","last_unstale":"2024-04-20T21:31:29.190955+0000","last_undegraded":"2024-04-20T21:31:29.190955+0000","last_fullsized":"2024-04-20T21:31:29.190955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:54:26.335431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.735177+0000","last_change":"2024-04-20T21:31:26.682311+0000","last_active":"2024-04-20T21:31:29.735177+0000","last_peered":"2024-04-20T21:31:29.735177+0000","last_clean":"2024-04-20T21:31:29.735177+0000","last_became_active":"2024-04-20T21:31:26.681996+0000","last_became_peered":"2024-04-20T21:31:26.681996+0000","last_unstale":"2024-04-20T21:31:29.735177+0000","last_undegraded":"2024-04-20T21:31:29.735177+0000","last_fullsized":"2024-04-20T21:31:29.735177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:32:37.659946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516725+0000","last_change":"2024-04-20T21:31:27.510922+0000","last_active":"2024-04-20T21:31:28.516725+0000","last_peered":"2024-04-20T21:31:28.516725+0000","last_clean":"2024-04-20T21:31:28.516725+0000","last_became_active":"2024-04-20T21:31:27.510553+0000","last_became_peered":"2024-04-20T21:31:27.510553+0000","last_unstale":"2024-04-20T21:31:28.516725+0000","last_undegraded":"2024-04-20T21:31:28.516725+0000","last_fullsized":"2024-04-20T21:31:28.516725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:23.754768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.680388+0000","last_change":"2024-04-20T21:31:26.697010+0000","last_active":"2024-04-20T21:31:28.680388+0000","last_peered":"2024-04-20T21:31:28.680388+0000","last_clean":"2024-04-20T21:31:28.680388+0000","last_became_active":"2024-04-20T21:31:26.696112+0000","last_became_peered":"2024-04-20T21:31:26.696112+0000","last_unstale":"2024-04-20T21:31:28.680388+0000","last_undegraded":"2024-04-20T21:31:28.680388+0000","last_fullsized":"2024-04-20T21:31:28.680388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:57:02.715884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192499+0000","last_change":"2024-04-20T21:31:27.545025+0000","last_active":"2024-04-20T21:31:29.192499+0000","last_peered":"2024-04-20T21:31:29.192499+0000","last_clean":"2024-04-20T21:31:29.192499+0000","last_became_active":"2024-04-20T21:31:27.544630+0000","last_became_peered":"2024-04-20T21:31:27.544630+0000","last_unstale":"2024-04-20T21:31:29.192499+0000","last_undegraded":"2024-04-20T21:31:29.192499+0000","last_fullsized":"2024-04-20T21:31:29.192499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:57.292481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191876+0000","last_change":"2024-04-20T21:31:26.682909+0000","last_active":"2024-04-20T21:31:29.191876+0000","last_peered":"2024-04-20T21:31:29.191876+0000","last_clean":"2024-04-20T21:31:29.191876+0000","last_became_active":"2024-04-20T21:31:26.682597+0000","last_became_peered":"2024-04-20T21:31:26.682597+0000","last_unstale":"2024-04-20T21:31:29.191876+0000","last_undegraded":"2024-04-20T21:31:29.191876+0000","last_fullsized":"2024-04-20T21:31:29.191876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:31.889513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192506+0000","last_change":"2024-04-20T21:31:27.530217+0000","last_active":"2024-04-20T21:31:29.192506+0000","last_peered":"2024-04-20T21:31:29.192506+0000","last_clean":"2024-04-20T21:31:29.192506+0000","last_became_active":"2024-04-20T21:31:27.529780+0000","last_became_peered":"2024-04-20T21:31:27.529780+0000","last_unstale":"2024-04-20T21:31:29.192506+0000","last_undegraded":"2024-04-20T21:31:29.192506+0000","last_fullsized":"2024-04-20T21:31:29.192506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:21.613460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191676+0000","last_change":"2024-04-20T21:31:27.518321+0000","last_active":"2024-04-20T21:31:29.191676+0000","last_peered":"2024-04-20T21:31:29.191676+0000","last_clean":"2024-04-20T21:31:29.191676+0000","last_became_active":"2024-04-20T21:31:27.518011+0000","last_became_peered":"2024-04-20T21:31:27.518011+0000","last_unstale":"2024-04-20T21:31:29.191676+0000","last_undegraded":"2024-04-20T21:31:29.191676+0000","last_fullsized":"2024-04-20T21:31:29.191676+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:50:55.457996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.517080+0000","last_change":"2024-04-20T21:31:26.692845+0000","last_active":"2024-04-20T21:31:28.517080+0000","last_peered":"2024-04-20T21:31:28.517080+0000","last_clean":"2024-04-20T21:31:28.517080+0000","last_became_active":"2024-04-20T21:31:26.691915+0000","last_became_peered":"2024-04-20T21:31:26.691915+0000","last_unstale":"2024-04-20T21:31:28.517080+0000","last_undegraded":"2024-04-20T21:31:28.517080+0000","last_fullsized":"2024-04-20T21:31:28.517080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:24:00.436955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.686312+0000","last_change":"2024-04-20T21:31:27.541623+0000","last_active":"2024-04-20T21:31:28.686312+0000","last_peered":"2024-04-20T21:31:28.686312+0000","last_clean":"2024-04-20T21:31:28.686312+0000","last_became_active":"2024-04-20T21:31:27.540953+0000","last_became_peered":"2024-04-20T21:31:27.540953+0000","last_unstale":"2024-04-20T21:31:28.686312+0000","last_undegraded":"2024-04-20T21:31:28.686312+0000","last_fullsized":"2024-04-20T21:31:28.686312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:03:15.091886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516047+0000","last_change":"2024-04-20T21:31:26.687373+0000","last_active":"2024-04-20T21:31:28.516047+0000","last_peered":"2024-04-20T21:31:28.516047+0000","last_clean":"2024-04-20T21:31:28.516047+0000","last_became_active":"2024-04-20T21:31:26.687009+0000","last_became_peered":"2024-04-20T21:31:26.687009+0000","last_unstale":"2024-04-20T21:31:28.516047+0000","last_undegraded":"2024-04-20T21:31:28.516047+0000","last_fullsized":"2024-04-20T21:31:28.516047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:23:42.708129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":346,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191964+0000","last_change":"2024-04-20T21:30:06.995711+0000","last_active":"2024-04-20T21:31:29.191964+0000","last_peered":"2024-04-20T21:31:29.191964+0000","last_clean":"2024-04-20T21:31:29.191964+0000","last_became_active":"2024-04-20T21:30:06.995280+0000","last_became_peered":"2024-04-20T21:30:06.995280+0000","last_unstale":"2024-04-20T21:31:29.191964+0000","last_undegraded":"2024-04-20T21:31:29.191964+0000","last_fullsized":"2024-04-20T21:31:29.191964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_clean_scrub_stamp":"2024-04-20T21:26:16.477494+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:49:26.014145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.517237+0000","last_change":"2024-04-20T21:31:27.514064+0000","last_active":"2024-04-20T21:31:28.517237+0000","last_peered":"2024-04-20T21:31:28.517237+0000","last_clean":"2024-04-20T21:31:28.517237+0000","last_became_active":"2024-04-20T21:31:27.513416+0000","last_became_peered":"2024-04-20T21:31:27.513416+0000","last_unstale":"2024-04-20T21:31:28.517237+0000","last_undegraded":"2024-04-20T21:31:28.517237+0000","last_fullsized":"2024-04-20T21:31:28.517237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:12.658265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192230+0000","last_change":"2024-04-20T21:31:26.680948+0000","last_active":"2024-04-20T21:31:29.192230+0000","last_peered":"2024-04-20T21:31:29.192230+0000","last_clean":"2024-04-20T21:31:29.192230+0000","last_became_active":"2024-04-20T21:31:26.680624+0000","last_became_peered":"2024-04-20T21:31:26.680624+0000","last_unstale":"2024-04-20T21:31:29.192230+0000","last_undegraded":"2024-04-20T21:31:29.192230+0000","last_fullsized":"2024-04-20T21:31:29.192230+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:02.750166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.684052+0000","last_change":"2024-04-20T21:31:26.697632+0000","last_active":"2024-04-20T21:31:28.684052+0000","last_peered":"2024-04-20T21:31:28.684052+0000","last_clean":"2024-04-20T21:31:28.684052+0000","last_became_active":"2024-04-20T21:31:26.697393+0000","last_became_peered":"2024-04-20T21:31:26.697393+0000","last_unstale":"2024-04-20T21:31:28.684052+0000","last_undegraded":"2024-04-20T21:31:28.684052+0000","last_fullsized":"2024-04-20T21:31:28.684052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:08:39.856082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.517845+0000","last_change":"2024-04-20T21:31:27.526613+0000","last_active":"2024-04-20T21:31:28.517845+0000","last_peered":"2024-04-20T21:31:28.517845+0000","last_clean":"2024-04-20T21:31:28.517845+0000","last_became_active":"2024-04-20T21:31:27.526234+0000","last_became_peered":"2024-04-20T21:31:27.526234+0000","last_unstale":"2024-04-20T21:31:28.517845+0000","last_undegraded":"2024-04-20T21:31:28.517845+0000","last_fullsized":"2024-04-20T21:31:28.517845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:06:44.114924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.683912+0000","last_change":"2024-04-20T21:31:26.700040+0000","last_active":"2024-04-20T21:31:28.683912+0000","last_peered":"2024-04-20T21:31:28.683912+0000","last_clean":"2024-04-20T21:31:28.683912+0000","last_became_active":"2024-04-20T21:31:26.699559+0000","last_became_peered":"2024-04-20T21:31:26.699559+0000","last_unstale":"2024-04-20T21:31:28.683912+0000","last_undegraded":"2024-04-20T21:31:28.683912+0000","last_fullsized":"2024-04-20T21:31:28.683912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:03.058778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.686164+0000","last_change":"2024-04-20T21:31:27.538361+0000","last_active":"2024-04-20T21:31:28.686164+0000","last_peered":"2024-04-20T21:31:28.686164+0000","last_clean":"2024-04-20T21:31:28.686164+0000","last_became_active":"2024-04-20T21:31:27.537494+0000","last_became_peered":"2024-04-20T21:31:27.537494+0000","last_unstale":"2024-04-20T21:31:28.686164+0000","last_undegraded":"2024-04-20T21:31:28.686164+0000","last_fullsized":"2024-04-20T21:31:28.686164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:46.212759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516325+0000","last_change":"2024-04-20T21:31:26.686960+0000","last_active":"2024-04-20T21:31:28.516325+0000","last_peered":"2024-04-20T21:31:28.516325+0000","last_clean":"2024-04-20T21:31:28.516325+0000","last_became_active":"2024-04-20T21:31:26.686565+0000","last_became_peered":"2024-04-20T21:31:26.686565+0000","last_unstale":"2024-04-20T21:31:28.516325+0000","last_undegraded":"2024-04-20T21:31:28.516325+0000","last_fullsized":"2024-04-20T21:31:28.516325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.608834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.194136+0000","last_change":"2024-04-20T21:31:27.546978+0000","last_active":"2024-04-20T21:31:29.194136+0000","last_peered":"2024-04-20T21:31:29.194136+0000","last_clean":"2024-04-20T21:31:29.194136+0000","last_became_active":"2024-04-20T21:31:27.545941+0000","last_became_peered":"2024-04-20T21:31:27.545941+0000","last_unstale":"2024-04-20T21:31:29.194136+0000","last_undegraded":"2024-04-20T21:31:29.194136+0000","last_fullsized":"2024-04-20T21:31:29.194136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:08.363519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190766+0000","last_change":"2024-04-20T21:31:26.690797+0000","last_active":"2024-04-20T21:31:29.190766+0000","last_peered":"2024-04-20T21:31:29.190766+0000","last_clean":"2024-04-20T21:31:29.190766+0000","last_became_active":"2024-04-20T21:31:26.690486+0000","last_became_peered":"2024-04-20T21:31:26.690486+0000","last_unstale":"2024-04-20T21:31:29.190766+0000","last_undegraded":"2024-04-20T21:31:29.190766+0000","last_fullsized":"2024-04-20T21:31:29.190766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:17:33.943212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.683652+0000","last_change":"2024-04-20T21:31:27.507175+0000","last_active":"2024-04-20T21:31:28.683652+0000","last_peered":"2024-04-20T21:31:28.683652+0000","last_clean":"2024-04-20T21:31:28.683652+0000","last_became_active":"2024-04-20T21:31:27.506808+0000","last_became_peered":"2024-04-20T21:31:27.506808+0000","last_unstale":"2024-04-20T21:31:28.683652+0000","last_undegraded":"2024-04-20T21:31:28.683652+0000","last_fullsized":"2024-04-20T21:31:28.683652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:38:15.212880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.684949+0000","last_change":"2024-04-20T21:31:26.698302+0000","last_active":"2024-04-20T21:31:28.684949+0000","last_peered":"2024-04-20T21:31:28.684949+0000","last_clean":"2024-04-20T21:31:28.684949+0000","last_became_active":"2024-04-20T21:31:26.696209+0000","last_became_peered":"2024-04-20T21:31:26.696209+0000","last_unstale":"2024-04-20T21:31:28.684949+0000","last_undegraded":"2024-04-20T21:31:28.684949+0000","last_fullsized":"2024-04-20T21:31:28.684949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:43.495880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516519+0000","last_change":"2024-04-20T21:31:27.521594+0000","last_active":"2024-04-20T21:31:28.516519+0000","last_peered":"2024-04-20T21:31:28.516519+0000","last_clean":"2024-04-20T21:31:28.516519+0000","last_became_active":"2024-04-20T21:31:27.521185+0000","last_became_peered":"2024-04-20T21:31:27.521185+0000","last_unstale":"2024-04-20T21:31:28.516519+0000","last_undegraded":"2024-04-20T21:31:28.516519+0000","last_fullsized":"2024-04-20T21:31:28.516519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:22:08.647624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190795+0000","last_change":"2024-04-20T21:31:26.691536+0000","last_active":"2024-04-20T21:31:29.190795+0000","last_peered":"2024-04-20T21:31:29.190795+0000","last_clean":"2024-04-20T21:31:29.190795+0000","last_became_active":"2024-04-20T21:31:26.691202+0000","last_became_peered":"2024-04-20T21:31:26.691202+0000","last_unstale":"2024-04-20T21:31:29.190795+0000","last_undegraded":"2024-04-20T21:31:29.190795+0000","last_fullsized":"2024-04-20T21:31:29.190795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:58:59.331644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.685423+0000","last_change":"2024-04-20T21:31:27.543799+0000","last_active":"2024-04-20T21:31:28.685423+0000","last_peered":"2024-04-20T21:31:28.685423+0000","last_clean":"2024-04-20T21:31:28.685423+0000","last_became_active":"2024-04-20T21:31:27.542061+0000","last_became_peered":"2024-04-20T21:31:27.542061+0000","last_unstale":"2024-04-20T21:31:28.685423+0000","last_undegraded":"2024-04-20T21:31:28.685423+0000","last_fullsized":"2024-04-20T21:31:28.685423+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:14.648628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.735118+0000","last_change":"2024-04-20T21:31:26.685963+0000","last_active":"2024-04-20T21:31:29.735118+0000","last_peered":"2024-04-20T21:31:29.735118+0000","last_clean":"2024-04-20T21:31:29.735118+0000","last_became_active":"2024-04-20T21:31:26.685647+0000","last_became_peered":"2024-04-20T21:31:26.685647+0000","last_unstale":"2024-04-20T21:31:29.735118+0000","last_undegraded":"2024-04-20T21:31:29.735118+0000","last_fullsized":"2024-04-20T21:31:29.735118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:56.109426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191498+0000","last_change":"2024-04-20T21:31:27.518346+0000","last_active":"2024-04-20T21:31:29.191498+0000","last_peered":"2024-04-20T21:31:29.191498+0000","last_clean":"2024-04-20T21:31:29.191498+0000","last_became_active":"2024-04-20T21:31:27.517782+0000","last_became_peered":"2024-04-20T21:31:27.517782+0000","last_unstale":"2024-04-20T21:31:29.191498+0000","last_undegraded":"2024-04-20T21:31:29.191498+0000","last_fullsized":"2024-04-20T21:31:29.191498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:21.674663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190809+0000","last_change":"2024-04-20T21:31:26.700335+0000","last_active":"2024-04-20T21:31:29.190809+0000","last_peered":"2024-04-20T21:31:29.190809+0000","last_clean":"2024-04-20T21:31:29.190809+0000","last_became_active":"2024-04-20T21:31:26.699965+0000","last_became_peered":"2024-04-20T21:31:26.699965+0000","last_unstale":"2024-04-20T21:31:29.190809+0000","last_undegraded":"2024-04-20T21:31:29.190809+0000","last_fullsized":"2024-04-20T21:31:29.190809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:21:40.626706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191836+0000","last_change":"2024-04-20T21:31:27.510920+0000","last_active":"2024-04-20T21:31:29.191836+0000","last_peered":"2024-04-20T21:31:29.191836+0000","last_clean":"2024-04-20T21:31:29.191836+0000","last_became_active":"2024-04-20T21:31:27.510564+0000","last_became_peered":"2024-04-20T21:31:27.510564+0000","last_unstale":"2024-04-20T21:31:29.191836+0000","last_undegraded":"2024-04-20T21:31:29.191836+0000","last_fullsized":"2024-04-20T21:31:29.191836+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:05.196178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:30.303208+0000","last_change":"2024-04-20T21:31:26.693159+0000","last_active":"2024-04-20T21:31:30.303208+0000","last_peered":"2024-04-20T21:31:30.303208+0000","last_clean":"2024-04-20T21:31:30.303208+0000","last_became_active":"2024-04-20T21:31:26.692902+0000","last_became_peered":"2024-04-20T21:31:26.692902+0000","last_unstale":"2024-04-20T21:31:30.303208+0000","last_undegraded":"2024-04-20T21:31:30.303208+0000","last_fullsized":"2024-04-20T21:31:30.303208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:54.704343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.657322+0000","last_change":"2024-04-20T21:31:27.541557+0000","last_active":"2024-04-20T21:31:29.657322+0000","last_peered":"2024-04-20T21:31:29.657322+0000","last_clean":"2024-04-20T21:31:29.657322+0000","last_became_active":"2024-04-20T21:31:27.541161+0000","last_became_peered":"2024-04-20T21:31:27.541161+0000","last_unstale":"2024-04-20T21:31:29.657322+0000","last_undegraded":"2024-04-20T21:31:29.657322+0000","last_fullsized":"2024-04-20T21:31:29.657322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:55:25.558918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.518067+0000","last_change":"2024-04-20T21:31:26.688019+0000","last_active":"2024-04-20T21:31:28.518067+0000","last_peered":"2024-04-20T21:31:28.518067+0000","last_clean":"2024-04-20T21:31:28.518067+0000","last_became_active":"2024-04-20T21:31:26.686370+0000","last_became_peered":"2024-04-20T21:31:26.686370+0000","last_unstale":"2024-04-20T21:31:28.518067+0000","last_undegraded":"2024-04-20T21:31:28.518067+0000","last_fullsized":"2024-04-20T21:31:28.518067+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:42:44.059365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192085+0000","last_change":"2024-04-20T21:31:27.511592+0000","last_active":"2024-04-20T21:31:29.192085+0000","last_peered":"2024-04-20T21:31:29.192085+0000","last_clean":"2024-04-20T21:31:29.192085+0000","last_became_active":"2024-04-20T21:31:27.511311+0000","last_became_peered":"2024-04-20T21:31:27.511311+0000","last_unstale":"2024-04-20T21:31:29.192085+0000","last_undegraded":"2024-04-20T21:31:29.192085+0000","last_fullsized":"2024-04-20T21:31:29.192085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:35:27.665578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516616+0000","last_change":"2024-04-20T21:31:27.511827+0000","last_active":"2024-04-20T21:31:28.516616+0000","last_peered":"2024-04-20T21:31:28.516616+0000","last_clean":"2024-04-20T21:31:28.516616+0000","last_became_active":"2024-04-20T21:31:27.511344+0000","last_became_peered":"2024-04-20T21:31:27.511344+0000","last_unstale":"2024-04-20T21:31:28.516616+0000","last_undegraded":"2024-04-20T21:31:28.516616+0000","last_fullsized":"2024-04-20T21:31:28.516616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:11:37.973050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190634+0000","last_change":"2024-04-20T21:31:27.510747+0000","last_active":"2024-04-20T21:31:29.190634+0000","last_peered":"2024-04-20T21:31:29.190634+0000","last_clean":"2024-04-20T21:31:29.190634+0000","last_became_active":"2024-04-20T21:31:27.510253+0000","last_became_peered":"2024-04-20T21:31:27.510253+0000","last_unstale":"2024-04-20T21:31:29.190634+0000","last_undegraded":"2024-04-20T21:31:29.190634+0000","last_fullsized":"2024-04-20T21:31:29.190634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:37:13.855840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190674+0000","last_change":"2024-04-20T21:31:27.541185+0000","last_active":"2024-04-20T21:31:29.190674+0000","last_peered":"2024-04-20T21:31:29.190674+0000","last_clean":"2024-04-20T21:31:29.190674+0000","last_became_active":"2024-04-20T21:31:27.540881+0000","last_became_peered":"2024-04-20T21:31:27.540881+0000","last_unstale":"2024-04-20T21:31:29.190674+0000","last_undegraded":"2024-04-20T21:31:29.190674+0000","last_fullsized":"2024-04-20T21:31:29.190674+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:46:41.897081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.193900+0000","last_change":"2024-04-20T21:31:27.527204+0000","last_active":"2024-04-20T21:31:29.193900+0000","last_peered":"2024-04-20T21:31:29.193900+0000","last_clean":"2024-04-20T21:31:29.193900+0000","last_became_active":"2024-04-20T21:31:27.526805+0000","last_became_peered":"2024-04-20T21:31:27.526805+0000","last_unstale":"2024-04-20T21:31:29.193900+0000","last_undegraded":"2024-04-20T21:31:29.193900+0000","last_fullsized":"2024-04-20T21:31:29.193900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:43.032609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.517700+0000","last_change":"2024-04-20T21:31:27.525521+0000","last_active":"2024-04-20T21:31:28.517700+0000","last_peered":"2024-04-20T21:31:28.517700+0000","last_clean":"2024-04-20T21:31:28.517700+0000","last_became_active":"2024-04-20T21:31:27.521892+0000","last_became_peered":"2024-04-20T21:31:27.521892+0000","last_unstale":"2024-04-20T21:31:28.517700+0000","last_undegraded":"2024-04-20T21:31:28.517700+0000","last_fullsized":"2024-04-20T21:31:28.517700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:32:47.683115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516776+0000","last_change":"2024-04-20T21:31:27.509828+0000","last_active":"2024-04-20T21:31:28.516776+0000","last_peered":"2024-04-20T21:31:28.516776+0000","last_clean":"2024-04-20T21:31:28.516776+0000","last_became_active":"2024-04-20T21:31:27.509269+0000","last_became_peered":"2024-04-20T21:31:27.509269+0000","last_unstale":"2024-04-20T21:31:28.516776+0000","last_undegraded":"2024-04-20T21:31:28.516776+0000","last_fullsized":"2024-04-20T21:31:28.516776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:25:54.492533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191333+0000","last_change":"2024-04-20T21:31:27.512542+0000","last_active":"2024-04-20T21:31:29.191333+0000","last_peered":"2024-04-20T21:31:29.191333+0000","last_clean":"2024-04-20T21:31:29.191333+0000","last_became_active":"2024-04-20T21:31:27.511907+0000","last_became_peered":"2024-04-20T21:31:27.511907+0000","last_unstale":"2024-04-20T21:31:29.191333+0000","last_undegraded":"2024-04-20T21:31:29.191333+0000","last_fullsized":"2024-04-20T21:31:29.191333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:10:09.265471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.685888+0000","last_change":"2024-04-20T21:31:27.538697+0000","last_active":"2024-04-20T21:31:28.685888+0000","last_peered":"2024-04-20T21:31:28.685888+0000","last_clean":"2024-04-20T21:31:28.685888+0000","last_became_active":"2024-04-20T21:31:27.537972+0000","last_became_peered":"2024-04-20T21:31:27.537972+0000","last_unstale":"2024-04-20T21:31:28.685888+0000","last_undegraded":"2024-04-20T21:31:28.685888+0000","last_fullsized":"2024-04-20T21:31:28.685888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:51.974485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191538+0000","last_change":"2024-04-20T21:31:27.538092+0000","last_active":"2024-04-20T21:31:29.191538+0000","last_peered":"2024-04-20T21:31:29.191538+0000","last_clean":"2024-04-20T21:31:29.191538+0000","last_became_active":"2024-04-20T21:31:27.537821+0000","last_became_peered":"2024-04-20T21:31:27.537821+0000","last_unstale":"2024-04-20T21:31:29.191538+0000","last_undegraded":"2024-04-20T21:31:29.191538+0000","last_fullsized":"2024-04-20T21:31:29.191538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:52.773583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.685298+0000","last_change":"2024-04-20T21:31:27.535498+0000","last_active":"2024-04-20T21:31:28.685298+0000","last_peered":"2024-04-20T21:31:28.685298+0000","last_clean":"2024-04-20T21:31:28.685298+0000","last_became_active":"2024-04-20T21:31:27.535162+0000","last_became_peered":"2024-04-20T21:31:27.535162+0000","last_unstale":"2024-04-20T21:31:28.685298+0000","last_undegraded":"2024-04-20T21:31:28.685298+0000","last_fullsized":"2024-04-20T21:31:28.685298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:46.638825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191242+0000","last_change":"2024-04-20T21:31:27.537776+0000","last_active":"2024-04-20T21:31:29.191242+0000","last_peered":"2024-04-20T21:31:29.191242+0000","last_clean":"2024-04-20T21:31:29.191242+0000","last_became_active":"2024-04-20T21:31:27.537428+0000","last_became_peered":"2024-04-20T21:31:27.537428+0000","last_unstale":"2024-04-20T21:31:29.191242+0000","last_undegraded":"2024-04-20T21:31:29.191242+0000","last_fullsized":"2024-04-20T21:31:29.191242+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:15:03.344277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.518248+0000","last_change":"2024-04-20T21:31:27.535943+0000","last_active":"2024-04-20T21:31:28.518248+0000","last_peered":"2024-04-20T21:31:28.518248+0000","last_clean":"2024-04-20T21:31:28.518248+0000","last_became_active":"2024-04-20T21:31:27.533694+0000","last_became_peered":"2024-04-20T21:31:27.533694+0000","last_unstale":"2024-04-20T21:31:28.518248+0000","last_undegraded":"2024-04-20T21:31:28.518248+0000","last_fullsized":"2024-04-20T21:31:28.518248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:42.603232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516673+0000","last_change":"2024-04-20T21:31:27.511345+0000","last_active":"2024-04-20T21:31:28.516673+0000","last_peered":"2024-04-20T21:31:28.516673+0000","last_clean":"2024-04-20T21:31:28.516673+0000","last_became_active":"2024-04-20T21:31:27.510984+0000","last_became_peered":"2024-04-20T21:31:27.510984+0000","last_unstale":"2024-04-20T21:31:28.516673+0000","last_undegraded":"2024-04-20T21:31:28.516673+0000","last_fullsized":"2024-04-20T21:31:28.516673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:19.965636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":21693,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":15,"num_read_kb":18,"num_write":54,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":63094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":47,"ondisk_log_size":47,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285583,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":451856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547219,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1063430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710745,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":452438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972385,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":452460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266724,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1042806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430252,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":451973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":958464,"data_stored":453468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":451946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":490083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":451960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":451878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691889,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36856,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708296,"statfs":{"total":95995035648,"available":95957295104,"internally_reserved":0,"allocated":1556480,"data_stored":1042523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":38227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":21089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:31:35.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:31:36.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.253+0000 7fe10a400640 1 -- 172.21.15.22:0/3944474298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe104075fa0 msgr2=0x7fe1040743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.253+0000 7fe10a400640 1 --2- 172.21.15.22:0/3944474298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe104075fa0 0x7fe1040743f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009900 tx=0x7fe0f402bc30 comp rx=0 tx=0).stop 2024-04-20T21:31:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.253+0000 7fe10a400640 1 -- 172.21.15.22:0/3944474298 shutdown_connections 2024-04-20T21:31:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.253+0000 7fe10a400640 1 --2- 172.21.15.22:0/3944474298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe104109cc0 0x7fe10410c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.253+0000 7fe10a400640 1 --2- 172.21.15.22:0/3944474298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1040749c0 0x7fe104074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.253+0000 7fe10a400640 1 --2- 172.21.15.22:0/3944474298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe104075fa0 0x7fe1040743f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.253+0000 7fe10a400640 1 -- 172.21.15.22:0/3944474298 >> 172.21.15.22:0/3944474298 conn(0x7fe1040fd370 msgr2=0x7fe1040ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.253+0000 7fe10a400640 1 -- 172.21.15.22:0/3944474298 shutdown_connections 2024-04-20T21:31:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.253+0000 7fe10a400640 1 -- 172.21.15.22:0/3944474298 wait complete. 2024-04-20T21:31:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.254+0000 7fe10a400640 1 Processor -- start 2024-04-20T21:31:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.255+0000 7fe10a400640 1 -- start start 2024-04-20T21:31:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.255+0000 7fe10a400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1040749c0 0x7fe10419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.255+0000 7fe10a400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe104075fa0 0x7fe10419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.255+0000 7fe10a400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe104109cc0 0x7fe1041a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.255+0000 7fe10a400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe10410fcd0 con 0x7fe1040749c0 2024-04-20T21:31:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.255+0000 7fe10a400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe10410fb50 con 0x7fe104075fa0 2024-04-20T21:31:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.255+0000 7fe10a400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe10410fe50 con 0x7fe104109cc0 2024-04-20T21:31:36.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.255+0000 7fe0fbe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe104075fa0 0x7fe10419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.256+0000 7fe109a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe104109cc0 0x7fe1041a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.256+0000 7fe0fbe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe104075fa0 0x7fe10419a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:49136/0 (socket says 172.21.15.22:49136) 2024-04-20T21:31:36.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.256+0000 7fe0fbe00640 1 -- 172.21.15.22:0/637127433 learned_addr learned my addr 172.21.15.22:0/637127433 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:36.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.256+0000 7fe109000640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1040749c0 0x7fe10419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.256+0000 7fe0fbe00640 1 -- 172.21.15.22:0/637127433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe104109cc0 msgr2=0x7fe1041a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.256+0000 7fe0fbe00640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe104109cc0 0x7fe1041a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.256+0000 7fe0fbe00640 1 -- 172.21.15.22:0/637127433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1040749c0 msgr2=0x7fe10419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.256+0000 7fe0fbe00640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1040749c0 0x7fe10419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.256+0000 7fe0fbe00640 1 -- 172.21.15.22:0/637127433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1041a1f40 con 0x7fe104075fa0 2024-04-20T21:31:36.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.256+0000 7fe109a00640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe104109cc0 0x7fe1041a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:36.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.256+0000 7fe0fbe00640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe104075fa0 0x7fe10419a860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00e9f0 tx=0x7fe0ec00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:36.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.257+0000 7fe102000640 1 -- 172.21.15.22:0/637127433 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec00ccc0 con 0x7fe104075fa0 2024-04-20T21:31:36.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.257+0000 7fe102000640 1 -- 172.21.15.22:0/637127433 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ec00ce60 con 0x7fe104075fa0 2024-04-20T21:31:36.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.257+0000 7fe10a400640 1 -- 172.21.15.22:0/637127433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1041a2230 con 0x7fe104075fa0 2024-04-20T21:31:36.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.257+0000 7fe102000640 1 -- 172.21.15.22:0/637127433 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec010690 con 0x7fe104075fa0 2024-04-20T21:31:36.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.257+0000 7fe10a400640 1 -- 172.21.15.22:0/637127433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1041a2730 con 0x7fe104075fa0 2024-04-20T21:31:36.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.258+0000 7fe10a400640 1 -- 172.21.15.22:0/637127433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1041089f0 con 0x7fe104075fa0 2024-04-20T21:31:36.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.260+0000 7fe102000640 1 -- 172.21.15.22:0/637127433 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe0ec0046c0 con 0x7fe104075fa0 2024-04-20T21:31:36.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.262+0000 7fe102000640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe0e80a3c40 0x7fe0e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.262+0000 7fe102000640 1 -- 172.21.15.22:0/637127433 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fe0ec0d09f0 con 0x7fe104075fa0 2024-04-20T21:31:36.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.262+0000 7fe109000640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe0e80a3c40 0x7fe0e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.263+0000 7fe109000640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe0e80a3c40 0x7fe0e80a6100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe0f402c240 tx=0x7fe0f402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:36.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.264+0000 7fe102000640 1 -- 172.21.15.22:0/637127433 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe0ec09d320 con 0x7fe104075fa0 2024-04-20T21:31:36.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:36 smithi033 ceph-mon[21626]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:36.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:36 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3905375300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:36.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:36 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/4040134540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:36.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:36 smithi022 ceph-mon[16843]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:36.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:36 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3905375300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:36.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:36 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/4040134540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:36.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.466+0000 7fe10a400640 1 -- 172.21.15.22:0/637127433 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1040623a0 con 0x7fe0e80a3c40 2024-04-20T21:31:36.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.484+0000 7fe102000640 1 -- 172.21.15.22:0/637127433 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258685 (secure 0 0 0) 0x7fe1040623a0 con 0x7fe0e80a3c40 2024-04-20T21:31:36.485 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:36.489 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-20T21:31:36.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.492+0000 7fe10a400640 1 -- 172.21.15.22:0/637127433 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe0e80a3c40 msgr2=0x7fe0e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.492+0000 7fe10a400640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe0e80a3c40 0x7fe0e80a6100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe0f402c240 tx=0x7fe0f402df90 comp rx=0 tx=0).stop 2024-04-20T21:31:36.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.492+0000 7fe10a400640 1 -- 172.21.15.22:0/637127433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe104075fa0 msgr2=0x7fe10419a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.492+0000 7fe10a400640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe104075fa0 0x7fe10419a860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00e9f0 tx=0x7fe0ec00edf0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.492+0000 7fe10a400640 1 -- 172.21.15.22:0/637127433 shutdown_connections 2024-04-20T21:31:36.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.492+0000 7fe10a400640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe0e80a3c40 0x7fe0e80a6100 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.492+0000 7fe10a400640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe104109cc0 0x7fe1041a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.492+0000 7fe10a400640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe104075fa0 0x7fe10419a860 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.492+0000 7fe10a400640 1 --2- 172.21.15.22:0/637127433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1040749c0 0x7fe10419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.492+0000 7fe10a400640 1 -- 172.21.15.22:0/637127433 >> 172.21.15.22:0/637127433 conn(0x7fe1040fd370 msgr2=0x7fe104105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:36.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.492+0000 7fe10a400640 1 -- 172.21.15.22:0/637127433 shutdown_connections 2024-04-20T21:31:36.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:36.492+0000 7fe10a400640 1 -- 172.21.15.22:0/637127433 wait complete. 2024-04-20T21:31:36.513 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-20T21:31:35.653227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":612295,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":151,"num_read_kb":137,"num_write":237,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378824,"kb_used_data":12988,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563000,"statfs":{"total":1151940427776,"available":1151552512000,"internally_reserved":0,"allocated":13299712,"data_stored":7256821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21927,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":15,"num_read_kb":18,"num_write":56,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.163438"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516987+0000","last_change":"2024-04-20T21:31:27.527151+0000","last_active":"2024-04-20T21:31:28.516987+0000","last_peered":"2024-04-20T21:31:28.516987+0000","last_clean":"2024-04-20T21:31:28.516987+0000","last_became_active":"2024-04-20T21:31:27.526739+0000","last_became_peered":"2024-04-20T21:31:27.526739+0000","last_unstale":"2024-04-20T21:31:28.516987+0000","last_undegraded":"2024-04-20T21:31:28.516987+0000","last_fullsized":"2024-04-20T21:31:28.516987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:16:39.204523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.194016+0000","last_change":"2024-04-20T21:31:27.527437+0000","last_active":"2024-04-20T21:31:29.194016+0000","last_peered":"2024-04-20T21:31:29.194016+0000","last_clean":"2024-04-20T21:31:29.194016+0000","last_became_active":"2024-04-20T21:31:27.527040+0000","last_became_peered":"2024-04-20T21:31:27.527040+0000","last_unstale":"2024-04-20T21:31:29.194016+0000","last_undegraded":"2024-04-20T21:31:29.194016+0000","last_fullsized":"2024-04-20T21:31:29.194016+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:53:39.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.193528+0000","last_change":"2024-04-20T21:31:27.531671+0000","last_active":"2024-04-20T21:31:29.193528+0000","last_peered":"2024-04-20T21:31:29.193528+0000","last_clean":"2024-04-20T21:31:29.193528+0000","last_became_active":"2024-04-20T21:31:27.531267+0000","last_became_peered":"2024-04-20T21:31:27.531267+0000","last_unstale":"2024-04-20T21:31:29.193528+0000","last_undegraded":"2024-04-20T21:31:29.193528+0000","last_fullsized":"2024-04-20T21:31:29.193528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:47.002481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516388+0000","last_change":"2024-04-20T21:31:27.512033+0000","last_active":"2024-04-20T21:31:28.516388+0000","last_peered":"2024-04-20T21:31:28.516388+0000","last_clean":"2024-04-20T21:31:28.516388+0000","last_became_active":"2024-04-20T21:31:27.510801+0000","last_became_peered":"2024-04-20T21:31:27.510801+0000","last_unstale":"2024-04-20T21:31:28.516388+0000","last_undegraded":"2024-04-20T21:31:28.516388+0000","last_fullsized":"2024-04-20T21:31:28.516388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:43:25.407173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.683813+0000","last_change":"2024-04-20T21:31:27.545862+0000","last_active":"2024-04-20T21:31:28.683813+0000","last_peered":"2024-04-20T21:31:28.683813+0000","last_clean":"2024-04-20T21:31:28.683813+0000","last_became_active":"2024-04-20T21:31:27.545500+0000","last_became_peered":"2024-04-20T21:31:27.545500+0000","last_unstale":"2024-04-20T21:31:28.683813+0000","last_undegraded":"2024-04-20T21:31:28.683813+0000","last_fullsized":"2024-04-20T21:31:28.683813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:37.640777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191640+0000","last_change":"2024-04-20T21:31:27.526408+0000","last_active":"2024-04-20T21:31:29.191640+0000","last_peered":"2024-04-20T21:31:29.191640+0000","last_clean":"2024-04-20T21:31:29.191640+0000","last_became_active":"2024-04-20T21:31:27.526015+0000","last_became_peered":"2024-04-20T21:31:27.526015+0000","last_unstale":"2024-04-20T21:31:29.191640+0000","last_undegraded":"2024-04-20T21:31:29.191640+0000","last_fullsized":"2024-04-20T21:31:29.191640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:56.904696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192281+0000","last_change":"2024-04-20T21:31:27.526272+0000","last_active":"2024-04-20T21:31:29.192281+0000","last_peered":"2024-04-20T21:31:29.192281+0000","last_clean":"2024-04-20T21:31:29.192281+0000","last_became_active":"2024-04-20T21:31:27.525852+0000","last_became_peered":"2024-04-20T21:31:27.525852+0000","last_unstale":"2024-04-20T21:31:29.192281+0000","last_undegraded":"2024-04-20T21:31:29.192281+0000","last_fullsized":"2024-04-20T21:31:29.192281+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:33:57.811339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.684874+0000","last_change":"2024-04-20T21:31:27.543548+0000","last_active":"2024-04-20T21:31:28.684874+0000","last_peered":"2024-04-20T21:31:28.684874+0000","last_clean":"2024-04-20T21:31:28.684874+0000","last_became_active":"2024-04-20T21:31:27.541487+0000","last_became_peered":"2024-04-20T21:31:27.541487+0000","last_unstale":"2024-04-20T21:31:28.684874+0000","last_undegraded":"2024-04-20T21:31:28.684874+0000","last_fullsized":"2024-04-20T21:31:28.684874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:23:21.009261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191372+0000","last_change":"2024-04-20T21:31:27.514810+0000","last_active":"2024-04-20T21:31:29.191372+0000","last_peered":"2024-04-20T21:31:29.191372+0000","last_clean":"2024-04-20T21:31:29.191372+0000","last_became_active":"2024-04-20T21:31:27.512548+0000","last_became_peered":"2024-04-20T21:31:27.512548+0000","last_unstale":"2024-04-20T21:31:29.191372+0000","last_undegraded":"2024-04-20T21:31:29.191372+0000","last_fullsized":"2024-04-20T21:31:29.191372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:11:30.375294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192277+0000","last_change":"2024-04-20T21:31:27.510303+0000","last_active":"2024-04-20T21:31:29.192277+0000","last_peered":"2024-04-20T21:31:29.192277+0000","last_clean":"2024-04-20T21:31:29.192277+0000","last_became_active":"2024-04-20T21:31:27.509907+0000","last_became_peered":"2024-04-20T21:31:27.509907+0000","last_unstale":"2024-04-20T21:31:29.192277+0000","last_undegraded":"2024-04-20T21:31:29.192277+0000","last_fullsized":"2024-04-20T21:31:29.192277+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:31.160785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191604+0000","last_change":"2024-04-20T21:31:27.513749+0000","last_active":"2024-04-20T21:31:29.191604+0000","last_peered":"2024-04-20T21:31:29.191604+0000","last_clean":"2024-04-20T21:31:29.191604+0000","last_became_active":"2024-04-20T21:31:27.513404+0000","last_became_peered":"2024-04-20T21:31:27.513404+0000","last_unstale":"2024-04-20T21:31:29.191604+0000","last_undegraded":"2024-04-20T21:31:29.191604+0000","last_fullsized":"2024-04-20T21:31:29.191604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:26.279697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516215+0000","last_change":"2024-04-20T21:31:27.507727+0000","last_active":"2024-04-20T21:31:28.516215+0000","last_peered":"2024-04-20T21:31:28.516215+0000","last_clean":"2024-04-20T21:31:28.516215+0000","last_became_active":"2024-04-20T21:31:27.507363+0000","last_became_peered":"2024-04-20T21:31:27.507363+0000","last_unstale":"2024-04-20T21:31:28.516215+0000","last_undegraded":"2024-04-20T21:31:28.516215+0000","last_fullsized":"2024-04-20T21:31:28.516215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:01.139963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.518273+0000","last_change":"2024-04-20T21:31:27.512941+0000","last_active":"2024-04-20T21:31:28.518273+0000","last_peered":"2024-04-20T21:31:28.518273+0000","last_clean":"2024-04-20T21:31:28.518273+0000","last_became_active":"2024-04-20T21:31:27.512571+0000","last_became_peered":"2024-04-20T21:31:27.512571+0000","last_unstale":"2024-04-20T21:31:28.518273+0000","last_undegraded":"2024-04-20T21:31:28.518273+0000","last_fullsized":"2024-04-20T21:31:28.518273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:30.977624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.193494+0000","last_change":"2024-04-20T21:31:27.521439+0000","last_active":"2024-04-20T21:31:29.193494+0000","last_peered":"2024-04-20T21:31:29.193494+0000","last_clean":"2024-04-20T21:31:29.193494+0000","last_became_active":"2024-04-20T21:31:27.520245+0000","last_became_peered":"2024-04-20T21:31:27.520245+0000","last_unstale":"2024-04-20T21:31:29.193494+0000","last_undegraded":"2024-04-20T21:31:29.193494+0000","last_fullsized":"2024-04-20T21:31:29.193494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:53:26.510845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.193165+0000","last_change":"2024-04-20T21:31:27.530544+0000","last_active":"2024-04-20T21:31:29.193165+0000","last_peered":"2024-04-20T21:31:29.193165+0000","last_clean":"2024-04-20T21:31:29.193165+0000","last_became_active":"2024-04-20T21:31:27.530195+0000","last_became_peered":"2024-04-20T21:31:27.530195+0000","last_unstale":"2024-04-20T21:31:29.193165+0000","last_undegraded":"2024-04-20T21:31:29.193165+0000","last_fullsized":"2024-04-20T21:31:29.193165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:28:23.472948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190775+0000","last_change":"2024-04-20T21:31:27.521533+0000","last_active":"2024-04-20T21:31:29.190775+0000","last_peered":"2024-04-20T21:31:29.190775+0000","last_clean":"2024-04-20T21:31:29.190775+0000","last_became_active":"2024-04-20T21:31:27.520920+0000","last_became_peered":"2024-04-20T21:31:27.520920+0000","last_unstale":"2024-04-20T21:31:29.190775+0000","last_undegraded":"2024-04-20T21:31:29.190775+0000","last_fullsized":"2024-04-20T21:31:29.190775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:28:13.605238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516788+0000","last_change":"2024-04-20T21:31:27.527484+0000","last_active":"2024-04-20T21:31:28.516788+0000","last_peered":"2024-04-20T21:31:28.516788+0000","last_clean":"2024-04-20T21:31:28.516788+0000","last_became_active":"2024-04-20T21:31:27.527048+0000","last_became_peered":"2024-04-20T21:31:27.527048+0000","last_unstale":"2024-04-20T21:31:28.516788+0000","last_undegraded":"2024-04-20T21:31:28.516788+0000","last_fullsized":"2024-04-20T21:31:28.516788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:35:56.467661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.680864+0000","last_change":"2024-04-20T21:31:27.527870+0000","last_active":"2024-04-20T21:31:28.680864+0000","last_peered":"2024-04-20T21:31:28.680864+0000","last_clean":"2024-04-20T21:31:28.680864+0000","last_became_active":"2024-04-20T21:31:27.525918+0000","last_became_peered":"2024-04-20T21:31:27.525918+0000","last_unstale":"2024-04-20T21:31:28.680864+0000","last_undegraded":"2024-04-20T21:31:28.680864+0000","last_fullsized":"2024-04-20T21:31:28.680864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:34.425370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191210+0000","last_change":"2024-04-20T21:31:27.544987+0000","last_active":"2024-04-20T21:31:29.191210+0000","last_peered":"2024-04-20T21:31:29.191210+0000","last_clean":"2024-04-20T21:31:29.191210+0000","last_became_active":"2024-04-20T21:31:27.544628+0000","last_became_peered":"2024-04-20T21:31:27.544628+0000","last_unstale":"2024-04-20T21:31:29.191210+0000","last_undegraded":"2024-04-20T21:31:29.191210+0000","last_fullsized":"2024-04-20T21:31:29.191210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:05:42.396775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.681074+0000","last_change":"2024-04-20T21:31:26.702803+0000","last_active":"2024-04-20T21:31:28.681074+0000","last_peered":"2024-04-20T21:31:28.681074+0000","last_clean":"2024-04-20T21:31:28.681074+0000","last_became_active":"2024-04-20T21:31:26.695781+0000","last_became_peered":"2024-04-20T21:31:26.695781+0000","last_unstale":"2024-04-20T21:31:28.681074+0000","last_undegraded":"2024-04-20T21:31:28.681074+0000","last_fullsized":"2024-04-20T21:31:28.681074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:49:17.131832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.686168+0000","last_change":"2024-04-20T21:31:27.535264+0000","last_active":"2024-04-20T21:31:28.686168+0000","last_peered":"2024-04-20T21:31:28.686168+0000","last_clean":"2024-04-20T21:31:28.686168+0000","last_became_active":"2024-04-20T21:31:27.534888+0000","last_became_peered":"2024-04-20T21:31:27.534888+0000","last_unstale":"2024-04-20T21:31:28.686168+0000","last_undegraded":"2024-04-20T21:31:28.686168+0000","last_fullsized":"2024-04-20T21:31:28.686168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:04:20.951916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.686277+0000","last_change":"2024-04-20T21:31:26.700378+0000","last_active":"2024-04-20T21:31:28.686277+0000","last_peered":"2024-04-20T21:31:28.686277+0000","last_clean":"2024-04-20T21:31:28.686277+0000","last_became_active":"2024-04-20T21:31:26.700077+0000","last_became_peered":"2024-04-20T21:31:26.700077+0000","last_unstale":"2024-04-20T21:31:28.686277+0000","last_undegraded":"2024-04-20T21:31:28.686277+0000","last_fullsized":"2024-04-20T21:31:28.686277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:34:21.646402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.194041+0000","last_change":"2024-04-20T21:31:27.531069+0000","last_active":"2024-04-20T21:31:29.194041+0000","last_peered":"2024-04-20T21:31:29.194041+0000","last_clean":"2024-04-20T21:31:29.194041+0000","last_became_active":"2024-04-20T21:31:27.530457+0000","last_became_peered":"2024-04-20T21:31:27.530457+0000","last_unstale":"2024-04-20T21:31:29.194041+0000","last_undegraded":"2024-04-20T21:31:29.194041+0000","last_fullsized":"2024-04-20T21:31:29.194041+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:46:59.829623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.684155+0000","last_change":"2024-04-20T21:31:26.684150+0000","last_active":"2024-04-20T21:31:28.684155+0000","last_peered":"2024-04-20T21:31:28.684155+0000","last_clean":"2024-04-20T21:31:28.684155+0000","last_became_active":"2024-04-20T21:31:26.683823+0000","last_became_peered":"2024-04-20T21:31:26.683823+0000","last_unstale":"2024-04-20T21:31:28.684155+0000","last_undegraded":"2024-04-20T21:31:28.684155+0000","last_fullsized":"2024-04-20T21:31:28.684155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:14.413606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191606+0000","last_change":"2024-04-20T21:31:27.507499+0000","last_active":"2024-04-20T21:31:29.191606+0000","last_peered":"2024-04-20T21:31:29.191606+0000","last_clean":"2024-04-20T21:31:29.191606+0000","last_became_active":"2024-04-20T21:31:27.506922+0000","last_became_peered":"2024-04-20T21:31:27.506922+0000","last_unstale":"2024-04-20T21:31:29.191606+0000","last_undegraded":"2024-04-20T21:31:29.191606+0000","last_fullsized":"2024-04-20T21:31:29.191606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:40.865232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.683982+0000","last_change":"2024-04-20T21:31:26.683100+0000","last_active":"2024-04-20T21:31:28.683982+0000","last_peered":"2024-04-20T21:31:28.683982+0000","last_clean":"2024-04-20T21:31:28.683982+0000","last_became_active":"2024-04-20T21:31:26.682086+0000","last_became_peered":"2024-04-20T21:31:26.682086+0000","last_unstale":"2024-04-20T21:31:28.683982+0000","last_undegraded":"2024-04-20T21:31:28.683982+0000","last_fullsized":"2024-04-20T21:31:28.683982+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:26.215174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191591+0000","last_change":"2024-04-20T21:31:27.517839+0000","last_active":"2024-04-20T21:31:29.191591+0000","last_peered":"2024-04-20T21:31:29.191591+0000","last_clean":"2024-04-20T21:31:29.191591+0000","last_became_active":"2024-04-20T21:31:27.517044+0000","last_became_peered":"2024-04-20T21:31:27.517044+0000","last_unstale":"2024-04-20T21:31:29.191591+0000","last_undegraded":"2024-04-20T21:31:29.191591+0000","last_fullsized":"2024-04-20T21:31:29.191591+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:50.042665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190859+0000","last_change":"2024-04-20T21:31:26.686151+0000","last_active":"2024-04-20T21:31:29.190859+0000","last_peered":"2024-04-20T21:31:29.190859+0000","last_clean":"2024-04-20T21:31:29.190859+0000","last_became_active":"2024-04-20T21:31:26.685827+0000","last_became_peered":"2024-04-20T21:31:26.685827+0000","last_unstale":"2024-04-20T21:31:29.190859+0000","last_undegraded":"2024-04-20T21:31:29.190859+0000","last_fullsized":"2024-04-20T21:31:29.190859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:41:17.263420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.683760+0000","last_change":"2024-04-20T21:31:27.526387+0000","last_active":"2024-04-20T21:31:28.683760+0000","last_peered":"2024-04-20T21:31:28.683760+0000","last_clean":"2024-04-20T21:31:28.683760+0000","last_became_active":"2024-04-20T21:31:27.525912+0000","last_became_peered":"2024-04-20T21:31:27.525912+0000","last_unstale":"2024-04-20T21:31:28.683760+0000","last_undegraded":"2024-04-20T21:31:28.683760+0000","last_fullsized":"2024-04-20T21:31:28.683760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:34:24.471806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191443+0000","last_change":"2024-04-20T21:31:26.684573+0000","last_active":"2024-04-20T21:31:29.191443+0000","last_peered":"2024-04-20T21:31:29.191443+0000","last_clean":"2024-04-20T21:31:29.191443+0000","last_became_active":"2024-04-20T21:31:26.684163+0000","last_became_peered":"2024-04-20T21:31:26.684163+0000","last_unstale":"2024-04-20T21:31:29.191443+0000","last_undegraded":"2024-04-20T21:31:29.191443+0000","last_fullsized":"2024-04-20T21:31:29.191443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:39.445068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191067+0000","last_change":"2024-04-20T21:31:27.543680+0000","last_active":"2024-04-20T21:31:29.191067+0000","last_peered":"2024-04-20T21:31:29.191067+0000","last_clean":"2024-04-20T21:31:29.191067+0000","last_became_active":"2024-04-20T21:31:27.543379+0000","last_became_peered":"2024-04-20T21:31:27.543379+0000","last_unstale":"2024-04-20T21:31:29.191067+0000","last_undegraded":"2024-04-20T21:31:29.191067+0000","last_fullsized":"2024-04-20T21:31:29.191067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:58:17.168813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.517404+0000","last_change":"2024-04-20T21:31:26.689665+0000","last_active":"2024-04-20T21:31:28.517404+0000","last_peered":"2024-04-20T21:31:28.517404+0000","last_clean":"2024-04-20T21:31:28.517404+0000","last_became_active":"2024-04-20T21:31:26.689175+0000","last_became_peered":"2024-04-20T21:31:26.689175+0000","last_unstale":"2024-04-20T21:31:28.517404+0000","last_undegraded":"2024-04-20T21:31:28.517404+0000","last_fullsized":"2024-04-20T21:31:28.517404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:21.342700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.685136+0000","last_change":"2024-04-20T21:31:27.544137+0000","last_active":"2024-04-20T21:31:28.685136+0000","last_peered":"2024-04-20T21:31:28.685136+0000","last_clean":"2024-04-20T21:31:28.685136+0000","last_became_active":"2024-04-20T21:31:27.542841+0000","last_became_peered":"2024-04-20T21:31:27.542841+0000","last_unstale":"2024-04-20T21:31:28.685136+0000","last_undegraded":"2024-04-20T21:31:28.685136+0000","last_fullsized":"2024-04-20T21:31:28.685136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:04.610144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190846+0000","last_change":"2024-04-20T21:31:26.681937+0000","last_active":"2024-04-20T21:31:29.190846+0000","last_peered":"2024-04-20T21:31:29.190846+0000","last_clean":"2024-04-20T21:31:29.190846+0000","last_became_active":"2024-04-20T21:31:26.681621+0000","last_became_peered":"2024-04-20T21:31:26.681621+0000","last_unstale":"2024-04-20T21:31:29.190846+0000","last_undegraded":"2024-04-20T21:31:29.190846+0000","last_fullsized":"2024-04-20T21:31:29.190846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:06.641608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.679993+0000","last_change":"2024-04-20T21:31:27.540225+0000","last_active":"2024-04-20T21:31:28.679993+0000","last_peered":"2024-04-20T21:31:28.679993+0000","last_clean":"2024-04-20T21:31:28.679993+0000","last_became_active":"2024-04-20T21:31:27.539860+0000","last_became_peered":"2024-04-20T21:31:27.539860+0000","last_unstale":"2024-04-20T21:31:28.679993+0000","last_undegraded":"2024-04-20T21:31:28.679993+0000","last_fullsized":"2024-04-20T21:31:28.679993+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:50.146761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192405+0000","last_change":"2024-04-20T21:31:26.701816+0000","last_active":"2024-04-20T21:31:29.192405+0000","last_peered":"2024-04-20T21:31:29.192405+0000","last_clean":"2024-04-20T21:31:29.192405+0000","last_became_active":"2024-04-20T21:31:26.701530+0000","last_became_peered":"2024-04-20T21:31:26.701530+0000","last_unstale":"2024-04-20T21:31:29.192405+0000","last_undegraded":"2024-04-20T21:31:29.192405+0000","last_fullsized":"2024-04-20T21:31:29.192405+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:31.175330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191307+0000","last_change":"2024-04-20T21:31:27.527222+0000","last_active":"2024-04-20T21:31:29.191307+0000","last_peered":"2024-04-20T21:31:29.191307+0000","last_clean":"2024-04-20T21:31:29.191307+0000","last_became_active":"2024-04-20T21:31:27.526779+0000","last_became_peered":"2024-04-20T21:31:27.526779+0000","last_unstale":"2024-04-20T21:31:29.191307+0000","last_undegraded":"2024-04-20T21:31:29.191307+0000","last_fullsized":"2024-04-20T21:31:29.191307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:13:54.500516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191164+0000","last_change":"2024-04-20T21:31:26.685969+0000","last_active":"2024-04-20T21:31:29.191164+0000","last_peered":"2024-04-20T21:31:29.191164+0000","last_clean":"2024-04-20T21:31:29.191164+0000","last_became_active":"2024-04-20T21:31:26.685580+0000","last_became_peered":"2024-04-20T21:31:26.685580+0000","last_unstale":"2024-04-20T21:31:29.191164+0000","last_undegraded":"2024-04-20T21:31:29.191164+0000","last_fullsized":"2024-04-20T21:31:29.191164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:22:50.558526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.194009+0000","last_change":"2024-04-20T21:31:27.525862+0000","last_active":"2024-04-20T21:31:29.194009+0000","last_peered":"2024-04-20T21:31:29.194009+0000","last_clean":"2024-04-20T21:31:29.194009+0000","last_became_active":"2024-04-20T21:31:27.525372+0000","last_became_peered":"2024-04-20T21:31:27.525372+0000","last_unstale":"2024-04-20T21:31:29.194009+0000","last_undegraded":"2024-04-20T21:31:29.194009+0000","last_fullsized":"2024-04-20T21:31:29.194009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:44:15.527137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191413+0000","last_change":"2024-04-20T21:31:26.691091+0000","last_active":"2024-04-20T21:31:29.191413+0000","last_peered":"2024-04-20T21:31:29.191413+0000","last_clean":"2024-04-20T21:31:29.191413+0000","last_became_active":"2024-04-20T21:31:26.690608+0000","last_became_peered":"2024-04-20T21:31:26.690608+0000","last_unstale":"2024-04-20T21:31:29.191413+0000","last_undegraded":"2024-04-20T21:31:29.191413+0000","last_fullsized":"2024-04-20T21:31:29.191413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:44:25.336193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191871+0000","last_change":"2024-04-20T21:31:27.507900+0000","last_active":"2024-04-20T21:31:29.191871+0000","last_peered":"2024-04-20T21:31:29.191871+0000","last_clean":"2024-04-20T21:31:29.191871+0000","last_became_active":"2024-04-20T21:31:27.507360+0000","last_became_peered":"2024-04-20T21:31:27.507360+0000","last_unstale":"2024-04-20T21:31:29.191871+0000","last_undegraded":"2024-04-20T21:31:29.191871+0000","last_fullsized":"2024-04-20T21:31:29.191871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:27.110877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'13","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:33.053729+0000","last_change":"2024-04-20T21:31:26.685736+0000","last_active":"2024-04-20T21:31:33.053729+0000","last_peered":"2024-04-20T21:31:33.053729+0000","last_clean":"2024-04-20T21:31:33.053729+0000","last_became_active":"2024-04-20T21:31:26.685309+0000","last_became_peered":"2024-04-20T21:31:26.685309+0000","last_unstale":"2024-04-20T21:31:33.053729+0000","last_undegraded":"2024-04-20T21:31:33.053729+0000","last_fullsized":"2024-04-20T21:31:33.053729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:06.501753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":12,"num_write":13,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.684010+0000","last_change":"2024-04-20T21:31:27.507532+0000","last_active":"2024-04-20T21:31:28.684010+0000","last_peered":"2024-04-20T21:31:28.684010+0000","last_clean":"2024-04-20T21:31:28.684010+0000","last_became_active":"2024-04-20T21:31:27.507188+0000","last_became_peered":"2024-04-20T21:31:27.507188+0000","last_unstale":"2024-04-20T21:31:28.684010+0000","last_undegraded":"2024-04-20T21:31:28.684010+0000","last_fullsized":"2024-04-20T21:31:28.684010+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:58.814116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:30.478395+0000","last_change":"2024-04-20T21:31:26.701361+0000","last_active":"2024-04-20T21:31:30.478395+0000","last_peered":"2024-04-20T21:31:30.478395+0000","last_clean":"2024-04-20T21:31:30.478395+0000","last_became_active":"2024-04-20T21:31:26.701068+0000","last_became_peered":"2024-04-20T21:31:26.701068+0000","last_unstale":"2024-04-20T21:31:30.478395+0000","last_undegraded":"2024-04-20T21:31:30.478395+0000","last_fullsized":"2024-04-20T21:31:30.478395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:22:51.745294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.517816+0000","last_change":"2024-04-20T21:31:27.542189+0000","last_active":"2024-04-20T21:31:28.517816+0000","last_peered":"2024-04-20T21:31:28.517816+0000","last_clean":"2024-04-20T21:31:28.517816+0000","last_became_active":"2024-04-20T21:31:27.541800+0000","last_became_peered":"2024-04-20T21:31:27.541800+0000","last_unstale":"2024-04-20T21:31:28.517816+0000","last_undegraded":"2024-04-20T21:31:28.517816+0000","last_fullsized":"2024-04-20T21:31:28.517816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:31.898443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191008+0000","last_change":"2024-04-20T21:31:26.687652+0000","last_active":"2024-04-20T21:31:29.191008+0000","last_peered":"2024-04-20T21:31:29.191008+0000","last_clean":"2024-04-20T21:31:29.191008+0000","last_became_active":"2024-04-20T21:31:26.687333+0000","last_became_peered":"2024-04-20T21:31:26.687333+0000","last_unstale":"2024-04-20T21:31:29.191008+0000","last_undegraded":"2024-04-20T21:31:29.191008+0000","last_fullsized":"2024-04-20T21:31:29.191008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:12.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192615+0000","last_change":"2024-04-20T21:31:27.522535+0000","last_active":"2024-04-20T21:31:29.192615+0000","last_peered":"2024-04-20T21:31:29.192615+0000","last_clean":"2024-04-20T21:31:29.192615+0000","last_became_active":"2024-04-20T21:31:27.521821+0000","last_became_peered":"2024-04-20T21:31:27.521821+0000","last_unstale":"2024-04-20T21:31:29.192615+0000","last_undegraded":"2024-04-20T21:31:29.192615+0000","last_fullsized":"2024-04-20T21:31:29.192615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:47:53.401129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516185+0000","last_change":"2024-04-20T21:31:26.681758+0000","last_active":"2024-04-20T21:31:28.516185+0000","last_peered":"2024-04-20T21:31:28.516185+0000","last_clean":"2024-04-20T21:31:28.516185+0000","last_became_active":"2024-04-20T21:31:26.681395+0000","last_became_peered":"2024-04-20T21:31:26.681395+0000","last_unstale":"2024-04-20T21:31:28.516185+0000","last_undegraded":"2024-04-20T21:31:28.516185+0000","last_fullsized":"2024-04-20T21:31:28.516185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:33:00.924879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191875+0000","last_change":"2024-04-20T21:31:27.512873+0000","last_active":"2024-04-20T21:31:29.191875+0000","last_peered":"2024-04-20T21:31:29.191875+0000","last_clean":"2024-04-20T21:31:29.191875+0000","last_became_active":"2024-04-20T21:31:27.512554+0000","last_became_peered":"2024-04-20T21:31:27.512554+0000","last_unstale":"2024-04-20T21:31:29.191875+0000","last_undegraded":"2024-04-20T21:31:29.191875+0000","last_fullsized":"2024-04-20T21:31:29.191875+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:26:06.670726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.683579+0000","last_change":"2024-04-20T21:31:26.691210+0000","last_active":"2024-04-20T21:31:28.683579+0000","last_peered":"2024-04-20T21:31:28.683579+0000","last_clean":"2024-04-20T21:31:28.683579+0000","last_became_active":"2024-04-20T21:31:26.690859+0000","last_became_peered":"2024-04-20T21:31:26.690859+0000","last_unstale":"2024-04-20T21:31:28.683579+0000","last_undegraded":"2024-04-20T21:31:28.683579+0000","last_fullsized":"2024-04-20T21:31:28.683579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:36.789429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190955+0000","last_change":"2024-04-20T21:31:27.517593+0000","last_active":"2024-04-20T21:31:29.190955+0000","last_peered":"2024-04-20T21:31:29.190955+0000","last_clean":"2024-04-20T21:31:29.190955+0000","last_became_active":"2024-04-20T21:31:27.516580+0000","last_became_peered":"2024-04-20T21:31:27.516580+0000","last_unstale":"2024-04-20T21:31:29.190955+0000","last_undegraded":"2024-04-20T21:31:29.190955+0000","last_fullsized":"2024-04-20T21:31:29.190955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:54:26.335431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.735177+0000","last_change":"2024-04-20T21:31:26.682311+0000","last_active":"2024-04-20T21:31:29.735177+0000","last_peered":"2024-04-20T21:31:29.735177+0000","last_clean":"2024-04-20T21:31:29.735177+0000","last_became_active":"2024-04-20T21:31:26.681996+0000","last_became_peered":"2024-04-20T21:31:26.681996+0000","last_unstale":"2024-04-20T21:31:29.735177+0000","last_undegraded":"2024-04-20T21:31:29.735177+0000","last_fullsized":"2024-04-20T21:31:29.735177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:32:37.659946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516725+0000","last_change":"2024-04-20T21:31:27.510922+0000","last_active":"2024-04-20T21:31:28.516725+0000","last_peered":"2024-04-20T21:31:28.516725+0000","last_clean":"2024-04-20T21:31:28.516725+0000","last_became_active":"2024-04-20T21:31:27.510553+0000","last_became_peered":"2024-04-20T21:31:27.510553+0000","last_unstale":"2024-04-20T21:31:28.516725+0000","last_undegraded":"2024-04-20T21:31:28.516725+0000","last_fullsized":"2024-04-20T21:31:28.516725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:23.754768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.680388+0000","last_change":"2024-04-20T21:31:26.697010+0000","last_active":"2024-04-20T21:31:28.680388+0000","last_peered":"2024-04-20T21:31:28.680388+0000","last_clean":"2024-04-20T21:31:28.680388+0000","last_became_active":"2024-04-20T21:31:26.696112+0000","last_became_peered":"2024-04-20T21:31:26.696112+0000","last_unstale":"2024-04-20T21:31:28.680388+0000","last_undegraded":"2024-04-20T21:31:28.680388+0000","last_fullsized":"2024-04-20T21:31:28.680388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:57:02.715884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192499+0000","last_change":"2024-04-20T21:31:27.545025+0000","last_active":"2024-04-20T21:31:29.192499+0000","last_peered":"2024-04-20T21:31:29.192499+0000","last_clean":"2024-04-20T21:31:29.192499+0000","last_became_active":"2024-04-20T21:31:27.544630+0000","last_became_peered":"2024-04-20T21:31:27.544630+0000","last_unstale":"2024-04-20T21:31:29.192499+0000","last_undegraded":"2024-04-20T21:31:29.192499+0000","last_fullsized":"2024-04-20T21:31:29.192499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:57.292481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191876+0000","last_change":"2024-04-20T21:31:26.682909+0000","last_active":"2024-04-20T21:31:29.191876+0000","last_peered":"2024-04-20T21:31:29.191876+0000","last_clean":"2024-04-20T21:31:29.191876+0000","last_became_active":"2024-04-20T21:31:26.682597+0000","last_became_peered":"2024-04-20T21:31:26.682597+0000","last_unstale":"2024-04-20T21:31:29.191876+0000","last_undegraded":"2024-04-20T21:31:29.191876+0000","last_fullsized":"2024-04-20T21:31:29.191876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:31.889513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192506+0000","last_change":"2024-04-20T21:31:27.530217+0000","last_active":"2024-04-20T21:31:29.192506+0000","last_peered":"2024-04-20T21:31:29.192506+0000","last_clean":"2024-04-20T21:31:29.192506+0000","last_became_active":"2024-04-20T21:31:27.529780+0000","last_became_peered":"2024-04-20T21:31:27.529780+0000","last_unstale":"2024-04-20T21:31:29.192506+0000","last_undegraded":"2024-04-20T21:31:29.192506+0000","last_fullsized":"2024-04-20T21:31:29.192506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:21.613460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191676+0000","last_change":"2024-04-20T21:31:27.518321+0000","last_active":"2024-04-20T21:31:29.191676+0000","last_peered":"2024-04-20T21:31:29.191676+0000","last_clean":"2024-04-20T21:31:29.191676+0000","last_became_active":"2024-04-20T21:31:27.518011+0000","last_became_peered":"2024-04-20T21:31:27.518011+0000","last_unstale":"2024-04-20T21:31:29.191676+0000","last_undegraded":"2024-04-20T21:31:29.191676+0000","last_fullsized":"2024-04-20T21:31:29.191676+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:50:55.457996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.517080+0000","last_change":"2024-04-20T21:31:26.692845+0000","last_active":"2024-04-20T21:31:28.517080+0000","last_peered":"2024-04-20T21:31:28.517080+0000","last_clean":"2024-04-20T21:31:28.517080+0000","last_became_active":"2024-04-20T21:31:26.691915+0000","last_became_peered":"2024-04-20T21:31:26.691915+0000","last_unstale":"2024-04-20T21:31:28.517080+0000","last_undegraded":"2024-04-20T21:31:28.517080+0000","last_fullsized":"2024-04-20T21:31:28.517080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:24:00.436955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.686312+0000","last_change":"2024-04-20T21:31:27.541623+0000","last_active":"2024-04-20T21:31:28.686312+0000","last_peered":"2024-04-20T21:31:28.686312+0000","last_clean":"2024-04-20T21:31:28.686312+0000","last_became_active":"2024-04-20T21:31:27.540953+0000","last_became_peered":"2024-04-20T21:31:27.540953+0000","last_unstale":"2024-04-20T21:31:28.686312+0000","last_undegraded":"2024-04-20T21:31:28.686312+0000","last_fullsized":"2024-04-20T21:31:28.686312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:03:15.091886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516047+0000","last_change":"2024-04-20T21:31:26.687373+0000","last_active":"2024-04-20T21:31:28.516047+0000","last_peered":"2024-04-20T21:31:28.516047+0000","last_clean":"2024-04-20T21:31:28.516047+0000","last_became_active":"2024-04-20T21:31:26.687009+0000","last_became_peered":"2024-04-20T21:31:26.687009+0000","last_unstale":"2024-04-20T21:31:28.516047+0000","last_undegraded":"2024-04-20T21:31:28.516047+0000","last_fullsized":"2024-04-20T21:31:28.516047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:23:42.708129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":346,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191964+0000","last_change":"2024-04-20T21:30:06.995711+0000","last_active":"2024-04-20T21:31:29.191964+0000","last_peered":"2024-04-20T21:31:29.191964+0000","last_clean":"2024-04-20T21:31:29.191964+0000","last_became_active":"2024-04-20T21:30:06.995280+0000","last_became_peered":"2024-04-20T21:30:06.995280+0000","last_unstale":"2024-04-20T21:31:29.191964+0000","last_undegraded":"2024-04-20T21:31:29.191964+0000","last_fullsized":"2024-04-20T21:31:29.191964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_clean_scrub_stamp":"2024-04-20T21:26:16.477494+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:49:26.014145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.517237+0000","last_change":"2024-04-20T21:31:27.514064+0000","last_active":"2024-04-20T21:31:28.517237+0000","last_peered":"2024-04-20T21:31:28.517237+0000","last_clean":"2024-04-20T21:31:28.517237+0000","last_became_active":"2024-04-20T21:31:27.513416+0000","last_became_peered":"2024-04-20T21:31:27.513416+0000","last_unstale":"2024-04-20T21:31:28.517237+0000","last_undegraded":"2024-04-20T21:31:28.517237+0000","last_fullsized":"2024-04-20T21:31:28.517237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:12.658265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192230+0000","last_change":"2024-04-20T21:31:26.680948+0000","last_active":"2024-04-20T21:31:29.192230+0000","last_peered":"2024-04-20T21:31:29.192230+0000","last_clean":"2024-04-20T21:31:29.192230+0000","last_became_active":"2024-04-20T21:31:26.680624+0000","last_became_peered":"2024-04-20T21:31:26.680624+0000","last_unstale":"2024-04-20T21:31:29.192230+0000","last_undegraded":"2024-04-20T21:31:29.192230+0000","last_fullsized":"2024-04-20T21:31:29.192230+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:02.750166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.684052+0000","last_change":"2024-04-20T21:31:26.697632+0000","last_active":"2024-04-20T21:31:28.684052+0000","last_peered":"2024-04-20T21:31:28.684052+0000","last_clean":"2024-04-20T21:31:28.684052+0000","last_became_active":"2024-04-20T21:31:26.697393+0000","last_became_peered":"2024-04-20T21:31:26.697393+0000","last_unstale":"2024-04-20T21:31:28.684052+0000","last_undegraded":"2024-04-20T21:31:28.684052+0000","last_fullsized":"2024-04-20T21:31:28.684052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:08:39.856082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.517845+0000","last_change":"2024-04-20T21:31:27.526613+0000","last_active":"2024-04-20T21:31:28.517845+0000","last_peered":"2024-04-20T21:31:28.517845+0000","last_clean":"2024-04-20T21:31:28.517845+0000","last_became_active":"2024-04-20T21:31:27.526234+0000","last_became_peered":"2024-04-20T21:31:27.526234+0000","last_unstale":"2024-04-20T21:31:28.517845+0000","last_undegraded":"2024-04-20T21:31:28.517845+0000","last_fullsized":"2024-04-20T21:31:28.517845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:06:44.114924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.683912+0000","last_change":"2024-04-20T21:31:26.700040+0000","last_active":"2024-04-20T21:31:28.683912+0000","last_peered":"2024-04-20T21:31:28.683912+0000","last_clean":"2024-04-20T21:31:28.683912+0000","last_became_active":"2024-04-20T21:31:26.699559+0000","last_became_peered":"2024-04-20T21:31:26.699559+0000","last_unstale":"2024-04-20T21:31:28.683912+0000","last_undegraded":"2024-04-20T21:31:28.683912+0000","last_fullsized":"2024-04-20T21:31:28.683912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:03.058778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.686164+0000","last_change":"2024-04-20T21:31:27.538361+0000","last_active":"2024-04-20T21:31:28.686164+0000","last_peered":"2024-04-20T21:31:28.686164+0000","last_clean":"2024-04-20T21:31:28.686164+0000","last_became_active":"2024-04-20T21:31:27.537494+0000","last_became_peered":"2024-04-20T21:31:27.537494+0000","last_unstale":"2024-04-20T21:31:28.686164+0000","last_undegraded":"2024-04-20T21:31:28.686164+0000","last_fullsized":"2024-04-20T21:31:28.686164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:46.212759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516325+0000","last_change":"2024-04-20T21:31:26.686960+0000","last_active":"2024-04-20T21:31:28.516325+0000","last_peered":"2024-04-20T21:31:28.516325+0000","last_clean":"2024-04-20T21:31:28.516325+0000","last_became_active":"2024-04-20T21:31:26.686565+0000","last_became_peered":"2024-04-20T21:31:26.686565+0000","last_unstale":"2024-04-20T21:31:28.516325+0000","last_undegraded":"2024-04-20T21:31:28.516325+0000","last_fullsized":"2024-04-20T21:31:28.516325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.608834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.194136+0000","last_change":"2024-04-20T21:31:27.546978+0000","last_active":"2024-04-20T21:31:29.194136+0000","last_peered":"2024-04-20T21:31:29.194136+0000","last_clean":"2024-04-20T21:31:29.194136+0000","last_became_active":"2024-04-20T21:31:27.545941+0000","last_became_peered":"2024-04-20T21:31:27.545941+0000","last_unstale":"2024-04-20T21:31:29.194136+0000","last_undegraded":"2024-04-20T21:31:29.194136+0000","last_fullsized":"2024-04-20T21:31:29.194136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:08.363519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190766+0000","last_change":"2024-04-20T21:31:26.690797+0000","last_active":"2024-04-20T21:31:29.190766+0000","last_peered":"2024-04-20T21:31:29.190766+0000","last_clean":"2024-04-20T21:31:29.190766+0000","last_became_active":"2024-04-20T21:31:26.690486+0000","last_became_peered":"2024-04-20T21:31:26.690486+0000","last_unstale":"2024-04-20T21:31:29.190766+0000","last_undegraded":"2024-04-20T21:31:29.190766+0000","last_fullsized":"2024-04-20T21:31:29.190766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:17:33.943212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.683652+0000","last_change":"2024-04-20T21:31:27.507175+0000","last_active":"2024-04-20T21:31:28.683652+0000","last_peered":"2024-04-20T21:31:28.683652+0000","last_clean":"2024-04-20T21:31:28.683652+0000","last_became_active":"2024-04-20T21:31:27.506808+0000","last_became_peered":"2024-04-20T21:31:27.506808+0000","last_unstale":"2024-04-20T21:31:28.683652+0000","last_undegraded":"2024-04-20T21:31:28.683652+0000","last_fullsized":"2024-04-20T21:31:28.683652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:38:15.212880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.684949+0000","last_change":"2024-04-20T21:31:26.698302+0000","last_active":"2024-04-20T21:31:28.684949+0000","last_peered":"2024-04-20T21:31:28.684949+0000","last_clean":"2024-04-20T21:31:28.684949+0000","last_became_active":"2024-04-20T21:31:26.696209+0000","last_became_peered":"2024-04-20T21:31:26.696209+0000","last_unstale":"2024-04-20T21:31:28.684949+0000","last_undegraded":"2024-04-20T21:31:28.684949+0000","last_fullsized":"2024-04-20T21:31:28.684949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:43.495880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516519+0000","last_change":"2024-04-20T21:31:27.521594+0000","last_active":"2024-04-20T21:31:28.516519+0000","last_peered":"2024-04-20T21:31:28.516519+0000","last_clean":"2024-04-20T21:31:28.516519+0000","last_became_active":"2024-04-20T21:31:27.521185+0000","last_became_peered":"2024-04-20T21:31:27.521185+0000","last_unstale":"2024-04-20T21:31:28.516519+0000","last_undegraded":"2024-04-20T21:31:28.516519+0000","last_fullsized":"2024-04-20T21:31:28.516519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:22:08.647624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190795+0000","last_change":"2024-04-20T21:31:26.691536+0000","last_active":"2024-04-20T21:31:29.190795+0000","last_peered":"2024-04-20T21:31:29.190795+0000","last_clean":"2024-04-20T21:31:29.190795+0000","last_became_active":"2024-04-20T21:31:26.691202+0000","last_became_peered":"2024-04-20T21:31:26.691202+0000","last_unstale":"2024-04-20T21:31:29.190795+0000","last_undegraded":"2024-04-20T21:31:29.190795+0000","last_fullsized":"2024-04-20T21:31:29.190795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:58:59.331644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.685423+0000","last_change":"2024-04-20T21:31:27.543799+0000","last_active":"2024-04-20T21:31:28.685423+0000","last_peered":"2024-04-20T21:31:28.685423+0000","last_clean":"2024-04-20T21:31:28.685423+0000","last_became_active":"2024-04-20T21:31:27.542061+0000","last_became_peered":"2024-04-20T21:31:27.542061+0000","last_unstale":"2024-04-20T21:31:28.685423+0000","last_undegraded":"2024-04-20T21:31:28.685423+0000","last_fullsized":"2024-04-20T21:31:28.685423+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:14.648628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.735118+0000","last_change":"2024-04-20T21:31:26.685963+0000","last_active":"2024-04-20T21:31:29.735118+0000","last_peered":"2024-04-20T21:31:29.735118+0000","last_clean":"2024-04-20T21:31:29.735118+0000","last_became_active":"2024-04-20T21:31:26.685647+0000","last_became_peered":"2024-04-20T21:31:26.685647+0000","last_unstale":"2024-04-20T21:31:29.735118+0000","last_undegraded":"2024-04-20T21:31:29.735118+0000","last_fullsized":"2024-04-20T21:31:29.735118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:56.109426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191498+0000","last_change":"2024-04-20T21:31:27.518346+0000","last_active":"2024-04-20T21:31:29.191498+0000","last_peered":"2024-04-20T21:31:29.191498+0000","last_clean":"2024-04-20T21:31:29.191498+0000","last_became_active":"2024-04-20T21:31:27.517782+0000","last_became_peered":"2024-04-20T21:31:27.517782+0000","last_unstale":"2024-04-20T21:31:29.191498+0000","last_undegraded":"2024-04-20T21:31:29.191498+0000","last_fullsized":"2024-04-20T21:31:29.191498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:21.674663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190809+0000","last_change":"2024-04-20T21:31:26.700335+0000","last_active":"2024-04-20T21:31:29.190809+0000","last_peered":"2024-04-20T21:31:29.190809+0000","last_clean":"2024-04-20T21:31:29.190809+0000","last_became_active":"2024-04-20T21:31:26.699965+0000","last_became_peered":"2024-04-20T21:31:26.699965+0000","last_unstale":"2024-04-20T21:31:29.190809+0000","last_undegraded":"2024-04-20T21:31:29.190809+0000","last_fullsized":"2024-04-20T21:31:29.190809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:21:40.626706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191836+0000","last_change":"2024-04-20T21:31:27.510920+0000","last_active":"2024-04-20T21:31:29.191836+0000","last_peered":"2024-04-20T21:31:29.191836+0000","last_clean":"2024-04-20T21:31:29.191836+0000","last_became_active":"2024-04-20T21:31:27.510564+0000","last_became_peered":"2024-04-20T21:31:27.510564+0000","last_unstale":"2024-04-20T21:31:29.191836+0000","last_undegraded":"2024-04-20T21:31:29.191836+0000","last_fullsized":"2024-04-20T21:31:29.191836+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:05.196178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:30.303208+0000","last_change":"2024-04-20T21:31:26.693159+0000","last_active":"2024-04-20T21:31:30.303208+0000","last_peered":"2024-04-20T21:31:30.303208+0000","last_clean":"2024-04-20T21:31:30.303208+0000","last_became_active":"2024-04-20T21:31:26.692902+0000","last_became_peered":"2024-04-20T21:31:26.692902+0000","last_unstale":"2024-04-20T21:31:30.303208+0000","last_undegraded":"2024-04-20T21:31:30.303208+0000","last_fullsized":"2024-04-20T21:31:30.303208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:54.704343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.657322+0000","last_change":"2024-04-20T21:31:27.541557+0000","last_active":"2024-04-20T21:31:29.657322+0000","last_peered":"2024-04-20T21:31:29.657322+0000","last_clean":"2024-04-20T21:31:29.657322+0000","last_became_active":"2024-04-20T21:31:27.541161+0000","last_became_peered":"2024-04-20T21:31:27.541161+0000","last_unstale":"2024-04-20T21:31:29.657322+0000","last_undegraded":"2024-04-20T21:31:29.657322+0000","last_fullsized":"2024-04-20T21:31:29.657322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:55:25.558918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.518067+0000","last_change":"2024-04-20T21:31:26.688019+0000","last_active":"2024-04-20T21:31:28.518067+0000","last_peered":"2024-04-20T21:31:28.518067+0000","last_clean":"2024-04-20T21:31:28.518067+0000","last_became_active":"2024-04-20T21:31:26.686370+0000","last_became_peered":"2024-04-20T21:31:26.686370+0000","last_unstale":"2024-04-20T21:31:28.518067+0000","last_undegraded":"2024-04-20T21:31:28.518067+0000","last_fullsized":"2024-04-20T21:31:28.518067+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:42:44.059365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.192085+0000","last_change":"2024-04-20T21:31:27.511592+0000","last_active":"2024-04-20T21:31:29.192085+0000","last_peered":"2024-04-20T21:31:29.192085+0000","last_clean":"2024-04-20T21:31:29.192085+0000","last_became_active":"2024-04-20T21:31:27.511311+0000","last_became_peered":"2024-04-20T21:31:27.511311+0000","last_unstale":"2024-04-20T21:31:29.192085+0000","last_undegraded":"2024-04-20T21:31:29.192085+0000","last_fullsized":"2024-04-20T21:31:29.192085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:35:27.665578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516616+0000","last_change":"2024-04-20T21:31:27.511827+0000","last_active":"2024-04-20T21:31:28.516616+0000","last_peered":"2024-04-20T21:31:28.516616+0000","last_clean":"2024-04-20T21:31:28.516616+0000","last_became_active":"2024-04-20T21:31:27.511344+0000","last_became_peered":"2024-04-20T21:31:27.511344+0000","last_unstale":"2024-04-20T21:31:28.516616+0000","last_undegraded":"2024-04-20T21:31:28.516616+0000","last_fullsized":"2024-04-20T21:31:28.516616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:11:37.973050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190634+0000","last_change":"2024-04-20T21:31:27.510747+0000","last_active":"2024-04-20T21:31:29.190634+0000","last_peered":"2024-04-20T21:31:29.190634+0000","last_clean":"2024-04-20T21:31:29.190634+0000","last_became_active":"2024-04-20T21:31:27.510253+0000","last_became_peered":"2024-04-20T21:31:27.510253+0000","last_unstale":"2024-04-20T21:31:29.190634+0000","last_undegraded":"2024-04-20T21:31:29.190634+0000","last_fullsized":"2024-04-20T21:31:29.190634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:37:13.855840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.190674+0000","last_change":"2024-04-20T21:31:27.541185+0000","last_active":"2024-04-20T21:31:29.190674+0000","last_peered":"2024-04-20T21:31:29.190674+0000","last_clean":"2024-04-20T21:31:29.190674+0000","last_became_active":"2024-04-20T21:31:27.540881+0000","last_became_peered":"2024-04-20T21:31:27.540881+0000","last_unstale":"2024-04-20T21:31:29.190674+0000","last_undegraded":"2024-04-20T21:31:29.190674+0000","last_fullsized":"2024-04-20T21:31:29.190674+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:46:41.897081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.193900+0000","last_change":"2024-04-20T21:31:27.527204+0000","last_active":"2024-04-20T21:31:29.193900+0000","last_peered":"2024-04-20T21:31:29.193900+0000","last_clean":"2024-04-20T21:31:29.193900+0000","last_became_active":"2024-04-20T21:31:27.526805+0000","last_became_peered":"2024-04-20T21:31:27.526805+0000","last_unstale":"2024-04-20T21:31:29.193900+0000","last_undegraded":"2024-04-20T21:31:29.193900+0000","last_fullsized":"2024-04-20T21:31:29.193900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:43.032609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.517700+0000","last_change":"2024-04-20T21:31:27.525521+0000","last_active":"2024-04-20T21:31:28.517700+0000","last_peered":"2024-04-20T21:31:28.517700+0000","last_clean":"2024-04-20T21:31:28.517700+0000","last_became_active":"2024-04-20T21:31:27.521892+0000","last_became_peered":"2024-04-20T21:31:27.521892+0000","last_unstale":"2024-04-20T21:31:28.517700+0000","last_undegraded":"2024-04-20T21:31:28.517700+0000","last_fullsized":"2024-04-20T21:31:28.517700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:32:47.683115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516776+0000","last_change":"2024-04-20T21:31:27.509828+0000","last_active":"2024-04-20T21:31:28.516776+0000","last_peered":"2024-04-20T21:31:28.516776+0000","last_clean":"2024-04-20T21:31:28.516776+0000","last_became_active":"2024-04-20T21:31:27.509269+0000","last_became_peered":"2024-04-20T21:31:27.509269+0000","last_unstale":"2024-04-20T21:31:28.516776+0000","last_undegraded":"2024-04-20T21:31:28.516776+0000","last_fullsized":"2024-04-20T21:31:28.516776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:25:54.492533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191333+0000","last_change":"2024-04-20T21:31:27.512542+0000","last_active":"2024-04-20T21:31:29.191333+0000","last_peered":"2024-04-20T21:31:29.191333+0000","last_clean":"2024-04-20T21:31:29.191333+0000","last_became_active":"2024-04-20T21:31:27.511907+0000","last_became_peered":"2024-04-20T21:31:27.511907+0000","last_unstale":"2024-04-20T21:31:29.191333+0000","last_undegraded":"2024-04-20T21:31:29.191333+0000","last_fullsized":"2024-04-20T21:31:29.191333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:10:09.265471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.685888+0000","last_change":"2024-04-20T21:31:27.538697+0000","last_active":"2024-04-20T21:31:28.685888+0000","last_peered":"2024-04-20T21:31:28.685888+0000","last_clean":"2024-04-20T21:31:28.685888+0000","last_became_active":"2024-04-20T21:31:27.537972+0000","last_became_peered":"2024-04-20T21:31:27.537972+0000","last_unstale":"2024-04-20T21:31:28.685888+0000","last_undegraded":"2024-04-20T21:31:28.685888+0000","last_fullsized":"2024-04-20T21:31:28.685888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:51.974485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191538+0000","last_change":"2024-04-20T21:31:27.538092+0000","last_active":"2024-04-20T21:31:29.191538+0000","last_peered":"2024-04-20T21:31:29.191538+0000","last_clean":"2024-04-20T21:31:29.191538+0000","last_became_active":"2024-04-20T21:31:27.537821+0000","last_became_peered":"2024-04-20T21:31:27.537821+0000","last_unstale":"2024-04-20T21:31:29.191538+0000","last_undegraded":"2024-04-20T21:31:29.191538+0000","last_fullsized":"2024-04-20T21:31:29.191538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:52.773583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.685298+0000","last_change":"2024-04-20T21:31:27.535498+0000","last_active":"2024-04-20T21:31:28.685298+0000","last_peered":"2024-04-20T21:31:28.685298+0000","last_clean":"2024-04-20T21:31:28.685298+0000","last_became_active":"2024-04-20T21:31:27.535162+0000","last_became_peered":"2024-04-20T21:31:27.535162+0000","last_unstale":"2024-04-20T21:31:28.685298+0000","last_undegraded":"2024-04-20T21:31:28.685298+0000","last_fullsized":"2024-04-20T21:31:28.685298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:46.638825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:29.191242+0000","last_change":"2024-04-20T21:31:27.537776+0000","last_active":"2024-04-20T21:31:29.191242+0000","last_peered":"2024-04-20T21:31:29.191242+0000","last_clean":"2024-04-20T21:31:29.191242+0000","last_became_active":"2024-04-20T21:31:27.537428+0000","last_became_peered":"2024-04-20T21:31:27.537428+0000","last_unstale":"2024-04-20T21:31:29.191242+0000","last_undegraded":"2024-04-20T21:31:29.191242+0000","last_fullsized":"2024-04-20T21:31:29.191242+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:15:03.344277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.518248+0000","last_change":"2024-04-20T21:31:27.535943+0000","last_active":"2024-04-20T21:31:28.518248+0000","last_peered":"2024-04-20T21:31:28.518248+0000","last_clean":"2024-04-20T21:31:28.518248+0000","last_became_active":"2024-04-20T21:31:27.533694+0000","last_became_peered":"2024-04-20T21:31:27.533694+0000","last_unstale":"2024-04-20T21:31:28.518248+0000","last_undegraded":"2024-04-20T21:31:28.518248+0000","last_fullsized":"2024-04-20T21:31:28.518248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:42.603232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:28.516673+0000","last_change":"2024-04-20T21:31:27.511345+0000","last_active":"2024-04-20T21:31:28.516673+0000","last_peered":"2024-04-20T21:31:28.516673+0000","last_clean":"2024-04-20T21:31:28.516673+0000","last_became_active":"2024-04-20T21:31:27.510984+0000","last_became_peered":"2024-04-20T21:31:27.510984+0000","last_unstale":"2024-04-20T21:31:28.516673+0000","last_undegraded":"2024-04-20T21:31:28.516673+0000","last_fullsized":"2024-04-20T21:31:28.516673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:19.965636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":21693,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":15,"num_read_kb":18,"num_write":54,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":63094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":47,"ondisk_log_size":47,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285583,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":451856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547219,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1063430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710745,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":452438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972385,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":452460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266724,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1042806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430252,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":451973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":958464,"data_stored":453468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":451946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":490083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":451960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":451878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691889,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36856,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708296,"statfs":{"total":95995035648,"available":95957295104,"internally_reserved":0,"allocated":1556480,"data_stored":1042523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":38227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":21089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:31:36.519 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T21:31:36.520 INFO:tasks.cephadm:Setting up client nodes... 2024-04-20T21:31:36.521 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-20T21:31:36.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:36 smithi122 ceph-mon[22147]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:36.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:36 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3905375300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:36.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:36 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/4040134540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:37.018 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:37.327 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:37 smithi022 ceph-mon[16843]: pgmap v321: 97 pgs: 97 active+clean; 598 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-04-20T21:31:37.327 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:37 smithi022 ceph-mon[16843]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:31:37.327 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:37 smithi022 ceph-mon[16843]: from='client.24713 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:31:37.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:37 smithi122 ceph-mon[22147]: pgmap v321: 97 pgs: 97 active+clean; 598 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-04-20T21:31:37.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:37 smithi122 ceph-mon[22147]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:31:37.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:37 smithi122 ceph-mon[22147]: from='client.24713 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:31:37.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:37 smithi033 ceph-mon[21626]: pgmap v321: 97 pgs: 97 active+clean; 598 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-04-20T21:31:37.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:37 smithi033 ceph-mon[21626]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:31:37.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:37 smithi033 ceph-mon[21626]: from='client.24713 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:31:39.296 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:39 smithi022 ceph-mon[16843]: pgmap v322: 97 pgs: 97 active+clean; 598 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-04-20T21:31:39.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:39 smithi122 ceph-mon[22147]: pgmap v322: 97 pgs: 97 active+clean; 598 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-04-20T21:31:39.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:39 smithi033 ceph-mon[21626]: pgmap v322: 97 pgs: 97 active+clean; 598 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-04-20T21:31:39.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.719+0000 7f515d200700 1 -- 172.21.15.22:0/2242398992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580754b0 msgr2=0x7f5158075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:39.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.719+0000 7f515d200700 1 --2- 172.21.15.22:0/2242398992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580754b0 0x7f5158075930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f51400098f0 tx=0x7f5140009c00 comp rx=0 tx=0).stop 2024-04-20T21:31:39.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.719+0000 7f515d200700 1 -- 172.21.15.22:0/2242398992 shutdown_connections 2024-04-20T21:31:39.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.719+0000 7f515d200700 1 --2- 172.21.15.22:0/2242398992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f515810f0b0 0x7f51581114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.719+0000 7f515d200700 1 --2- 172.21.15.22:0/2242398992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5158076a80 0x7f5158074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.719+0000 7f515d200700 1 --2- 172.21.15.22:0/2242398992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580754b0 0x7f5158075930 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.719+0000 7f515d200700 1 -- 172.21.15.22:0/2242398992 >> 172.21.15.22:0/2242398992 conn(0x7f51580fe070 msgr2=0x7f51581004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:39.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.719+0000 7f515d200700 1 -- 172.21.15.22:0/2242398992 shutdown_connections 2024-04-20T21:31:39.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.720+0000 7f515d200700 1 -- 172.21.15.22:0/2242398992 wait complete. 2024-04-20T21:31:39.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.721+0000 7f515d200700 1 Processor -- start 2024-04-20T21:31:39.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.721+0000 7f515d200700 1 -- start start 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.721+0000 7f515d200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580754b0 0x7f515819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.721+0000 7f515d200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5158076a80 0x7f515819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.721+0000 7f515d200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515810f0b0 0x7f51581a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f515d200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5158114d30 con 0x7f515810f0b0 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f515d200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5158114bb0 con 0x7f51580754b0 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f515d200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5158114eb0 con 0x7f5158076a80 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f514d400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5158076a80 0x7f515819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f514d400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5158076a80 0x7f515819f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42908/0 (socket says 172.21.15.22:42908) 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f514d400700 1 -- 172.21.15.22:0/2853505606 learned_addr learned my addr 172.21.15.22:0/2853505606 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f5155e00700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580754b0 0x7f515819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f5156800700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515810f0b0 0x7f51581a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f514d400700 1 -- 172.21.15.22:0/2853505606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580754b0 msgr2=0x7f515819f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:39.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f514d400700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580754b0 0x7f515819f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f514d400700 1 -- 172.21.15.22:0/2853505606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515810f0b0 msgr2=0x7f51581a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:39.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f514d400700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515810f0b0 0x7f51581a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f514d400700 1 -- 172.21.15.22:0/2853505606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51581a6f30 con 0x7f5158076a80 2024-04-20T21:31:39.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.722+0000 7f5156800700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515810f0b0 0x7f51581a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:39.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.723+0000 7f514d400700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5158076a80 0x7f515819f810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5140009230 tx=0x7f5140005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:39.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.723+0000 7f514f400700 1 -- 172.21.15.22:0/2853505606 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5140025070 con 0x7f5158076a80 2024-04-20T21:31:39.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.723+0000 7f514f400700 1 -- 172.21.15.22:0/2853505606 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514000f950 con 0x7f5158076a80 2024-04-20T21:31:39.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.723+0000 7f515d200700 1 -- 172.21.15.22:0/2853505606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51581a71c0 con 0x7f5158076a80 2024-04-20T21:31:39.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.723+0000 7f514f400700 1 -- 172.21.15.22:0/2853505606 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5140003d30 con 0x7f5158076a80 2024-04-20T21:31:39.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.723+0000 7f515d200700 1 -- 172.21.15.22:0/2853505606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51581a7670 con 0x7f5158076a80 2024-04-20T21:31:39.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.724+0000 7f515d200700 1 -- 172.21.15.22:0/2853505606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5158050f90 con 0x7f5158076a80 2024-04-20T21:31:39.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.726+0000 7f514f400700 1 -- 172.21.15.22:0/2853505606 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5140028070 con 0x7f5158076a80 2024-04-20T21:31:39.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.728+0000 7f514f400700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f513c0a3e00 0x7f513c0a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:39.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.728+0000 7f514f400700 1 -- 172.21.15.22:0/2853505606 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f51400d1130 con 0x7f5158076a80 2024-04-20T21:31:39.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.728+0000 7f5155e00700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f513c0a3e00 0x7f513c0a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:39.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.729+0000 7f5155e00700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f513c0a3e00 0x7f513c0a62c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5158076e60 tx=0x7f5148009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:39.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:39.731+0000 7f514f400700 1 -- 172.21.15.22:0/2853505606 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f514002f020 con 0x7f5158076a80 2024-04-20T21:31:40.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.063+0000 7f515d200700 1 -- 172.21.15.22:0/2853505606 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f515802e030 con 0x7f5158076a80 2024-04-20T21:31:40.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.073+0000 7f514f400700 1 -- 172.21.15.22:0/2853505606 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f515802e030 con 0x7f5158076a80 2024-04-20T21:31:40.073 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-04-20T21:31:40.073 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQA8NCRmV8/vAxAANntxeGMVSfubsv0ezNLW0g== 2024-04-20T21:31:40.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.076+0000 7f515d200700 1 -- 172.21.15.22:0/2853505606 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f513c0a3e00 msgr2=0x7f513c0a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:40.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.076+0000 7f515d200700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f513c0a3e00 0x7f513c0a62c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5158076e60 tx=0x7f5148009310 comp rx=0 tx=0).stop 2024-04-20T21:31:40.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.076+0000 7f515d200700 1 -- 172.21.15.22:0/2853505606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5158076a80 msgr2=0x7f515819f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:40.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.076+0000 7f515d200700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5158076a80 0x7f515819f810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5140009230 tx=0x7f5140005250 comp rx=0 tx=0).stop 2024-04-20T21:31:40.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.077+0000 7f515d200700 1 -- 172.21.15.22:0/2853505606 shutdown_connections 2024-04-20T21:31:40.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.077+0000 7f515d200700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f513c0a3e00 0x7f513c0a62c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.077+0000 7f515d200700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f515810f0b0 0x7f51581a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.077+0000 7f515d200700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580754b0 0x7f515819f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.077+0000 7f515d200700 1 --2- 172.21.15.22:0/2853505606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5158076a80 0x7f515819f810 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.077+0000 7f515d200700 1 -- 172.21.15.22:0/2853505606 >> 172.21.15.22:0/2853505606 conn(0x7f51580fe070 msgr2=0x7f5158102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.077+0000 7f515d200700 1 -- 172.21.15.22:0/2853505606 shutdown_connections 2024-04-20T21:31:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:40.077+0000 7f515d200700 1 -- 172.21.15.22:0/2853505606 wait complete. 2024-04-20T21:31:40.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:40 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2853505606' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T21:31:40.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:40 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T21:31:40.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:40 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T21:31:40.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:40 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2853505606' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T21:31:40.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:40 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T21:31:40.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:40 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T21:31:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:40 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2853505606' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T21:31:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:40 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T21:31:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:40 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T21:31:40.881 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:31:40.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-20T21:31:40.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T21:31:41.026 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T21:31:41.031 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-20T21:31:41.032 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-20T21:31:41.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:41 smithi022 ceph-mon[16843]: pgmap v323: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-04-20T21:31:41.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:41 smithi122 ceph-mon[22147]: pgmap v323: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-04-20T21:31:41.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:41 smithi033 ceph-mon[21626]: pgmap v323: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-04-20T21:31:41.559 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:43.245 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:43 smithi022 ceph-mon[16843]: pgmap v324: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-20T21:31:43.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:43 smithi122 ceph-mon[22147]: pgmap v324: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-20T21:31:43.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:43 smithi033 ceph-mon[21626]: pgmap v324: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-20T21:31:43.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.842+0000 7f920ee00700 1 -- 172.21.15.22:0/2104154520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208107950 msgr2=0x7f9208107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.842+0000 7f920ee00700 1 --2- 172.21.15.22:0/2104154520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208107950 0x7f9208107d30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91f80099c0 tx=0x7f91f8009cd0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.843+0000 7f920ee00700 1 -- 172.21.15.22:0/2104154520 shutdown_connections 2024-04-20T21:31:43.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.843+0000 7f920ee00700 1 --2- 172.21.15.22:0/2104154520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92081034c0 0x7f920810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.843+0000 7f920ee00700 1 --2- 172.21.15.22:0/2104154520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208107950 0x7f9208107d30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.843+0000 7f920ee00700 1 --2- 172.21.15.22:0/2104154520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208102b00 0x7f9208102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.843+0000 7f920ee00700 1 -- 172.21.15.22:0/2104154520 >> 172.21.15.22:0/2104154520 conn(0x7f9208079bb0 msgr2=0x7f9208079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:43.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.843+0000 7f920ee00700 1 -- 172.21.15.22:0/2104154520 shutdown_connections 2024-04-20T21:31:43.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.843+0000 7f920ee00700 1 -- 172.21.15.22:0/2104154520 wait complete. 2024-04-20T21:31:43.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.844+0000 7f920ee00700 1 Processor -- start 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.844+0000 7f920ee00700 1 -- start start 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f920ee00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208102b00 0x7f920819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f920ee00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92081034c0 0x7f920819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f920ee00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208107950 0x7f920819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f920ee00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9208112e00 con 0x7f9208107950 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f920ee00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9208112c80 con 0x7f9208102b00 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f920ee00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9208112f80 con 0x7f92081034c0 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f91ff400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92081034c0 0x7f920819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f9207e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208102b00 0x7f920819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f920ce00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208107950 0x7f920819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f91ff400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92081034c0 0x7f920819b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36086/0 (socket says 172.21.15.22:36086) 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f91ff400700 1 -- 172.21.15.22:0/1596650306 learned_addr learned my addr 172.21.15.22:0/1596650306 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f9207e00700 1 -- 172.21.15.22:0/1596650306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92081034c0 msgr2=0x7f920819b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f9207e00700 1 --2- 172.21.15.22:0/1596650306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92081034c0 0x7f920819b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f9207e00700 1 -- 172.21.15.22:0/1596650306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208107950 msgr2=0x7f920819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f9207e00700 1 --2- 172.21.15.22:0/1596650306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208107950 0x7f920819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.845+0000 7f9207e00700 1 -- 172.21.15.22:0/1596650306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92081a0440 con 0x7f9208102b00 2024-04-20T21:31:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.846+0000 7f9207e00700 1 --2- 172.21.15.22:0/1596650306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208102b00 0x7f920819b430 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91f80098e0 tx=0x7f91f8003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:43.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.846+0000 7f9205600700 1 -- 172.21.15.22:0/1596650306 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8003680 con 0x7f9208102b00 2024-04-20T21:31:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.846+0000 7f920ee00700 1 -- 172.21.15.22:0/1596650306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92081a06d0 con 0x7f9208102b00 2024-04-20T21:31:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.846+0000 7f9205600700 1 -- 172.21.15.22:0/1596650306 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f80055a0 con 0x7f9208102b00 2024-04-20T21:31:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.846+0000 7f9205600700 1 -- 172.21.15.22:0/1596650306 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f800fc90 con 0x7f9208102b00 2024-04-20T21:31:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.846+0000 7f920ee00700 1 -- 172.21.15.22:0/1596650306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92081a7fb0 con 0x7f9208102b00 2024-04-20T21:31:43.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.847+0000 7f920ee00700 1 -- 172.21.15.22:0/1596650306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92081041c0 con 0x7f9208102b00 2024-04-20T21:31:43.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.849+0000 7f9205600700 1 -- 172.21.15.22:0/1596650306 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f91f800f460 con 0x7f9208102b00 2024-04-20T21:31:43.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.851+0000 7f9205600700 1 --2- 172.21.15.22:0/1596650306 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91ec0a3e70 0x7f91ec0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.851+0000 7f9205600700 1 -- 172.21.15.22:0/1596650306 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f91f80d1330 con 0x7f9208102b00 2024-04-20T21:31:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.851+0000 7f91ff400700 1 --2- 172.21.15.22:0/1596650306 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91ec0a3e70 0x7f91ec0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.852+0000 7f91ff400700 1 --2- 172.21.15.22:0/1596650306 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91ec0a3e70 0x7f91ec0a6330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9208076d90 tx=0x7f91f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:43.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:43.853+0000 7f9205600700 1 -- 172.21.15.22:0/1596650306 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f91f8013070 con 0x7f9208102b00 2024-04-20T21:31:44.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.151+0000 7f920ee00700 1 -- 172.21.15.22:0/1596650306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f920819c790 con 0x7f9208102b00 2024-04-20T21:31:44.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.156+0000 7f9205600700 1 -- 172.21.15.22:0/1596650306 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264998 (secure 0 0 0) 0x7f91f809dcf0 con 0x7f9208102b00 2024-04-20T21:31:44.160 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:44.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.161+0000 7f920ee00700 1 -- 172.21.15.22:0/1596650306 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91ec0a3e70 msgr2=0x7f91ec0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:44.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.161+0000 7f920ee00700 1 --2- 172.21.15.22:0/1596650306 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91ec0a3e70 0x7f91ec0a6330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9208076d90 tx=0x7f91f0009260 comp rx=0 tx=0).stop 2024-04-20T21:31:44.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.161+0000 7f920ee00700 1 -- 172.21.15.22:0/1596650306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208102b00 msgr2=0x7f920819b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:44.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.161+0000 7f920ee00700 1 --2- 172.21.15.22:0/1596650306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208102b00 0x7f920819b430 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91f80098e0 tx=0x7f91f8003a90 comp rx=0 tx=0).stop 2024-04-20T21:31:44.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.161+0000 7f920ee00700 1 -- 172.21.15.22:0/1596650306 shutdown_connections 2024-04-20T21:31:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.161+0000 7f920ee00700 1 --2- 172.21.15.22:0/1596650306 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91ec0a3e70 0x7f91ec0a6330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.161+0000 7f920ee00700 1 --2- 172.21.15.22:0/1596650306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208107950 0x7f920819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.161+0000 7f920ee00700 1 --2- 172.21.15.22:0/1596650306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208102b00 0x7f920819b430 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.161+0000 7f920ee00700 1 --2- 172.21.15.22:0/1596650306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92081034c0 0x7f920819b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.161+0000 7f920ee00700 1 -- 172.21.15.22:0/1596650306 >> 172.21.15.22:0/1596650306 conn(0x7f9208079bb0 msgr2=0x7f9208100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.162+0000 7f920ee00700 1 -- 172.21.15.22:0/1596650306 shutdown_connections 2024-04-20T21:31:44.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:44.162+0000 7f920ee00700 1 -- 172.21.15.22:0/1596650306 wait complete. 2024-04-20T21:31:45.131 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1911146506},{"type":"v1","addr":"172.21.15.22:6801","nonce":1911146506}]},"active_addr":"172.21.15.22:6801/1911146506","active_change":"2024-04-20T21:23:29.510389+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.22:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":1408863783}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":3485677447}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":1815958949}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":3359736029}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":3452973972}]}]} 2024-04-20T21:31:45.137 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-20T21:31:45.137 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-20T21:31:45.137 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-20T21:31:45.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:45 smithi022 ceph-mon[16843]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.7 KiB/s wr, 6 op/s 2024-04-20T21:31:45.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:45 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1596650306' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T21:31:45.522 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:45 smithi122 ceph-mon[22147]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.7 KiB/s wr, 6 op/s 2024-04-20T21:31:45.523 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:45 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1596650306' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T21:31:45.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:45 smithi033 ceph-mon[21626]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.7 KiB/s wr, 6 op/s 2024-04-20T21:31:45.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:45 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1596650306' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T21:31:45.632 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:47.368 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:47 smithi022 ceph-mon[16843]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-04-20T21:31:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:47 smithi122 ceph-mon[22147]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-04-20T21:31:47.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:47 smithi033 ceph-mon[21626]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-04-20T21:31:47.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.863+0000 7f2a21800700 1 -- 172.21.15.22:0/2254565207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a1c101e90 msgr2=0x7f2a1c107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:47.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.863+0000 7f2a21800700 1 --2- 172.21.15.22:0/2254565207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a1c101e90 0x7f2a1c107670 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2a08009990 tx=0x7f2a08009ca0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.864+0000 7f2a21800700 1 -- 172.21.15.22:0/2254565207 shutdown_connections 2024-04-20T21:31:47.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.864+0000 7f2a21800700 1 --2- 172.21.15.22:0/2254565207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a1c107d50 0x7f2a1c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.864+0000 7f2a21800700 1 --2- 172.21.15.22:0/2254565207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a1c1014e0 0x7f2a1c1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.864+0000 7f2a21800700 1 --2- 172.21.15.22:0/2254565207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a1c101e90 0x7f2a1c107670 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.864+0000 7f2a21800700 1 -- 172.21.15.22:0/2254565207 >> 172.21.15.22:0/2254565207 conn(0x7f2a1c076c00 msgr2=0x7f2a1c077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:47.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.864+0000 7f2a21800700 1 -- 172.21.15.22:0/2254565207 shutdown_connections 2024-04-20T21:31:47.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.864+0000 7f2a21800700 1 -- 172.21.15.22:0/2254565207 wait complete. 2024-04-20T21:31:47.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.865+0000 7f2a21800700 1 Processor -- start 2024-04-20T21:31:47.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.865+0000 7f2a21800700 1 -- start start 2024-04-20T21:31:47.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a21800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a1c1014e0 0x7f2a1c102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:47.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a21800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a1c101e90 0x7f2a1c1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:47.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a21800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a1c107d50 0x7f2a1c102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:47.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a21800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c112d20 con 0x7f2a1c107d50 2024-04-20T21:31:47.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a21800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c112ba0 con 0x7f2a1c101e90 2024-04-20T21:31:47.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a21800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c112ea0 con 0x7f2a1c1014e0 2024-04-20T21:31:47.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a1be00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a1c1014e0 0x7f2a1c102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:47.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a1be00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a1c1014e0 0x7f2a1c102460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36108/0 (socket says 172.21.15.22:36108) 2024-04-20T21:31:47.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a1b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a1c101e90 0x7f2a1c1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:47.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a1be00700 1 -- 172.21.15.22:0/142748119 learned_addr learned my addr 172.21.15.22:0/142748119 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:47.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a20e00700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a1c107d50 0x7f2a1c102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:47.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a1b400700 1 -- 172.21.15.22:0/142748119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a1c1014e0 msgr2=0x7f2a1c102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:47.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a1b400700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a1c1014e0 0x7f2a1c102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a1b400700 1 -- 172.21.15.22:0/142748119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a1c107d50 msgr2=0x7f2a1c102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:47.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a1b400700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a1c107d50 0x7f2a1c102ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a1b400700 1 -- 172.21.15.22:0/142748119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c1a7660 con 0x7f2a1c101e90 2024-04-20T21:31:47.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.866+0000 7f2a1be00700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a1c1014e0 0x7f2a1c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:47.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.867+0000 7f2a1b400700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a1c101e90 0x7f2a1c1029a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2a080098e0 tx=0x7f2a0800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:47.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.867+0000 7f2a18c00700 1 -- 172.21.15.22:0/142748119 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08027070 con 0x7f2a1c101e90 2024-04-20T21:31:47.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.867+0000 7f2a21800700 1 -- 172.21.15.22:0/142748119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1c1a78f0 con 0x7f2a1c101e90 2024-04-20T21:31:47.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.867+0000 7f2a18c00700 1 -- 172.21.15.22:0/142748119 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a08003d70 con 0x7f2a1c101e90 2024-04-20T21:31:47.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.867+0000 7f2a18c00700 1 -- 172.21.15.22:0/142748119 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0800f460 con 0x7f2a1c101e90 2024-04-20T21:31:47.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.867+0000 7f2a21800700 1 -- 172.21.15.22:0/142748119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1c1a7da0 con 0x7f2a1c101e90 2024-04-20T21:31:47.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.868+0000 7f2a21800700 1 -- 172.21.15.22:0/142748119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a1c069740 con 0x7f2a1c101e90 2024-04-20T21:31:47.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.871+0000 7f2a18c00700 1 -- 172.21.15.22:0/142748119 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2a0802a070 con 0x7f2a1c101e90 2024-04-20T21:31:47.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.872+0000 7f2a18c00700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2a000a3cd0 0x7f2a000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:47.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.873+0000 7f2a18c00700 1 -- 172.21.15.22:0/142748119 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f2a080d1420 con 0x7f2a1c101e90 2024-04-20T21:31:47.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.873+0000 7f2a1be00700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2a000a3cd0 0x7f2a000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:47.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.873+0000 7f2a1be00700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2a000a3cd0 0x7f2a000a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c011550 tx=0x7f2a0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:47.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:47.877+0000 7f2a18c00700 1 -- 172.21.15.22:0/142748119 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2a0805eff0 con 0x7f2a1c101e90 2024-04-20T21:31:48.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.110+0000 7f2a21800700 1 -- 172.21.15.22:0/142748119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a1c02df90 con 0x7f2a1c101e90 2024-04-20T21:31:48.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.112+0000 7f2a18c00700 1 -- 172.21.15.22:0/142748119 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22196 (secure 0 0 0) 0x7f2a08013070 con 0x7f2a1c101e90 2024-04-20T21:31:48.112 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:48.112 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":88,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:31:28.510347+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:31:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.115+0000 7f2a21800700 1 -- 172.21.15.22:0/142748119 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2a000a3cd0 msgr2=0x7f2a000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.115+0000 7f2a21800700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2a000a3cd0 0x7f2a000a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c011550 tx=0x7f2a0c009260 comp rx=0 tx=0).stop 2024-04-20T21:31:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.115+0000 7f2a21800700 1 -- 172.21.15.22:0/142748119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a1c101e90 msgr2=0x7f2a1c1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.115+0000 7f2a21800700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a1c101e90 0x7f2a1c1029a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2a080098e0 tx=0x7f2a0800bab0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.116+0000 7f2a21800700 1 -- 172.21.15.22:0/142748119 shutdown_connections 2024-04-20T21:31:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.116+0000 7f2a21800700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2a000a3cd0 0x7f2a000a6190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.116+0000 7f2a21800700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a1c107d50 0x7f2a1c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.116+0000 7f2a21800700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a1c1014e0 0x7f2a1c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.116+0000 7f2a21800700 1 --2- 172.21.15.22:0/142748119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a1c101e90 0x7f2a1c1029a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:48.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.116+0000 7f2a21800700 1 -- 172.21.15.22:0/142748119 >> 172.21.15.22:0/142748119 conn(0x7f2a1c076c00 msgr2=0x7f2a1c100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:48.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.116+0000 7f2a21800700 1 -- 172.21.15.22:0/142748119 shutdown_connections 2024-04-20T21:31:48.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:48.116+0000 7f2a21800700 1 -- 172.21.15.22:0/142748119 wait complete. 2024-04-20T21:31:48.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:48 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/142748119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:48.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:48 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/142748119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:48.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:48 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/142748119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:48.931 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-20T21:31:48.931 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-20T21:31:49.414 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:49 smithi022 ceph-mon[16843]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-04-20T21:31:49.416 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:49.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:49 smithi122 ceph-mon[22147]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-04-20T21:31:49.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:49 smithi033 ceph-mon[21626]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-04-20T21:31:51.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:51 smithi022 ceph-mon[16843]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-04-20T21:31:51.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:51 smithi122 ceph-mon[22147]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-04-20T21:31:51.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:51 smithi033 ceph-mon[21626]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-04-20T21:31:51.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.729+0000 7fba4ca00700 1 -- 172.21.15.22:0/868134877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48074c80 msgr2=0x7fba48075060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.729+0000 7fba4ca00700 1 --2- 172.21.15.22:0/868134877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48074c80 0x7fba48075060 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba38009990 tx=0x7fba38009ca0 comp rx=0 tx=0).stop 2024-04-20T21:31:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.729+0000 7fba4ca00700 1 -- 172.21.15.22:0/868134877 shutdown_connections 2024-04-20T21:31:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.729+0000 7fba4ca00700 1 --2- 172.21.15.22:0/868134877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba4810f4b0 0x7fba481118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.729+0000 7fba4ca00700 1 --2- 172.21.15.22:0/868134877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48074c80 0x7fba48075060 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.729+0000 7fba4ca00700 1 --2- 172.21.15.22:0/868134877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba480755a0 0x7fba4810ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.729+0000 7fba4ca00700 1 -- 172.21.15.22:0/868134877 >> 172.21.15.22:0/868134877 conn(0x7fba48079bb0 msgr2=0x7fba48079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.729+0000 7fba4ca00700 1 -- 172.21.15.22:0/868134877 shutdown_connections 2024-04-20T21:31:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.729+0000 7fba4ca00700 1 -- 172.21.15.22:0/868134877 wait complete. 2024-04-20T21:31:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.731+0000 7fba4ca00700 1 Processor -- start 2024-04-20T21:31:51.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.731+0000 7fba4ca00700 1 -- start start 2024-04-20T21:31:51.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.731+0000 7fba4ca00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48074c80 0x7fba4819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.731+0000 7fba4ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba480755a0 0x7fba4819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba4ca00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba4810f4b0 0x7fba4819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba4ca00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba48115010 con 0x7fba4810f4b0 2024-04-20T21:31:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba4ca00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba48114e90 con 0x7fba480755a0 2024-04-20T21:31:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba4ca00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba48115190 con 0x7fba48074c80 2024-04-20T21:31:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba45e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48074c80 0x7fba4819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba45e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48074c80 0x7fba4819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36114/0 (socket says 172.21.15.22:36114) 2024-04-20T21:31:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba45e00700 1 -- 172.21.15.22:0/4055044666 learned_addr learned my addr 172.21.15.22:0/4055044666 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:31:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba45400700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba480755a0 0x7fba4819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba46800700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba4810f4b0 0x7fba4819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba45e00700 1 -- 172.21.15.22:0/4055044666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba480755a0 msgr2=0x7fba4819b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba45e00700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba480755a0 0x7fba4819b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba45e00700 1 -- 172.21.15.22:0/4055044666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba4810f4b0 msgr2=0x7fba4819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba45e00700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba4810f4b0 0x7fba4819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba45e00700 1 -- 172.21.15.22:0/4055044666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba481a0410 con 0x7fba48074c80 2024-04-20T21:31:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.732+0000 7fba46800700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba4810f4b0 0x7fba4819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.733+0000 7fba45e00700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48074c80 0x7fba4819b400 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fba38000c00 tx=0x7fba3800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.733+0000 7fba3ea00700 1 -- 172.21.15.22:0/4055044666 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba38027070 con 0x7fba48074c80 2024-04-20T21:31:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.733+0000 7fba3ea00700 1 -- 172.21.15.22:0/4055044666 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba38003d70 con 0x7fba48074c80 2024-04-20T21:31:51.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.733+0000 7fba4ca00700 1 -- 172.21.15.22:0/4055044666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba481a06a0 con 0x7fba48074c80 2024-04-20T21:31:51.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.733+0000 7fba3ea00700 1 -- 172.21.15.22:0/4055044666 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3800f460 con 0x7fba48074c80 2024-04-20T21:31:51.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.733+0000 7fba4ca00700 1 -- 172.21.15.22:0/4055044666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba481a7f80 con 0x7fba48074c80 2024-04-20T21:31:51.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.734+0000 7fba4ca00700 1 -- 172.21.15.22:0/4055044666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba48069740 con 0x7fba48074c80 2024-04-20T21:31:51.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.736+0000 7fba3ea00700 1 -- 172.21.15.22:0/4055044666 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fba3802a070 con 0x7fba48074c80 2024-04-20T21:31:51.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.738+0000 7fba3ea00700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d50 0x7fba2c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:51.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.738+0000 7fba3ea00700 1 -- 172.21.15.22:0/4055044666 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fba380d17e0 con 0x7fba48074c80 2024-04-20T21:31:51.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.738+0000 7fba45400700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d50 0x7fba2c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:51.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.739+0000 7fba45400700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d50 0x7fba2c0a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fba48110f20 tx=0x7fba3400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:51.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.740+0000 7fba3ea00700 1 -- 172.21.15.22:0/4055044666 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fba38013070 con 0x7fba48074c80 2024-04-20T21:31:51.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.977+0000 7fba4ca00700 1 -- 172.21.15.22:0/4055044666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba481a8150 con 0x7fba48074c80 2024-04-20T21:31:51.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.979+0000 7fba3ea00700 1 -- 172.21.15.22:0/4055044666 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22196 (secure 0 0 0) 0x7fba3809e1a0 con 0x7fba48074c80 2024-04-20T21:31:51.979 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:31:51.979 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":88,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:31:28.510347+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:31:51.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.982+0000 7fba4ca00700 1 -- 172.21.15.22:0/4055044666 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d50 msgr2=0x7fba2c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:51.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.982+0000 7fba4ca00700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d50 0x7fba2c0a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fba48110f20 tx=0x7fba3400b3c0 comp rx=0 tx=0).stop 2024-04-20T21:31:51.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.982+0000 7fba4ca00700 1 -- 172.21.15.22:0/4055044666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48074c80 msgr2=0x7fba4819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.982+0000 7fba4ca00700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48074c80 0x7fba4819b400 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fba38000c00 tx=0x7fba3800ba90 comp rx=0 tx=0).stop 2024-04-20T21:31:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.982+0000 7fba4ca00700 1 -- 172.21.15.22:0/4055044666 shutdown_connections 2024-04-20T21:31:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.982+0000 7fba4ca00700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d50 0x7fba2c0a6210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.982+0000 7fba4ca00700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba4810f4b0 0x7fba4819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.982+0000 7fba4ca00700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48074c80 0x7fba4819b400 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.982+0000 7fba4ca00700 1 --2- 172.21.15.22:0/4055044666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba480755a0 0x7fba4819b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.982+0000 7fba4ca00700 1 -- 172.21.15.22:0/4055044666 >> 172.21.15.22:0/4055044666 conn(0x7fba48079bb0 msgr2=0x7fba480ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.983+0000 7fba4ca00700 1 -- 172.21.15.22:0/4055044666 shutdown_connections 2024-04-20T21:31:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:31:51.983+0000 7fba4ca00700 1 -- 172.21.15.22:0/4055044666 wait complete. 2024-04-20T21:31:52.323 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:52 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/4055044666' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:52 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/4055044666' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:52 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/4055044666' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:52.890 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-20T21:31:52.891 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-20T21:31:52.891 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-20T21:31:52.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-20T21:31:52.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-20T21:31:52.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-20T21:31:52.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-20T21:31:52.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-20T21:31:52.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-20T21:31:52.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-20T21:31:52.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-20T21:31:52.895 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-20T21:31:53.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:53 smithi022 ceph-mon[16843]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-20T21:31:53.454 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:53 smithi033 ceph-mon[21626]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-20T21:31:53.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:53 smithi122 ceph-mon[22147]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-20T21:31:53.900 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:53.914 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:53.931 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:54.073 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:54.082 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:54.083 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:55.429 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:55 smithi022 ceph-mon[16843]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-20T21:31:55.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:55 smithi122 ceph-mon[22147]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-20T21:31:55.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:55 smithi033 ceph-mon[21626]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-20T21:31:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:57.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:57 smithi022 ceph-mon[16843]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T21:31:57.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:57 smithi122 ceph-mon[22147]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T21:31:57.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:57 smithi033 ceph-mon[21626]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T21:31:58.659 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:58.662 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:59.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:31:59 smithi122 ceph-mon[22147]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T21:31:59.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:31:59 smithi033 ceph-mon[21626]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T21:31:59.599 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:31:59 smithi022 ceph-mon[16843]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T21:32:00.220 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:00.227 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.257+0000 7fd1b3e00700 1 -- 172.21.15.22:0/219145528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4108980 msgr2=0x7fd1b4108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.257+0000 7fd1b3e00700 1 --2- 172.21.15.22:0/219145528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4108980 0x7fd1b4108d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd1a40098f0 tx=0x7fd1a4009c00 comp rx=0 tx=0).stop 2024-04-20T21:32:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.257+0000 7fd1b3e00700 1 -- 172.21.15.22:0/219145528 shutdown_connections 2024-04-20T21:32:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.257+0000 7fd1b3e00700 1 --2- 172.21.15.22:0/219145528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b4103340 0x7fd1b410f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.257+0000 7fd1b3e00700 1 --2- 172.21.15.22:0/219145528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4108980 0x7fd1b4108d60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.257+0000 7fd1b3e00700 1 --2- 172.21.15.22:0/219145528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b4102980 0x7fd1b4102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.257+0000 7fd1b3e00700 1 -- 172.21.15.22:0/219145528 >> 172.21.15.22:0/219145528 conn(0x7fd1b40fe4d0 msgr2=0x7fd1b41008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.257+0000 7fd1b3e00700 1 -- 172.21.15.22:0/219145528 shutdown_connections 2024-04-20T21:32:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.257+0000 7fd1b3e00700 1 -- 172.21.15.22:0/219145528 wait complete. 2024-04-20T21:32:00.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.258+0000 7fd1b3e00700 1 Processor -- start 2024-04-20T21:32:00.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.259+0000 7fd1b3e00700 1 -- start start 2024-04-20T21:32:00.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.259+0000 7fd1b3e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b4102980 0x7fd1b419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:00.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.259+0000 7fd1b3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b4103340 0x7fd1b419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:00.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.259+0000 7fd1b3e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4108980 0x7fd1b419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:00.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.259+0000 7fd1b3e00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4112d10 con 0x7fd1b4103340 2024-04-20T21:32:00.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.259+0000 7fd1b3e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4112b90 con 0x7fd1b4108980 2024-04-20T21:32:00.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.259+0000 7fd1b3e00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4112e90 con 0x7fd1b4102980 2024-04-20T21:32:00.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.260+0000 7fd1b2a00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b4102980 0x7fd1b419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:00.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.260+0000 7fd1b2a00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b4102980 0x7fd1b419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46348/0 (socket says 172.21.15.22:46348) 2024-04-20T21:32:00.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.260+0000 7fd1b2a00700 1 -- 172.21.15.22:0/1676819736 learned_addr learned my addr 172.21.15.22:0/1676819736 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:00.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.260+0000 7fd1b3400700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4108980 0x7fd1b419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:00.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.260+0000 7fd1b2000700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b4103340 0x7fd1b419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:00.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.261+0000 7fd1b3400700 1 -- 172.21.15.22:0/1676819736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b4102980 msgr2=0x7fd1b419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:00.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.261+0000 7fd1b3400700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b4102980 0x7fd1b419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.261+0000 7fd1b3400700 1 -- 172.21.15.22:0/1676819736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b4103340 msgr2=0x7fd1b419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:00.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.261+0000 7fd1b3400700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b4103340 0x7fd1b419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.261+0000 7fd1b3400700 1 -- 172.21.15.22:0/1676819736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b41a0390 con 0x7fd1b4108980 2024-04-20T21:32:00.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.261+0000 7fd1b3400700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4108980 0x7fd1b419fc50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac007f10 tx=0x7fd1ac00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:00.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.262+0000 7fd1ab400700 1 -- 172.21.15.22:0/1676819736 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac00dbd0 con 0x7fd1b4108980 2024-04-20T21:32:00.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.262+0000 7fd1ab400700 1 -- 172.21.15.22:0/1676819736 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ac00dd70 con 0x7fd1b4108980 2024-04-20T21:32:00.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.262+0000 7fd1ab400700 1 -- 172.21.15.22:0/1676819736 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac015400 con 0x7fd1b4108980 2024-04-20T21:32:00.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.262+0000 7fd1b3e00700 1 -- 172.21.15.22:0/1676819736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b41a0680 con 0x7fd1b4108980 2024-04-20T21:32:00.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.262+0000 7fd1b3e00700 1 -- 172.21.15.22:0/1676819736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b41a7f00 con 0x7fd1b4108980 2024-04-20T21:32:00.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.263+0000 7fd1b3e00700 1 -- 172.21.15.22:0/1676819736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd1b40ffd00 con 0x7fd1b4108980 2024-04-20T21:32:00.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.265+0000 7fd1ab400700 1 -- 172.21.15.22:0/1676819736 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd1ac005020 con 0x7fd1b4108980 2024-04-20T21:32:00.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.267+0000 7fd1ab400700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd1980a3d50 0x7fd1980a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:00.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.267+0000 7fd1ab400700 1 -- 172.21.15.22:0/1676819736 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd1ac0cfce0 con 0x7fd1b4108980 2024-04-20T21:32:00.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.268+0000 7fd1b2a00700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd1980a3d50 0x7fd1980a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:00.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.268+0000 7fd1ab400700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204] conn(0x7fd1980ad530 0x7fd1980af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:00.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.268+0000 7fd1ab400700 1 -- 172.21.15.22:0/1676819736 --> [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1980b0000 con 0x7fd1980ad530 2024-04-20T21:32:00.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.268+0000 7fd1ab400700 1 -- 172.21.15.22:0/1676819736 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd1ac0d2300 con 0x7fd1b4108980 2024-04-20T21:32:00.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.268+0000 7fd1b2000700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204] conn(0x7fd1980ad530 0x7fd1980af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:00.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.269+0000 7fd1b2a00700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd1980a3d50 0x7fd1980a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400c010 tx=0x7fd1a4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:00.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.269+0000 7fd1b2000700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204] conn(0x7fd1980ad530 0x7fd1980af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:00.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.271+0000 7fd1ab400700 1 -- 172.21.15.22:0/1676819736 <== osd.11 v2:172.21.15.122:6824/3622105204 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1980b0000 con 0x7fd1980ad530 2024-04-20T21:32:00.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.290+0000 7fd1b3e00700 1 -- 172.21.15.22:0/1676819736 --> [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd1b41040c0 con 0x7fd1980ad530 2024-04-20T21:32:00.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.291+0000 7fd1ab400700 1 -- 172.21.15.22:0/1676819736 <== osd.11 v2:172.21.15.122:6824/3622105204 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd1b41040c0 con 0x7fd1980ad530 2024-04-20T21:32:00.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.292+0000 7fd1b3e00700 1 -- 172.21.15.22:0/1676819736 >> [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204] conn(0x7fd1980ad530 msgr2=0x7fd1980af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:00.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.292+0000 7fd1b3e00700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204] conn(0x7fd1980ad530 0x7fd1980af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.292+0000 7fd1b3e00700 1 -- 172.21.15.22:0/1676819736 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd1980a3d50 msgr2=0x7fd1980a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:00.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.292+0000 7fd1b3e00700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd1980a3d50 0x7fd1980a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400c010 tx=0x7fd1a4019040 comp rx=0 tx=0).stop 2024-04-20T21:32:00.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.292+0000 7fd1b3e00700 1 -- 172.21.15.22:0/1676819736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4108980 msgr2=0x7fd1b419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:00.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.292+0000 7fd1b3e00700 1 --2- 172.21.15.22:0/1676819736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4108980 0x7fd1b419fc50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac007f10 tx=0x7fd1ac00d3b0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.292+0000 7fd1b3400700 1 -- 172.21.15.22:0/1676819736 reap_dead start 2024-04-20T21:32:00.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.292+0000 7fd1b3e00700 1 -- 172.21.15.22:0/1676819736 shutdown_connections 2024-04-20T21:32:00.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.292+0000 7fd1b3e00700 1 -- 172.21.15.22:0/1676819736 >> 172.21.15.22:0/1676819736 conn(0x7fd1b40fe4d0 msgr2=0x7fd1b410e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:00.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.292+0000 7fd1b3e00700 1 -- 172.21.15.22:0/1676819736 shutdown_connections 2024-04-20T21:32:00.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:00.292+0000 7fd1b3e00700 1 -- 172.21.15.22:0/1676819736 wait complete. 2024-04-20T21:32:01.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:01 smithi022 ceph-mon[16843]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:32:01.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:01 smithi122 ceph-mon[22147]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:32:01.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:01 smithi033 ceph-mon[21626]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:32:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.827+0000 7f71ff800700 1 -- 172.21.15.22:0/947964163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8108a40 msgr2=0x7f71f8108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.827+0000 7f71ff800700 1 --2- 172.21.15.22:0/947964163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8108a40 0x7f71f8108e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009990 tx=0x7f71ec009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.827+0000 7f71ff800700 1 -- 172.21.15.22:0/947964163 shutdown_connections 2024-04-20T21:32:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.827+0000 7f71ff800700 1 --2- 172.21.15.22:0/947964163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f806b100 0x7f71f810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.827+0000 7f71ff800700 1 --2- 172.21.15.22:0/947964163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8108a40 0x7f71f8108e20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.827+0000 7f71ff800700 1 --2- 172.21.15.22:0/947964163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71f806a760 0x7f71f806abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.827+0000 7f71ff800700 1 -- 172.21.15.22:0/947964163 >> 172.21.15.22:0/947964163 conn(0x7f71f8078900 msgr2=0x7f71f8078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.827+0000 7f71ff800700 1 -- 172.21.15.22:0/947964163 shutdown_connections 2024-04-20T21:32:02.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.828+0000 7f71ff800700 1 -- 172.21.15.22:0/947964163 wait complete. 2024-04-20T21:32:02.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.829+0000 7f71ff800700 1 Processor -- start 2024-04-20T21:32:02.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.830+0000 7f71ff800700 1 -- start start 2024-04-20T21:32:02.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.830+0000 7f71ff800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71f806a760 0x7f71f8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:02.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.830+0000 7f71ff800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f806b100 0x7f71f81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:02.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.830+0000 7f71ff800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8108a40 0x7f71f819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:02.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.830+0000 7f71ff800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8112d30 con 0x7f71f8108a40 2024-04-20T21:32:02.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.830+0000 7f71ff800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8112bb0 con 0x7f71f806b100 2024-04-20T21:32:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.830+0000 7f71ff800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8112eb0 con 0x7f71f806a760 2024-04-20T21:32:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.830+0000 7f71fce00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71f806a760 0x7f71f8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.830+0000 7f71fd800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8108a40 0x7f71f819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.830+0000 7f71fce00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71f806a760 0x7f71f8199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:58790/0 (socket says 172.21.15.22:58790) 2024-04-20T21:32:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.831+0000 7f71fce00700 1 -- 172.21.15.22:0/2446697564 learned_addr learned my addr 172.21.15.22:0/2446697564 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.831+0000 7f71f7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f806b100 0x7f71f81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:02.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.831+0000 7f71fce00700 1 -- 172.21.15.22:0/2446697564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f806b100 msgr2=0x7f71f81997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.831+0000 7f71fce00700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f806b100 0x7f71f81997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.831+0000 7f71fce00700 1 -- 172.21.15.22:0/2446697564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8108a40 msgr2=0x7f71f819dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.831+0000 7f71fce00700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8108a40 0x7f71f819dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.831+0000 7f71fce00700 1 -- 172.21.15.22:0/2446697564 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f819e220 con 0x7f71f806a760 2024-04-20T21:32:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.831+0000 7f71f7e00700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f806b100 0x7f71f81997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.831+0000 7f71fce00700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71f806a760 0x7f71f8199260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00bae0 tx=0x7f71ec00bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.832+0000 7f71f5600700 1 -- 172.21.15.22:0/2446697564 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec026070 con 0x7f71f806a760 2024-04-20T21:32:02.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.832+0000 7f71f5600700 1 -- 172.21.15.22:0/2446697564 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ec003d70 con 0x7f71f806a760 2024-04-20T21:32:02.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.832+0000 7f71f5600700 1 -- 172.21.15.22:0/2446697564 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00fe50 con 0x7f71f806a760 2024-04-20T21:32:02.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.832+0000 7f71ff800700 1 -- 172.21.15.22:0/2446697564 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f819e4b0 con 0x7f71f806a760 2024-04-20T21:32:02.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.832+0000 7f71ff800700 1 -- 172.21.15.22:0/2446697564 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f81a5d90 con 0x7f71f806a760 2024-04-20T21:32:02.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.833+0000 7f71ff800700 1 -- 172.21.15.22:0/2446697564 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71f80631e0 con 0x7f71f806a760 2024-04-20T21:32:02.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.835+0000 7f71f5600700 1 -- 172.21.15.22:0/2446697564 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f71ec029070 con 0x7f71f806a760 2024-04-20T21:32:02.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.837+0000 7f71f5600700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f71e00a3ef0 0x7f71e00a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:02.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.837+0000 7f71f5600700 1 -- 172.21.15.22:0/2446697564 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f71ec0d1890 con 0x7f71f806a760 2024-04-20T21:32:02.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.837+0000 7f71f7e00700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f71e00a3ef0 0x7f71e00a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:02.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.837+0000 7f71f5600700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534] conn(0x7f71e00ad5a0 0x7f71e00af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:02.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.838+0000 7f71f5600700 1 -- 172.21.15.22:0/2446697564 --> [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71ec028e50 con 0x7f71e00ad5a0 2024-04-20T21:32:02.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.838+0000 7f71f5600700 1 -- 172.21.15.22:0/2446697564 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f71ec0d1c40 con 0x7f71f806a760 2024-04-20T21:32:02.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.838+0000 7f71fd800700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534] conn(0x7f71e00ad5a0 0x7f71e00af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:02.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.838+0000 7f71f7e00700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f71e00a3ef0 0x7f71e00a63b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f71f8078f70 tx=0x7f71e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:02.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.839+0000 7f71fd800700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534] conn(0x7f71e00ad5a0 0x7f71e00af9c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:02.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.839+0000 7f71f5600700 1 -- 172.21.15.22:0/2446697564 <== osd.2 v2:172.21.15.122:6800/4001148534 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71ec028e50 con 0x7f71e00ad5a0 2024-04-20T21:32:02.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.859+0000 7f71ff800700 1 -- 172.21.15.22:0/2446697564 --> [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71f80636b0 con 0x7f71e00ad5a0 2024-04-20T21:32:02.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.860+0000 7f71f5600700 1 -- 172.21.15.22:0/2446697564 <== osd.2 v2:172.21.15.122:6800/4001148534 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f71f80636b0 con 0x7f71e00ad5a0 2024-04-20T21:32:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.860+0000 7f71ff800700 1 -- 172.21.15.22:0/2446697564 >> [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534] conn(0x7f71e00ad5a0 msgr2=0x7f71e00af9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.860+0000 7f71ff800700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534] conn(0x7f71e00ad5a0 0x7f71e00af9c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.860+0000 7f71ff800700 1 -- 172.21.15.22:0/2446697564 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f71e00a3ef0 msgr2=0x7f71e00a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.861+0000 7f71ff800700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f71e00a3ef0 0x7f71e00a63b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f71f8078f70 tx=0x7f71e4009260 comp rx=0 tx=0).stop 2024-04-20T21:32:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.861+0000 7f71ff800700 1 -- 172.21.15.22:0/2446697564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71f806a760 msgr2=0x7f71f8199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.861+0000 7f71ff800700 1 --2- 172.21.15.22:0/2446697564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71f806a760 0x7f71f8199260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00bae0 tx=0x7f71ec00bb10 comp rx=0 tx=0).stop 2024-04-20T21:32:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.861+0000 7f71fd800700 1 -- 172.21.15.22:0/2446697564 reap_dead start 2024-04-20T21:32:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.861+0000 7f71ff800700 1 -- 172.21.15.22:0/2446697564 shutdown_connections 2024-04-20T21:32:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.861+0000 7f71ff800700 1 -- 172.21.15.22:0/2446697564 >> 172.21.15.22:0/2446697564 conn(0x7f71f8078900 msgr2=0x7f71f80ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:02.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.861+0000 7f71ff800700 1 -- 172.21.15.22:0/2446697564 shutdown_connections 2024-04-20T21:32:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:02.861+0000 7f71ff800700 1 -- 172.21.15.22:0/2446697564 wait complete. 2024-04-20T21:32:03.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:03 smithi022 ceph-mon[16843]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T21:32:03.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:03 smithi122 ceph-mon[22147]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T21:32:03.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:03 smithi033 ceph-mon[21626]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T21:32:05.303 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:05 smithi022 ceph-mon[16843]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:32:05.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:05 smithi122 ceph-mon[22147]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:32:05.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:05 smithi033 ceph-mon[21626]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:32:07.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.081+0000 7ff93a200700 1 -- 172.21.15.22:0/3140357405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934074c80 msgr2=0x7ff934075060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:07.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.081+0000 7ff93a200700 1 --2- 172.21.15.22:0/3140357405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934074c80 0x7ff934075060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff924009990 tx=0x7ff924009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.081+0000 7ff93a200700 1 -- 172.21.15.22:0/3140357405 shutdown_connections 2024-04-20T21:32:07.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.081+0000 7ff93a200700 1 --2- 172.21.15.22:0/3140357405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff93410f480 0x7ff934111870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.081+0000 7ff93a200700 1 --2- 172.21.15.22:0/3140357405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934074c80 0x7ff934075060 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.081+0000 7ff93a200700 1 --2- 172.21.15.22:0/3140357405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9340755a0 0x7ff93410ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.081+0000 7ff93a200700 1 -- 172.21.15.22:0/3140357405 >> 172.21.15.22:0/3140357405 conn(0x7ff934079b20 msgr2=0x7ff934079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:07.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.081+0000 7ff93a200700 1 -- 172.21.15.22:0/3140357405 shutdown_connections 2024-04-20T21:32:07.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.081+0000 7ff93a200700 1 -- 172.21.15.22:0/3140357405 wait complete. 2024-04-20T21:32:07.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.082+0000 7ff93a200700 1 Processor -- start 2024-04-20T21:32:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.083+0000 7ff93a200700 1 -- start start 2024-04-20T21:32:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.083+0000 7ff93a200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff934074c80 0x7ff93419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.083+0000 7ff93a200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9340755a0 0x7ff93419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.083+0000 7ff93a200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff93410f480 0x7ff93419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.083+0000 7ff93a200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff934114fd0 con 0x7ff9340755a0 2024-04-20T21:32:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.083+0000 7ff93a200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff934114e50 con 0x7ff93410f480 2024-04-20T21:32:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.083+0000 7ff93a200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff934115150 con 0x7ff934074c80 2024-04-20T21:32:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.083+0000 7ff92aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9340755a0 0x7ff93419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.083+0000 7ff933400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff934074c80 0x7ff93419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.083+0000 7ff933400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff934074c80 0x7ff93419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:58848/0 (socket says 172.21.15.22:58848) 2024-04-20T21:32:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.083+0000 7ff933400700 1 -- 172.21.15.22:0/2746527476 learned_addr learned my addr 172.21.15.22:0/2746527476 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:07.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.084+0000 7ff933e00700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff93410f480 0x7ff93419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:07.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.084+0000 7ff933e00700 1 -- 172.21.15.22:0/2746527476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff934074c80 msgr2=0x7ff93419b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:07.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.084+0000 7ff933e00700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff934074c80 0x7ff93419b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.084+0000 7ff933e00700 1 -- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9340755a0 msgr2=0x7ff93419b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:07.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.084+0000 7ff933e00700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9340755a0 0x7ff93419b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.084+0000 7ff933e00700 1 -- 172.21.15.22:0/2746527476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9341a03e0 con 0x7ff93410f480 2024-04-20T21:32:07.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.084+0000 7ff92aa00700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9340755a0 0x7ff93419b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:07.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.084+0000 7ff933e00700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff93410f480 0x7ff93419fca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff92c007ee0 tx=0x7ff92c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:07.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.085+0000 7ff930c00700 1 -- 172.21.15.22:0/2746527476 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c004db0 con 0x7ff93410f480 2024-04-20T21:32:07.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.085+0000 7ff93a200700 1 -- 172.21.15.22:0/2746527476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9341a06d0 con 0x7ff93410f480 2024-04-20T21:32:07.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.085+0000 7ff930c00700 1 -- 172.21.15.22:0/2746527476 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff92c009e20 con 0x7ff93410f480 2024-04-20T21:32:07.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.085+0000 7ff93a200700 1 -- 172.21.15.22:0/2746527476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9341a7f50 con 0x7ff93410f480 2024-04-20T21:32:07.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.085+0000 7ff930c00700 1 -- 172.21.15.22:0/2746527476 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c015400 con 0x7ff93410f480 2024-04-20T21:32:07.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.086+0000 7ff93a200700 1 -- 172.21.15.22:0/2746527476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff910001030 con 0x7ff93410f480 2024-04-20T21:32:07.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.088+0000 7ff930c00700 1 -- 172.21.15.22:0/2746527476 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff92c00db20 con 0x7ff93410f480 2024-04-20T21:32:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.090+0000 7ff930c00700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff9180a3cd0 0x7ff9180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.090+0000 7ff930c00700 1 -- 172.21.15.22:0/2746527476 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7ff92c0d0210 con 0x7ff93410f480 2024-04-20T21:32:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.090+0000 7ff933400700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff9180a3cd0 0x7ff9180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.090+0000 7ff930c00700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045] conn(0x7ff9180ad4b0 0x7ff9180af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:07.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.090+0000 7ff930c00700 1 -- 172.21.15.22:0/2746527476 --> [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9180aff80 con 0x7ff9180ad4b0 2024-04-20T21:32:07.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.090+0000 7ff930c00700 1 -- 172.21.15.22:0/2746527476 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7ff92c0d05c0 con 0x7ff93410f480 2024-04-20T21:32:07.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.090+0000 7ff92aa00700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045] conn(0x7ff9180ad4b0 0x7ff9180af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:07.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.091+0000 7ff933400700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff9180a3cd0 0x7ff9180a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff924006010 tx=0x7ff92400b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:07.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.091+0000 7ff92aa00700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045] conn(0x7ff9180ad4b0 0x7ff9180af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:07.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.092+0000 7ff930c00700 1 -- 172.21.15.22:0/2746527476 <== osd.3 v2:172.21.15.22:6810/1762015045 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9180aff80 con 0x7ff9180ad4b0 2024-04-20T21:32:07.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.112+0000 7ff93a200700 1 -- 172.21.15.22:0/2746527476 --> [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff910002e60 con 0x7ff9180ad4b0 2024-04-20T21:32:07.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.112+0000 7ff930c00700 1 -- 172.21.15.22:0/2746527476 <== osd.3 v2:172.21.15.22:6810/1762015045 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff910002e60 con 0x7ff9180ad4b0 2024-04-20T21:32:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.113+0000 7ff93a200700 1 -- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045] conn(0x7ff9180ad4b0 msgr2=0x7ff9180af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.113+0000 7ff93a200700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045] conn(0x7ff9180ad4b0 0x7ff9180af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.113+0000 7ff93a200700 1 -- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff9180a3cd0 msgr2=0x7ff9180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.113+0000 7ff93a200700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff9180a3cd0 0x7ff9180a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff924006010 tx=0x7ff92400b560 comp rx=0 tx=0).stop 2024-04-20T21:32:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.113+0000 7ff93a200700 1 -- 172.21.15.22:0/2746527476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff93410f480 msgr2=0x7ff93419fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:07.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.113+0000 7ff93a200700 1 --2- 172.21.15.22:0/2746527476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff93410f480 0x7ff93419fca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff92c007ee0 tx=0x7ff92c00d3b0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.113+0000 7ff933e00700 1 -- 172.21.15.22:0/2746527476 reap_dead start 2024-04-20T21:32:07.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.113+0000 7ff93a200700 1 -- 172.21.15.22:0/2746527476 shutdown_connections 2024-04-20T21:32:07.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.113+0000 7ff93a200700 1 -- 172.21.15.22:0/2746527476 >> 172.21.15.22:0/2746527476 conn(0x7ff934079b20 msgr2=0x7ff9340ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:07.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.114+0000 7ff93a200700 1 -- 172.21.15.22:0/2746527476 shutdown_connections 2024-04-20T21:32:07.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:07.114+0000 7ff93a200700 1 -- 172.21.15.22:0/2746527476 wait complete. 2024-04-20T21:32:07.222 INFO:teuthology.orchestra.run.smithi022.stdout:356482285588 2024-04-20T21:32:07.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-20T21:32:07.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:07 smithi022 ceph-mon[16843]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:07.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:07 smithi122 ceph-mon[22147]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:07.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:07 smithi033 ceph-mon[21626]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:08.452 INFO:teuthology.orchestra.run.smithi022.stdout:111669149762 2024-04-20T21:32:08.452 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-20T21:32:08.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.555+0000 7fea8e600700 1 -- 172.21.15.22:0/499461618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea8810e3c0 msgr2=0x7fea88115f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:08.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.555+0000 7fea8e600700 1 --2- 172.21.15.22:0/499461618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea8810e3c0 0x7fea88115f90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea80009fc0 tx=0x7fea8000b330 comp rx=0 tx=0).stop 2024-04-20T21:32:08.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.555+0000 7fea8e600700 1 -- 172.21.15.22:0/499461618 shutdown_connections 2024-04-20T21:32:08.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.555+0000 7fea8e600700 1 --2- 172.21.15.22:0/499461618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea8810e3c0 0x7fea88115f90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:08.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.555+0000 7fea8e600700 1 --2- 172.21.15.22:0/499461618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881092e0 0x7fea881096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:08.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.555+0000 7fea8e600700 1 --2- 172.21.15.22:0/499461618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea88109c90 0x7fea8810dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:08.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.555+0000 7fea8e600700 1 -- 172.21.15.22:0/499461618 >> 172.21.15.22:0/499461618 conn(0x7fea88076220 msgr2=0x7fea88107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:08.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.555+0000 7fea8e600700 1 -- 172.21.15.22:0/499461618 shutdown_connections 2024-04-20T21:32:08.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.555+0000 7fea8e600700 1 -- 172.21.15.22:0/499461618 wait complete. 2024-04-20T21:32:08.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.556+0000 7fea8e600700 1 Processor -- start 2024-04-20T21:32:08.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.557+0000 7fea8e600700 1 -- start start 2024-04-20T21:32:08.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.557+0000 7fea8e600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea881092e0 0x7fea881a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:08.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.557+0000 7fea8e600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea88109c90 0x7fea881a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:08.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.557+0000 7fea8e600700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea8810e3c0 0x7fea8819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:08.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.557+0000 7fea8e600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fea88119400 con 0x7fea881092e0 2024-04-20T21:32:08.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.557+0000 7fea8e600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea88119280 con 0x7fea88109c90 2024-04-20T21:32:08.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.557+0000 7fea8e600700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea88119580 con 0x7fea8810e3c0 2024-04-20T21:32:08.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.557+0000 7fea87400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea881092e0 0x7fea881a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:08.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.557+0000 7fea87400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea881092e0 0x7fea881a5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38422/0 (socket says 172.21.15.22:38422) 2024-04-20T21:32:08.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.557+0000 7fea87400700 1 -- 172.21.15.22:0/188119138 learned_addr learned my addr 172.21.15.22:0/188119138 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:08.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.558+0000 7fea86a00700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea88109c90 0x7fea881a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:08.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.558+0000 7fea87e00700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea8810e3c0 0x7fea8819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:08.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.558+0000 7fea87400700 1 -- 172.21.15.22:0/188119138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea8810e3c0 msgr2=0x7fea8819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:08.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.558+0000 7fea87400700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea8810e3c0 0x7fea8819f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:08.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.558+0000 7fea87400700 1 -- 172.21.15.22:0/188119138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea88109c90 msgr2=0x7fea881a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:08.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.558+0000 7fea87400700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea88109c90 0x7fea881a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.558+0000 7fea87400700 1 -- 172.21.15.22:0/188119138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8819fd20 con 0x7fea881092e0 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.558+0000 7fea87400700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea881092e0 0x7fea881a5400 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fea7800e9a0 tx=0x7fea7800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.559+0000 7fea7fe00700 1 -- 172.21.15.22:0/188119138 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7800c9c0 con 0x7fea881092e0 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.559+0000 7fea8e600700 1 -- 172.21.15.22:0/188119138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea881a0010 con 0x7fea881092e0 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.559+0000 7fea7fe00700 1 -- 172.21.15.22:0/188119138 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea7800cb60 con 0x7fea881092e0 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.559+0000 7fea8e600700 1 -- 172.21.15.22:0/188119138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea881ac2d0 con 0x7fea881092e0 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.559+0000 7fea7fe00700 1 -- 172.21.15.22:0/188119138 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78018620 con 0x7fea881092e0 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.560+0000 7fea8e600700 1 -- 172.21.15.22:0/188119138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea88077160 con 0x7fea881092e0 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.562+0000 7fea7fe00700 1 -- 172.21.15.22:0/188119138 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fea7800cd10 con 0x7fea881092e0 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.563+0000 7fea7fe00700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fea6c0a3cd0 0x7fea6c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.563+0000 7fea7fe00700 1 -- 172.21.15.22:0/188119138 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fea780d0aa0 con 0x7fea881092e0 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.564+0000 7fea86a00700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fea6c0a3cd0 0x7fea6c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:08.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.564+0000 7fea7fe00700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998] conn(0x7fea6c0ad4b0 0x7fea6c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:08.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.564+0000 7fea7fe00700 1 -- 172.21.15.22:0/188119138 --> [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea7800be50 con 0x7fea6c0ad4b0 2024-04-20T21:32:08.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.564+0000 7fea7fe00700 1 -- 172.21.15.22:0/188119138 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fea780d0e50 con 0x7fea881092e0 2024-04-20T21:32:08.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.564+0000 7fea87e00700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998] conn(0x7fea6c0ad4b0 0x7fea6c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:08.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.564+0000 7fea86a00700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fea6c0a3cd0 0x7fea6c0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea8810f5d0 tx=0x7fea74009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:08.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.565+0000 7fea87e00700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998] conn(0x7fea6c0ad4b0 0x7fea6c0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:08.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.565+0000 7fea7fe00700 1 -- 172.21.15.22:0/188119138 <== osd.4 v2:172.21.15.33:6808/1935934998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea7800be50 con 0x7fea6c0ad4b0 2024-04-20T21:32:08.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.584+0000 7fea8e600700 1 -- 172.21.15.22:0/188119138 --> [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea8810a7c0 con 0x7fea6c0ad4b0 2024-04-20T21:32:08.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.584+0000 7fea7fe00700 1 -- 172.21.15.22:0/188119138 <== osd.4 v2:172.21.15.33:6808/1935934998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea8810a7c0 con 0x7fea6c0ad4b0 2024-04-20T21:32:08.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.585+0000 7fea8e600700 1 -- 172.21.15.22:0/188119138 >> [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998] conn(0x7fea6c0ad4b0 msgr2=0x7fea6c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:08.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.585+0000 7fea8e600700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998] conn(0x7fea6c0ad4b0 0x7fea6c0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:08.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.585+0000 7fea8e600700 1 -- 172.21.15.22:0/188119138 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fea6c0a3cd0 msgr2=0x7fea6c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:08.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.585+0000 7fea8e600700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fea6c0a3cd0 0x7fea6c0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea8810f5d0 tx=0x7fea74009310 comp rx=0 tx=0).stop 2024-04-20T21:32:08.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.585+0000 7fea8e600700 1 -- 172.21.15.22:0/188119138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea881092e0 msgr2=0x7fea881a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:08.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.585+0000 7fea8e600700 1 --2- 172.21.15.22:0/188119138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea881092e0 0x7fea881a5400 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fea7800e9a0 tx=0x7fea7800ecb0 comp rx=0 tx=0).stop 2024-04-20T21:32:08.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.585+0000 7fea87e00700 1 -- 172.21.15.22:0/188119138 reap_dead start 2024-04-20T21:32:08.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.585+0000 7fea8e600700 1 -- 172.21.15.22:0/188119138 shutdown_connections 2024-04-20T21:32:08.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.585+0000 7fea8e600700 1 -- 172.21.15.22:0/188119138 >> 172.21.15.22:0/188119138 conn(0x7fea88076220 msgr2=0x7fea88076700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:08.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.586+0000 7fea8e600700 1 -- 172.21.15.22:0/188119138 shutdown_connections 2024-04-20T21:32:08.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:06.586+0000 7fea8e600700 1 -- 172.21.15.22:0/188119138 wait complete. 2024-04-20T21:32:09.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:09 smithi122 ceph-mon[22147]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:09.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:09 smithi033 ceph-mon[21626]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:09.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:09 smithi022 ceph-mon[16843]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:10.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.250+0000 7fd65dc00700 1 -- 172.21.15.22:0/792793903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd65806a760 msgr2=0x7fd65806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:10.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.250+0000 7fd65dc00700 1 --2- 172.21.15.22:0/792793903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd65806a760 0x7fd65806ab40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd648009990 tx=0x7fd648009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.250+0000 7fd65dc00700 1 -- 172.21.15.22:0/792793903 shutdown_connections 2024-04-20T21:32:10.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.250+0000 7fd65dc00700 1 --2- 172.21.15.22:0/792793903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd65810f500 0x7fd6581118f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.250+0000 7fd65dc00700 1 --2- 172.21.15.22:0/792793903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd65806a760 0x7fd65806ab40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.250+0000 7fd65dc00700 1 --2- 172.21.15.22:0/792793903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd65806b080 0x7fd65810efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.250+0000 7fd65dc00700 1 -- 172.21.15.22:0/792793903 >> 172.21.15.22:0/792793903 conn(0x7fd658078900 msgr2=0x7fd658078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:10.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.250+0000 7fd65dc00700 1 -- 172.21.15.22:0/792793903 shutdown_connections 2024-04-20T21:32:10.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.250+0000 7fd65dc00700 1 -- 172.21.15.22:0/792793903 wait complete. 2024-04-20T21:32:10.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.251+0000 7fd65dc00700 1 Processor -- start 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.251+0000 7fd65dc00700 1 -- start start 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.252+0000 7fd65dc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd65806a760 0x7fd65819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.252+0000 7fd65dc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd65806b080 0x7fd65819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.252+0000 7fd65dc00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd65810f500 0x7fd65819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.252+0000 7fd65dc00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd658114f60 con 0x7fd65806a760 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.252+0000 7fd65dc00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd658114de0 con 0x7fd65806b080 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.252+0000 7fd65dc00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581150e0 con 0x7fd65810f500 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.252+0000 7fd656a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd65806a760 0x7fd65819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.252+0000 7fd656a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd65806a760 0x7fd65819b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38458/0 (socket says 172.21.15.22:38458) 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.252+0000 7fd656a00700 1 -- 172.21.15.22:0/1524981877 learned_addr learned my addr 172.21.15.22:0/1524981877 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.252+0000 7fd656000700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd65806b080 0x7fd65819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.253+0000 7fd657400700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd65810f500 0x7fd65819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:10.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.253+0000 7fd656a00700 1 -- 172.21.15.22:0/1524981877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd65810f500 msgr2=0x7fd65819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.253+0000 7fd656a00700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd65810f500 0x7fd65819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.253+0000 7fd656a00700 1 -- 172.21.15.22:0/1524981877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd65806b080 msgr2=0x7fd65819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.253+0000 7fd656a00700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd65806b080 0x7fd65819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.253+0000 7fd656a00700 1 -- 172.21.15.22:0/1524981877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6581a04b0 con 0x7fd65806a760 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.253+0000 7fd656000700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd65806b080 0x7fd65819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.253+0000 7fd657400700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd65810f500 0x7fd65819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.253+0000 7fd656a00700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd65806a760 0x7fd65819b4f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd64800bae0 tx=0x7fd64800bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.253+0000 7fd64f400700 1 -- 172.21.15.22:0/1524981877 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648026070 con 0x7fd65806a760 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.253+0000 7fd65dc00700 1 -- 172.21.15.22:0/1524981877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6581a0740 con 0x7fd65806a760 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.254+0000 7fd64f400700 1 -- 172.21.15.22:0/1524981877 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd648003d70 con 0x7fd65806a760 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.254+0000 7fd64f400700 1 -- 172.21.15.22:0/1524981877 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648021410 con 0x7fd65806a760 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.254+0000 7fd65dc00700 1 -- 172.21.15.22:0/1524981877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6581a8020 con 0x7fd65806a760 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.255+0000 7fd65dc00700 1 -- 172.21.15.22:0/1524981877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd658063160 con 0x7fd65806a760 2024-04-20T21:32:10.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.257+0000 7fd64f400700 1 -- 172.21.15.22:0/1524981877 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd648029070 con 0x7fd65806a760 2024-04-20T21:32:10.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.258+0000 7fd64f400700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd63c0a3cd0 0x7fd63c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:10.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.258+0000 7fd64f400700 1 -- 172.21.15.22:0/1524981877 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd6480d1df0 con 0x7fd65806a760 2024-04-20T21:32:10.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.258+0000 7fd656000700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd63c0a3cd0 0x7fd63c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:10.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.259+0000 7fd64f400700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324] conn(0x7fd63c0ad4b0 0x7fd63c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:10.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.259+0000 7fd64f400700 1 -- 172.21.15.22:0/1524981877 --> [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd648004e70 con 0x7fd63c0ad4b0 2024-04-20T21:32:10.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.259+0000 7fd64f400700 1 -- 172.21.15.22:0/1524981877 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd6480d21a0 con 0x7fd65806a760 2024-04-20T21:32:10.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.259+0000 7fd657400700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324] conn(0x7fd63c0ad4b0 0x7fd63c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.259+0000 7fd656000700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd63c0a3cd0 0x7fd63c0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd658102f10 tx=0x7fd644009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.260+0000 7fd657400700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324] conn(0x7fd63c0ad4b0 0x7fd63c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.261+0000 7fd64f400700 1 -- 172.21.15.22:0/1524981877 <== osd.0 v2:172.21.15.22:6802/2391966324 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd648004e70 con 0x7fd63c0ad4b0 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.280+0000 7fd65dc00700 1 -- 172.21.15.22:0/1524981877 --> [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd658063560 con 0x7fd63c0ad4b0 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.280+0000 7fd64f400700 1 -- 172.21.15.22:0/1524981877 <== osd.0 v2:172.21.15.22:6802/2391966324 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd658063560 con 0x7fd63c0ad4b0 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.281+0000 7fd65dc00700 1 -- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324] conn(0x7fd63c0ad4b0 msgr2=0x7fd63c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.281+0000 7fd65dc00700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324] conn(0x7fd63c0ad4b0 0x7fd63c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.281+0000 7fd65dc00700 1 -- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd63c0a3cd0 msgr2=0x7fd63c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.281+0000 7fd65dc00700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd63c0a3cd0 0x7fd63c0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd658102f10 tx=0x7fd644009260 comp rx=0 tx=0).stop 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.281+0000 7fd65dc00700 1 -- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd65806a760 msgr2=0x7fd65819b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.281+0000 7fd65dc00700 1 --2- 172.21.15.22:0/1524981877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd65806a760 0x7fd65819b4f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd64800bae0 tx=0x7fd64800bb10 comp rx=0 tx=0).stop 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.281+0000 7fd657400700 1 -- 172.21.15.22:0/1524981877 reap_dead start 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.281+0000 7fd65dc00700 1 -- 172.21.15.22:0/1524981877 shutdown_connections 2024-04-20T21:32:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.281+0000 7fd65dc00700 1 -- 172.21.15.22:0/1524981877 >> 172.21.15.22:0/1524981877 conn(0x7fd658078900 msgr2=0x7fd6581109f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:10.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.281+0000 7fd65dc00700 1 -- 172.21.15.22:0/1524981877 shutdown_connections 2024-04-20T21:32:10.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.281+0000 7fd65dc00700 1 -- 172.21.15.22:0/1524981877 wait complete. 2024-04-20T21:32:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.159+0000 7ff1bc000700 1 -- 172.21.15.22:0/2601041989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b4109fb0 msgr2=0x7ff1b4111b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.159+0000 7ff1bc000700 1 --2- 172.21.15.22:0/2601041989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b4109fb0 0x7ff1b4111b80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009f90 tx=0x7ff1ac00b330 comp rx=0 tx=0).stop 2024-04-20T21:32:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.159+0000 7ff1bc000700 1 -- 172.21.15.22:0/2601041989 shutdown_connections 2024-04-20T21:32:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.159+0000 7ff1bc000700 1 --2- 172.21.15.22:0/2601041989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b4109fb0 0x7ff1b4111b80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.159+0000 7ff1bc000700 1 --2- 172.21.15.22:0/2601041989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b4104ed0 0x7ff1b41052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.159+0000 7ff1bc000700 1 --2- 172.21.15.22:0/2601041989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1b4105880 0x7ff1b41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.159+0000 7ff1bc000700 1 -- 172.21.15.22:0/2601041989 >> 172.21.15.22:0/2601041989 conn(0x7ff1b4100760 msgr2=0x7ff1b4102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.160+0000 7ff1bc000700 1 -- 172.21.15.22:0/2601041989 shutdown_connections 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.160+0000 7ff1bc000700 1 -- 172.21.15.22:0/2601041989 wait complete. 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.161+0000 7ff1bc000700 1 Processor -- start 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.161+0000 7ff1bc000700 1 -- start start 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.161+0000 7ff1bc000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b4104ed0 0x7ff1b419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.161+0000 7ff1bc000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b4105880 0x7ff1b419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.161+0000 7ff1bc000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1b4109fb0 0x7ff1b419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.161+0000 7ff1bc000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4114fe0 con 0x7ff1b4104ed0 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.161+0000 7ff1bc000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4114e60 con 0x7ff1b4105880 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.161+0000 7ff1bc000700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4115160 con 0x7ff1b4109fb0 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.161+0000 7ff1b9600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b4104ed0 0x7ff1b419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.161+0000 7ff1b9600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b4104ed0 0x7ff1b419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38508/0 (socket says 172.21.15.22:38508) 2024-04-20T21:32:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.161+0000 7ff1b9600700 1 -- 172.21.15.22:0/254978858 learned_addr learned my addr 172.21.15.22:0/254978858 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.162+0000 7ff1b8c00700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b4105880 0x7ff1b419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.162+0000 7ff1ba000700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1b4109fb0 0x7ff1b419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.162+0000 7ff1b9600700 1 -- 172.21.15.22:0/254978858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1b4109fb0 msgr2=0x7ff1b419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.162+0000 7ff1b9600700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1b4109fb0 0x7ff1b419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.162+0000 7ff1b9600700 1 -- 172.21.15.22:0/254978858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b4105880 msgr2=0x7ff1b419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.162+0000 7ff1b9600700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b4105880 0x7ff1b419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.162+0000 7ff1b9600700 1 -- 172.21.15.22:0/254978858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b41a0380 con 0x7ff1b4104ed0 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.162+0000 7ff1b8c00700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1b4105880 0x7ff1b419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.162+0000 7ff1ba000700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1b4109fb0 0x7ff1b419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.162+0000 7ff1b9600700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b4104ed0 0x7ff1b419b370 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800ec10 tx=0x7ff1a800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.163+0000 7ff1b2000700 1 -- 172.21.15.22:0/254978858 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a800cc90 con 0x7ff1b4104ed0 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.163+0000 7ff1b2000700 1 -- 172.21.15.22:0/254978858 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a800ce30 con 0x7ff1b4104ed0 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.163+0000 7ff1b2000700 1 -- 172.21.15.22:0/254978858 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8010690 con 0x7ff1b4104ed0 2024-04-20T21:32:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.163+0000 7ff1bc000700 1 -- 172.21.15.22:0/254978858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b41a0670 con 0x7ff1b4104ed0 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.163+0000 7ff1bc000700 1 -- 172.21.15.22:0/254978858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b41a7ef0 con 0x7ff1b4104ed0 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.164+0000 7ff1bc000700 1 -- 172.21.15.22:0/254978858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1b4101a80 con 0x7ff1b4104ed0 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.166+0000 7ff1b2000700 1 -- 172.21.15.22:0/254978858 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff1a8003680 con 0x7ff1b4104ed0 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.167+0000 7ff1b2000700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff19c0a3d50 0x7ff19c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.167+0000 7ff1b2000700 1 -- 172.21.15.22:0/254978858 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7ff1a80d0a30 con 0x7ff1b4104ed0 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.168+0000 7ff1b8c00700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff19c0a3d50 0x7ff19c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.168+0000 7ff1b2000700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982] conn(0x7ff19c0ad530 0x7ff19c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.168+0000 7ff1b2000700 1 -- 172.21.15.22:0/254978858 --> [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1a8011e50 con 0x7ff19c0ad530 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.168+0000 7ff1b2000700 1 -- 172.21.15.22:0/254978858 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7ff1a80d0de0 con 0x7ff1b4104ed0 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.168+0000 7ff1ba000700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982] conn(0x7ff19c0ad530 0x7ff19c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.168+0000 7ff1b8c00700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff19c0a3d50 0x7ff19c0a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff1b410b4b0 tx=0x7ff1a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.169+0000 7ff1ba000700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982] conn(0x7ff19c0ad530 0x7ff19c0af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.170+0000 7ff1b2000700 1 -- 172.21.15.22:0/254978858 <== osd.7 v2:172.21.15.33:6816/539722982 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1a8011e50 con 0x7ff19c0ad530 2024-04-20T21:32:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.189+0000 7ff1bc000700 1 -- 172.21.15.22:0/254978858 --> [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff1b41063b0 con 0x7ff19c0ad530 2024-04-20T21:32:11.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.189+0000 7ff1b2000700 1 -- 172.21.15.22:0/254978858 <== osd.7 v2:172.21.15.33:6816/539722982 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff1b41063b0 con 0x7ff19c0ad530 2024-04-20T21:32:11.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.190+0000 7ff1bc000700 1 -- 172.21.15.22:0/254978858 >> [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982] conn(0x7ff19c0ad530 msgr2=0x7ff19c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.190+0000 7ff1bc000700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982] conn(0x7ff19c0ad530 0x7ff19c0af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.190+0000 7ff1bc000700 1 -- 172.21.15.22:0/254978858 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff19c0a3d50 msgr2=0x7ff19c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.190+0000 7ff1bc000700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff19c0a3d50 0x7ff19c0a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff1b410b4b0 tx=0x7ff1a0009260 comp rx=0 tx=0).stop 2024-04-20T21:32:11.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.190+0000 7ff1bc000700 1 -- 172.21.15.22:0/254978858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b4104ed0 msgr2=0x7ff1b419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.190+0000 7ff1bc000700 1 --2- 172.21.15.22:0/254978858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b4104ed0 0x7ff1b419b370 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800ec10 tx=0x7ff1a800ef20 comp rx=0 tx=0).stop 2024-04-20T21:32:11.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.190+0000 7ff1ba000700 1 -- 172.21.15.22:0/254978858 reap_dead start 2024-04-20T21:32:11.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.190+0000 7ff1bc000700 1 -- 172.21.15.22:0/254978858 shutdown_connections 2024-04-20T21:32:11.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.190+0000 7ff1bc000700 1 -- 172.21.15.22:0/254978858 >> 172.21.15.22:0/254978858 conn(0x7ff1b4100760 msgr2=0x7ff1b41010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:11.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.190+0000 7ff1bc000700 1 -- 172.21.15.22:0/254978858 shutdown_connections 2024-04-20T21:32:11.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.190+0000 7ff1bc000700 1 -- 172.21.15.22:0/254978858 wait complete. 2024-04-20T21:32:11.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:11 smithi122 ceph-mon[22147]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:11.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:11 smithi033 ceph-mon[21626]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:11.629 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:11 smithi022 ceph-mon[16843]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:11.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.676+0000 7f1e6b200700 1 -- 172.21.15.22:0/4040893441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e64102aa0 msgr2=0x7f1e64102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.676+0000 7f1e6b200700 1 --2- 172.21.15.22:0/4040893441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e64102aa0 0x7f1e64102f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009f90 tx=0x7f1e4c00b330 comp rx=0 tx=0).stop 2024-04-20T21:32:11.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.676+0000 7f1e6b200700 1 -- 172.21.15.22:0/4040893441 shutdown_connections 2024-04-20T21:32:11.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.676+0000 7f1e6b200700 1 --2- 172.21.15.22:0/4040893441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e64103460 0x7f1e6410f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.676+0000 7f1e6b200700 1 --2- 172.21.15.22:0/4040893441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e64108a60 0x7f1e64108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.676+0000 7f1e6b200700 1 --2- 172.21.15.22:0/4040893441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e64102aa0 0x7f1e64102f20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.676+0000 7f1e6b200700 1 -- 172.21.15.22:0/4040893441 >> 172.21.15.22:0/4040893441 conn(0x7f1e64079bb0 msgr2=0x7f1e64079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.676+0000 7f1e6b200700 1 -- 172.21.15.22:0/4040893441 shutdown_connections 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.676+0000 7f1e6b200700 1 -- 172.21.15.22:0/4040893441 wait complete. 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.678+0000 7f1e6b200700 1 Processor -- start 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.678+0000 7f1e6b200700 1 -- start start 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.678+0000 7f1e6b200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e64102aa0 0x7f1e64199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.678+0000 7f1e6b200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e64103460 0x7f1e641997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.678+0000 7f1e6b200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e64108a60 0x7f1e6419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.678+0000 7f1e6b200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1e64112db0 con 0x7f1e64108a60 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.678+0000 7f1e6b200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e64112c30 con 0x7f1e64103460 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.678+0000 7f1e6b200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e64112f30 con 0x7f1e64102aa0 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.679+0000 7f1e63e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e64102aa0 0x7f1e64199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.679+0000 7f1e63e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e64102aa0 0x7f1e64199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:58898/0 (socket says 172.21.15.22:58898) 2024-04-20T21:32:11.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.679+0000 7f1e63e00700 1 -- 172.21.15.22:0/3001664411 learned_addr learned my addr 172.21.15.22:0/3001664411 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.679+0000 7f1e5b400700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e64103460 0x7f1e641997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.679+0000 7f1e69200700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e64108a60 0x7f1e6419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.679+0000 7f1e5b400700 1 -- 172.21.15.22:0/3001664411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e64102aa0 msgr2=0x7f1e64199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.679+0000 7f1e5b400700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e64102aa0 0x7f1e64199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.679+0000 7f1e5b400700 1 -- 172.21.15.22:0/3001664411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e64108a60 msgr2=0x7f1e6419db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.679+0000 7f1e5b400700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e64108a60 0x7f1e6419db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.679+0000 7f1e5b400700 1 -- 172.21.15.22:0/3001664411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6419e270 con 0x7f1e64103460 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.680+0000 7f1e5b400700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e64103460 0x7f1e641997a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009f90 tx=0x7f1e4c004f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.680+0000 7f1e61600700 1 -- 172.21.15.22:0/3001664411 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c021070 con 0x7f1e64103460 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.680+0000 7f1e61600700 1 -- 172.21.15.22:0/3001664411 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e4c007ac0 con 0x7f1e64103460 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.680+0000 7f1e6b200700 1 -- 172.21.15.22:0/3001664411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6419e500 con 0x7f1e64103460 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.680+0000 7f1e61600700 1 -- 172.21.15.22:0/3001664411 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c0052a0 con 0x7f1e64103460 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.680+0000 7f1e6b200700 1 -- 172.21.15.22:0/3001664411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e641a5de0 con 0x7f1e64103460 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.681+0000 7f1e6b200700 1 -- 172.21.15.22:0/3001664411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e640631e0 con 0x7f1e64103460 2024-04-20T21:32:11.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.683+0000 7f1e61600700 1 -- 172.21.15.22:0/3001664411 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1e4c024070 con 0x7f1e64103460 2024-04-20T21:32:11.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.685+0000 7f1e61600700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f1e480a3cd0 0x7f1e480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:11.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.685+0000 7f1e61600700 1 -- 172.21.15.22:0/3001664411 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f1e4c00f020 con 0x7f1e64103460 2024-04-20T21:32:11.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.685+0000 7f1e63e00700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f1e480a3cd0 0x7f1e480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:11.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.685+0000 7f1e61600700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441] conn(0x7f1e480ad4b0 0x7f1e480af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:11.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.685+0000 7f1e61600700 1 -- 172.21.15.22:0/3001664411 --> [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e4c01ae50 con 0x7f1e480ad4b0 2024-04-20T21:32:11.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.685+0000 7f1e61600700 1 -- 172.21.15.22:0/3001664411 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f1e4c0d1ef0 con 0x7f1e64103460 2024-04-20T21:32:11.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.686+0000 7f1e63e00700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f1e480a3cd0 0x7f1e480a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009ca0 tx=0x7f1e54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:11.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.686+0000 7f1e69200700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441] conn(0x7f1e480ad4b0 0x7f1e480af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:11.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.686+0000 7f1e69200700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441] conn(0x7f1e480ad4b0 0x7f1e480af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:11.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.687+0000 7f1e61600700 1 -- 172.21.15.22:0/3001664411 <== osd.1 v2:172.21.15.33:6800/2822429441 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e4c01ae50 con 0x7f1e480ad4b0 2024-04-20T21:32:11.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.705+0000 7f1e6b200700 1 -- 172.21.15.22:0/3001664411 --> [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1e64063660 con 0x7f1e480ad4b0 2024-04-20T21:32:11.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.706+0000 7f1e61600700 1 -- 172.21.15.22:0/3001664411 <== osd.1 v2:172.21.15.33:6800/2822429441 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1e64063660 con 0x7f1e480ad4b0 2024-04-20T21:32:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.707+0000 7f1e6b200700 1 -- 172.21.15.22:0/3001664411 >> [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441] conn(0x7f1e480ad4b0 msgr2=0x7f1e480af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.707+0000 7f1e6b200700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441] conn(0x7f1e480ad4b0 0x7f1e480af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.707+0000 7f1e6b200700 1 -- 172.21.15.22:0/3001664411 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f1e480a3cd0 msgr2=0x7f1e480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.707+0000 7f1e6b200700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f1e480a3cd0 0x7f1e480a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009ca0 tx=0x7f1e54009260 comp rx=0 tx=0).stop 2024-04-20T21:32:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.707+0000 7f1e6b200700 1 -- 172.21.15.22:0/3001664411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e64103460 msgr2=0x7f1e641997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.707+0000 7f1e6b200700 1 --2- 172.21.15.22:0/3001664411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e64103460 0x7f1e641997a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009f90 tx=0x7f1e4c004f20 comp rx=0 tx=0).stop 2024-04-20T21:32:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.707+0000 7f1e69200700 1 -- 172.21.15.22:0/3001664411 reap_dead start 2024-04-20T21:32:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.707+0000 7f1e6b200700 1 -- 172.21.15.22:0/3001664411 shutdown_connections 2024-04-20T21:32:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.707+0000 7f1e6b200700 1 -- 172.21.15.22:0/3001664411 >> 172.21.15.22:0/3001664411 conn(0x7f1e64079bb0 msgr2=0x7f1e64101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.707+0000 7f1e6b200700 1 -- 172.21.15.22:0/3001664411 shutdown_connections 2024-04-20T21:32:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:10.707+0000 7f1e6b200700 1 -- 172.21.15.22:0/3001664411 wait complete. 2024-04-20T21:32:12.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.809+0000 7f09c9800700 1 -- 172.21.15.22:0/3225257460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09c4103330 msgr2=0x7f09c410f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:12.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.809+0000 7f09c9800700 1 --2- 172.21.15.22:0/3225257460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09c4103330 0x7f09c410f840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009f90 tx=0x7f09bc00b330 comp rx=0 tx=0).stop 2024-04-20T21:32:12.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.809+0000 7f09c9800700 1 -- 172.21.15.22:0/3225257460 shutdown_connections 2024-04-20T21:32:12.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.809+0000 7f09c9800700 1 --2- 172.21.15.22:0/3225257460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09c4103330 0x7f09c410f840 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.809+0000 7f09c9800700 1 --2- 172.21.15.22:0/3225257460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09c4108970 0x7f09c4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.809+0000 7f09c9800700 1 --2- 172.21.15.22:0/3225257460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09c4102970 0x7f09c4102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.809+0000 7f09c9800700 1 -- 172.21.15.22:0/3225257460 >> 172.21.15.22:0/3225257460 conn(0x7f09c40fe4a0 msgr2=0x7f09c41008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:12.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.809+0000 7f09c9800700 1 -- 172.21.15.22:0/3225257460 shutdown_connections 2024-04-20T21:32:12.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.809+0000 7f09c9800700 1 -- 172.21.15.22:0/3225257460 wait complete. 2024-04-20T21:32:12.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.810+0000 7f09c9800700 1 Processor -- start 2024-04-20T21:32:12.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.811+0000 7f09c9800700 1 -- start start 2024-04-20T21:32:12.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.811+0000 7f09c9800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09c4102970 0x7f09c41990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:12.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.811+0000 7f09c9800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09c4103330 0x7f09c4199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:12.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.811+0000 7f09c9800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09c4108970 0x7f09c419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:12.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.811+0000 7f09c9800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4112cd0 con 0x7f09c4103330 2024-04-20T21:32:12.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.811+0000 7f09c9800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4112b50 con 0x7f09c4102970 2024-04-20T21:32:12.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.811+0000 7f09c9800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4112e50 con 0x7f09c4108970 2024-04-20T21:32:12.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.811+0000 7f09c3400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09c4103330 0x7f09c4199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:12.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.811+0000 7f09c8e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09c4108970 0x7f09c419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:12.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.811+0000 7f09c3400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09c4103330 0x7f09c4199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38520/0 (socket says 172.21.15.22:38520) 2024-04-20T21:32:12.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.811+0000 7f09c3400700 1 -- 172.21.15.22:0/1913740062 learned_addr learned my addr 172.21.15.22:0/1913740062 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:12.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.811+0000 7f09c3e00700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09c4102970 0x7f09c41990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:12.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.812+0000 7f09c8e00700 1 -- 172.21.15.22:0/1913740062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09c4102970 msgr2=0x7f09c41990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:12.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.812+0000 7f09c8e00700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09c4102970 0x7f09c41990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.812+0000 7f09c8e00700 1 -- 172.21.15.22:0/1913740062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09c4103330 msgr2=0x7f09c4199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:12.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.812+0000 7f09c8e00700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09c4103330 0x7f09c4199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.812+0000 7f09c8e00700 1 -- 172.21.15.22:0/1913740062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c419e0d0 con 0x7f09c4108970 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.812+0000 7f09c3e00700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09c4102970 0x7f09c41990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.812+0000 7f09c8e00700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09c4108970 0x7f09c419d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09bc015000 tx=0x7f09bc004f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.813+0000 7f09c0c00700 1 -- 172.21.15.22:0/1913740062 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc021070 con 0x7f09c4108970 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.813+0000 7f09c9800700 1 -- 172.21.15.22:0/1913740062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c419e360 con 0x7f09c4108970 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.813+0000 7f09c0c00700 1 -- 172.21.15.22:0/1913740062 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09bc006020 con 0x7f09c4108970 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.813+0000 7f09c0c00700 1 -- 172.21.15.22:0/1913740062 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc005040 con 0x7f09c4108970 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.813+0000 7f09c9800700 1 -- 172.21.15.22:0/1913740062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c41a5c40 con 0x7f09c4108970 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.814+0000 7f09c9800700 1 -- 172.21.15.22:0/1913740062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09c40631a0 con 0x7f09c4108970 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.816+0000 7f09c0c00700 1 -- 172.21.15.22:0/1913740062 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f09bc024070 con 0x7f09c4108970 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.817+0000 7f09c0c00700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f09a80a3cd0 0x7f09a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.817+0000 7f09c0c00700 1 -- 172.21.15.22:0/1913740062 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f09bc00f020 con 0x7f09c4108970 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.818+0000 7f09c3e00700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f09a80a3cd0 0x7f09a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:12.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.818+0000 7f09c0c00700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286] conn(0x7f09a80ad4b0 0x7f09a80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:12.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.818+0000 7f09c0c00700 1 -- 172.21.15.22:0/1913740062 --> [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09bc00c850 con 0x7f09a80ad4b0 2024-04-20T21:32:12.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.818+0000 7f09c0c00700 1 -- 172.21.15.22:0/1913740062 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f09bc0d1dc0 con 0x7f09c4108970 2024-04-20T21:32:12.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.818+0000 7f09c3400700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286] conn(0x7f09a80ad4b0 0x7f09a80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:12.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.818+0000 7f09c3e00700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f09a80a3cd0 0x7f09a80a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005fd0 tx=0x7f09b4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:12.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.819+0000 7f09c3400700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286] conn(0x7f09a80ad4b0 0x7f09a80af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:12.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.820+0000 7f09c0c00700 1 -- 172.21.15.22:0/1913740062 <== osd.6 v2:172.21.15.22:6818/1631721286 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09bc00c850 con 0x7f09a80ad4b0 2024-04-20T21:32:12.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.838+0000 7f09c9800700 1 -- 172.21.15.22:0/1913740062 --> [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f09c40635d0 con 0x7f09a80ad4b0 2024-04-20T21:32:12.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.839+0000 7f09c0c00700 1 -- 172.21.15.22:0/1913740062 <== osd.6 v2:172.21.15.22:6818/1631721286 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f09c40635d0 con 0x7f09a80ad4b0 2024-04-20T21:32:12.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.840+0000 7f09c9800700 1 -- 172.21.15.22:0/1913740062 >> [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286] conn(0x7f09a80ad4b0 msgr2=0x7f09a80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:12.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.840+0000 7f09c9800700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286] conn(0x7f09a80ad4b0 0x7f09a80af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.840+0000 7f09c9800700 1 -- 172.21.15.22:0/1913740062 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f09a80a3cd0 msgr2=0x7f09a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:12.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.840+0000 7f09c9800700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f09a80a3cd0 0x7f09a80a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005fd0 tx=0x7f09b4009310 comp rx=0 tx=0).stop 2024-04-20T21:32:12.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.840+0000 7f09c9800700 1 -- 172.21.15.22:0/1913740062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09c4108970 msgr2=0x7f09c419d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:12.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.840+0000 7f09c9800700 1 --2- 172.21.15.22:0/1913740062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09c4108970 0x7f09c419d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09bc015000 tx=0x7f09bc004f00 comp rx=0 tx=0).stop 2024-04-20T21:32:12.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.840+0000 7f09c8e00700 1 -- 172.21.15.22:0/1913740062 reap_dead start 2024-04-20T21:32:12.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.840+0000 7f09c9800700 1 -- 172.21.15.22:0/1913740062 shutdown_connections 2024-04-20T21:32:12.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.840+0000 7f09c9800700 1 -- 172.21.15.22:0/1913740062 >> 172.21.15.22:0/1913740062 conn(0x7f09c40fe4a0 msgr2=0x7f09c40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:12.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.840+0000 7f09c9800700 1 -- 172.21.15.22:0/1913740062 shutdown_connections 2024-04-20T21:32:12.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:11.840+0000 7f09c9800700 1 -- 172.21.15.22:0/1913740062 wait complete. 2024-04-20T21:32:12.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.997+0000 7f0e2b200700 1 -- 172.21.15.22:0/1376445763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2410cf70 msgr2=0x7f0e2410f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:12.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.997+0000 7f0e2b200700 1 --2- 172.21.15.22:0/1376445763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2410cf70 0x7f0e2410f430 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000b0a0 tx=0x7f0e2000b3b0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.997+0000 7f0e2b200700 1 -- 172.21.15.22:0/1376445763 shutdown_connections 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.997+0000 7f0e2b200700 1 --2- 172.21.15.22:0/1376445763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2410cf70 0x7f0e2410f430 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.997+0000 7f0e2b200700 1 --2- 172.21.15.22:0/1376445763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e24103b40 0x7f0e24103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.997+0000 7f0e2b200700 1 --2- 172.21.15.22:0/1376445763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e24104460 0x7f0e2410c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.997+0000 7f0e2b200700 1 -- 172.21.15.22:0/1376445763 >> 172.21.15.22:0/1376445763 conn(0x7f0e240fd370 msgr2=0x7f0e240ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.997+0000 7f0e2b200700 1 -- 172.21.15.22:0/1376445763 shutdown_connections 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.997+0000 7f0e2b200700 1 -- 172.21.15.22:0/1376445763 wait complete. 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.998+0000 7f0e2b200700 1 Processor -- start 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.998+0000 7f0e2b200700 1 -- start start 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e2b200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e24103b40 0x7f0e2419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e2b200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e24104460 0x7f0e2419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e2b200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2410cf70 0x7f0e2419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e2b200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0e24112cc0 con 0x7f0e2410cf70 2024-04-20T21:32:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e2b200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e24112b40 con 0x7f0e24103b40 2024-04-20T21:32:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e2b200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e24112e40 con 0x7f0e24104460 2024-04-20T21:32:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e29e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e24103b40 0x7f0e2419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e29e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e24103b40 0x7f0e2419b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:42196/0 (socket says 172.21.15.22:42196) 2024-04-20T21:32:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e29e00700 1 -- 172.21.15.22:0/3703105141 learned_addr learned my addr 172.21.15.22:0/3703105141 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e29400700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e24104460 0x7f0e2419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e2a800700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2410cf70 0x7f0e2419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e29e00700 1 -- 172.21.15.22:0/3703105141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e24104460 msgr2=0x7f0e2419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e29e00700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e24104460 0x7f0e2419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e29e00700 1 -- 172.21.15.22:0/3703105141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2410cf70 msgr2=0x7f0e2419fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e29e00700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2410cf70 0x7f0e2419fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e29e00700 1 -- 172.21.15.22:0/3703105141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e241a02f0 con 0x7f0e24103b40 2024-04-20T21:32:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:04.999+0000 7f0e2a800700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2410cf70 0x7f0e2419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:32:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.000+0000 7f0e29e00700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e24103b40 0x7f0e2419b2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800e9a0 tx=0x7f0e1800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.000+0000 7f0e1ea00700 1 -- 172.21.15.22:0/3703105141 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1800cbd0 con 0x7f0e24103b40 2024-04-20T21:32:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.000+0000 7f0e2b200700 1 -- 172.21.15.22:0/3703105141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e241a05e0 con 0x7f0e24103b40 2024-04-20T21:32:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.000+0000 7f0e1ea00700 1 -- 172.21.15.22:0/3703105141 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e1800cd70 con 0x7f0e24103b40 2024-04-20T21:32:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.000+0000 7f0e1ea00700 1 -- 172.21.15.22:0/3703105141 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18018750 con 0x7f0e24103b40 2024-04-20T21:32:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.000+0000 7f0e2b200700 1 -- 172.21.15.22:0/3703105141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e241a7e60 con 0x7f0e24103b40 2024-04-20T21:32:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.001+0000 7f0e2b200700 1 -- 172.21.15.22:0/3703105141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e24103240 con 0x7f0e24103b40 2024-04-20T21:32:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.003+0000 7f0e1ea00700 1 -- 172.21.15.22:0/3703105141 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0e18010430 con 0x7f0e24103b40 2024-04-20T21:32:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.005+0000 7f0e1ea00700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f0e0c0a3d50 0x7f0e0c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.005+0000 7f0e1ea00700 1 -- 172.21.15.22:0/3703105141 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f0e180d0ed0 con 0x7f0e24103b40 2024-04-20T21:32:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.005+0000 7f0e29400700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f0e0c0a3d50 0x7f0e0c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.005+0000 7f0e1ea00700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470] conn(0x7f0e0c0ad530 0x7f0e0c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:12.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.005+0000 7f0e1ea00700 1 -- 172.21.15.22:0/3703105141 --> [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e18004500 con 0x7f0e0c0ad530 2024-04-20T21:32:12.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.005+0000 7f0e1ea00700 1 -- 172.21.15.22:0/3703105141 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0e180d1280 con 0x7f0e24103b40 2024-04-20T21:32:12.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.006+0000 7f0e2a800700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470] conn(0x7f0e0c0ad530 0x7f0e0c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:12.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.006+0000 7f0e29400700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f0e0c0a3d50 0x7f0e0c0a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0e2410e220 tx=0x7f0e14007320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:12.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.006+0000 7f0e2a800700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470] conn(0x7f0e0c0ad530 0x7f0e0c0af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:12.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.007+0000 7f0e1ea00700 1 -- 172.21.15.22:0/3703105141 <== osd.5 v2:172.21.15.122:6808/1738459470 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e18004500 con 0x7f0e0c0ad530 2024-04-20T21:32:12.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.025+0000 7f0e2b200700 1 -- 172.21.15.22:0/3703105141 --> [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0e2406b010 con 0x7f0e0c0ad530 2024-04-20T21:32:12.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.026+0000 7f0e1ea00700 1 -- 172.21.15.22:0/3703105141 <== osd.5 v2:172.21.15.122:6808/1738459470 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0e2406b010 con 0x7f0e0c0ad530 2024-04-20T21:32:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.026+0000 7f0e2b200700 1 -- 172.21.15.22:0/3703105141 >> [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470] conn(0x7f0e0c0ad530 msgr2=0x7f0e0c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.026+0000 7f0e2b200700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470] conn(0x7f0e0c0ad530 0x7f0e0c0af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.026+0000 7f0e2b200700 1 -- 172.21.15.22:0/3703105141 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f0e0c0a3d50 msgr2=0x7f0e0c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.026+0000 7f0e2b200700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f0e0c0a3d50 0x7f0e0c0a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0e2410e220 tx=0x7f0e14007320 comp rx=0 tx=0).stop 2024-04-20T21:32:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.026+0000 7f0e2b200700 1 -- 172.21.15.22:0/3703105141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e24103b40 msgr2=0x7f0e2419b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.026+0000 7f0e2b200700 1 --2- 172.21.15.22:0/3703105141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e24103b40 0x7f0e2419b2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800e9a0 tx=0x7f0e1800ecb0 comp rx=0 tx=0).stop 2024-04-20T21:32:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.027+0000 7f0e2a800700 1 -- 172.21.15.22:0/3703105141 reap_dead start 2024-04-20T21:32:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.027+0000 7f0e2b200700 1 -- 172.21.15.22:0/3703105141 shutdown_connections 2024-04-20T21:32:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.027+0000 7f0e2b200700 1 -- 172.21.15.22:0/3703105141 >> 172.21.15.22:0/3703105141 conn(0x7f0e240fd370 msgr2=0x7f0e2410f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.027+0000 7f0e2b200700 1 -- 172.21.15.22:0/3703105141 shutdown_connections 2024-04-20T21:32:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:05.027+0000 7f0e2b200700 1 -- 172.21.15.22:0/3703105141 wait complete. 2024-04-20T21:32:13.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:13 smithi022 ceph-mon[16843]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:13.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:13 smithi122 ceph-mon[22147]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:13.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:13 smithi033 ceph-mon[21626]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:14.854 INFO:teuthology.orchestra.run.smithi022.stdout:206158430260 2024-04-20T21:32:14.854 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-20T21:32:15.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:15 smithi022 ceph-mon[16843]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:15.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:15 smithi122 ceph-mon[22147]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:15.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:15 smithi033 ceph-mon[21626]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:16.180 INFO:teuthology.orchestra.run.smithi022.stdout:55834574928 2024-04-20T21:32:16.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-20T21:32:16.444 INFO:teuthology.orchestra.run.smithi022.stdout:171798691895 2024-04-20T21:32:16.445 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-20T21:32:16.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.921+0000 7f545ea00700 1 -- 172.21.15.22:0/1651137813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5458108a60 msgr2=0x7f5458108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:16.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.921+0000 7f545ea00700 1 --2- 172.21.15.22:0/1651137813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5458108a60 0x7f5458108e40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f54480098f0 tx=0x7f5448009c00 comp rx=0 tx=0).stop 2024-04-20T21:32:16.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.922+0000 7f545ea00700 1 -- 172.21.15.22:0/1651137813 shutdown_connections 2024-04-20T21:32:16.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.922+0000 7f545ea00700 1 --2- 172.21.15.22:0/1651137813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5458103460 0x7f545810f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:16.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.922+0000 7f545ea00700 1 --2- 172.21.15.22:0/1651137813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5458108a60 0x7f5458108e40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:16.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.922+0000 7f545ea00700 1 --2- 172.21.15.22:0/1651137813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5458102aa0 0x7f5458102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:16.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.922+0000 7f545ea00700 1 -- 172.21.15.22:0/1651137813 >> 172.21.15.22:0/1651137813 conn(0x7f5458079bb0 msgr2=0x7f5458079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:16.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.922+0000 7f545ea00700 1 -- 172.21.15.22:0/1651137813 shutdown_connections 2024-04-20T21:32:16.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.922+0000 7f545ea00700 1 -- 172.21.15.22:0/1651137813 wait complete. 2024-04-20T21:32:16.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.923+0000 7f545ea00700 1 Processor -- start 2024-04-20T21:32:16.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.923+0000 7f545ea00700 1 -- start start 2024-04-20T21:32:16.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.924+0000 7f545ea00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5458102aa0 0x7f5458074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:16.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.924+0000 7f545ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5458103460 0x7f5458075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:16.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.924+0000 7f545ea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5458108a60 0x7f54580757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:16.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.924+0000 7f545ea00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5458112dc0 con 0x7f5458108a60 2024-04-20T21:32:16.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.924+0000 7f545ea00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5458112c40 con 0x7f5458103460 2024-04-20T21:32:16.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.924+0000 7f545ea00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5458112f40 con 0x7f5458102aa0 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.924+0000 7f545ca00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5458108a60 0x7f54580757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.924+0000 7f5457e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5458102aa0 0x7f5458074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.924+0000 7f545ca00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5458108a60 0x7f54580757d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37260/0 (socket says 172.21.15.22:37260) 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.924+0000 7f545ca00700 1 -- 172.21.15.22:0/2635547217 learned_addr learned my addr 172.21.15.22:0/2635547217 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.924+0000 7f544fe00700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5458103460 0x7f5458075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.925+0000 7f5457e00700 1 -- 172.21.15.22:0/2635547217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5458103460 msgr2=0x7f5458075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.925+0000 7f5457e00700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5458103460 0x7f5458075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.925+0000 7f5457e00700 1 -- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5458108a60 msgr2=0x7f54580757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.925+0000 7f5457e00700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5458108a60 0x7f54580757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.925+0000 7f5457e00700 1 -- 172.21.15.22:0/2635547217 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54581a7860 con 0x7f5458102aa0 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.925+0000 7f545ca00700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5458108a60 0x7f54580757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.925+0000 7f5457e00700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5458102aa0 0x7f5458074d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5448000c00 tx=0x7f544800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:16.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.925+0000 7f5455600700 1 -- 172.21.15.22:0/2635547217 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448005140 con 0x7f5458102aa0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.925+0000 7f545ea00700 1 -- 172.21.15.22:0/2635547217 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54581a7af0 con 0x7f5458102aa0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.925+0000 7f5455600700 1 -- 172.21.15.22:0/2635547217 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f544800b560 con 0x7f5458102aa0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.925+0000 7f5455600700 1 -- 172.21.15.22:0/2635547217 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448003a20 con 0x7f5458102aa0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.926+0000 7f545ea00700 1 -- 172.21.15.22:0/2635547217 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54581a8020 con 0x7f5458102aa0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.926+0000 7f545ea00700 1 -- 172.21.15.22:0/2635547217 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54580630e0 con 0x7f5458102aa0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.929+0000 7f5455600700 1 -- 172.21.15.22:0/2635547217 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f544800ba10 con 0x7f5458102aa0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.930+0000 7f5455600700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f543c0a3cd0 0x7f543c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.930+0000 7f5455600700 1 -- 172.21.15.22:0/2635547217 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f54480d16e0 con 0x7f5458102aa0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.930+0000 7f544fe00700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f543c0a3cd0 0x7f543c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.930+0000 7f5455600700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711] conn(0x7f543c0ad4b0 0x7f543c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.930+0000 7f5455600700 1 -- 172.21.15.22:0/2635547217 --> [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f544800bde0 con 0x7f543c0ad4b0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.930+0000 7f5455600700 1 -- 172.21.15.22:0/2635547217 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f54480d1a90 con 0x7f5458102aa0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.931+0000 7f545ca00700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711] conn(0x7f543c0ad4b0 0x7f543c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.931+0000 7f544fe00700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f543c0a3cd0 0x7f543c0a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f545807a2d0 tx=0x7f5440009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:16.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.931+0000 7f545ca00700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711] conn(0x7f543c0ad4b0 0x7f543c0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:16.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.932+0000 7f5455600700 1 -- 172.21.15.22:0/2635547217 <== osd.9 v2:172.21.15.22:6826/1246646711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f544800bde0 con 0x7f543c0ad4b0 2024-04-20T21:32:16.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.951+0000 7f545ea00700 1 -- 172.21.15.22:0/2635547217 --> [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5458063510 con 0x7f543c0ad4b0 2024-04-20T21:32:16.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.952+0000 7f5455600700 1 -- 172.21.15.22:0/2635547217 <== osd.9 v2:172.21.15.22:6826/1246646711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5458063510 con 0x7f543c0ad4b0 2024-04-20T21:32:16.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.952+0000 7f545ea00700 1 -- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711] conn(0x7f543c0ad4b0 msgr2=0x7f543c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:16.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.952+0000 7f545ea00700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711] conn(0x7f543c0ad4b0 0x7f543c0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:16.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.952+0000 7f545ea00700 1 -- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f543c0a3cd0 msgr2=0x7f543c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:16.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.952+0000 7f545ea00700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f543c0a3cd0 0x7f543c0a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f545807a2d0 tx=0x7f5440009310 comp rx=0 tx=0).stop 2024-04-20T21:32:16.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.952+0000 7f545ea00700 1 -- 172.21.15.22:0/2635547217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5458102aa0 msgr2=0x7f5458074d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:16.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.952+0000 7f545ea00700 1 --2- 172.21.15.22:0/2635547217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5458102aa0 0x7f5458074d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5448000c00 tx=0x7f544800fa00 comp rx=0 tx=0).stop 2024-04-20T21:32:16.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.952+0000 7f545ca00700 1 -- 172.21.15.22:0/2635547217 reap_dead start 2024-04-20T21:32:16.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.953+0000 7f545ea00700 1 -- 172.21.15.22:0/2635547217 shutdown_connections 2024-04-20T21:32:16.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.953+0000 7f545ea00700 1 -- 172.21.15.22:0/2635547217 >> 172.21.15.22:0/2635547217 conn(0x7f5458079bb0 msgr2=0x7f54581018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:16.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.953+0000 7f545ea00700 1 -- 172.21.15.22:0/2635547217 shutdown_connections 2024-04-20T21:32:16.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:15.953+0000 7f545ea00700 1 -- 172.21.15.22:0/2635547217 wait complete. 2024-04-20T21:32:17.122 INFO:teuthology.orchestra.run.smithi022.stdout:146028888126 2024-04-20T21:32:17.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-20T21:32:17.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:17 smithi022 ceph-mon[16843]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:17.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:17 smithi122 ceph-mon[22147]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:17.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:17 smithi033 ceph-mon[21626]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:17.566 INFO:teuthology.orchestra.run.smithi022.stdout:227633266732 2024-04-20T21:32:17.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-20T21:32:18.310 INFO:teuthology.orchestra.run.smithi022.stdout:77309411400 2024-04-20T21:32:18.310 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-20T21:32:18.964 INFO:teuthology.orchestra.run.smithi022.stdout:300647710754 2024-04-20T21:32:18.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-20T21:32:19.416 INFO:teuthology.orchestra.run.smithi022.stdout:34359738452 2024-04-20T21:32:19.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-20T21:32:19.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:19 smithi022 ceph-mon[16843]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:19.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:19 smithi122 ceph-mon[22147]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:19.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:19 smithi033 ceph-mon[21626]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:19.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.645+0000 7faef4a00700 1 -- 172.21.15.22:0/2520918467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef0108a30 msgr2=0x7faef0108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:19.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.645+0000 7faef4a00700 1 --2- 172.21.15.22:0/2520918467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef0108a30 0x7faef0108e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faee0009990 tx=0x7faee0009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.645+0000 7faef4a00700 1 -- 172.21.15.22:0/2520918467 shutdown_connections 2024-04-20T21:32:19.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.645+0000 7faef4a00700 1 --2- 172.21.15.22:0/2520918467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faef006b990 0x7faef010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.645+0000 7faef4a00700 1 --2- 172.21.15.22:0/2520918467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef0108a30 0x7faef0108e10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.645+0000 7faef4a00700 1 --2- 172.21.15.22:0/2520918467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faef006aff0 0x7faef006b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.645+0000 7faef4a00700 1 -- 172.21.15.22:0/2520918467 >> 172.21.15.22:0/2520918467 conn(0x7faef00788c0 msgr2=0x7faef0078cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:19.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.645+0000 7faef4a00700 1 -- 172.21.15.22:0/2520918467 shutdown_connections 2024-04-20T21:32:19.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.645+0000 7faef4a00700 1 -- 172.21.15.22:0/2520918467 wait complete. 2024-04-20T21:32:19.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.647+0000 7faef4a00700 1 Processor -- start 2024-04-20T21:32:19.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.647+0000 7faef4a00700 1 -- start start 2024-04-20T21:32:19.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.647+0000 7faef4a00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faef006aff0 0x7faef019b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:19.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.647+0000 7faef4a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faef006b990 0x7faef019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:19.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.647+0000 7faef4a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef0108a30 0x7faef019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:19.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.647+0000 7faef4a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faef0112d50 con 0x7faef0108a30 2024-04-20T21:32:19.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.647+0000 7faef4a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faef0112bd0 con 0x7faef006b990 2024-04-20T21:32:19.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.647+0000 7faef4a00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faef0112ed0 con 0x7faef006aff0 2024-04-20T21:32:19.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.648+0000 7faeede00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faef006aff0 0x7faef019b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:19.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.648+0000 7faeede00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faef006aff0 0x7faef019b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:47100/0 (socket says 172.21.15.22:47100) 2024-04-20T21:32:19.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.648+0000 7faeede00700 1 -- 172.21.15.22:0/4094112571 learned_addr learned my addr 172.21.15.22:0/4094112571 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:19.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.648+0000 7faeee800700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef0108a30 0x7faef019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:19.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.648+0000 7faeed400700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faef006b990 0x7faef019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:19.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.648+0000 7faeede00700 1 -- 172.21.15.22:0/4094112571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faef006b990 msgr2=0x7faef019ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:19.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.648+0000 7faeede00700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faef006b990 0x7faef019ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.648+0000 7faeede00700 1 -- 172.21.15.22:0/4094112571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef0108a30 msgr2=0x7faef019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:19.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.648+0000 7faeede00700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faef0108a30 0x7faef019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.648+0000 7faeede00700 1 -- 172.21.15.22:0/4094112571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef01a04a0 con 0x7faef006aff0 2024-04-20T21:32:19.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.648+0000 7faeed400700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faef006b990 0x7faef019ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:32:19.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.648+0000 7faeede00700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faef006aff0 0x7faef019b4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faee0000c00 tx=0x7faee000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:19.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.649+0000 7faee6a00700 1 -- 172.21.15.22:0/4094112571 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0027070 con 0x7faef006aff0 2024-04-20T21:32:19.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.649+0000 7faee6a00700 1 -- 172.21.15.22:0/4094112571 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee0003d70 con 0x7faef006aff0 2024-04-20T21:32:19.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.649+0000 7faee6a00700 1 -- 172.21.15.22:0/4094112571 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee000f460 con 0x7faef006aff0 2024-04-20T21:32:19.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.649+0000 7faef4a00700 1 -- 172.21.15.22:0/4094112571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef01a0730 con 0x7faef006aff0 2024-04-20T21:32:19.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.649+0000 7faef4a00700 1 -- 172.21.15.22:0/4094112571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef01a8010 con 0x7faef006aff0 2024-04-20T21:32:19.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.650+0000 7faef4a00700 1 -- 172.21.15.22:0/4094112571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faef0063160 con 0x7faef006aff0 2024-04-20T21:32:19.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.652+0000 7faee6a00700 1 -- 172.21.15.22:0/4094112571 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faee002a070 con 0x7faef006aff0 2024-04-20T21:32:19.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.653+0000 7faee6a00700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7faed40a3cd0 0x7faed40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:19.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.654+0000 7faee6a00700 1 -- 172.21.15.22:0/4094112571 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7faee00d1ab0 con 0x7faef006aff0 2024-04-20T21:32:19.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.654+0000 7faeed400700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7faed40a3cd0 0x7faed40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:19.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.654+0000 7faee6a00700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244] conn(0x7faed40ad4b0 0x7faed40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:19.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.654+0000 7faee6a00700 1 -- 172.21.15.22:0/4094112571 --> [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faee0003bc0 con 0x7faed40ad4b0 2024-04-20T21:32:19.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.654+0000 7faee6a00700 1 -- 172.21.15.22:0/4094112571 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7faee00d1e60 con 0x7faef006aff0 2024-04-20T21:32:19.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.655+0000 7faeee800700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244] conn(0x7faed40ad4b0 0x7faed40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:19.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.655+0000 7faeed400700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7faed40a3cd0 0x7faed40a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faef0108050 tx=0x7faedc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:19.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.655+0000 7faeee800700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244] conn(0x7faed40ad4b0 0x7faed40af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:19.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.656+0000 7faee6a00700 1 -- 172.21.15.22:0/4094112571 <== osd.10 v2:172.21.15.33:6824/1408592244 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faee0003bc0 con 0x7faed40ad4b0 2024-04-20T21:32:19.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.675+0000 7faef4a00700 1 -- 172.21.15.22:0/4094112571 --> [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faef00635e0 con 0x7faed40ad4b0 2024-04-20T21:32:19.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.676+0000 7faee6a00700 1 -- 172.21.15.22:0/4094112571 <== osd.10 v2:172.21.15.33:6824/1408592244 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faef00635e0 con 0x7faed40ad4b0 2024-04-20T21:32:19.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.676+0000 7faef4a00700 1 -- 172.21.15.22:0/4094112571 >> [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244] conn(0x7faed40ad4b0 msgr2=0x7faed40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:19.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.676+0000 7faef4a00700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244] conn(0x7faed40ad4b0 0x7faed40af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.676+0000 7faef4a00700 1 -- 172.21.15.22:0/4094112571 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7faed40a3cd0 msgr2=0x7faed40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:19.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.676+0000 7faef4a00700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7faed40a3cd0 0x7faed40a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faef0108050 tx=0x7faedc009260 comp rx=0 tx=0).stop 2024-04-20T21:32:19.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.676+0000 7faef4a00700 1 -- 172.21.15.22:0/4094112571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faef006aff0 msgr2=0x7faef019b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:19.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.676+0000 7faef4a00700 1 --2- 172.21.15.22:0/4094112571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faef006aff0 0x7faef019b4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faee0000c00 tx=0x7faee000bab0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.676+0000 7faeee800700 1 -- 172.21.15.22:0/4094112571 reap_dead start 2024-04-20T21:32:19.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.677+0000 7faef4a00700 1 -- 172.21.15.22:0/4094112571 shutdown_connections 2024-04-20T21:32:19.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.677+0000 7faef4a00700 1 -- 172.21.15.22:0/4094112571 >> 172.21.15.22:0/4094112571 conn(0x7faef00788c0 msgr2=0x7faef010de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:19.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.677+0000 7faef4a00700 1 -- 172.21.15.22:0/4094112571 shutdown_connections 2024-04-20T21:32:19.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:19.677+0000 7faef4a00700 1 -- 172.21.15.22:0/4094112571 wait complete. 2024-04-20T21:32:19.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.068+0000 7f8e13e00700 1 -- 172.21.15.22:0/1380374800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e0c104ea0 msgr2=0x7f8e0c105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:19.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.068+0000 7f8e13e00700 1 --2- 172.21.15.22:0/1380374800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e0c104ea0 0x7f8e0c105280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8e000099c0 tx=0x7f8e00009cd0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.068+0000 7f8e13e00700 1 -- 172.21.15.22:0/1380374800 shutdown_connections 2024-04-20T21:32:19.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.068+0000 7f8e13e00700 1 --2- 172.21.15.22:0/1380374800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c109f80 0x7f8e0c111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.068+0000 7f8e13e00700 1 --2- 172.21.15.22:0/1380374800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e0c104ea0 0x7f8e0c105280 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.068+0000 7f8e13e00700 1 --2- 172.21.15.22:0/1380374800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c105850 0x7f8e0c1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.068+0000 7f8e13e00700 1 -- 172.21.15.22:0/1380374800 >> 172.21.15.22:0/1380374800 conn(0x7f8e0c100730 msgr2=0x7f8e0c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.069+0000 7f8e13e00700 1 -- 172.21.15.22:0/1380374800 shutdown_connections 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.069+0000 7f8e13e00700 1 -- 172.21.15.22:0/1380374800 wait complete. 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.070+0000 7f8e13e00700 1 Processor -- start 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.070+0000 7f8e13e00700 1 -- start start 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.071+0000 7f8e13e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e0c104ea0 0x7f8e0c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.071+0000 7f8e13e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c105850 0x7f8e0c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.071+0000 7f8e13e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c109f80 0x7f8e0c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.071+0000 7f8e13e00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c114f80 con 0x7f8e0c104ea0 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.071+0000 7f8e13e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c114e00 con 0x7f8e0c109f80 2024-04-20T21:32:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.071+0000 7f8e13e00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c115100 con 0x7f8e0c105850 2024-04-20T21:32:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.071+0000 7f8e11400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e0c104ea0 0x7f8e0c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.071+0000 7f8e11400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e0c104ea0 0x7f8e0c19b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37274/0 (socket says 172.21.15.22:37274) 2024-04-20T21:32:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.071+0000 7f8e11400700 1 -- 172.21.15.22:0/460994518 learned_addr learned my addr 172.21.15.22:0/460994518 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.071+0000 7f8e10a00700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c105850 0x7f8e0c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.071+0000 7f8e11e00700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c109f80 0x7f8e0c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.072+0000 7f8e11400700 1 -- 172.21.15.22:0/460994518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c105850 msgr2=0x7f8e0c19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.072+0000 7f8e11400700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c105850 0x7f8e0c19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.072+0000 7f8e11400700 1 -- 172.21.15.22:0/460994518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c109f80 msgr2=0x7f8e0c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.072+0000 7f8e11400700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c109f80 0x7f8e0c19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.072+0000 7f8e11400700 1 -- 172.21.15.22:0/460994518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c1a0420 con 0x7f8e0c104ea0 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.072+0000 7f8e11400700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e0c104ea0 0x7f8e0c19b410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8e00006010 tx=0x7f8e0000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.072+0000 7f8e0a000700 1 -- 172.21.15.22:0/460994518 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00003680 con 0x7f8e0c104ea0 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.072+0000 7f8e0a000700 1 -- 172.21.15.22:0/460994518 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e000055b0 con 0x7f8e0c104ea0 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.072+0000 7f8e0a000700 1 -- 172.21.15.22:0/460994518 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00003cc0 con 0x7f8e0c104ea0 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.072+0000 7f8e13e00700 1 -- 172.21.15.22:0/460994518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c1a06b0 con 0x7f8e0c104ea0 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.073+0000 7f8e13e00700 1 -- 172.21.15.22:0/460994518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c1a7f90 con 0x7f8e0c104ea0 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.074+0000 7f8e13e00700 1 -- 172.21.15.22:0/460994518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e0c1a0b40 con 0x7f8e0c104ea0 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.076+0000 7f8e0a000700 1 -- 172.21.15.22:0/460994518 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8e0000fb80 con 0x7f8e0c104ea0 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.077+0000 7f8e0a000700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8df40a3ef0 0x7f8df40a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.077+0000 7f8e0a000700 1 -- 172.21.15.22:0/460994518 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f8e000d1510 con 0x7f8e0c104ea0 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.077+0000 7f8e10a00700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8df40a3ef0 0x7f8df40a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.078+0000 7f8e0a000700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556] conn(0x7f8df40ad6d0 0x7f8df40afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.078+0000 7f8e0a000700 1 -- 172.21.15.22:0/460994518 --> [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e0002ce50 con 0x7f8df40ad6d0 2024-04-20T21:32:19.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.078+0000 7f8e0a000700 1 -- 172.21.15.22:0/460994518 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f8e000d18c0 con 0x7f8e0c104ea0 2024-04-20T21:32:19.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.078+0000 7f8e10a00700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8df40a3ef0 0x7f8df40a63b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c101bc0 tx=0x7f8df8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:19.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.078+0000 7f8e11e00700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556] conn(0x7f8df40ad6d0 0x7f8df40afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:19.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.079+0000 7f8e11e00700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556] conn(0x7f8df40ad6d0 0x7f8df40afaf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:19.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.080+0000 7f8e0a000700 1 -- 172.21.15.22:0/460994518 <== osd.8 v2:172.21.15.122:6816/2876209556 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e0002ce50 con 0x7f8df40ad6d0 2024-04-20T21:32:19.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.098+0000 7f8e13e00700 1 -- 172.21.15.22:0/460994518 --> [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8e0c0634c0 con 0x7f8df40ad6d0 2024-04-20T21:32:19.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.099+0000 7f8e0a000700 1 -- 172.21.15.22:0/460994518 <== osd.8 v2:172.21.15.122:6816/2876209556 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8e0c0634c0 con 0x7f8df40ad6d0 2024-04-20T21:32:19.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.099+0000 7f8e13e00700 1 -- 172.21.15.22:0/460994518 >> [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556] conn(0x7f8df40ad6d0 msgr2=0x7f8df40afaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:19.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.099+0000 7f8e13e00700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556] conn(0x7f8df40ad6d0 0x7f8df40afaf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:19.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.100+0000 7f8e13e00700 1 -- 172.21.15.22:0/460994518 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8df40a3ef0 msgr2=0x7f8df40a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:19.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.100+0000 7f8e13e00700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8df40a3ef0 0x7f8df40a63b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c101bc0 tx=0x7f8df8009260 comp rx=0 tx=0).stop 2024-04-20T21:32:19.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.100+0000 7f8e13e00700 1 -- 172.21.15.22:0/460994518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e0c104ea0 msgr2=0x7f8e0c19b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:19.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.100+0000 7f8e13e00700 1 --2- 172.21.15.22:0/460994518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e0c104ea0 0x7f8e0c19b410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8e00006010 tx=0x7f8e0000f690 comp rx=0 tx=0).stop 2024-04-20T21:32:19.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.100+0000 7f8e11e00700 1 -- 172.21.15.22:0/460994518 reap_dead start 2024-04-20T21:32:19.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.100+0000 7f8e13e00700 1 -- 172.21.15.22:0/460994518 shutdown_connections 2024-04-20T21:32:19.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.100+0000 7f8e13e00700 1 -- 172.21.15.22:0/460994518 >> 172.21.15.22:0/460994518 conn(0x7f8e0c100730 msgr2=0x7f8e0c1024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:19.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.100+0000 7f8e13e00700 1 -- 172.21.15.22:0/460994518 shutdown_connections 2024-04-20T21:32:19.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:18.100+0000 7f8e13e00700 1 -- 172.21.15.22:0/460994518 wait complete. 2024-04-20T21:32:20.459 INFO:teuthology.orchestra.run.smithi022.stdout:266287972394 2024-04-20T21:32:20.459 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-20T21:32:21.125 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:21.143 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:21.150 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:21.160 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:21.213 INFO:teuthology.orchestra.run.smithi022.stdout:322122547229 2024-04-20T21:32:21.214 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-20T21:32:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:21.327 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:21.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:21 smithi022 ceph-mon[16843]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:21.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:21 smithi122 ceph-mon[22147]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:21.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:21 smithi033 ceph-mon[21626]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:23.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:23 smithi022 ceph-mon[16843]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-20T21:32:23.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:23 smithi122 ceph-mon[22147]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-20T21:32:23.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:23 smithi033 ceph-mon[21626]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-20T21:32:23.665 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:25.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:25 smithi022 ceph-mon[16843]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:25.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:25 smithi122 ceph-mon[22147]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:25.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:25 smithi033 ceph-mon[21626]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:27.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:27 smithi122 ceph-mon[22147]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:27.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:27 smithi033 ceph-mon[21626]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:27.562 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:27 smithi022 ceph-mon[16843]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:28.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:28 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:32:28.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:28 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:32:28.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:28 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:32:29.299 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:29 smithi022 ceph-mon[16843]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:29.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:29 smithi122 ceph-mon[22147]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:29.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:29 smithi033 ceph-mon[21626]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:30.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:30.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:30.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:30.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-20T21:32:30.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-04-20T21:32:30.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:30.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-04-20T21:32:30.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-20T21:32:30.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-20T21:32:30.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-20T21:32:30.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-04-20T21:32:30.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-20T21:32:30.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-20T21:32:30.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-20T21:32:30.806 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:30.806 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:30 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2936218958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T21:32:30.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:30.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:30.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:30.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-20T21:32:30.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-04-20T21:32:30.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:30.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-04-20T21:32:30.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-20T21:32:30.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-20T21:32:30.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-20T21:32:30.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-04-20T21:32:30.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-20T21:32:30.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-20T21:32:30.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-20T21:32:30.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:30.908 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:30 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2936218958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T21:32:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-20T21:32:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-04-20T21:32:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-04-20T21:32:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-20T21:32:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-20T21:32:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-20T21:32:31.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-04-20T21:32:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-20T21:32:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-20T21:32:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-20T21:32:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:31.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:30 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2936218958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T21:32:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.716+0000 7f9be6a00700 1 -- 172.21.15.22:0/2523522049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be0108aa0 msgr2=0x7f9be0108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.716+0000 7f9be6a00700 1 --2- 172.21.15.22:0/2523522049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be0108aa0 0x7f9be0108e80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009990 tx=0x7f9bd0009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.716+0000 7f9be6a00700 1 -- 172.21.15.22:0/2523522049 shutdown_connections 2024-04-20T21:32:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.716+0000 7f9be6a00700 1 --2- 172.21.15.22:0/2523522049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be006bac0 0x7f9be010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.716+0000 7f9be6a00700 1 --2- 172.21.15.22:0/2523522049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be0108aa0 0x7f9be0108e80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.716+0000 7f9be6a00700 1 --2- 172.21.15.22:0/2523522049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be006b100 0x7f9be006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.716+0000 7f9be6a00700 1 -- 172.21.15.22:0/2523522049 >> 172.21.15.22:0/2523522049 conn(0x7f9be00786b0 msgr2=0x7f9be0078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.716+0000 7f9be6a00700 1 -- 172.21.15.22:0/2523522049 shutdown_connections 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.716+0000 7f9be6a00700 1 -- 172.21.15.22:0/2523522049 wait complete. 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.718+0000 7f9be6a00700 1 Processor -- start 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.718+0000 7f9be6a00700 1 -- start start 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.718+0000 7f9be6a00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be006b100 0x7f9be019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.718+0000 7f9be6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be006bac0 0x7f9be019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.718+0000 7f9be6a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be0108aa0 0x7f9be019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.718+0000 7f9be6a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9be0112dd0 con 0x7f9be006bac0 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.718+0000 7f9be6a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9be0112c50 con 0x7f9be0108aa0 2024-04-20T21:32:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.718+0000 7f9be6a00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9be0112f50 con 0x7f9be006b100 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.718+0000 7f9bdfe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be006b100 0x7f9be019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.719+0000 7f9bdfe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be006b100 0x7f9be019b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42540/0 (socket says 172.21.15.22:42540) 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.719+0000 7f9bdfe00700 1 -- 172.21.15.22:0/2936218958 learned_addr learned my addr 172.21.15.22:0/2936218958 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.719+0000 7f9bdf400700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be006bac0 0x7f9be019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.719+0000 7f9be4a00700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be0108aa0 0x7f9be019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.719+0000 7f9bdfe00700 1 -- 172.21.15.22:0/2936218958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be0108aa0 msgr2=0x7f9be019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.719+0000 7f9bdfe00700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be0108aa0 0x7f9be019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.719+0000 7f9bdfe00700 1 -- 172.21.15.22:0/2936218958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be006bac0 msgr2=0x7f9be019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.719+0000 7f9bdfe00700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be006bac0 0x7f9be019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.719+0000 7f9bdfe00700 1 -- 172.21.15.22:0/2936218958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be01a04e0 con 0x7f9be006b100 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.719+0000 7f9bdfe00700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be006b100 0x7f9be019b4d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0005910 tx=0x7f9bd000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.720+0000 7f9bdcc00700 1 -- 172.21.15.22:0/2936218958 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0027070 con 0x7f9be006b100 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.720+0000 7f9bdcc00700 1 -- 172.21.15.22:0/2936218958 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bd0003d70 con 0x7f9be006b100 2024-04-20T21:32:31.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.720+0000 7f9be6a00700 1 -- 172.21.15.22:0/2936218958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be01a0770 con 0x7f9be006b100 2024-04-20T21:32:31.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.720+0000 7f9bdcc00700 1 -- 172.21.15.22:0/2936218958 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd000f460 con 0x7f9be006b100 2024-04-20T21:32:31.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.720+0000 7f9be6a00700 1 -- 172.21.15.22:0/2936218958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be01a8050 con 0x7f9be006b100 2024-04-20T21:32:31.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.721+0000 7f9be6a00700 1 -- 172.21.15.22:0/2936218958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be0069740 con 0x7f9be006b100 2024-04-20T21:32:31.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.723+0000 7f9bdcc00700 1 -- 172.21.15.22:0/2936218958 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9bd002a070 con 0x7f9be006b100 2024-04-20T21:32:31.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.724+0000 7f9bdcc00700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f9bc40a3d50 0x7f9bc40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:31.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.724+0000 7f9bdcc00700 1 -- 172.21.15.22:0/2936218958 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f9bd00d1270 con 0x7f9be006b100 2024-04-20T21:32:31.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.724+0000 7f9bdf400700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f9bc40a3d50 0x7f9bc40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:31.155 INFO:teuthology.orchestra.run.smithi022.stdout:171798691900 2024-04-20T21:32:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.725+0000 7f9bdf400700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f9bc40a3d50 0x7f9bc40a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9be0103c50 tx=0x7f9bcc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.728+0000 7f9bdcc00700 1 -- 172.21.15.22:0/2936218958 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9bd005eff0 con 0x7f9be006b100 2024-04-20T21:32:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.975+0000 7f9be6a00700 1 -- 172.21.15.22:0/2936218958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9be01a8220 con 0x7f9be006b100 2024-04-20T21:32:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.975+0000 7f9bdcc00700 1 -- 172.21.15.22:0/2936218958 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9bd0013070 con 0x7f9be006b100 2024-04-20T21:32:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.978+0000 7f9be6a00700 1 -- 172.21.15.22:0/2936218958 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f9bc40a3d50 msgr2=0x7f9bc40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.978+0000 7f9be6a00700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f9bc40a3d50 0x7f9bc40a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9be0103c50 tx=0x7f9bcc0073d0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.978+0000 7f9be6a00700 1 -- 172.21.15.22:0/2936218958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be006b100 msgr2=0x7f9be019b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.978+0000 7f9be6a00700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be006b100 0x7f9be019b4d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0005910 tx=0x7f9bd000bab0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.978+0000 7f9be6a00700 1 -- 172.21.15.22:0/2936218958 shutdown_connections 2024-04-20T21:32:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.978+0000 7f9be6a00700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f9bc40a3d50 0x7f9bc40a6210 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.978+0000 7f9be6a00700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be0108aa0 0x7f9be019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.978+0000 7f9be6a00700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be006b100 0x7f9be019b4d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.978+0000 7f9be6a00700 1 --2- 172.21.15.22:0/2936218958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be006bac0 0x7f9be019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.978+0000 7f9be6a00700 1 -- 172.21.15.22:0/2936218958 >> 172.21.15.22:0/2936218958 conn(0x7f9be00786b0 msgr2=0x7f9be010dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:31.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.978+0000 7f9be6a00700 1 -- 172.21.15.22:0/2936218958 shutdown_connections 2024-04-20T21:32:31.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:29.978+0000 7f9be6a00700 1 -- 172.21.15.22:0/2936218958 wait complete. 2024-04-20T21:32:31.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-20T21:32:31.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-04-20T21:32:31.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-04-20T21:32:31.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-20T21:32:31.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-20T21:32:31.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-20T21:32:31.826 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-04-20T21:32:31.826 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-20T21:32:31.826 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-20T21:32:31.826 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-20T21:32:31.826 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:31 smithi022 ceph-mon[16843]: osdmap e89: 12 total, 12 up, 12 in 2024-04-20T21:32:31.889 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691895 got 171798691900 for osd.5 2024-04-20T21:32:31.889 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:32.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-20T21:32:32.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-04-20T21:32:32.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-04-20T21:32:32.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-20T21:32:32.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-20T21:32:32.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-20T21:32:32.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-04-20T21:32:32.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-20T21:32:32.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-20T21:32:32.026 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-20T21:32:32.027 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:31 smithi122 ceph-mon[22147]: osdmap e89: 12 total, 12 up, 12 in 2024-04-20T21:32:32.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-20T21:32:32.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-04-20T21:32:32.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-04-20T21:32:32.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-20T21:32:32.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-20T21:32:32.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-20T21:32:32.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-04-20T21:32:32.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-20T21:32:32.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-20T21:32:32.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-20T21:32:32.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:31 smithi033 ceph-mon[21626]: osdmap e89: 12 total, 12 up, 12 in 2024-04-20T21:32:32.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.587+0000 7f8410800700 1 -- 172.21.15.22:0/4142876349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8408103460 msgr2=0x7f840810f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.587+0000 7f8410800700 1 --2- 172.21.15.22:0/4142876349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8408103460 0x7f840810f8c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f840400b0a0 tx=0x7f840400b3b0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.587+0000 7f8410800700 1 -- 172.21.15.22:0/4142876349 shutdown_connections 2024-04-20T21:32:32.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.587+0000 7f8410800700 1 --2- 172.21.15.22:0/4142876349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8408103460 0x7f840810f8c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.587+0000 7f8410800700 1 --2- 172.21.15.22:0/4142876349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8408108a60 0x7f8408108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.587+0000 7f8410800700 1 --2- 172.21.15.22:0/4142876349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8408102aa0 0x7f8408102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.587+0000 7f8410800700 1 -- 172.21.15.22:0/4142876349 >> 172.21.15.22:0/4142876349 conn(0x7f8408079bb0 msgr2=0x7f8408079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:32.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.587+0000 7f8410800700 1 -- 172.21.15.22:0/4142876349 shutdown_connections 2024-04-20T21:32:32.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.587+0000 7f8410800700 1 -- 172.21.15.22:0/4142876349 wait complete. 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.588+0000 7f8410800700 1 Processor -- start 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.589+0000 7f8410800700 1 -- start start 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.589+0000 7f8410800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8408102aa0 0x7f840819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.589+0000 7f8410800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8408103460 0x7f840819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.589+0000 7f8410800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8408108a60 0x7f840819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.589+0000 7f8410800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8408112db0 con 0x7f8408108a60 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.589+0000 7f8410800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8408112c30 con 0x7f8408103460 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.589+0000 7f8410800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8408112f30 con 0x7f8408102aa0 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.589+0000 7f840e800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8408108a60 0x7f840819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.589+0000 7f840e800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8408108a60 0x7f840819fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54868/0 (socket says 172.21.15.22:54868) 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.589+0000 7f840e800700 1 -- 172.21.15.22:0/2436977191 learned_addr learned my addr 172.21.15.22:0/2436977191 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.590+0000 7f840de00700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8408102aa0 0x7f840819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.590+0000 7f840d400700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8408103460 0x7f840819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:32.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.590+0000 7f840e800700 1 -- 172.21.15.22:0/2436977191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8408102aa0 msgr2=0x7f840819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.590+0000 7f840e800700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8408102aa0 0x7f840819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.590+0000 7f840e800700 1 -- 172.21.15.22:0/2436977191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8408103460 msgr2=0x7f840819b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.590+0000 7f840e800700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8408103460 0x7f840819b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.590+0000 7f840e800700 1 -- 172.21.15.22:0/2436977191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84081a0450 con 0x7f8408108a60 2024-04-20T21:32:32.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.590+0000 7f840e800700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8408108a60 0x7f840819fd10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f840400b070 tx=0x7f8404004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:32.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.591+0000 7f8402a00700 1 -- 172.21.15.22:0/2436977191 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404021070 con 0x7f8408108a60 2024-04-20T21:32:32.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.591+0000 7f8402a00700 1 -- 172.21.15.22:0/2436977191 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8404003c00 con 0x7f8408108a60 2024-04-20T21:32:32.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.591+0000 7f8410800700 1 -- 172.21.15.22:0/2436977191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84081a06e0 con 0x7f8408108a60 2024-04-20T21:32:32.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.591+0000 7f8402a00700 1 -- 172.21.15.22:0/2436977191 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404004740 con 0x7f8408108a60 2024-04-20T21:32:32.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.591+0000 7f8410800700 1 -- 172.21.15.22:0/2436977191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84081a7fc0 con 0x7f8408108a60 2024-04-20T21:32:32.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.592+0000 7f8410800700 1 -- 172.21.15.22:0/2436977191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8408069740 con 0x7f8408108a60 2024-04-20T21:32:32.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.594+0000 7f8402a00700 1 -- 172.21.15.22:0/2436977191 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8404007980 con 0x7f8408108a60 2024-04-20T21:32:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.596+0000 7f8402a00700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f83f00a3cd0 0x7f83f00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.596+0000 7f8402a00700 1 -- 172.21.15.22:0/2436977191 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f840400f020 con 0x7f8408108a60 2024-04-20T21:32:32.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.596+0000 7f840de00700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f83f00a3cd0 0x7f83f00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.597+0000 7f840de00700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f83f00a3cd0 0x7f83f00a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009be0 tx=0x7f83fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:32.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.599+0000 7f8402a00700 1 -- 172.21.15.22:0/2436977191 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f840402a020 con 0x7f8408108a60 2024-04-20T21:32:32.715 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:32 smithi022 ceph-mon[16843]: osdmap e90: 12 total, 12 up, 12 in 2024-04-20T21:32:32.715 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:32 smithi022 ceph-mon[16843]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-20T21:32:32.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.846+0000 7f8410800700 1 -- 172.21.15.22:0/2436977191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f84081a8190 con 0x7f8408108a60 2024-04-20T21:32:32.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.847+0000 7f8402a00700 1 -- 172.21.15.22:0/2436977191 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f840409e570 con 0x7f8408108a60 2024-04-20T21:32:32.846 INFO:teuthology.orchestra.run.smithi022.stdout:356482285595 2024-04-20T21:32:32.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.850+0000 7f8410800700 1 -- 172.21.15.22:0/2436977191 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f83f00a3cd0 msgr2=0x7f83f00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.850+0000 7f8410800700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f83f00a3cd0 0x7f83f00a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009be0 tx=0x7f83fc009260 comp rx=0 tx=0).stop 2024-04-20T21:32:32.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.850+0000 7f8410800700 1 -- 172.21.15.22:0/2436977191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8408108a60 msgr2=0x7f840819fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.850+0000 7f8410800700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8408108a60 0x7f840819fd10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f840400b070 tx=0x7f8404004c60 comp rx=0 tx=0).stop 2024-04-20T21:32:32.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.850+0000 7f8410800700 1 -- 172.21.15.22:0/2436977191 shutdown_connections 2024-04-20T21:32:32.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.850+0000 7f8410800700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f83f00a3cd0 0x7f83f00a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.850+0000 7f8410800700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8408108a60 0x7f840819fd10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.850+0000 7f8410800700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8408102aa0 0x7f840819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.850+0000 7f8410800700 1 --2- 172.21.15.22:0/2436977191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8408103460 0x7f840819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.850+0000 7f8410800700 1 -- 172.21.15.22:0/2436977191 >> 172.21.15.22:0/2436977191 conn(0x7f8408079bb0 msgr2=0x7f840810dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:32.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.851+0000 7f8410800700 1 -- 172.21.15.22:0/2436977191 shutdown_connections 2024-04-20T21:32:32.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:32.851+0000 7f8410800700 1 -- 172.21.15.22:0/2436977191 wait complete. 2024-04-20T21:32:33.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:32 smithi122 ceph-mon[22147]: osdmap e90: 12 total, 12 up, 12 in 2024-04-20T21:32:33.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:32 smithi122 ceph-mon[22147]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-20T21:32:33.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:32 smithi033 ceph-mon[21626]: osdmap e90: 12 total, 12 up, 12 in 2024-04-20T21:32:33.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:32 smithi033 ceph-mon[21626]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-20T21:32:33.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:33 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2436977191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-20T21:32:34.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:33 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2436977191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-20T21:32:34.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:33 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2436977191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-20T21:32:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.196+0000 7f6983200700 1 -- 172.21.15.22:0/1452739340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697c0755a0 msgr2=0x7f697c10ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.196+0000 7f6983200700 1 --2- 172.21.15.22:0/1452739340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697c0755a0 0x7f697c10ef10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6968009990 tx=0x7f6968009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.196+0000 7f6983200700 1 -- 172.21.15.22:0/1452739340 shutdown_connections 2024-04-20T21:32:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.196+0000 7f6983200700 1 --2- 172.21.15.22:0/1452739340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c10f450 0x7f697c111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.196+0000 7f6983200700 1 --2- 172.21.15.22:0/1452739340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c074c80 0x7f697c075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.196+0000 7f6983200700 1 --2- 172.21.15.22:0/1452739340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697c0755a0 0x7f697c10ef10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.196+0000 7f6983200700 1 -- 172.21.15.22:0/1452739340 >> 172.21.15.22:0/1452739340 conn(0x7f697c079bb0 msgr2=0x7f697c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.196+0000 7f6983200700 1 -- 172.21.15.22:0/1452739340 shutdown_connections 2024-04-20T21:32:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.196+0000 7f6983200700 1 -- 172.21.15.22:0/1452739340 wait complete. 2024-04-20T21:32:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.197+0000 7f6983200700 1 Processor -- start 2024-04-20T21:32:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.198+0000 7f6983200700 1 -- start start 2024-04-20T21:32:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.198+0000 7f6983200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697c074c80 0x7f697c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.198+0000 7f6983200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c0755a0 0x7f697c19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.198+0000 7f6983200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c10f450 0x7f697c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.198+0000 7f6983200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f697c114fd0 con 0x7f697c074c80 2024-04-20T21:32:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.198+0000 7f6983200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f697c114e50 con 0x7f697c0755a0 2024-04-20T21:32:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.198+0000 7f6983200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f697c115150 con 0x7f697c10f450 2024-04-20T21:32:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.198+0000 7f697be00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697c074c80 0x7f697c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.198+0000 7f697be00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697c074c80 0x7f697c19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54892/0 (socket says 172.21.15.22:54892) 2024-04-20T21:32:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.198+0000 7f697be00700 1 -- 172.21.15.22:0/3339099299 learned_addr learned my addr 172.21.15.22:0/3339099299 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.198+0000 7f697b400700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c0755a0 0x7f697c19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.199+0000 7f6981200700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c10f450 0x7f697c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.199+0000 7f697be00700 1 -- 172.21.15.22:0/3339099299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c10f450 msgr2=0x7f697c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.199+0000 7f697be00700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c10f450 0x7f697c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.199+0000 7f697be00700 1 -- 172.21.15.22:0/3339099299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c0755a0 msgr2=0x7f697c19b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.199+0000 7f697be00700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c0755a0 0x7f697c19b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.199+0000 7f697be00700 1 -- 172.21.15.22:0/3339099299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c1a0430 con 0x7f697c074c80 2024-04-20T21:32:34.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.199+0000 7f697be00700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697c074c80 0x7f697c19b420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f696c00eb30 tx=0x7f696c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:34.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.200+0000 7f6978c00700 1 -- 172.21.15.22:0/3339099299 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c00cbd0 con 0x7f697c074c80 2024-04-20T21:32:34.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.200+0000 7f6983200700 1 -- 172.21.15.22:0/3339099299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c1a0720 con 0x7f697c074c80 2024-04-20T21:32:34.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.200+0000 7f6978c00700 1 -- 172.21.15.22:0/3339099299 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f696c004500 con 0x7f697c074c80 2024-04-20T21:32:34.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.200+0000 7f6978c00700 1 -- 172.21.15.22:0/3339099299 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c010430 con 0x7f697c074c80 2024-04-20T21:32:34.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.200+0000 7f6983200700 1 -- 172.21.15.22:0/3339099299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c1a7fa0 con 0x7f697c074c80 2024-04-20T21:32:34.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.201+0000 7f6983200700 1 -- 172.21.15.22:0/3339099299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697c069740 con 0x7f697c074c80 2024-04-20T21:32:34.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.203+0000 7f6978c00700 1 -- 172.21.15.22:0/3339099299 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f696c004020 con 0x7f697c074c80 2024-04-20T21:32:34.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.204+0000 7f6978c00700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f69600a3dd0 0x7f69600a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.204+0000 7f6978c00700 1 -- 172.21.15.22:0/3339099299 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f696c0d08d0 con 0x7f697c074c80 2024-04-20T21:32:34.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.205+0000 7f697b400700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f69600a3dd0 0x7f69600a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.205+0000 7f697b400700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f69600a3dd0 0x7f69600a6290 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f697c111010 tx=0x7f696800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:34.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.209+0000 7f6978c00700 1 -- 172.21.15.22:0/3339099299 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f696c09d0d0 con 0x7f697c074c80 2024-04-20T21:32:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.457+0000 7f6983200700 1 -- 172.21.15.22:0/3339099299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f697c1a8260 con 0x7f697c074c80 2024-04-20T21:32:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.458+0000 7f6978c00700 1 -- 172.21.15.22:0/3339099299 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f696c09c9d0 con 0x7f697c074c80 2024-04-20T21:32:34.456 INFO:teuthology.orchestra.run.smithi022.stdout:111669149767 2024-04-20T21:32:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.461+0000 7f6983200700 1 -- 172.21.15.22:0/3339099299 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f69600a3dd0 msgr2=0x7f69600a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.461+0000 7f6983200700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f69600a3dd0 0x7f69600a6290 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f697c111010 tx=0x7f696800b260 comp rx=0 tx=0).stop 2024-04-20T21:32:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.461+0000 7f6983200700 1 -- 172.21.15.22:0/3339099299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697c074c80 msgr2=0x7f697c19b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.461+0000 7f6983200700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697c074c80 0x7f697c19b420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f696c00eb30 tx=0x7f696c00ee40 comp rx=0 tx=0).stop 2024-04-20T21:32:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.461+0000 7f6983200700 1 -- 172.21.15.22:0/3339099299 shutdown_connections 2024-04-20T21:32:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.461+0000 7f6983200700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f69600a3dd0 0x7f69600a6290 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.461+0000 7f6983200700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f697c10f450 0x7f697c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.461+0000 7f6983200700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697c074c80 0x7f697c19b420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.461+0000 7f6983200700 1 --2- 172.21.15.22:0/3339099299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c0755a0 0x7f697c19b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.461+0000 7f6983200700 1 -- 172.21.15.22:0/3339099299 >> 172.21.15.22:0/3339099299 conn(0x7f697c079bb0 msgr2=0x7f697c0ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:34.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.461+0000 7f6983200700 1 -- 172.21.15.22:0/3339099299 shutdown_connections 2024-04-20T21:32:34.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.461+0000 7f6983200700 1 -- 172.21.15.22:0/3339099299 wait complete. 2024-04-20T21:32:34.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:34 smithi022 ceph-mon[16843]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:34.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:34 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/341768318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T21:32:34.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:34 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3339099299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T21:32:34.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.960+0000 7fed94600700 1 -- 172.21.15.22:0/2294353678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c104ed0 msgr2=0x7fed8c1052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.960+0000 7fed94600700 1 --2- 172.21.15.22:0/2294353678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c104ed0 0x7fed8c1052b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fed800099c0 tx=0x7fed80009cd0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.960+0000 7fed94600700 1 -- 172.21.15.22:0/2294353678 shutdown_connections 2024-04-20T21:32:34.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.960+0000 7fed94600700 1 --2- 172.21.15.22:0/2294353678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed8c109fb0 0x7fed8c111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.960+0000 7fed94600700 1 --2- 172.21.15.22:0/2294353678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c104ed0 0x7fed8c1052b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.960+0000 7fed94600700 1 --2- 172.21.15.22:0/2294353678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed8c105880 0x7fed8c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.960+0000 7fed94600700 1 -- 172.21.15.22:0/2294353678 >> 172.21.15.22:0/2294353678 conn(0x7fed8c100720 msgr2=0x7fed8c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:34.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.960+0000 7fed94600700 1 -- 172.21.15.22:0/2294353678 shutdown_connections 2024-04-20T21:32:34.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.960+0000 7fed94600700 1 -- 172.21.15.22:0/2294353678 wait complete. 2024-04-20T21:32:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.961+0000 7fed94600700 1 Processor -- start 2024-04-20T21:32:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.961+0000 7fed94600700 1 -- start start 2024-04-20T21:32:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.962+0000 7fed94600700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed8c104ed0 0x7fed8c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.962+0000 7fed94600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c105880 0x7fed8c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.962+0000 7fed94600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed8c109fb0 0x7fed8c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.962+0000 7fed94600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c115000 con 0x7fed8c105880 2024-04-20T21:32:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.962+0000 7fed94600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c114e80 con 0x7fed8c109fb0 2024-04-20T21:32:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.962+0000 7fed94600700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c115180 con 0x7fed8c104ed0 2024-04-20T21:32:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.962+0000 7fed91200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c105880 0x7fed8c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.962+0000 7fed91c00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed8c104ed0 0x7fed8c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.962+0000 7fed91200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c105880 0x7fed8c19b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54918/0 (socket says 172.21.15.22:54918) 2024-04-20T21:32:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.962+0000 7fed91200700 1 -- 172.21.15.22:0/3780579870 learned_addr learned my addr 172.21.15.22:0/3780579870 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.962+0000 7fed92600700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed8c109fb0 0x7fed8c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.963+0000 7fed91200700 1 -- 172.21.15.22:0/3780579870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed8c104ed0 msgr2=0x7fed8c19b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.963+0000 7fed91200700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed8c104ed0 0x7fed8c19b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.963+0000 7fed91200700 1 -- 172.21.15.22:0/3780579870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed8c109fb0 msgr2=0x7fed8c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.963+0000 7fed91200700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed8c109fb0 0x7fed8c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.963+0000 7fed91200700 1 -- 172.21.15.22:0/3780579870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c1a0460 con 0x7fed8c105880 2024-04-20T21:32:34.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.963+0000 7fed91200700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c105880 0x7fed8c19b990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fed7800e970 tx=0x7fed7800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:34.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.963+0000 7fed8a000700 1 -- 172.21.15.22:0/3780579870 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78004a30 con 0x7fed8c105880 2024-04-20T21:32:34.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.963+0000 7fed8a000700 1 -- 172.21.15.22:0/3780579870 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed7800ccc0 con 0x7fed8c105880 2024-04-20T21:32:34.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.963+0000 7fed94600700 1 -- 172.21.15.22:0/3780579870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8c1a0750 con 0x7fed8c105880 2024-04-20T21:32:34.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.964+0000 7fed8a000700 1 -- 172.21.15.22:0/3780579870 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78005650 con 0x7fed8c105880 2024-04-20T21:32:34.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.964+0000 7fed94600700 1 -- 172.21.15.22:0/3780579870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8c1a7fd0 con 0x7fed8c105880 2024-04-20T21:32:34.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.965+0000 7fed94600700 1 -- 172.21.15.22:0/3780579870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8c069740 con 0x7fed8c105880 2024-04-20T21:32:34.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.967+0000 7fed8a000700 1 -- 172.21.15.22:0/3780579870 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fed78004bd0 con 0x7fed8c105880 2024-04-20T21:32:34.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.968+0000 7fed8a000700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fed740a3cd0 0x7fed740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.968+0000 7fed8a000700 1 -- 172.21.15.22:0/3780579870 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fed780d0820 con 0x7fed8c105880 2024-04-20T21:32:34.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.969+0000 7fed91c00700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fed740a3cd0 0x7fed740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.970+0000 7fed91c00700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fed740a3cd0 0x7fed740a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fed80000c00 tx=0x7fed80005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:34.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.973+0000 7fed8a000700 1 -- 172.21.15.22:0/3780579870 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fed7809d020 con 0x7fed8c105880 2024-04-20T21:32:35.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:34 smithi122 ceph-mon[22147]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:35.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:34 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/341768318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T21:32:35.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:34 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3339099299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T21:32:35.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:34 smithi033 ceph-mon[21626]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:35.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:34 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/341768318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T21:32:35.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:34 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3339099299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T21:32:35.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.853+0000 7fd042400700 1 -- 172.21.15.22:0/694308563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd03c103460 msgr2=0x7fd03c10f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.853+0000 7fd042400700 1 --2- 172.21.15.22:0/694308563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd03c103460 0x7fd03c10f8c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd03400b0a0 tx=0x7fd03400b3b0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.853+0000 7fd042400700 1 -- 172.21.15.22:0/694308563 shutdown_connections 2024-04-20T21:32:35.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.853+0000 7fd042400700 1 --2- 172.21.15.22:0/694308563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd03c103460 0x7fd03c10f8c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.853+0000 7fd042400700 1 --2- 172.21.15.22:0/694308563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd03c108a60 0x7fd03c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.853+0000 7fd042400700 1 --2- 172.21.15.22:0/694308563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03c102aa0 0x7fd03c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.853+0000 7fd042400700 1 -- 172.21.15.22:0/694308563 >> 172.21.15.22:0/694308563 conn(0x7fd03c079bb0 msgr2=0x7fd03c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.853+0000 7fd042400700 1 -- 172.21.15.22:0/694308563 shutdown_connections 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.853+0000 7fd042400700 1 -- 172.21.15.22:0/694308563 wait complete. 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.854+0000 7fd042400700 1 Processor -- start 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.855+0000 7fd042400700 1 -- start start 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.855+0000 7fd042400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd03c102aa0 0x7fd03c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.855+0000 7fd042400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03c103460 0x7fd03c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.855+0000 7fd042400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd03c108a60 0x7fd03c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.855+0000 7fd042400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c112db0 con 0x7fd03c108a60 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.855+0000 7fd042400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c112c30 con 0x7fd03c103460 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.855+0000 7fd042400700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c112f30 con 0x7fd03c102aa0 2024-04-20T21:32:35.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.856+0000 7fd03b400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd03c102aa0 0x7fd03c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.856+0000 7fd03b400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd03c102aa0 0x7fd03c199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:47950/0 (socket says 172.21.15.22:47950) 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.856+0000 7fd03b400700 1 -- 172.21.15.22:0/341768318 learned_addr learned my addr 172.21.15.22:0/341768318 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.856+0000 7fd03be00700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd03c108a60 0x7fd03c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.856+0000 7fd032a00700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03c103460 0x7fd03c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.856+0000 7fd03b400700 1 -- 172.21.15.22:0/341768318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03c103460 msgr2=0x7fd03c199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.856+0000 7fd03b400700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03c103460 0x7fd03c199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.856+0000 7fd03b400700 1 -- 172.21.15.22:0/341768318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd03c108a60 msgr2=0x7fd03c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.856+0000 7fd03b400700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd03c108a60 0x7fd03c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.856+0000 7fd03b400700 1 -- 172.21.15.22:0/341768318 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c19e210 con 0x7fd03c102aa0 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.856+0000 7fd032a00700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03c103460 0x7fd03c199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.856+0000 7fd03b400700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd03c102aa0 0x7fd03c199200 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00e9a0 tx=0x7fd02c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.857+0000 7fd038c00700 1 -- 172.21.15.22:0/341768318 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c00cbd0 con 0x7fd03c102aa0 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.857+0000 7fd038c00700 1 -- 172.21.15.22:0/341768318 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02c00cd70 con 0x7fd03c102aa0 2024-04-20T21:32:35.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.857+0000 7fd042400700 1 -- 172.21.15.22:0/341768318 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03c19e500 con 0x7fd03c102aa0 2024-04-20T21:32:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.857+0000 7fd038c00700 1 -- 172.21.15.22:0/341768318 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c018750 con 0x7fd03c102aa0 2024-04-20T21:32:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.857+0000 7fd042400700 1 -- 172.21.15.22:0/341768318 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03c1a5d80 con 0x7fd03c102aa0 2024-04-20T21:32:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.858+0000 7fd042400700 1 -- 172.21.15.22:0/341768318 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd03c069740 con 0x7fd03c102aa0 2024-04-20T21:32:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.860+0000 7fd038c00700 1 -- 172.21.15.22:0/341768318 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd02c010430 con 0x7fd03c102aa0 2024-04-20T21:32:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.862+0000 7fd038c00700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd0200a3cd0 0x7fd0200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.862+0000 7fd038c00700 1 -- 172.21.15.22:0/341768318 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fd02c0d0840 con 0x7fd03c102aa0 2024-04-20T21:32:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.862+0000 7fd032a00700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd0200a3cd0 0x7fd0200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.863+0000 7fd032a00700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd0200a3cd0 0x7fd0200a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd03c07a1b0 tx=0x7fd024009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:33.865+0000 7fd038c00700 1 -- 172.21.15.22:0/341768318 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd02c09d040 con 0x7fd03c102aa0 2024-04-20T21:32:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.120+0000 7fd042400700 1 -- 172.21.15.22:0/341768318 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd03c02df90 con 0x7fd03c102aa0 2024-04-20T21:32:35.133 INFO:teuthology.orchestra.run.smithi022.stdout:146028888131 2024-04-20T21:32:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.121+0000 7fd038c00700 1 -- 172.21.15.22:0/341768318 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd02c09c940 con 0x7fd03c102aa0 2024-04-20T21:32:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.123+0000 7fd042400700 1 -- 172.21.15.22:0/341768318 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd0200a3cd0 msgr2=0x7fd0200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.123+0000 7fd042400700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd0200a3cd0 0x7fd0200a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd03c07a1b0 tx=0x7fd024009260 comp rx=0 tx=0).stop 2024-04-20T21:32:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.123+0000 7fd042400700 1 -- 172.21.15.22:0/341768318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd03c102aa0 msgr2=0x7fd03c199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.123+0000 7fd042400700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd03c102aa0 0x7fd03c199200 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00e9a0 tx=0x7fd02c00ecb0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.123+0000 7fd042400700 1 -- 172.21.15.22:0/341768318 shutdown_connections 2024-04-20T21:32:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.123+0000 7fd042400700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd0200a3cd0 0x7fd0200a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.123+0000 7fd042400700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd03c108a60 0x7fd03c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.123+0000 7fd042400700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd03c102aa0 0x7fd03c199200 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.123+0000 7fd042400700 1 --2- 172.21.15.22:0/341768318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03c103460 0x7fd03c199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.124+0000 7fd042400700 1 -- 172.21.15.22:0/341768318 >> 172.21.15.22:0/341768318 conn(0x7fd03c079bb0 msgr2=0x7fd03c1016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.124+0000 7fd042400700 1 -- 172.21.15.22:0/341768318 shutdown_connections 2024-04-20T21:32:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:34.124+0000 7fd042400700 1 -- 172.21.15.22:0/341768318 wait complete. 2024-04-20T21:32:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.226+0000 7fed94600700 1 -- 172.21.15.22:0/3780579870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fed8c02df90 con 0x7fed8c105880 2024-04-20T21:32:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.227+0000 7fed8a000700 1 -- 172.21.15.22:0/3780579870 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fed7809c920 con 0x7fed8c105880 2024-04-20T21:32:35.226 INFO:teuthology.orchestra.run.smithi022.stdout:227633266736 2024-04-20T21:32:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.230+0000 7fed94600700 1 -- 172.21.15.22:0/3780579870 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fed740a3cd0 msgr2=0x7fed740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.230+0000 7fed94600700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fed740a3cd0 0x7fed740a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fed80000c00 tx=0x7fed80005fb0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.230+0000 7fed94600700 1 -- 172.21.15.22:0/3780579870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c105880 msgr2=0x7fed8c19b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.230+0000 7fed94600700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c105880 0x7fed8c19b990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fed7800e970 tx=0x7fed7800ec80 comp rx=0 tx=0).stop 2024-04-20T21:32:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.230+0000 7fed94600700 1 -- 172.21.15.22:0/3780579870 shutdown_connections 2024-04-20T21:32:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.230+0000 7fed94600700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fed740a3cd0 0x7fed740a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.230+0000 7fed94600700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed8c109fb0 0x7fed8c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.230+0000 7fed94600700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed8c104ed0 0x7fed8c19b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.230+0000 7fed94600700 1 --2- 172.21.15.22:0/3780579870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c105880 0x7fed8c19b990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.230+0000 7fed94600700 1 -- 172.21.15.22:0/3780579870 >> 172.21.15.22:0/3780579870 conn(0x7fed8c100720 msgr2=0x7fed8c102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.230+0000 7fed94600700 1 -- 172.21.15.22:0/3780579870 shutdown_connections 2024-04-20T21:32:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:35.231+0000 7fed94600700 1 -- 172.21.15.22:0/3780579870 wait complete. 2024-04-20T21:32:35.789 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:35.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:35 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3780579870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-20T21:32:36.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:35 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3780579870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-20T21:32:36.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:35 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3780579870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-20T21:32:36.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:36 smithi022 ceph-mon[16843]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:37.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:36 smithi122 ceph-mon[22147]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:37.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:36 smithi033 ceph-mon[21626]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:37.354 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149762 got 111669149767 for osd.3 2024-04-20T21:32:37.354 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:37.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.455+0000 7f3b37800700 1 -- 172.21.15.22:0/1544666024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b30104ed0 msgr2=0x7f3b301052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.455+0000 7f3b37800700 1 --2- 172.21.15.22:0/1544666024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b30104ed0 0x7f3b301052b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3b240099c0 tx=0x7f3b24009cd0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.455+0000 7f3b37800700 1 -- 172.21.15.22:0/1544666024 shutdown_connections 2024-04-20T21:32:37.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.455+0000 7f3b37800700 1 --2- 172.21.15.22:0/1544666024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b30109fb0 0x7f3b30111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.455+0000 7f3b37800700 1 --2- 172.21.15.22:0/1544666024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b30104ed0 0x7f3b301052b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.455+0000 7f3b37800700 1 --2- 172.21.15.22:0/1544666024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b30105880 0x7f3b301098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.455+0000 7f3b37800700 1 -- 172.21.15.22:0/1544666024 >> 172.21.15.22:0/1544666024 conn(0x7f3b30100760 msgr2=0x7f3b30102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.455+0000 7f3b37800700 1 -- 172.21.15.22:0/1544666024 shutdown_connections 2024-04-20T21:32:37.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.455+0000 7f3b37800700 1 -- 172.21.15.22:0/1544666024 wait complete. 2024-04-20T21:32:37.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.456+0000 7f3b37800700 1 Processor -- start 2024-04-20T21:32:37.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.457+0000 7f3b37800700 1 -- start start 2024-04-20T21:32:37.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.457+0000 7f3b37800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b30104ed0 0x7f3b3019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.457+0000 7f3b37800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b30105880 0x7f3b3019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.457+0000 7f3b37800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b30109fb0 0x7f3b3019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.457+0000 7f3b37800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3b30114f90 con 0x7f3b30104ed0 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.457+0000 7f3b37800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b30114e10 con 0x7f3b30105880 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.457+0000 7f3b37800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3b30115110 con 0x7f3b30109fb0 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.458+0000 7f3b34e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b30104ed0 0x7f3b3019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.458+0000 7f3b34e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b30104ed0 0x7f3b3019b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54922/0 (socket says 172.21.15.22:54922) 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.458+0000 7f3b34e00700 1 -- 172.21.15.22:0/460869320 learned_addr learned my addr 172.21.15.22:0/460869320 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.458+0000 7f3b2fe00700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b30105880 0x7f3b3019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.458+0000 7f3b35800700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b30109fb0 0x7f3b3019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.458+0000 7f3b34e00700 1 -- 172.21.15.22:0/460869320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b30109fb0 msgr2=0x7f3b3019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.458+0000 7f3b34e00700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b30109fb0 0x7f3b3019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.458+0000 7f3b34e00700 1 -- 172.21.15.22:0/460869320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b30105880 msgr2=0x7f3b3019b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.458+0000 7f3b34e00700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b30105880 0x7f3b3019b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.458+0000 7f3b34e00700 1 -- 172.21.15.22:0/460869320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b301a03d0 con 0x7f3b30104ed0 2024-04-20T21:32:37.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.458+0000 7f3b2fe00700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b30105880 0x7f3b3019b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:37.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.459+0000 7f3b34e00700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b30104ed0 0x7f3b3019b3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3b24006010 tx=0x7f3b24003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.459+0000 7f3b2d600700 1 -- 172.21.15.22:0/460869320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24003680 con 0x7f3b30104ed0 2024-04-20T21:32:37.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.459+0000 7f3b37800700 1 -- 172.21.15.22:0/460869320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b301a0660 con 0x7f3b30104ed0 2024-04-20T21:32:37.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.459+0000 7f3b2d600700 1 -- 172.21.15.22:0/460869320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b240055a0 con 0x7f3b30104ed0 2024-04-20T21:32:37.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.459+0000 7f3b2d600700 1 -- 172.21.15.22:0/460869320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2400fe50 con 0x7f3b30104ed0 2024-04-20T21:32:37.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.459+0000 7f3b37800700 1 -- 172.21.15.22:0/460869320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b301a7f40 con 0x7f3b30104ed0 2024-04-20T21:32:37.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.461+0000 7f3b37800700 1 -- 172.21.15.22:0/460869320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b301063b0 con 0x7f3b30104ed0 2024-04-20T21:32:37.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.463+0000 7f3b2d600700 1 -- 172.21.15.22:0/460869320 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3b2400f580 con 0x7f3b30104ed0 2024-04-20T21:32:37.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.465+0000 7f3b2d600700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3b180a4010 0x7f3b180a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.465+0000 7f3b2d600700 1 -- 172.21.15.22:0/460869320 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f3b240d1270 con 0x7f3b30104ed0 2024-04-20T21:32:37.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.465+0000 7f3b2fe00700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3b180a4010 0x7f3b180a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.466+0000 7f3b2fe00700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3b180a4010 0x7f3b180a64d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b3010b510 tx=0x7f3b1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.469+0000 7f3b2d600700 1 -- 172.21.15.22:0/460869320 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3b24013070 con 0x7f3b30104ed0 2024-04-20T21:32:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.701+0000 7f3b37800700 1 -- 172.21.15.22:0/460869320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3b3002df90 con 0x7f3b30104ed0 2024-04-20T21:32:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.702+0000 7f3b2d600700 1 -- 172.21.15.22:0/460869320 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3b2409db00 con 0x7f3b30104ed0 2024-04-20T21:32:37.701 INFO:teuthology.orchestra.run.smithi022.stdout:55834574933 2024-04-20T21:32:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.705+0000 7f3b37800700 1 -- 172.21.15.22:0/460869320 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3b180a4010 msgr2=0x7f3b180a64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.705+0000 7f3b37800700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3b180a4010 0x7f3b180a64d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b3010b510 tx=0x7f3b1c009260 comp rx=0 tx=0).stop 2024-04-20T21:32:37.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.705+0000 7f3b37800700 1 -- 172.21.15.22:0/460869320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b30104ed0 msgr2=0x7f3b3019b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.705+0000 7f3b37800700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b30104ed0 0x7f3b3019b3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3b24006010 tx=0x7f3b24003a90 comp rx=0 tx=0).stop 2024-04-20T21:32:37.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.706+0000 7f3b37800700 1 -- 172.21.15.22:0/460869320 shutdown_connections 2024-04-20T21:32:37.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.706+0000 7f3b37800700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f3b180a4010 0x7f3b180a64d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.706+0000 7f3b37800700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b30109fb0 0x7f3b3019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.706+0000 7f3b37800700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b30104ed0 0x7f3b3019b3c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.706+0000 7f3b37800700 1 --2- 172.21.15.22:0/460869320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b30105880 0x7f3b3019b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.706+0000 7f3b37800700 1 -- 172.21.15.22:0/460869320 >> 172.21.15.22:0/460869320 conn(0x7f3b30100760 msgr2=0x7f3b30102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.706+0000 7f3b37800700 1 -- 172.21.15.22:0/460869320 shutdown_connections 2024-04-20T21:32:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:37.706+0000 7f3b37800700 1 -- 172.21.15.22:0/460869320 wait complete. 2024-04-20T21:32:37.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:37 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/460869320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T21:32:38.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:37 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/460869320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T21:32:38.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:37 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/460869320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T21:32:38.099 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888126 got 146028888131 for osd.4 2024-04-20T21:32:38.100 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:38.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.489+0000 7f96a9c00700 1 -- 172.21.15.22:0/1869297674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a4108a40 msgr2=0x7f96a4108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:38.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.489+0000 7f96a9c00700 1 --2- 172.21.15.22:0/1869297674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a4108a40 0x7f96a4108e20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f969c009990 tx=0x7f969c009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:38.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.489+0000 7f96a9c00700 1 -- 172.21.15.22:0/1869297674 shutdown_connections 2024-04-20T21:32:38.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.489+0000 7f96a9c00700 1 --2- 172.21.15.22:0/1869297674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a406b100 0x7f96a410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:38.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.489+0000 7f96a9c00700 1 --2- 172.21.15.22:0/1869297674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a4108a40 0x7f96a4108e20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:38.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.489+0000 7f96a9c00700 1 --2- 172.21.15.22:0/1869297674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a406a760 0x7f96a406abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:38.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.489+0000 7f96a9c00700 1 -- 172.21.15.22:0/1869297674 >> 172.21.15.22:0/1869297674 conn(0x7f96a4078900 msgr2=0x7f96a4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:38.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.489+0000 7f96a9c00700 1 -- 172.21.15.22:0/1869297674 shutdown_connections 2024-04-20T21:32:38.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.489+0000 7f96a9c00700 1 -- 172.21.15.22:0/1869297674 wait complete. 2024-04-20T21:32:38.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.490+0000 7f96a9c00700 1 Processor -- start 2024-04-20T21:32:38.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.490+0000 7f96a9c00700 1 -- start start 2024-04-20T21:32:38.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a9c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a406a760 0x7f96a419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:38.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a9c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a406b100 0x7f96a419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:38.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a9c00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a4108a40 0x7f96a419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:38.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a9c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4112d70 con 0x7f96a406b100 2024-04-20T21:32:38.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a9c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4112bf0 con 0x7f96a406a760 2024-04-20T21:32:38.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a9c00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4112ef0 con 0x7f96a4108a40 2024-04-20T21:32:38.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a2000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a406b100 0x7f96a419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:38.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a2000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a406b100 0x7f96a419b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54948/0 (socket says 172.21.15.22:54948) 2024-04-20T21:32:38.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a2000700 1 -- 172.21.15.22:0/1948017545 learned_addr learned my addr 172.21.15.22:0/1948017545 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:38.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a2a00700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a406a760 0x7f96a419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:38.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a3400700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a4108a40 0x7f96a419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:38.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a2000700 1 -- 172.21.15.22:0/1948017545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a4108a40 msgr2=0x7f96a419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:38.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a2000700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a4108a40 0x7f96a419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:38.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a2000700 1 -- 172.21.15.22:0/1948017545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a406a760 msgr2=0x7f96a419b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:38.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a2000700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a406a760 0x7f96a419b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:38.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a2000700 1 -- 172.21.15.22:0/1948017545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a41a0410 con 0x7f96a406b100 2024-04-20T21:32:38.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.491+0000 7f96a2a00700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a406a760 0x7f96a419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:38.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.492+0000 7f96a2000700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a406b100 0x7f96a419b940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f968c00ef10 tx=0x7f968c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:38.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.492+0000 7f969b400700 1 -- 172.21.15.22:0/1948017545 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c00c9c0 con 0x7f96a406b100 2024-04-20T21:32:38.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.492+0000 7f969b400700 1 -- 172.21.15.22:0/1948017545 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968c019070 con 0x7f96a406b100 2024-04-20T21:32:38.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.492+0000 7f969b400700 1 -- 172.21.15.22:0/1948017545 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c007e60 con 0x7f96a406b100 2024-04-20T21:32:38.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.492+0000 7f96a9c00700 1 -- 172.21.15.22:0/1948017545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a41a06a0 con 0x7f96a406b100 2024-04-20T21:32:38.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.492+0000 7f96a9c00700 1 -- 172.21.15.22:0/1948017545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a41a7f80 con 0x7f96a406b100 2024-04-20T21:32:38.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.494+0000 7f96a9c00700 1 -- 172.21.15.22:0/1948017545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a4069740 con 0x7f96a406b100 2024-04-20T21:32:38.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.495+0000 7f969b400700 1 -- 172.21.15.22:0/1948017545 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f968c012050 con 0x7f96a406b100 2024-04-20T21:32:38.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.497+0000 7f969b400700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f96880a3d50 0x7f96880a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:38.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.497+0000 7f969b400700 1 -- 172.21.15.22:0/1948017545 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f968c0d0580 con 0x7f96a406b100 2024-04-20T21:32:38.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.497+0000 7f96a2a00700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f96880a3d50 0x7f96880a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:38.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.498+0000 7f96a2a00700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f96880a3d50 0x7f96880a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f969c00b2e0 tx=0x7f969c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.501+0000 7f969b400700 1 -- 172.21.15.22:0/1948017545 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f968c010040 con 0x7f96a406b100 2024-04-20T21:32:38.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.747+0000 7f96a9c00700 1 -- 172.21.15.22:0/1948017545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f96a402df90 con 0x7f96a406b100 2024-04-20T21:32:38.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.748+0000 7f969b400700 1 -- 172.21.15.22:0/1948017545 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f968c09cd80 con 0x7f96a406b100 2024-04-20T21:32:38.747 INFO:teuthology.orchestra.run.smithi022.stdout:77309411407 2024-04-20T21:32:38.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.751+0000 7f96a9c00700 1 -- 172.21.15.22:0/1948017545 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f96880a3d50 msgr2=0x7f96880a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:38.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.751+0000 7f96a9c00700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f96880a3d50 0x7f96880a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f969c00b2e0 tx=0x7f969c005fd0 comp rx=0 tx=0).stop 2024-04-20T21:32:38.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.751+0000 7f96a9c00700 1 -- 172.21.15.22:0/1948017545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a406b100 msgr2=0x7f96a419b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:38.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.751+0000 7f96a9c00700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a406b100 0x7f96a419b940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f968c00ef10 tx=0x7f968c00c3a0 comp rx=0 tx=0).stop 2024-04-20T21:32:38.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.751+0000 7f96a9c00700 1 -- 172.21.15.22:0/1948017545 shutdown_connections 2024-04-20T21:32:38.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.751+0000 7f96a9c00700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f96880a3d50 0x7f96880a6210 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:38.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.751+0000 7f96a9c00700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a4108a40 0x7f96a419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:38.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.751+0000 7f96a9c00700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a406a760 0x7f96a419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:38.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.751+0000 7f96a9c00700 1 --2- 172.21.15.22:0/1948017545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96a406b100 0x7f96a419b940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:38.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.751+0000 7f96a9c00700 1 -- 172.21.15.22:0/1948017545 >> 172.21.15.22:0/1948017545 conn(0x7f96a4078900 msgr2=0x7f96a410de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:38.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.751+0000 7f96a9c00700 1 -- 172.21.15.22:0/1948017545 shutdown_connections 2024-04-20T21:32:38.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:38.752+0000 7f96a9c00700 1 -- 172.21.15.22:0/1948017545 wait complete. 2024-04-20T21:32:38.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:38 smithi022 ceph-mon[16843]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:38.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:38 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1948017545' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T21:32:39.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:38 smithi122 ceph-mon[22147]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:39.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:38 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1948017545' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T21:32:39.032 INFO:tasks.cephadm.ceph_manager.ceph:need seq 356482285588 got 356482285595 for osd.11 2024-04-20T21:32:39.032 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:39.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:38 smithi033 ceph-mon[21626]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:39.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:38 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1948017545' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T21:32:39.612 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574928 got 55834574933 for osd.1 2024-04-20T21:32:39.612 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:40.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.034+0000 7fb91d400700 1 -- 172.21.15.22:0/916598436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb918104ed0 msgr2=0x7fb9181052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:40.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.034+0000 7fb91d400700 1 --2- 172.21.15.22:0/916598436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb918104ed0 0x7fb9181052b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b0d0 tx=0x7fb90800b3e0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.034+0000 7fb91d400700 1 -- 172.21.15.22:0/916598436 shutdown_connections 2024-04-20T21:32:40.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.034+0000 7fb91d400700 1 --2- 172.21.15.22:0/916598436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb918109fb0 0x7fb918111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.034+0000 7fb91d400700 1 --2- 172.21.15.22:0/916598436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb918104ed0 0x7fb9181052b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.034+0000 7fb91d400700 1 --2- 172.21.15.22:0/916598436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb918105880 0x7fb9181098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.034+0000 7fb91d400700 1 -- 172.21.15.22:0/916598436 >> 172.21.15.22:0/916598436 conn(0x7fb918100760 msgr2=0x7fb918102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:40.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.035+0000 7fb91d400700 1 -- 172.21.15.22:0/916598436 shutdown_connections 2024-04-20T21:32:40.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.035+0000 7fb91d400700 1 -- 172.21.15.22:0/916598436 wait complete. 2024-04-20T21:32:40.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.036+0000 7fb91d400700 1 Processor -- start 2024-04-20T21:32:40.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.036+0000 7fb91d400700 1 -- start start 2024-04-20T21:32:40.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.036+0000 7fb91d400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb918104ed0 0x7fb91819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:40.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.036+0000 7fb91d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb918105880 0x7fb91819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:40.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.036+0000 7fb91d400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb918109fb0 0x7fb91819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:40.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.036+0000 7fb91d400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb918114fc0 con 0x7fb918109fb0 2024-04-20T21:32:40.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.036+0000 7fb91d400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb918114e40 con 0x7fb918105880 2024-04-20T21:32:40.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.036+0000 7fb91d400700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb918115140 con 0x7fb918104ed0 2024-04-20T21:32:40.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.037+0000 7fb916800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb918104ed0 0x7fb91819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:40.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.037+0000 7fb916800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb918104ed0 0x7fb91819b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:48048/0 (socket says 172.21.15.22:48048) 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.037+0000 7fb916800700 1 -- 172.21.15.22:0/507584877 learned_addr learned my addr 172.21.15.22:0/507584877 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.037+0000 7fb915e00700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb918105880 0x7fb91819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.037+0000 7fb917200700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb918109fb0 0x7fb91819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.037+0000 7fb915e00700 1 -- 172.21.15.22:0/507584877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb918104ed0 msgr2=0x7fb91819b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.037+0000 7fb915e00700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb918104ed0 0x7fb91819b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.037+0000 7fb915e00700 1 -- 172.21.15.22:0/507584877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb918109fb0 msgr2=0x7fb91819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.037+0000 7fb915e00700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb918109fb0 0x7fb91819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.037+0000 7fb915e00700 1 -- 172.21.15.22:0/507584877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9181a0440 con 0x7fb918105880 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.038+0000 7fb916800700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb918104ed0 0x7fb91819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.038+0000 7fb915e00700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb918105880 0x7fb91819b970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb90400e9c0 tx=0x7fb90400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.038+0000 7fb90f400700 1 -- 172.21.15.22:0/507584877 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90400cbd0 con 0x7fb918105880 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.038+0000 7fb90f400700 1 -- 172.21.15.22:0/507584877 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb904004500 con 0x7fb918105880 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.038+0000 7fb91d400700 1 -- 172.21.15.22:0/507584877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9181a0730 con 0x7fb918105880 2024-04-20T21:32:40.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.038+0000 7fb91d400700 1 -- 172.21.15.22:0/507584877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9181a7fb0 con 0x7fb918105880 2024-04-20T21:32:40.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.038+0000 7fb90f400700 1 -- 172.21.15.22:0/507584877 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904005280 con 0x7fb918105880 2024-04-20T21:32:40.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.040+0000 7fb91d400700 1 -- 172.21.15.22:0/507584877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f8005360 con 0x7fb918105880 2024-04-20T21:32:40.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.042+0000 7fb90f400700 1 -- 172.21.15.22:0/507584877 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb904005420 con 0x7fb918105880 2024-04-20T21:32:40.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.043+0000 7fb90f400700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb8fc0a3d50 0x7fb8fc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:40.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.043+0000 7fb90f400700 1 -- 172.21.15.22:0/507584877 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fb9040d0380 con 0x7fb918105880 2024-04-20T21:32:40.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.043+0000 7fb916800700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb8fc0a3d50 0x7fb8fc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:40.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.044+0000 7fb916800700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb8fc0a3d50 0x7fb8fc0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb908009fd0 tx=0x7fb90800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:40.046 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:40.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.047+0000 7fb90f400700 1 -- 172.21.15.22:0/507584877 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb904016330 con 0x7fb918105880 2024-04-20T21:32:40.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.292+0000 7fb91d400700 1 -- 172.21.15.22:0/507584877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb8f8005920 con 0x7fb918105880 2024-04-20T21:32:40.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.293+0000 7fb90f400700 1 -- 172.21.15.22:0/507584877 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb90409cb80 con 0x7fb918105880 2024-04-20T21:32:40.292 INFO:teuthology.orchestra.run.smithi022.stdout:34359738457 2024-04-20T21:32:40.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.296+0000 7fb91d400700 1 -- 172.21.15.22:0/507584877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb8fc0a3d50 msgr2=0x7fb8fc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:40.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.296+0000 7fb91d400700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb8fc0a3d50 0x7fb8fc0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb908009fd0 tx=0x7fb90800b040 comp rx=0 tx=0).stop 2024-04-20T21:32:40.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.296+0000 7fb91d400700 1 -- 172.21.15.22:0/507584877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb918105880 msgr2=0x7fb91819b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:40.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.296+0000 7fb91d400700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb918105880 0x7fb91819b970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb90400e9c0 tx=0x7fb90400ecd0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.296+0000 7fb91d400700 1 -- 172.21.15.22:0/507584877 shutdown_connections 2024-04-20T21:32:40.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.296+0000 7fb91d400700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb8fc0a3d50 0x7fb8fc0a6210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.296+0000 7fb91d400700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb918109fb0 0x7fb91819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.296+0000 7fb91d400700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb918104ed0 0x7fb91819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.296+0000 7fb91d400700 1 --2- 172.21.15.22:0/507584877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb918105880 0x7fb91819b970 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.296+0000 7fb91d400700 1 -- 172.21.15.22:0/507584877 >> 172.21.15.22:0/507584877 conn(0x7fb918100760 msgr2=0x7fb9181024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:40.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.297+0000 7fb91d400700 1 -- 172.21.15.22:0/507584877 shutdown_connections 2024-04-20T21:32:40.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:40.297+0000 7fb91d400700 1 -- 172.21.15.22:0/507584877 wait complete. 2024-04-20T21:32:40.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:40 smithi022 ceph-mon[16843]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:40.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:40 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/507584877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T21:32:41.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:40 smithi122 ceph-mon[22147]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:41.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:40 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/507584877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T21:32:41.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:40 smithi033 ceph-mon[21626]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:41.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:40 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/507584877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T21:32:41.293 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266732 got 227633266736 for osd.7 2024-04-20T21:32:41.293 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:41.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.499+0000 7f0457200700 1 -- 172.21.15.22:0/1438158987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04580757a0 msgr2=0x7f045810ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.499+0000 7f0457200700 1 --2- 172.21.15.22:0/1438158987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04580757a0 0x7f045810ec60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0444009990 tx=0x7f0444009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.499+0000 7f0457200700 1 -- 172.21.15.22:0/1438158987 shutdown_connections 2024-04-20T21:32:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.499+0000 7f0457200700 1 --2- 172.21.15.22:0/1438158987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045810f1a0 0x7f0458111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.499+0000 7f0457200700 1 --2- 172.21.15.22:0/1438158987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0458074e80 0x7f0458075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.499+0000 7f0457200700 1 --2- 172.21.15.22:0/1438158987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04580757a0 0x7f045810ec60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.499+0000 7f0457200700 1 -- 172.21.15.22:0/1438158987 >> 172.21.15.22:0/1438158987 conn(0x7f04580fe470 msgr2=0x7f0458100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.499+0000 7f0457200700 1 -- 172.21.15.22:0/1438158987 shutdown_connections 2024-04-20T21:32:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.499+0000 7f0457200700 1 -- 172.21.15.22:0/1438158987 wait complete. 2024-04-20T21:32:41.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.500+0000 7f0457200700 1 Processor -- start 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.500+0000 7f0457200700 1 -- start start 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.501+0000 7f0457200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0458074e80 0x7f045819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.501+0000 7f0457200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04580757a0 0x7f045819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.501+0000 7f0457200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045810f1a0 0x7f045819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.501+0000 7f0457200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0458114e90 con 0x7f04580757a0 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.501+0000 7f0457200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0458114d10 con 0x7f045810f1a0 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.501+0000 7f0457200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0458115010 con 0x7f0458074e80 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.501+0000 7f0455e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0458074e80 0x7f045819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.501+0000 7f0455e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0458074e80 0x7f045819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:48060/0 (socket says 172.21.15.22:48060) 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.501+0000 7f0455e00700 1 -- 172.21.15.22:0/2542114661 learned_addr learned my addr 172.21.15.22:0/2542114661 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.501+0000 7f0455400700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04580757a0 0x7f045819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.501+0000 7f0456800700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045810f1a0 0x7f045819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.502+0000 7f0455e00700 1 -- 172.21.15.22:0/2542114661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045810f1a0 msgr2=0x7f045819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.502+0000 7f0455e00700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045810f1a0 0x7f045819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.502+0000 7f0455e00700 1 -- 172.21.15.22:0/2542114661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04580757a0 msgr2=0x7f045819b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.502+0000 7f0455e00700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04580757a0 0x7f045819b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.502+0000 7f0455e00700 1 -- 172.21.15.22:0/2542114661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04581a03a0 con 0x7f0458074e80 2024-04-20T21:32:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.502+0000 7f0456800700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045810f1a0 0x7f045819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:32:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.502+0000 7f0455e00700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0458074e80 0x7f045819b3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f044800e9c0 tx=0x7f044800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.502+0000 7f044ea00700 1 -- 172.21.15.22:0/2542114661 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044800cbd0 con 0x7f0458074e80 2024-04-20T21:32:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.502+0000 7f0457200700 1 -- 172.21.15.22:0/2542114661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04581a0690 con 0x7f0458074e80 2024-04-20T21:32:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.502+0000 7f044ea00700 1 -- 172.21.15.22:0/2542114661 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044800cd70 con 0x7f0458074e80 2024-04-20T21:32:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.502+0000 7f044ea00700 1 -- 172.21.15.22:0/2542114661 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448010430 con 0x7f0458074e80 2024-04-20T21:32:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.502+0000 7f0457200700 1 -- 172.21.15.22:0/2542114661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04581a7f10 con 0x7f0458074e80 2024-04-20T21:32:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.504+0000 7f0457200700 1 -- 172.21.15.22:0/2542114661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0458069740 con 0x7f0458074e80 2024-04-20T21:32:41.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.506+0000 7f044ea00700 1 -- 172.21.15.22:0/2542114661 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0448004750 con 0x7f0458074e80 2024-04-20T21:32:41.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.507+0000 7f044ea00700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f043c0a3cd0 0x7f043c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:41.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.507+0000 7f044ea00700 1 -- 172.21.15.22:0/2542114661 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f04480d08b0 con 0x7f0458074e80 2024-04-20T21:32:41.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.508+0000 7f0455400700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f043c0a3cd0 0x7f043c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:41.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.508+0000 7f0455400700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f043c0a3cd0 0x7f043c0a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04440098e0 tx=0x7f0444005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:41.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.512+0000 7f044ea00700 1 -- 172.21.15.22:0/2542114661 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f044809d0b0 con 0x7f0458074e80 2024-04-20T21:32:41.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.759+0000 7f0457200700 1 -- 172.21.15.22:0/2542114661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f045819c6f0 con 0x7f0458074e80 2024-04-20T21:32:41.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.760+0000 7f044ea00700 1 -- 172.21.15.22:0/2542114661 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f044809c9b0 con 0x7f0458074e80 2024-04-20T21:32:41.759 INFO:teuthology.orchestra.run.smithi022.stdout:300647710759 2024-04-20T21:32:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.763+0000 7f0457200700 1 -- 172.21.15.22:0/2542114661 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f043c0a3cd0 msgr2=0x7f043c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.763+0000 7f0457200700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f043c0a3cd0 0x7f043c0a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04440098e0 tx=0x7f0444005960 comp rx=0 tx=0).stop 2024-04-20T21:32:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.763+0000 7f0457200700 1 -- 172.21.15.22:0/2542114661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0458074e80 msgr2=0x7f045819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.763+0000 7f0457200700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0458074e80 0x7f045819b3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f044800e9c0 tx=0x7f044800ecd0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.763+0000 7f0457200700 1 -- 172.21.15.22:0/2542114661 shutdown_connections 2024-04-20T21:32:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.763+0000 7f0457200700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f043c0a3cd0 0x7f043c0a6190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.763+0000 7f0457200700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f045810f1a0 0x7f045819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.763+0000 7f0457200700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0458074e80 0x7f045819b3e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.763+0000 7f0457200700 1 --2- 172.21.15.22:0/2542114661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04580757a0 0x7f045819b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.763+0000 7f0457200700 1 -- 172.21.15.22:0/2542114661 >> 172.21.15.22:0/2542114661 conn(0x7f04580fe470 msgr2=0x7f04580ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.763+0000 7f0457200700 1 -- 172.21.15.22:0/2542114661 shutdown_connections 2024-04-20T21:32:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.763+0000 7f0457200700 1 -- 172.21.15.22:0/2542114661 wait complete. 2024-04-20T21:32:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.987+0000 7f267fe00700 1 -- 172.21.15.22:0/3730656820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f267806a760 msgr2=0x7f267806abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.987+0000 7f267fe00700 1 --2- 172.21.15.22:0/3730656820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f267806a760 0x7f267806abc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f266400b0a0 tx=0x7f266400b3b0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.987+0000 7f267fe00700 1 -- 172.21.15.22:0/3730656820 shutdown_connections 2024-04-20T21:32:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.987+0000 7f267fe00700 1 --2- 172.21.15.22:0/3730656820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806b100 0x7f267810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.987+0000 7f267fe00700 1 --2- 172.21.15.22:0/3730656820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2678108a40 0x7f2678108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.987+0000 7f267fe00700 1 --2- 172.21.15.22:0/3730656820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f267806a760 0x7f267806abc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.987+0000 7f267fe00700 1 -- 172.21.15.22:0/3730656820 >> 172.21.15.22:0/3730656820 conn(0x7f2678078900 msgr2=0x7f2678078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.988+0000 7f267fe00700 1 -- 172.21.15.22:0/3730656820 shutdown_connections 2024-04-20T21:32:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.988+0000 7f267fe00700 1 -- 172.21.15.22:0/3730656820 wait complete. 2024-04-20T21:32:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.989+0000 7f267fe00700 1 Processor -- start 2024-04-20T21:32:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.990+0000 7f267fe00700 1 -- start start 2024-04-20T21:32:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.990+0000 7f267fe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806a760 0x7f2678101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.990+0000 7f267fe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267806b100 0x7f2678101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.990+0000 7f267fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678108a40 0x7f26781024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.990+0000 7f267fe00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2678112d30 con 0x7f267806a760 2024-04-20T21:32:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.990+0000 7f267fe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2678112bb0 con 0x7f2678108a40 2024-04-20T21:32:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.990+0000 7f267fe00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2678112eb0 con 0x7f267806b100 2024-04-20T21:32:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.990+0000 7f267d400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806a760 0x7f2678101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.990+0000 7f267d400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806a760 0x7f2678101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55012/0 (socket says 172.21.15.22:55012) 2024-04-20T21:32:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.990+0000 7f267d400700 1 -- 172.21.15.22:0/2576964181 learned_addr learned my addr 172.21.15.22:0/2576964181 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.990+0000 7f267ca00700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267806b100 0x7f2678101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.991+0000 7f267de00700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678108a40 0x7f26781024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.991+0000 7f267d400700 1 -- 172.21.15.22:0/2576964181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267806b100 msgr2=0x7f2678101f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.991+0000 7f267d400700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267806b100 0x7f2678101f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.991+0000 7f267d400700 1 -- 172.21.15.22:0/2576964181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678108a40 msgr2=0x7f26781024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.991+0000 7f267d400700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678108a40 0x7f26781024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.991+0000 7f267d400700 1 -- 172.21.15.22:0/2576964181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26781a7740 con 0x7f267806a760 2024-04-20T21:32:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.991+0000 7f267ca00700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267806b100 0x7f2678101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.991+0000 7f267de00700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678108a40 0x7f26781024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.991+0000 7f267d400700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806a760 0x7f2678101a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f266800e9a0 tx=0x7f266800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.992+0000 7f2676000700 1 -- 172.21.15.22:0/2576964181 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266800ca10 con 0x7f267806a760 2024-04-20T21:32:41.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.992+0000 7f2676000700 1 -- 172.21.15.22:0/2576964181 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f266800cbb0 con 0x7f267806a760 2024-04-20T21:32:41.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.992+0000 7f267fe00700 1 -- 172.21.15.22:0/2576964181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26781a7a30 con 0x7f267806a760 2024-04-20T21:32:41.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.992+0000 7f2676000700 1 -- 172.21.15.22:0/2576964181 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2668018680 con 0x7f267806a760 2024-04-20T21:32:41.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.992+0000 7f267fe00700 1 -- 172.21.15.22:0/2576964181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26781a7f70 con 0x7f267806a760 2024-04-20T21:32:41.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.993+0000 7f267fe00700 1 -- 172.21.15.22:0/2576964181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2678069740 con 0x7f267806a760 2024-04-20T21:32:41.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.995+0000 7f2676000700 1 -- 172.21.15.22:0/2576964181 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2668010430 con 0x7f267806a760 2024-04-20T21:32:41.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.996+0000 7f2676000700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f265c0a3cd0 0x7f265c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.997+0000 7f2676000700 1 -- 172.21.15.22:0/2576964181 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f26680d0bd0 con 0x7f267806a760 2024-04-20T21:32:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.997+0000 7f267ca00700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f265c0a3cd0 0x7f265c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:41.998+0000 7f267ca00700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f265c0a3cd0 0x7f265c0a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2678078f70 tx=0x7f266400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:41.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.000+0000 7f2676000700 1 -- 172.21.15.22:0/2576964181 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f266809d3d0 con 0x7f267806a760 2024-04-20T21:32:42.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:41 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2542114661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-20T21:32:42.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.237+0000 7f267fe00700 1 -- 172.21.15.22:0/2576964181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f267802e150 con 0x7f267806a760 2024-04-20T21:32:42.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.238+0000 7f2676000700 1 -- 172.21.15.22:0/2576964181 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f266809ccd0 con 0x7f267806a760 2024-04-20T21:32:42.237 INFO:teuthology.orchestra.run.smithi022.stdout:206158430266 2024-04-20T21:32:42.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.240+0000 7f267fe00700 1 -- 172.21.15.22:0/2576964181 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f265c0a3cd0 msgr2=0x7f265c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:42.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.240+0000 7f267fe00700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f265c0a3cd0 0x7f265c0a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2678078f70 tx=0x7f266400b700 comp rx=0 tx=0).stop 2024-04-20T21:32:42.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.241+0000 7f267fe00700 1 -- 172.21.15.22:0/2576964181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806a760 msgr2=0x7f2678101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:42.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.241+0000 7f267fe00700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806a760 0x7f2678101a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f266800e9a0 tx=0x7f266800ecb0 comp rx=0 tx=0).stop 2024-04-20T21:32:42.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.241+0000 7f267fe00700 1 -- 172.21.15.22:0/2576964181 shutdown_connections 2024-04-20T21:32:42.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.241+0000 7f267fe00700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f265c0a3cd0 0x7f265c0a6190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:42.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.241+0000 7f267fe00700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678108a40 0x7f26781024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:42.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.241+0000 7f267fe00700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f267806a760 0x7f2678101a20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:42.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.241+0000 7f267fe00700 1 --2- 172.21.15.22:0/2576964181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267806b100 0x7f2678101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:42.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.241+0000 7f267fe00700 1 -- 172.21.15.22:0/2576964181 >> 172.21.15.22:0/2576964181 conn(0x7f2678078900 msgr2=0x7f26780ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:42.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.241+0000 7f267fe00700 1 -- 172.21.15.22:0/2576964181 shutdown_connections 2024-04-20T21:32:42.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:42.241+0000 7f267fe00700 1 -- 172.21.15.22:0/2576964181 wait complete. 2024-04-20T21:32:42.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:41 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2542114661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-20T21:32:42.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:41 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2542114661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-20T21:32:42.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:42 smithi022 ceph-mon[16843]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:42.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:42 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2576964181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-20T21:32:43.176 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:42 smithi122 ceph-mon[22147]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:43.176 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:42 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2576964181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-20T21:32:43.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:42 smithi033 ceph-mon[21626]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:43.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:42 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2576964181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-20T21:32:43.395 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710754 got 300647710759 for osd.9 2024-04-20T21:32:43.395 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:44.954 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:44 smithi022 ceph-mon[16843]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:44.954 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:44 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1066329877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-20T21:32:44.998 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430260 got 206158430266 for osd.6 2024-04-20T21:32:44.998 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:45.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:44 smithi122 ceph-mon[22147]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:45.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:44 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1066329877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-20T21:32:45.276 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738457 for osd.0 2024-04-20T21:32:45.276 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:45.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:44 smithi033 ceph-mon[21626]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-20T21:32:45.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:44 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1066329877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-20T21:32:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.169+0000 7f2ef3200700 1 -- 172.21.15.22:0/1372401002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ef4109ea0 msgr2=0x7f2ef4111a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.169+0000 7f2ef3200700 1 --2- 172.21.15.22:0/1372401002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ef4109ea0 0x7f2ef4111a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b0d0 tx=0x7f2eec00b3e0 comp rx=0 tx=0).stop 2024-04-20T21:32:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.169+0000 7f2ef3200700 1 -- 172.21.15.22:0/1372401002 shutdown_connections 2024-04-20T21:32:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.169+0000 7f2ef3200700 1 --2- 172.21.15.22:0/1372401002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ef4109ea0 0x7f2ef4111a70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.169+0000 7f2ef3200700 1 --2- 172.21.15.22:0/1372401002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ef4104dc0 0x7f2ef41051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.169+0000 7f2ef3200700 1 --2- 172.21.15.22:0/1372401002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ef4105770 0x7f2ef41097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.169+0000 7f2ef3200700 1 -- 172.21.15.22:0/1372401002 >> 172.21.15.22:0/1372401002 conn(0x7f2ef4100690 msgr2=0x7f2ef4102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.169+0000 7f2ef3200700 1 -- 172.21.15.22:0/1372401002 shutdown_connections 2024-04-20T21:32:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.169+0000 7f2ef3200700 1 -- 172.21.15.22:0/1372401002 wait complete. 2024-04-20T21:32:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.170+0000 7f2ef3200700 1 Processor -- start 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.170+0000 7f2ef3200700 1 -- start start 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.170+0000 7f2ef3200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ef4104dc0 0x7f2ef419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.170+0000 7f2ef3200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ef4105770 0x7f2ef419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef3200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ef4109ea0 0x7f2ef419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef3200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4114ea0 con 0x7f2ef4109ea0 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef3200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4114d20 con 0x7f2ef4105770 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef3200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4115020 con 0x7f2ef4104dc0 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef1e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ef4104dc0 0x7f2ef419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef1e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ef4104dc0 0x7f2ef419b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:41142/0 (socket says 172.21.15.22:41142) 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef1e00700 1 -- 172.21.15.22:0/1066329877 learned_addr learned my addr 172.21.15.22:0/1066329877 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef2800700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ef4109ea0 0x7f2ef419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef1400700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ef4105770 0x7f2ef419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef1e00700 1 -- 172.21.15.22:0/1066329877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ef4105770 msgr2=0x7f2ef419b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:45.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef1e00700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ef4105770 0x7f2ef419b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:45.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef1e00700 1 -- 172.21.15.22:0/1066329877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ef4109ea0 msgr2=0x7f2ef419fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:45.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef1e00700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ef4109ea0 0x7f2ef419fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:45.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.171+0000 7f2ef1e00700 1 -- 172.21.15.22:0/1066329877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef41a0280 con 0x7f2ef4104dc0 2024-04-20T21:32:45.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.172+0000 7f2ef1e00700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ef4104dc0 0x7f2ef419b270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400e9a0 tx=0x7f2ee400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:45.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.172+0000 7f2eeaa00700 1 -- 172.21.15.22:0/1066329877 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee400ca10 con 0x7f2ef4104dc0 2024-04-20T21:32:45.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.172+0000 7f2eeaa00700 1 -- 172.21.15.22:0/1066329877 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee400cbb0 con 0x7f2ef4104dc0 2024-04-20T21:32:45.667 INFO:teuthology.orchestra.run.smithi022.stdout:266287972399 2024-04-20T21:32:45.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.172+0000 7f2ef3200700 1 -- 172.21.15.22:0/1066329877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef41a0570 con 0x7f2ef4104dc0 2024-04-20T21:32:45.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.172+0000 7f2eeaa00700 1 -- 172.21.15.22:0/1066329877 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4018680 con 0x7f2ef4104dc0 2024-04-20T21:32:45.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.172+0000 7f2ef3200700 1 -- 172.21.15.22:0/1066329877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef41a7df0 con 0x7f2ef4104dc0 2024-04-20T21:32:45.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.173+0000 7f2ef3200700 1 -- 172.21.15.22:0/1066329877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef4069740 con 0x7f2ef4104dc0 2024-04-20T21:32:45.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.175+0000 7f2eeaa00700 1 -- 172.21.15.22:0/1066329877 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2ee4010430 con 0x7f2ef4104dc0 2024-04-20T21:32:45.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.176+0000 7f2eeaa00700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2ed80a3d50 0x7f2ed80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:45.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.177+0000 7f2eeaa00700 1 -- 172.21.15.22:0/1066329877 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f2ee40d0790 con 0x7f2ef4104dc0 2024-04-20T21:32:45.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.177+0000 7f2ef1400700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2ed80a3d50 0x7f2ed80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:45.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.177+0000 7f2ef1400700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2ed80a3d50 0x7f2ed80a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4103cf0 tx=0x7f2ee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.180+0000 7f2eeaa00700 1 -- 172.21.15.22:0/1066329877 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2ee409cf90 con 0x7f2ef4104dc0 2024-04-20T21:32:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.419+0000 7f2ef3200700 1 -- 172.21.15.22:0/1066329877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f2ef41a7fc0 con 0x7f2ef4104dc0 2024-04-20T21:32:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.420+0000 7f2eeaa00700 1 -- 172.21.15.22:0/1066329877 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2ee4085e20 con 0x7f2ef4104dc0 2024-04-20T21:32:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.424+0000 7f2ef3200700 1 -- 172.21.15.22:0/1066329877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2ed80a3d50 msgr2=0x7f2ed80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.424+0000 7f2ef3200700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2ed80a3d50 0x7f2ed80a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4103cf0 tx=0x7f2ee0009260 comp rx=0 tx=0).stop 2024-04-20T21:32:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.424+0000 7f2ef3200700 1 -- 172.21.15.22:0/1066329877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ef4104dc0 msgr2=0x7f2ef419b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.424+0000 7f2ef3200700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ef4104dc0 0x7f2ef419b270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400e9a0 tx=0x7f2ee400ecb0 comp rx=0 tx=0).stop 2024-04-20T21:32:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.424+0000 7f2ef3200700 1 -- 172.21.15.22:0/1066329877 shutdown_connections 2024-04-20T21:32:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.424+0000 7f2ef3200700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2ed80a3d50 0x7f2ed80a6210 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.424+0000 7f2ef3200700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ef4109ea0 0x7f2ef419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.424+0000 7f2ef3200700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ef4104dc0 0x7f2ef419b270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:45.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.424+0000 7f2ef3200700 1 --2- 172.21.15.22:0/1066329877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ef4105770 0x7f2ef419b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:45.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.424+0000 7f2ef3200700 1 -- 172.21.15.22:0/1066329877 >> 172.21.15.22:0/1066329877 conn(0x7f2ef4100690 msgr2=0x7f2ef41010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:45.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.424+0000 7f2ef3200700 1 -- 172.21.15.22:0/1066329877 shutdown_connections 2024-04-20T21:32:45.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:44.424+0000 7f2ef3200700 1 -- 172.21.15.22:0/1066329877 wait complete. 2024-04-20T21:32:46.475 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972394 got 266287972399 for osd.8 2024-04-20T21:32:46.475 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:46.836 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411407 for osd.2 2024-04-20T21:32:46.837 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:47.056 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:46 smithi022 ceph-mon[16843]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T21:32:47.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:46 smithi122 ceph-mon[22147]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T21:32:47.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:46 smithi033 ceph-mon[21626]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T21:32:48.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.374+0000 7f302ee00700 1 -- 172.21.15.22:0/545629771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30281019e0 msgr2=0x7f302810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:48.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.374+0000 7f302ee00700 1 --2- 172.21.15.22:0/545629771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30281019e0 0x7f302810f920 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f302000b0a0 tx=0x7f302000b3b0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.375+0000 7f302ee00700 1 -- 172.21.15.22:0/545629771 shutdown_connections 2024-04-20T21:32:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.375+0000 7f302ee00700 1 --2- 172.21.15.22:0/545629771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30281019e0 0x7f302810f920 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.375+0000 7f302ee00700 1 --2- 172.21.15.22:0/545629771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3028108ae0 0x7f3028108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.375+0000 7f302ee00700 1 --2- 172.21.15.22:0/545629771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3028101040 0x7f30281014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.375+0000 7f302ee00700 1 -- 172.21.15.22:0/545629771 >> 172.21.15.22:0/545629771 conn(0x7f3028077fc0 msgr2=0x7f30280ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.375+0000 7f302ee00700 1 -- 172.21.15.22:0/545629771 shutdown_connections 2024-04-20T21:32:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.375+0000 7f302ee00700 1 -- 172.21.15.22:0/545629771 wait complete. 2024-04-20T21:32:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.376+0000 7f302ee00700 1 Processor -- start 2024-04-20T21:32:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.376+0000 7f302ee00700 1 -- start start 2024-04-20T21:32:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.376+0000 7f302ee00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3028101040 0x7f3028102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.376+0000 7f302ee00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30281019e0 0x7f3028102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.376+0000 7f302ee00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3028108ae0 0x7f3028102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f302ee00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3028112d80 con 0x7f30281019e0 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f302ee00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3028112c00 con 0x7f3028101040 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f302ee00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3028112f00 con 0x7f3028108ae0 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f3027e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3028101040 0x7f3028102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f3027e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3028101040 0x7f3028102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38792/0 (socket says 172.21.15.22:38792) 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f3027e00700 1 -- 172.21.15.22:0/2967876657 learned_addr learned my addr 172.21.15.22:0/2967876657 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f3027400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30281019e0 0x7f3028102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f302ce00700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3028108ae0 0x7f3028102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f3027e00700 1 -- 172.21.15.22:0/2967876657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3028108ae0 msgr2=0x7f3028102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f3027e00700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3028108ae0 0x7f3028102ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f3027e00700 1 -- 172.21.15.22:0/2967876657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30281019e0 msgr2=0x7f3028102760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f3027e00700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30281019e0 0x7f3028102760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.377+0000 7f3027e00700 1 -- 172.21.15.22:0/2967876657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30281a7760 con 0x7f3028101040 2024-04-20T21:32:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.378+0000 7f3027e00700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3028101040 0x7f3028102220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f301800e9c0 tx=0x7f301800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.378+0000 7f3024c00700 1 -- 172.21.15.22:0/2967876657 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301800cbd0 con 0x7f3028101040 2024-04-20T21:32:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.378+0000 7f302ee00700 1 -- 172.21.15.22:0/2967876657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30281a7a50 con 0x7f3028101040 2024-04-20T21:32:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.378+0000 7f3024c00700 1 -- 172.21.15.22:0/2967876657 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301800ee60 con 0x7f3028101040 2024-04-20T21:32:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.378+0000 7f3024c00700 1 -- 172.21.15.22:0/2967876657 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018005630 con 0x7f3028101040 2024-04-20T21:32:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.378+0000 7f302ee00700 1 -- 172.21.15.22:0/2967876657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30281a7f40 con 0x7f3028101040 2024-04-20T21:32:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.380+0000 7f302ee00700 1 -- 172.21.15.22:0/2967876657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3028104360 con 0x7f3028101040 2024-04-20T21:32:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.382+0000 7f3024c00700 1 -- 172.21.15.22:0/2967876657 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3018004750 con 0x7f3028101040 2024-04-20T21:32:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.383+0000 7f3024c00700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f300c0a3cd0 0x7f300c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.383+0000 7f3024c00700 1 -- 172.21.15.22:0/2967876657 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f30180d07e0 con 0x7f3028101040 2024-04-20T21:32:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.383+0000 7f3027400700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f300c0a3cd0 0x7f300c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:48.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.384+0000 7f3027400700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f300c0a3cd0 0x7f300c0a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3014009680 tx=0x7f3014009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:48.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.387+0000 7f3024c00700 1 -- 172.21.15.22:0/2967876657 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f301809cfe0 con 0x7f3028101040 2024-04-20T21:32:48.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.640+0000 7f302ee00700 1 -- 172.21.15.22:0/2967876657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f302802df90 con 0x7f3028101040 2024-04-20T21:32:48.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.641+0000 7f3024c00700 1 -- 172.21.15.22:0/2967876657 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f301809c8e0 con 0x7f3028101040 2024-04-20T21:32:48.640 INFO:teuthology.orchestra.run.smithi022.stdout:322122547234 2024-04-20T21:32:48.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.645+0000 7f302ee00700 1 -- 172.21.15.22:0/2967876657 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f300c0a3cd0 msgr2=0x7f300c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:48.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.645+0000 7f302ee00700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f300c0a3cd0 0x7f300c0a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3014009680 tx=0x7f3014009260 comp rx=0 tx=0).stop 2024-04-20T21:32:48.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.645+0000 7f302ee00700 1 -- 172.21.15.22:0/2967876657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3028101040 msgr2=0x7f3028102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:48.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.645+0000 7f302ee00700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3028101040 0x7f3028102220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f301800e9c0 tx=0x7f301800ecd0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.645+0000 7f302ee00700 1 -- 172.21.15.22:0/2967876657 shutdown_connections 2024-04-20T21:32:48.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.645+0000 7f302ee00700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f300c0a3cd0 0x7f300c0a6190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.645+0000 7f302ee00700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3028108ae0 0x7f3028102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.645+0000 7f302ee00700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3028101040 0x7f3028102220 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.645+0000 7f302ee00700 1 --2- 172.21.15.22:0/2967876657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30281019e0 0x7f3028102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.645+0000 7f302ee00700 1 -- 172.21.15.22:0/2967876657 >> 172.21.15.22:0/2967876657 conn(0x7f3028077fc0 msgr2=0x7f30280ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:48.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.645+0000 7f302ee00700 1 -- 172.21.15.22:0/2967876657 shutdown_connections 2024-04-20T21:32:48.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:48.646+0000 7f302ee00700 1 -- 172.21.15.22:0/2967876657 wait complete. 2024-04-20T21:32:48.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:48 smithi022 ceph-mon[16843]: pgmap v359: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:48.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:48 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2967876657' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-20T21:32:49.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:48 smithi122 ceph-mon[22147]: pgmap v359: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:49.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:48 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2967876657' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-20T21:32:49.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:48 smithi033 ceph-mon[21626]: pgmap v359: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:49.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:48 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2967876657' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-20T21:32:49.462 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547229 got 322122547234 for osd.10 2024-04-20T21:32:49.462 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:49.462 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-20T21:32:49.463 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-20T21:32:49.904 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:51.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:50 smithi122 ceph-mon[22147]: pgmap v360: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:51.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:51.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:50 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:51.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:50 smithi033 ceph-mon[21626]: pgmap v360: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:51.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:51.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:50 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:51.084 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:50 smithi022 ceph-mon[16843]: pgmap v360: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:51.085 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:51.085 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:50 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.979+0000 7ff5d0000700 1 -- 172.21.15.22:0/190464558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5c8076f80 msgr2=0x7ff5c8077360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.979+0000 7ff5d0000700 1 --2- 172.21.15.22:0/190464558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5c8076f80 0x7ff5c8077360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0098f0 tx=0x7ff5bc009c00 comp rx=0 tx=0).stop 2024-04-20T21:32:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.979+0000 7ff5d0000700 1 -- 172.21.15.22:0/190464558 shutdown_connections 2024-04-20T21:32:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.979+0000 7ff5d0000700 1 --2- 172.21.15.22:0/190464558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c81138e0 0x7ff5c8115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.979+0000 7ff5d0000700 1 --2- 172.21.15.22:0/190464558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5c8076f80 0x7ff5c8077360 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.979+0000 7ff5d0000700 1 --2- 172.21.15.22:0/190464558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5c80778a0 0x7ff5c81133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.979+0000 7ff5d0000700 1 -- 172.21.15.22:0/190464558 >> 172.21.15.22:0/190464558 conn(0x7ff5c8100730 msgr2=0x7ff5c8102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.979+0000 7ff5d0000700 1 -- 172.21.15.22:0/190464558 shutdown_connections 2024-04-20T21:32:51.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.979+0000 7ff5d0000700 1 -- 172.21.15.22:0/190464558 wait complete. 2024-04-20T21:32:51.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:32:51.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:32:51.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:51 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:51.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.980+0000 7ff5d0000700 1 Processor -- start 2024-04-20T21:32:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.981+0000 7ff5d0000700 1 -- start start 2024-04-20T21:32:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.981+0000 7ff5d0000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5c8076f80 0x7ff5c81a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.981+0000 7ff5d0000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5c80778a0 0x7ff5c81a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.981+0000 7ff5d0000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c81138e0 0x7ff5c819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.981+0000 7ff5d0000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c81193f0 con 0x7ff5c81138e0 2024-04-20T21:32:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.981+0000 7ff5d0000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8119270 con 0x7ff5c80778a0 2024-04-20T21:32:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.981+0000 7ff5d0000700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8119570 con 0x7ff5c8076f80 2024-04-20T21:32:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.982+0000 7ff5cd600700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5c8076f80 0x7ff5c81a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.982+0000 7ff5cd600700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5c8076f80 0x7ff5c81a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:41174/0 (socket says 172.21.15.22:41174) 2024-04-20T21:32:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.982+0000 7ff5cd600700 1 -- 172.21.15.22:0/1560691674 learned_addr learned my addr 172.21.15.22:0/1560691674 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:51.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.982+0000 7ff5ccc00700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5c80778a0 0x7ff5c81a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.982+0000 7ff5ce000700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c81138e0 0x7ff5c819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.982+0000 7ff5cd600700 1 -- 172.21.15.22:0/1560691674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5c80778a0 msgr2=0x7ff5c81a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.982+0000 7ff5cd600700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5c80778a0 0x7ff5c81a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.982+0000 7ff5cd600700 1 -- 172.21.15.22:0/1560691674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c81138e0 msgr2=0x7ff5c819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.982+0000 7ff5cd600700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c81138e0 0x7ff5c819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.982+0000 7ff5cd600700 1 -- 172.21.15.22:0/1560691674 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c819fff0 con 0x7ff5c8076f80 2024-04-20T21:32:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.982+0000 7ff5ce000700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c81138e0 0x7ff5c819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:32:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.983+0000 7ff5cd600700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5c8076f80 0x7ff5c81a55f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc005950 tx=0x7ff5bc005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.983+0000 7ff5c6000700 1 -- 172.21.15.22:0/1560691674 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc0052e0 con 0x7ff5c8076f80 2024-04-20T21:32:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.983+0000 7ff5d0000700 1 -- 172.21.15.22:0/1560691674 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c81a0280 con 0x7ff5c8076f80 2024-04-20T21:32:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.983+0000 7ff5c6000700 1 -- 172.21.15.22:0/1560691674 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5bc00ba40 con 0x7ff5c8076f80 2024-04-20T21:32:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.983+0000 7ff5c6000700 1 -- 172.21.15.22:0/1560691674 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc003cb0 con 0x7ff5c8076f80 2024-04-20T21:32:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.983+0000 7ff5d0000700 1 -- 172.21.15.22:0/1560691674 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c81ac3a0 con 0x7ff5c8076f80 2024-04-20T21:32:51.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.984+0000 7ff5d0000700 1 -- 172.21.15.22:0/1560691674 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c80786c0 con 0x7ff5c8076f80 2024-04-20T21:32:51.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.986+0000 7ff5c6000700 1 -- 172.21.15.22:0/1560691674 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff5bc005510 con 0x7ff5c8076f80 2024-04-20T21:32:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.988+0000 7ff5c6000700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5b00a3cd0 0x7ff5b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.988+0000 7ff5c6000700 1 -- 172.21.15.22:0/1560691674 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7ff5bc0d1620 con 0x7ff5c8076f80 2024-04-20T21:32:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.988+0000 7ff5ccc00700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5b00a3cd0 0x7ff5b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:51.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.989+0000 7ff5ccc00700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5b00a3cd0 0x7ff5b00a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff5c81a0e80 tx=0x7ff5b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:51.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:51.992+0000 7ff5c6000700 1 -- 172.21.15.22:0/1560691674 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff5bc013030 con 0x7ff5c8076f80 2024-04-20T21:32:52.024 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:32:52.024 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:32:52.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:51 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:52.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:32:52.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:32:52.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:51 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' 2024-04-20T21:32:52.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.218+0000 7ff5d0000700 1 -- 172.21.15.22:0/1560691674 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5c81a0bf0 con 0x7ff5b00a3cd0 2024-04-20T21:32:52.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.227+0000 7ff5c6000700 1 -- 172.21.15.22:0/1560691674 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258641 (secure 0 0 0) 0x7ff5c81a0bf0 con 0x7ff5b00a3cd0 2024-04-20T21:32:52.231 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:32:52.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.233+0000 7ff5d0000700 1 -- 172.21.15.22:0/1560691674 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5b00a3cd0 msgr2=0x7ff5b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:52.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.234+0000 7ff5d0000700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5b00a3cd0 0x7ff5b00a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff5c81a0e80 tx=0x7ff5b400b410 comp rx=0 tx=0).stop 2024-04-20T21:32:52.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.234+0000 7ff5d0000700 1 -- 172.21.15.22:0/1560691674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5c8076f80 msgr2=0x7ff5c81a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:52.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.234+0000 7ff5d0000700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5c8076f80 0x7ff5c81a55f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc005950 tx=0x7ff5bc005040 comp rx=0 tx=0).stop 2024-04-20T21:32:52.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.234+0000 7ff5d0000700 1 -- 172.21.15.22:0/1560691674 shutdown_connections 2024-04-20T21:32:52.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.234+0000 7ff5d0000700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5b00a3cd0 0x7ff5b00a6190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:52.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.234+0000 7ff5d0000700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c81138e0 0x7ff5c819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:52.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.234+0000 7ff5d0000700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5c8076f80 0x7ff5c81a55f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:52.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.234+0000 7ff5d0000700 1 --2- 172.21.15.22:0/1560691674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5c80778a0 0x7ff5c81a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:52.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.234+0000 7ff5d0000700 1 -- 172.21.15.22:0/1560691674 >> 172.21.15.22:0/1560691674 conn(0x7ff5c8100730 msgr2=0x7ff5c8102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:52.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.234+0000 7ff5d0000700 1 -- 172.21.15.22:0/1560691674 shutdown_connections 2024-04-20T21:32:52.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:52.234+0000 7ff5d0000700 1 -- 172.21.15.22:0/1560691674 wait complete. 2024-04-20T21:32:52.235 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-20T21:32:53.100 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-04-20T21:32:51.689705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652446,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":306,"num_read_kb":266,"num_write":262,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379368,"kb_used_data":13500,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562456,"statfs":{"total":1151940427776,"available":1151551954944,"internally_reserved":0,"allocated":13824000,"data_stored":7586922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005534"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606348+0000","last_change":"2024-04-20T21:31:27.527151+0000","last_active":"2024-04-20T21:32:31.606348+0000","last_peered":"2024-04-20T21:32:31.606348+0000","last_clean":"2024-04-20T21:32:31.606348+0000","last_became_active":"2024-04-20T21:31:27.526739+0000","last_became_peered":"2024-04-20T21:31:27.526739+0000","last_unstale":"2024-04-20T21:32:31.606348+0000","last_undegraded":"2024-04-20T21:32:31.606348+0000","last_fullsized":"2024-04-20T21:32:31.606348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:16:39.204523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610559+0000","last_change":"2024-04-20T21:31:27.527437+0000","last_active":"2024-04-20T21:32:31.610559+0000","last_peered":"2024-04-20T21:32:31.610559+0000","last_clean":"2024-04-20T21:32:31.610559+0000","last_became_active":"2024-04-20T21:31:27.527040+0000","last_became_peered":"2024-04-20T21:31:27.527040+0000","last_unstale":"2024-04-20T21:32:31.610559+0000","last_undegraded":"2024-04-20T21:32:31.610559+0000","last_fullsized":"2024-04-20T21:32:31.610559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:53:39.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.612462+0000","last_change":"2024-04-20T21:31:27.531671+0000","last_active":"2024-04-20T21:32:31.612462+0000","last_peered":"2024-04-20T21:32:31.612462+0000","last_clean":"2024-04-20T21:32:31.612462+0000","last_became_active":"2024-04-20T21:31:27.531267+0000","last_became_peered":"2024-04-20T21:31:27.531267+0000","last_unstale":"2024-04-20T21:32:31.612462+0000","last_undegraded":"2024-04-20T21:32:31.612462+0000","last_fullsized":"2024-04-20T21:32:31.612462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:47.002481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.602598+0000","last_change":"2024-04-20T21:31:27.512033+0000","last_active":"2024-04-20T21:32:31.602598+0000","last_peered":"2024-04-20T21:32:31.602598+0000","last_clean":"2024-04-20T21:32:31.602598+0000","last_became_active":"2024-04-20T21:31:27.510801+0000","last_became_peered":"2024-04-20T21:31:27.510801+0000","last_unstale":"2024-04-20T21:32:31.602598+0000","last_undegraded":"2024-04-20T21:32:31.602598+0000","last_fullsized":"2024-04-20T21:32:31.602598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:43:25.407173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606987+0000","last_change":"2024-04-20T21:31:27.545862+0000","last_active":"2024-04-20T21:32:31.606987+0000","last_peered":"2024-04-20T21:32:31.606987+0000","last_clean":"2024-04-20T21:32:31.606987+0000","last_became_active":"2024-04-20T21:31:27.545500+0000","last_became_peered":"2024-04-20T21:31:27.545500+0000","last_unstale":"2024-04-20T21:32:31.606987+0000","last_undegraded":"2024-04-20T21:32:31.606987+0000","last_fullsized":"2024-04-20T21:32:31.606987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:37.640777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.995231+0000","last_change":"2024-04-20T21:31:27.526408+0000","last_active":"2024-04-20T21:32:31.995231+0000","last_peered":"2024-04-20T21:32:31.995231+0000","last_clean":"2024-04-20T21:32:31.995231+0000","last_became_active":"2024-04-20T21:31:27.526015+0000","last_became_peered":"2024-04-20T21:31:27.526015+0000","last_unstale":"2024-04-20T21:32:31.995231+0000","last_undegraded":"2024-04-20T21:32:31.995231+0000","last_fullsized":"2024-04-20T21:32:31.995231+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:56.904696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610098+0000","last_change":"2024-04-20T21:31:27.526272+0000","last_active":"2024-04-20T21:32:31.610098+0000","last_peered":"2024-04-20T21:32:31.610098+0000","last_clean":"2024-04-20T21:32:31.610098+0000","last_became_active":"2024-04-20T21:31:27.525852+0000","last_became_peered":"2024-04-20T21:31:27.525852+0000","last_unstale":"2024-04-20T21:32:31.610098+0000","last_undegraded":"2024-04-20T21:32:31.610098+0000","last_fullsized":"2024-04-20T21:32:31.610098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:33:57.811339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:32.000228+0000","last_change":"2024-04-20T21:32:31.998671+0000","last_active":"2024-04-20T21:32:32.000228+0000","last_peered":"2024-04-20T21:32:32.000228+0000","last_clean":"2024-04-20T21:32:32.000228+0000","last_became_active":"2024-04-20T21:32:31.996769+0000","last_became_peered":"2024-04-20T21:32:31.996769+0000","last_unstale":"2024-04-20T21:32:32.000228+0000","last_undegraded":"2024-04-20T21:32:32.000228+0000","last_fullsized":"2024-04-20T21:32:32.000228+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:03.135794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609584+0000","last_change":"2024-04-20T21:31:27.514810+0000","last_active":"2024-04-20T21:32:31.609584+0000","last_peered":"2024-04-20T21:32:31.609584+0000","last_clean":"2024-04-20T21:32:31.609584+0000","last_became_active":"2024-04-20T21:31:27.512548+0000","last_became_peered":"2024-04-20T21:31:27.512548+0000","last_unstale":"2024-04-20T21:32:31.609584+0000","last_undegraded":"2024-04-20T21:32:31.609584+0000","last_fullsized":"2024-04-20T21:32:31.609584+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:11:30.375294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.995577+0000","last_change":"2024-04-20T21:31:27.510303+0000","last_active":"2024-04-20T21:32:31.995577+0000","last_peered":"2024-04-20T21:32:31.995577+0000","last_clean":"2024-04-20T21:32:31.995577+0000","last_became_active":"2024-04-20T21:31:27.509907+0000","last_became_peered":"2024-04-20T21:31:27.509907+0000","last_unstale":"2024-04-20T21:32:31.995577+0000","last_undegraded":"2024-04-20T21:32:31.995577+0000","last_fullsized":"2024-04-20T21:32:31.995577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:31.160785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610593+0000","last_change":"2024-04-20T21:31:27.513749+0000","last_active":"2024-04-20T21:32:31.610593+0000","last_peered":"2024-04-20T21:32:31.610593+0000","last_clean":"2024-04-20T21:32:31.610593+0000","last_became_active":"2024-04-20T21:31:27.513404+0000","last_became_peered":"2024-04-20T21:31:27.513404+0000","last_unstale":"2024-04-20T21:32:31.610593+0000","last_undegraded":"2024-04-20T21:32:31.610593+0000","last_fullsized":"2024-04-20T21:32:31.610593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:26.279697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.602447+0000","last_change":"2024-04-20T21:31:27.507727+0000","last_active":"2024-04-20T21:32:31.602447+0000","last_peered":"2024-04-20T21:32:31.602447+0000","last_clean":"2024-04-20T21:32:31.602447+0000","last_became_active":"2024-04-20T21:31:27.507363+0000","last_became_peered":"2024-04-20T21:31:27.507363+0000","last_unstale":"2024-04-20T21:32:31.602447+0000","last_undegraded":"2024-04-20T21:32:31.602447+0000","last_fullsized":"2024-04-20T21:32:31.602447+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:01.139963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605691+0000","last_change":"2024-04-20T21:31:27.512941+0000","last_active":"2024-04-20T21:32:31.605691+0000","last_peered":"2024-04-20T21:32:31.605691+0000","last_clean":"2024-04-20T21:32:31.605691+0000","last_became_active":"2024-04-20T21:31:27.512571+0000","last_became_peered":"2024-04-20T21:31:27.512571+0000","last_unstale":"2024-04-20T21:32:31.605691+0000","last_undegraded":"2024-04-20T21:32:31.605691+0000","last_fullsized":"2024-04-20T21:32:31.605691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:30.977624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610949+0000","last_change":"2024-04-20T21:31:27.521439+0000","last_active":"2024-04-20T21:32:31.610949+0000","last_peered":"2024-04-20T21:32:31.610949+0000","last_clean":"2024-04-20T21:32:31.610949+0000","last_became_active":"2024-04-20T21:31:27.520245+0000","last_became_peered":"2024-04-20T21:31:27.520245+0000","last_unstale":"2024-04-20T21:32:31.610949+0000","last_undegraded":"2024-04-20T21:32:31.610949+0000","last_fullsized":"2024-04-20T21:32:31.610949+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:53:26.510845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.619209+0000","last_change":"2024-04-20T21:32:31.619209+0000","last_active":"2024-04-20T21:32:31.619209+0000","last_peered":"2024-04-20T21:32:31.619209+0000","last_clean":"2024-04-20T21:32:31.619209+0000","last_became_active":"2024-04-20T21:32:31.618527+0000","last_became_peered":"2024-04-20T21:32:31.618527+0000","last_unstale":"2024-04-20T21:32:31.619209+0000","last_undegraded":"2024-04-20T21:32:31.619209+0000","last_fullsized":"2024-04-20T21:32:31.619209+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:54:21.400896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.608731+0000","last_change":"2024-04-20T21:31:27.521533+0000","last_active":"2024-04-20T21:32:31.608731+0000","last_peered":"2024-04-20T21:32:31.608731+0000","last_clean":"2024-04-20T21:32:31.608731+0000","last_became_active":"2024-04-20T21:31:27.520920+0000","last_became_peered":"2024-04-20T21:31:27.520920+0000","last_unstale":"2024-04-20T21:32:31.608731+0000","last_undegraded":"2024-04-20T21:32:31.608731+0000","last_fullsized":"2024-04-20T21:32:31.608731+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:28:13.605238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606169+0000","last_change":"2024-04-20T21:31:27.527484+0000","last_active":"2024-04-20T21:32:31.606169+0000","last_peered":"2024-04-20T21:32:31.606169+0000","last_clean":"2024-04-20T21:32:31.606169+0000","last_became_active":"2024-04-20T21:31:27.527048+0000","last_became_peered":"2024-04-20T21:31:27.527048+0000","last_unstale":"2024-04-20T21:32:31.606169+0000","last_undegraded":"2024-04-20T21:32:31.606169+0000","last_fullsized":"2024-04-20T21:32:31.606169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:35:56.467661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.615976+0000","last_change":"2024-04-20T21:31:27.527870+0000","last_active":"2024-04-20T21:32:31.615976+0000","last_peered":"2024-04-20T21:32:31.615976+0000","last_clean":"2024-04-20T21:32:31.615976+0000","last_became_active":"2024-04-20T21:31:27.525918+0000","last_became_peered":"2024-04-20T21:31:27.525918+0000","last_unstale":"2024-04-20T21:32:31.615976+0000","last_undegraded":"2024-04-20T21:32:31.615976+0000","last_fullsized":"2024-04-20T21:32:31.615976+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:34.425370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610085+0000","last_change":"2024-04-20T21:31:27.544987+0000","last_active":"2024-04-20T21:32:31.610085+0000","last_peered":"2024-04-20T21:32:31.610085+0000","last_clean":"2024-04-20T21:32:31.610085+0000","last_became_active":"2024-04-20T21:31:27.544628+0000","last_became_peered":"2024-04-20T21:31:27.544628+0000","last_unstale":"2024-04-20T21:32:31.610085+0000","last_undegraded":"2024-04-20T21:32:31.610085+0000","last_fullsized":"2024-04-20T21:32:31.610085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:05:42.396775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.615021+0000","last_change":"2024-04-20T21:31:26.702803+0000","last_active":"2024-04-20T21:32:31.615021+0000","last_peered":"2024-04-20T21:32:31.615021+0000","last_clean":"2024-04-20T21:32:31.615021+0000","last_became_active":"2024-04-20T21:31:26.695781+0000","last_became_peered":"2024-04-20T21:31:26.695781+0000","last_unstale":"2024-04-20T21:32:31.615021+0000","last_undegraded":"2024-04-20T21:32:31.615021+0000","last_fullsized":"2024-04-20T21:32:31.615021+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:49:17.131832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.612344+0000","last_change":"2024-04-20T21:31:27.535264+0000","last_active":"2024-04-20T21:32:31.612344+0000","last_peered":"2024-04-20T21:32:31.612344+0000","last_clean":"2024-04-20T21:32:31.612344+0000","last_became_active":"2024-04-20T21:31:27.534888+0000","last_became_peered":"2024-04-20T21:31:27.534888+0000","last_unstale":"2024-04-20T21:32:31.612344+0000","last_undegraded":"2024-04-20T21:32:31.612344+0000","last_fullsized":"2024-04-20T21:32:31.612344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:04:20.951916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.612789+0000","last_change":"2024-04-20T21:31:26.700378+0000","last_active":"2024-04-20T21:32:31.612789+0000","last_peered":"2024-04-20T21:32:31.612789+0000","last_clean":"2024-04-20T21:32:31.612789+0000","last_became_active":"2024-04-20T21:31:26.700077+0000","last_became_peered":"2024-04-20T21:31:26.700077+0000","last_unstale":"2024-04-20T21:32:31.612789+0000","last_undegraded":"2024-04-20T21:32:31.612789+0000","last_fullsized":"2024-04-20T21:32:31.612789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:34:21.646402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.612348+0000","last_change":"2024-04-20T21:31:27.531069+0000","last_active":"2024-04-20T21:32:31.612348+0000","last_peered":"2024-04-20T21:32:31.612348+0000","last_clean":"2024-04-20T21:32:31.612348+0000","last_became_active":"2024-04-20T21:31:27.530457+0000","last_became_peered":"2024-04-20T21:31:27.530457+0000","last_unstale":"2024-04-20T21:32:31.612348+0000","last_undegraded":"2024-04-20T21:32:31.612348+0000","last_fullsized":"2024-04-20T21:32:31.612348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:46:59.829623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.607657+0000","last_change":"2024-04-20T21:31:26.684150+0000","last_active":"2024-04-20T21:32:31.607657+0000","last_peered":"2024-04-20T21:32:31.607657+0000","last_clean":"2024-04-20T21:32:31.607657+0000","last_became_active":"2024-04-20T21:31:26.683823+0000","last_became_peered":"2024-04-20T21:31:26.683823+0000","last_unstale":"2024-04-20T21:32:31.607657+0000","last_undegraded":"2024-04-20T21:32:31.607657+0000","last_fullsized":"2024-04-20T21:32:31.607657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:14.413606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609028+0000","last_change":"2024-04-20T21:31:27.507499+0000","last_active":"2024-04-20T21:32:31.609028+0000","last_peered":"2024-04-20T21:32:31.609028+0000","last_clean":"2024-04-20T21:32:31.609028+0000","last_became_active":"2024-04-20T21:31:27.506922+0000","last_became_peered":"2024-04-20T21:31:27.506922+0000","last_unstale":"2024-04-20T21:32:31.609028+0000","last_undegraded":"2024-04-20T21:32:31.609028+0000","last_fullsized":"2024-04-20T21:32:31.609028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:40.865232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.614078+0000","last_change":"2024-04-20T21:32:31.613577+0000","last_active":"2024-04-20T21:32:31.614078+0000","last_peered":"2024-04-20T21:32:31.614078+0000","last_clean":"2024-04-20T21:32:31.614078+0000","last_became_active":"2024-04-20T21:32:31.608943+0000","last_became_peered":"2024-04-20T21:32:31.608943+0000","last_unstale":"2024-04-20T21:32:31.614078+0000","last_undegraded":"2024-04-20T21:32:31.614078+0000","last_fullsized":"2024-04-20T21:32:31.614078+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:06.561157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604249+0000","last_change":"2024-04-20T21:31:27.517839+0000","last_active":"2024-04-20T21:32:31.604249+0000","last_peered":"2024-04-20T21:32:31.604249+0000","last_clean":"2024-04-20T21:32:31.604249+0000","last_became_active":"2024-04-20T21:31:27.517044+0000","last_became_peered":"2024-04-20T21:31:27.517044+0000","last_unstale":"2024-04-20T21:32:31.604249+0000","last_undegraded":"2024-04-20T21:32:31.604249+0000","last_fullsized":"2024-04-20T21:32:31.604249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:50.042665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604337+0000","last_change":"2024-04-20T21:31:26.686151+0000","last_active":"2024-04-20T21:32:31.604337+0000","last_peered":"2024-04-20T21:32:31.604337+0000","last_clean":"2024-04-20T21:32:31.604337+0000","last_became_active":"2024-04-20T21:31:26.685827+0000","last_became_peered":"2024-04-20T21:31:26.685827+0000","last_unstale":"2024-04-20T21:32:31.604337+0000","last_undegraded":"2024-04-20T21:32:31.604337+0000","last_fullsized":"2024-04-20T21:32:31.604337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:41:17.263420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.620563+0000","last_change":"2024-04-20T21:32:31.620563+0000","last_active":"2024-04-20T21:32:31.620563+0000","last_peered":"2024-04-20T21:32:31.620563+0000","last_clean":"2024-04-20T21:32:31.620563+0000","last_became_active":"2024-04-20T21:32:31.620145+0000","last_became_peered":"2024-04-20T21:32:31.620145+0000","last_unstale":"2024-04-20T21:32:31.620563+0000","last_undegraded":"2024-04-20T21:32:31.620563+0000","last_fullsized":"2024-04-20T21:32:31.620563+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:43.609490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.995117+0000","last_change":"2024-04-20T21:31:26.684573+0000","last_active":"2024-04-20T21:32:31.995117+0000","last_peered":"2024-04-20T21:32:31.995117+0000","last_clean":"2024-04-20T21:32:31.995117+0000","last_became_active":"2024-04-20T21:31:26.684163+0000","last_became_peered":"2024-04-20T21:31:26.684163+0000","last_unstale":"2024-04-20T21:32:31.995117+0000","last_undegraded":"2024-04-20T21:32:31.995117+0000","last_fullsized":"2024-04-20T21:32:31.995117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:39.445068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604500+0000","last_change":"2024-04-20T21:31:27.543680+0000","last_active":"2024-04-20T21:32:31.604500+0000","last_peered":"2024-04-20T21:32:31.604500+0000","last_clean":"2024-04-20T21:32:31.604500+0000","last_became_active":"2024-04-20T21:31:27.543379+0000","last_became_peered":"2024-04-20T21:31:27.543379+0000","last_unstale":"2024-04-20T21:32:31.604500+0000","last_undegraded":"2024-04-20T21:32:31.604500+0000","last_fullsized":"2024-04-20T21:32:31.604500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:58:17.168813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605053+0000","last_change":"2024-04-20T21:31:26.689665+0000","last_active":"2024-04-20T21:32:31.605053+0000","last_peered":"2024-04-20T21:32:31.605053+0000","last_clean":"2024-04-20T21:32:31.605053+0000","last_became_active":"2024-04-20T21:31:26.689175+0000","last_became_peered":"2024-04-20T21:31:26.689175+0000","last_unstale":"2024-04-20T21:32:31.605053+0000","last_undegraded":"2024-04-20T21:32:31.605053+0000","last_fullsized":"2024-04-20T21:32:31.605053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:21.342700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609476+0000","last_change":"2024-04-20T21:31:27.544137+0000","last_active":"2024-04-20T21:32:31.609476+0000","last_peered":"2024-04-20T21:32:31.609476+0000","last_clean":"2024-04-20T21:32:31.609476+0000","last_became_active":"2024-04-20T21:31:27.542841+0000","last_became_peered":"2024-04-20T21:31:27.542841+0000","last_unstale":"2024-04-20T21:32:31.609476+0000","last_undegraded":"2024-04-20T21:32:31.609476+0000","last_fullsized":"2024-04-20T21:32:31.609476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:04.610144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604714+0000","last_change":"2024-04-20T21:31:26.681937+0000","last_active":"2024-04-20T21:32:31.604714+0000","last_peered":"2024-04-20T21:32:31.604714+0000","last_clean":"2024-04-20T21:32:31.604714+0000","last_became_active":"2024-04-20T21:31:26.681621+0000","last_became_peered":"2024-04-20T21:31:26.681621+0000","last_unstale":"2024-04-20T21:32:31.604714+0000","last_undegraded":"2024-04-20T21:32:31.604714+0000","last_fullsized":"2024-04-20T21:32:31.604714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:06.641608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.612834+0000","last_change":"2024-04-20T21:31:27.540225+0000","last_active":"2024-04-20T21:32:31.612834+0000","last_peered":"2024-04-20T21:32:31.612834+0000","last_clean":"2024-04-20T21:32:31.612834+0000","last_became_active":"2024-04-20T21:31:27.539860+0000","last_became_peered":"2024-04-20T21:31:27.539860+0000","last_unstale":"2024-04-20T21:32:31.612834+0000","last_undegraded":"2024-04-20T21:32:31.612834+0000","last_fullsized":"2024-04-20T21:32:31.612834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:50.146761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.995600+0000","last_change":"2024-04-20T21:31:26.701816+0000","last_active":"2024-04-20T21:32:31.995600+0000","last_peered":"2024-04-20T21:32:31.995600+0000","last_clean":"2024-04-20T21:32:31.995600+0000","last_became_active":"2024-04-20T21:31:26.701530+0000","last_became_peered":"2024-04-20T21:31:26.701530+0000","last_unstale":"2024-04-20T21:32:31.995600+0000","last_undegraded":"2024-04-20T21:32:31.995600+0000","last_fullsized":"2024-04-20T21:32:31.995600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:31.175330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609579+0000","last_change":"2024-04-20T21:31:27.527222+0000","last_active":"2024-04-20T21:32:31.609579+0000","last_peered":"2024-04-20T21:32:31.609579+0000","last_clean":"2024-04-20T21:32:31.609579+0000","last_became_active":"2024-04-20T21:31:27.526779+0000","last_became_peered":"2024-04-20T21:31:27.526779+0000","last_unstale":"2024-04-20T21:32:31.609579+0000","last_undegraded":"2024-04-20T21:32:31.609579+0000","last_fullsized":"2024-04-20T21:32:31.609579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:13:54.500516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610604+0000","last_change":"2024-04-20T21:31:26.685969+0000","last_active":"2024-04-20T21:32:31.610604+0000","last_peered":"2024-04-20T21:32:31.610604+0000","last_clean":"2024-04-20T21:32:31.610604+0000","last_became_active":"2024-04-20T21:31:26.685580+0000","last_became_peered":"2024-04-20T21:31:26.685580+0000","last_unstale":"2024-04-20T21:32:31.610604+0000","last_undegraded":"2024-04-20T21:32:31.610604+0000","last_fullsized":"2024-04-20T21:32:31.610604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:22:50.558526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.612042+0000","last_change":"2024-04-20T21:31:27.525862+0000","last_active":"2024-04-20T21:32:31.612042+0000","last_peered":"2024-04-20T21:32:31.612042+0000","last_clean":"2024-04-20T21:32:31.612042+0000","last_became_active":"2024-04-20T21:31:27.525372+0000","last_became_peered":"2024-04-20T21:31:27.525372+0000","last_unstale":"2024-04-20T21:32:31.612042+0000","last_undegraded":"2024-04-20T21:32:31.612042+0000","last_fullsized":"2024-04-20T21:32:31.612042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:44:15.527137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604956+0000","last_change":"2024-04-20T21:31:26.691091+0000","last_active":"2024-04-20T21:32:31.604956+0000","last_peered":"2024-04-20T21:32:31.604956+0000","last_clean":"2024-04-20T21:32:31.604956+0000","last_became_active":"2024-04-20T21:31:26.690608+0000","last_became_peered":"2024-04-20T21:31:26.690608+0000","last_unstale":"2024-04-20T21:32:31.604956+0000","last_undegraded":"2024-04-20T21:32:31.604956+0000","last_fullsized":"2024-04-20T21:32:31.604956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:44:25.336193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609185+0000","last_change":"2024-04-20T21:31:27.507900+0000","last_active":"2024-04-20T21:32:31.609185+0000","last_peered":"2024-04-20T21:32:31.609185+0000","last_clean":"2024-04-20T21:32:31.609185+0000","last_became_active":"2024-04-20T21:31:27.507360+0000","last_became_peered":"2024-04-20T21:31:27.507360+0000","last_unstale":"2024-04-20T21:32:31.609185+0000","last_undegraded":"2024-04-20T21:32:31.609185+0000","last_fullsized":"2024-04-20T21:32:31.609185+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:27.110877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'26","reported_seq":209,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:48.157319+0000","last_change":"2024-04-20T21:31:26.685736+0000","last_active":"2024-04-20T21:32:48.157319+0000","last_peered":"2024-04-20T21:32:48.157319+0000","last_clean":"2024-04-20T21:32:48.157319+0000","last_became_active":"2024-04-20T21:31:26.685309+0000","last_became_peered":"2024-04-20T21:31:26.685309+0000","last_unstale":"2024-04-20T21:32:48.157319+0000","last_undegraded":"2024-04-20T21:32:48.157319+0000","last_fullsized":"2024-04-20T21:32:48.157319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:06.501753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":164,"num_read_kb":141,"num_write":26,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.607434+0000","last_change":"2024-04-20T21:31:27.507532+0000","last_active":"2024-04-20T21:32:31.607434+0000","last_peered":"2024-04-20T21:32:31.607434+0000","last_clean":"2024-04-20T21:32:31.607434+0000","last_became_active":"2024-04-20T21:31:27.507188+0000","last_became_peered":"2024-04-20T21:31:27.507188+0000","last_unstale":"2024-04-20T21:32:31.607434+0000","last_undegraded":"2024-04-20T21:32:31.607434+0000","last_fullsized":"2024-04-20T21:32:31.607434+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:58.814116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609522+0000","last_change":"2024-04-20T21:31:26.701361+0000","last_active":"2024-04-20T21:32:31.609522+0000","last_peered":"2024-04-20T21:32:31.609522+0000","last_clean":"2024-04-20T21:32:31.609522+0000","last_became_active":"2024-04-20T21:31:26.701068+0000","last_became_peered":"2024-04-20T21:31:26.701068+0000","last_unstale":"2024-04-20T21:32:31.609522+0000","last_undegraded":"2024-04-20T21:32:31.609522+0000","last_fullsized":"2024-04-20T21:32:31.609522+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:22:51.745294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604922+0000","last_change":"2024-04-20T21:31:27.542189+0000","last_active":"2024-04-20T21:32:31.604922+0000","last_peered":"2024-04-20T21:32:31.604922+0000","last_clean":"2024-04-20T21:32:31.604922+0000","last_became_active":"2024-04-20T21:31:27.541800+0000","last_became_peered":"2024-04-20T21:31:27.541800+0000","last_unstale":"2024-04-20T21:32:31.604922+0000","last_undegraded":"2024-04-20T21:32:31.604922+0000","last_fullsized":"2024-04-20T21:32:31.604922+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:31.898443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604003+0000","last_change":"2024-04-20T21:31:26.687652+0000","last_active":"2024-04-20T21:32:31.604003+0000","last_peered":"2024-04-20T21:32:31.604003+0000","last_clean":"2024-04-20T21:32:31.604003+0000","last_became_active":"2024-04-20T21:31:26.687333+0000","last_became_peered":"2024-04-20T21:31:26.687333+0000","last_unstale":"2024-04-20T21:32:31.604003+0000","last_undegraded":"2024-04-20T21:32:31.604003+0000","last_fullsized":"2024-04-20T21:32:31.604003+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:12.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610787+0000","last_change":"2024-04-20T21:31:27.522535+0000","last_active":"2024-04-20T21:32:31.610787+0000","last_peered":"2024-04-20T21:32:31.610787+0000","last_clean":"2024-04-20T21:32:31.610787+0000","last_became_active":"2024-04-20T21:31:27.521821+0000","last_became_peered":"2024-04-20T21:31:27.521821+0000","last_unstale":"2024-04-20T21:32:31.610787+0000","last_undegraded":"2024-04-20T21:32:31.610787+0000","last_fullsized":"2024-04-20T21:32:31.610787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:47:53.401129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.602154+0000","last_change":"2024-04-20T21:31:26.681758+0000","last_active":"2024-04-20T21:32:31.602154+0000","last_peered":"2024-04-20T21:32:31.602154+0000","last_clean":"2024-04-20T21:32:31.602154+0000","last_became_active":"2024-04-20T21:31:26.681395+0000","last_became_peered":"2024-04-20T21:31:26.681395+0000","last_unstale":"2024-04-20T21:32:31.602154+0000","last_undegraded":"2024-04-20T21:32:31.602154+0000","last_fullsized":"2024-04-20T21:32:31.602154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:33:00.924879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609948+0000","last_change":"2024-04-20T21:31:27.512873+0000","last_active":"2024-04-20T21:32:31.609948+0000","last_peered":"2024-04-20T21:32:31.609948+0000","last_clean":"2024-04-20T21:32:31.609948+0000","last_became_active":"2024-04-20T21:31:27.512554+0000","last_became_peered":"2024-04-20T21:31:27.512554+0000","last_unstale":"2024-04-20T21:32:31.609948+0000","last_undegraded":"2024-04-20T21:32:31.609948+0000","last_fullsized":"2024-04-20T21:32:31.609948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:26:06.670726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606898+0000","last_change":"2024-04-20T21:31:26.691210+0000","last_active":"2024-04-20T21:32:31.606898+0000","last_peered":"2024-04-20T21:32:31.606898+0000","last_clean":"2024-04-20T21:32:31.606898+0000","last_became_active":"2024-04-20T21:31:26.690859+0000","last_became_peered":"2024-04-20T21:31:26.690859+0000","last_unstale":"2024-04-20T21:32:31.606898+0000","last_undegraded":"2024-04-20T21:32:31.606898+0000","last_fullsized":"2024-04-20T21:32:31.606898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:36.789429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.616413+0000","last_change":"2024-04-20T21:32:31.616413+0000","last_active":"2024-04-20T21:32:31.616413+0000","last_peered":"2024-04-20T21:32:31.616413+0000","last_clean":"2024-04-20T21:32:31.616413+0000","last_became_active":"2024-04-20T21:32:31.616011+0000","last_became_peered":"2024-04-20T21:32:31.616011+0000","last_unstale":"2024-04-20T21:32:31.616413+0000","last_undegraded":"2024-04-20T21:32:31.616413+0000","last_fullsized":"2024-04-20T21:32:31.616413+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:11:05.630337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.607564+0000","last_change":"2024-04-20T21:31:26.682311+0000","last_active":"2024-04-20T21:32:31.607564+0000","last_peered":"2024-04-20T21:32:31.607564+0000","last_clean":"2024-04-20T21:32:31.607564+0000","last_became_active":"2024-04-20T21:31:26.681996+0000","last_became_peered":"2024-04-20T21:31:26.681996+0000","last_unstale":"2024-04-20T21:32:31.607564+0000","last_undegraded":"2024-04-20T21:32:31.607564+0000","last_fullsized":"2024-04-20T21:32:31.607564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:32:37.659946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605711+0000","last_change":"2024-04-20T21:31:27.510922+0000","last_active":"2024-04-20T21:32:31.605711+0000","last_peered":"2024-04-20T21:32:31.605711+0000","last_clean":"2024-04-20T21:32:31.605711+0000","last_became_active":"2024-04-20T21:31:27.510553+0000","last_became_peered":"2024-04-20T21:31:27.510553+0000","last_unstale":"2024-04-20T21:32:31.605711+0000","last_undegraded":"2024-04-20T21:32:31.605711+0000","last_fullsized":"2024-04-20T21:32:31.605711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:23.754768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.613686+0000","last_change":"2024-04-20T21:31:26.697010+0000","last_active":"2024-04-20T21:32:31.613686+0000","last_peered":"2024-04-20T21:32:31.613686+0000","last_clean":"2024-04-20T21:32:31.613686+0000","last_became_active":"2024-04-20T21:31:26.696112+0000","last_became_peered":"2024-04-20T21:31:26.696112+0000","last_unstale":"2024-04-20T21:32:31.613686+0000","last_undegraded":"2024-04-20T21:32:31.613686+0000","last_fullsized":"2024-04-20T21:32:31.613686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:57:02.715884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.995528+0000","last_change":"2024-04-20T21:31:27.545025+0000","last_active":"2024-04-20T21:32:31.995528+0000","last_peered":"2024-04-20T21:32:31.995528+0000","last_clean":"2024-04-20T21:32:31.995528+0000","last_became_active":"2024-04-20T21:31:27.544630+0000","last_became_peered":"2024-04-20T21:31:27.544630+0000","last_unstale":"2024-04-20T21:32:31.995528+0000","last_undegraded":"2024-04-20T21:32:31.995528+0000","last_fullsized":"2024-04-20T21:32:31.995528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:57.292481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610349+0000","last_change":"2024-04-20T21:31:26.682909+0000","last_active":"2024-04-20T21:32:31.610349+0000","last_peered":"2024-04-20T21:32:31.610349+0000","last_clean":"2024-04-20T21:32:31.610349+0000","last_became_active":"2024-04-20T21:31:26.682597+0000","last_became_peered":"2024-04-20T21:31:26.682597+0000","last_unstale":"2024-04-20T21:32:31.610349+0000","last_undegraded":"2024-04-20T21:32:31.610349+0000","last_fullsized":"2024-04-20T21:32:31.610349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:31.889513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.995498+0000","last_change":"2024-04-20T21:31:27.530217+0000","last_active":"2024-04-20T21:32:31.995498+0000","last_peered":"2024-04-20T21:32:31.995498+0000","last_clean":"2024-04-20T21:32:31.995498+0000","last_became_active":"2024-04-20T21:31:27.529780+0000","last_became_peered":"2024-04-20T21:31:27.529780+0000","last_unstale":"2024-04-20T21:32:31.995498+0000","last_undegraded":"2024-04-20T21:32:31.995498+0000","last_fullsized":"2024-04-20T21:32:31.995498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:21.613460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610727+0000","last_change":"2024-04-20T21:32:31.610727+0000","last_active":"2024-04-20T21:32:31.610727+0000","last_peered":"2024-04-20T21:32:31.610727+0000","last_clean":"2024-04-20T21:32:31.610727+0000","last_became_active":"2024-04-20T21:32:31.610321+0000","last_became_peered":"2024-04-20T21:32:31.610321+0000","last_unstale":"2024-04-20T21:32:31.610727+0000","last_undegraded":"2024-04-20T21:32:31.610727+0000","last_fullsized":"2024-04-20T21:32:31.610727+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:01:35.697137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604769+0000","last_change":"2024-04-20T21:31:26.692845+0000","last_active":"2024-04-20T21:32:31.604769+0000","last_peered":"2024-04-20T21:32:31.604769+0000","last_clean":"2024-04-20T21:32:31.604769+0000","last_became_active":"2024-04-20T21:31:26.691915+0000","last_became_peered":"2024-04-20T21:31:26.691915+0000","last_unstale":"2024-04-20T21:32:31.604769+0000","last_undegraded":"2024-04-20T21:32:31.604769+0000","last_fullsized":"2024-04-20T21:32:31.604769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:24:00.436955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610357+0000","last_change":"2024-04-20T21:31:27.541623+0000","last_active":"2024-04-20T21:32:31.610357+0000","last_peered":"2024-04-20T21:32:31.610357+0000","last_clean":"2024-04-20T21:32:31.610357+0000","last_became_active":"2024-04-20T21:31:27.540953+0000","last_became_peered":"2024-04-20T21:31:27.540953+0000","last_unstale":"2024-04-20T21:32:31.610357+0000","last_undegraded":"2024-04-20T21:32:31.610357+0000","last_fullsized":"2024-04-20T21:32:31.610357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:03:15.091886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.602251+0000","last_change":"2024-04-20T21:31:26.687373+0000","last_active":"2024-04-20T21:32:31.602251+0000","last_peered":"2024-04-20T21:32:31.602251+0000","last_clean":"2024-04-20T21:32:31.602251+0000","last_became_active":"2024-04-20T21:31:26.687009+0000","last_became_peered":"2024-04-20T21:31:26.687009+0000","last_unstale":"2024-04-20T21:32:31.602251+0000","last_undegraded":"2024-04-20T21:32:31.602251+0000","last_fullsized":"2024-04-20T21:32:31.602251+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:23:42.708129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610529+0000","last_change":"2024-04-20T21:30:06.995711+0000","last_active":"2024-04-20T21:32:31.610529+0000","last_peered":"2024-04-20T21:32:31.610529+0000","last_clean":"2024-04-20T21:32:31.610529+0000","last_became_active":"2024-04-20T21:30:06.995280+0000","last_became_peered":"2024-04-20T21:30:06.995280+0000","last_unstale":"2024-04-20T21:32:31.610529+0000","last_undegraded":"2024-04-20T21:32:31.610529+0000","last_fullsized":"2024-04-20T21:32:31.610529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_clean_scrub_stamp":"2024-04-20T21:26:16.477494+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:49:26.014145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604435+0000","last_change":"2024-04-20T21:31:27.514064+0000","last_active":"2024-04-20T21:32:31.604435+0000","last_peered":"2024-04-20T21:32:31.604435+0000","last_clean":"2024-04-20T21:32:31.604435+0000","last_became_active":"2024-04-20T21:31:27.513416+0000","last_became_peered":"2024-04-20T21:31:27.513416+0000","last_unstale":"2024-04-20T21:32:31.604435+0000","last_undegraded":"2024-04-20T21:32:31.604435+0000","last_fullsized":"2024-04-20T21:32:31.604435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:12.658265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610349+0000","last_change":"2024-04-20T21:31:26.680948+0000","last_active":"2024-04-20T21:32:31.610349+0000","last_peered":"2024-04-20T21:32:31.610349+0000","last_clean":"2024-04-20T21:32:31.610349+0000","last_became_active":"2024-04-20T21:31:26.680624+0000","last_became_peered":"2024-04-20T21:31:26.680624+0000","last_unstale":"2024-04-20T21:32:31.610349+0000","last_undegraded":"2024-04-20T21:32:31.610349+0000","last_fullsized":"2024-04-20T21:32:31.610349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:02.750166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.607783+0000","last_change":"2024-04-20T21:31:26.697632+0000","last_active":"2024-04-20T21:32:31.607783+0000","last_peered":"2024-04-20T21:32:31.607783+0000","last_clean":"2024-04-20T21:32:31.607783+0000","last_became_active":"2024-04-20T21:31:26.697393+0000","last_became_peered":"2024-04-20T21:31:26.697393+0000","last_unstale":"2024-04-20T21:32:31.607783+0000","last_undegraded":"2024-04-20T21:32:31.607783+0000","last_fullsized":"2024-04-20T21:32:31.607783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:08:39.856082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606331+0000","last_change":"2024-04-20T21:31:27.526613+0000","last_active":"2024-04-20T21:32:31.606331+0000","last_peered":"2024-04-20T21:32:31.606331+0000","last_clean":"2024-04-20T21:32:31.606331+0000","last_became_active":"2024-04-20T21:31:27.526234+0000","last_became_peered":"2024-04-20T21:31:27.526234+0000","last_unstale":"2024-04-20T21:32:31.606331+0000","last_undegraded":"2024-04-20T21:32:31.606331+0000","last_fullsized":"2024-04-20T21:32:31.606331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:06:44.114924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.625214+0000","last_change":"2024-04-20T21:32:31.625214+0000","last_active":"2024-04-20T21:32:31.625214+0000","last_peered":"2024-04-20T21:32:31.625214+0000","last_clean":"2024-04-20T21:32:31.625214+0000","last_became_active":"2024-04-20T21:32:31.619956+0000","last_became_peered":"2024-04-20T21:32:31.619956+0000","last_unstale":"2024-04-20T21:32:31.625214+0000","last_undegraded":"2024-04-20T21:32:31.625214+0000","last_fullsized":"2024-04-20T21:32:31.625214+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:19:41.620846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.611511+0000","last_change":"2024-04-20T21:31:27.538361+0000","last_active":"2024-04-20T21:32:31.611511+0000","last_peered":"2024-04-20T21:32:31.611511+0000","last_clean":"2024-04-20T21:32:31.611511+0000","last_became_active":"2024-04-20T21:31:27.537494+0000","last_became_peered":"2024-04-20T21:31:27.537494+0000","last_unstale":"2024-04-20T21:32:31.611511+0000","last_undegraded":"2024-04-20T21:32:31.611511+0000","last_fullsized":"2024-04-20T21:32:31.611511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:46.212759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.602516+0000","last_change":"2024-04-20T21:31:26.686960+0000","last_active":"2024-04-20T21:32:31.602516+0000","last_peered":"2024-04-20T21:32:31.602516+0000","last_clean":"2024-04-20T21:32:31.602516+0000","last_became_active":"2024-04-20T21:31:26.686565+0000","last_became_peered":"2024-04-20T21:31:26.686565+0000","last_unstale":"2024-04-20T21:32:31.602516+0000","last_undegraded":"2024-04-20T21:32:31.602516+0000","last_fullsized":"2024-04-20T21:32:31.602516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.608834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610858+0000","last_change":"2024-04-20T21:31:27.546978+0000","last_active":"2024-04-20T21:32:31.610858+0000","last_peered":"2024-04-20T21:32:31.610858+0000","last_clean":"2024-04-20T21:32:31.610858+0000","last_became_active":"2024-04-20T21:31:27.545941+0000","last_became_peered":"2024-04-20T21:31:27.545941+0000","last_unstale":"2024-04-20T21:32:31.610858+0000","last_undegraded":"2024-04-20T21:32:31.610858+0000","last_fullsized":"2024-04-20T21:32:31.610858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:08.363519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.608841+0000","last_change":"2024-04-20T21:31:26.690797+0000","last_active":"2024-04-20T21:32:31.608841+0000","last_peered":"2024-04-20T21:32:31.608841+0000","last_clean":"2024-04-20T21:32:31.608841+0000","last_became_active":"2024-04-20T21:31:26.690486+0000","last_became_peered":"2024-04-20T21:31:26.690486+0000","last_unstale":"2024-04-20T21:32:31.608841+0000","last_undegraded":"2024-04-20T21:32:31.608841+0000","last_fullsized":"2024-04-20T21:32:31.608841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:17:33.943212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606857+0000","last_change":"2024-04-20T21:31:27.507175+0000","last_active":"2024-04-20T21:32:31.606857+0000","last_peered":"2024-04-20T21:32:31.606857+0000","last_clean":"2024-04-20T21:32:31.606857+0000","last_became_active":"2024-04-20T21:31:27.506808+0000","last_became_peered":"2024-04-20T21:31:27.506808+0000","last_unstale":"2024-04-20T21:32:31.606857+0000","last_undegraded":"2024-04-20T21:32:31.606857+0000","last_fullsized":"2024-04-20T21:32:31.606857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:38:15.212880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610329+0000","last_change":"2024-04-20T21:31:26.698302+0000","last_active":"2024-04-20T21:32:31.610329+0000","last_peered":"2024-04-20T21:32:31.610329+0000","last_clean":"2024-04-20T21:32:31.610329+0000","last_became_active":"2024-04-20T21:31:26.696209+0000","last_became_peered":"2024-04-20T21:31:26.696209+0000","last_unstale":"2024-04-20T21:32:31.610329+0000","last_undegraded":"2024-04-20T21:32:31.610329+0000","last_fullsized":"2024-04-20T21:32:31.610329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:43.495880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605694+0000","last_change":"2024-04-20T21:31:27.521594+0000","last_active":"2024-04-20T21:32:31.605694+0000","last_peered":"2024-04-20T21:32:31.605694+0000","last_clean":"2024-04-20T21:32:31.605694+0000","last_became_active":"2024-04-20T21:31:27.521185+0000","last_became_peered":"2024-04-20T21:31:27.521185+0000","last_unstale":"2024-04-20T21:32:31.605694+0000","last_undegraded":"2024-04-20T21:32:31.605694+0000","last_fullsized":"2024-04-20T21:32:31.605694+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:22:08.647624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609271+0000","last_change":"2024-04-20T21:31:26.691536+0000","last_active":"2024-04-20T21:32:31.609271+0000","last_peered":"2024-04-20T21:32:31.609271+0000","last_clean":"2024-04-20T21:32:31.609271+0000","last_became_active":"2024-04-20T21:31:26.691202+0000","last_became_peered":"2024-04-20T21:31:26.691202+0000","last_unstale":"2024-04-20T21:32:31.609271+0000","last_undegraded":"2024-04-20T21:32:31.609271+0000","last_fullsized":"2024-04-20T21:32:31.609271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:58:59.331644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:32.000242+0000","last_change":"2024-04-20T21:32:31.998258+0000","last_active":"2024-04-20T21:32:32.000242+0000","last_peered":"2024-04-20T21:32:32.000242+0000","last_clean":"2024-04-20T21:32:32.000242+0000","last_became_active":"2024-04-20T21:32:31.997119+0000","last_became_peered":"2024-04-20T21:32:31.997119+0000","last_unstale":"2024-04-20T21:32:32.000242+0000","last_undegraded":"2024-04-20T21:32:32.000242+0000","last_fullsized":"2024-04-20T21:32:32.000242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:41:08.565434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.616245+0000","last_change":"2024-04-20T21:32:31.616245+0000","last_active":"2024-04-20T21:32:31.616245+0000","last_peered":"2024-04-20T21:32:31.616245+0000","last_clean":"2024-04-20T21:32:31.616245+0000","last_became_active":"2024-04-20T21:32:31.611535+0000","last_became_peered":"2024-04-20T21:32:31.611535+0000","last_unstale":"2024-04-20T21:32:31.616245+0000","last_undegraded":"2024-04-20T21:32:31.616245+0000","last_fullsized":"2024-04-20T21:32:31.616245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:28:17.609890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610597+0000","last_change":"2024-04-20T21:31:27.518346+0000","last_active":"2024-04-20T21:32:31.610597+0000","last_peered":"2024-04-20T21:32:31.610597+0000","last_clean":"2024-04-20T21:32:31.610597+0000","last_became_active":"2024-04-20T21:31:27.517782+0000","last_became_peered":"2024-04-20T21:31:27.517782+0000","last_unstale":"2024-04-20T21:32:31.610597+0000","last_undegraded":"2024-04-20T21:32:31.610597+0000","last_fullsized":"2024-04-20T21:32:31.610597+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:21.674663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.616197+0000","last_change":"2024-04-20T21:32:31.615626+0000","last_active":"2024-04-20T21:32:31.616197+0000","last_peered":"2024-04-20T21:32:31.616197+0000","last_clean":"2024-04-20T21:32:31.616197+0000","last_became_active":"2024-04-20T21:32:31.615181+0000","last_became_peered":"2024-04-20T21:32:31.615181+0000","last_unstale":"2024-04-20T21:32:31.616197+0000","last_undegraded":"2024-04-20T21:32:31.616197+0000","last_fullsized":"2024-04-20T21:32:31.616197+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:58:18.344583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610670+0000","last_change":"2024-04-20T21:31:27.510920+0000","last_active":"2024-04-20T21:32:31.610670+0000","last_peered":"2024-04-20T21:32:31.610670+0000","last_clean":"2024-04-20T21:32:31.610670+0000","last_became_active":"2024-04-20T21:31:27.510564+0000","last_became_peered":"2024-04-20T21:31:27.510564+0000","last_unstale":"2024-04-20T21:32:31.610670+0000","last_undegraded":"2024-04-20T21:32:31.610670+0000","last_fullsized":"2024-04-20T21:32:31.610670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:05.196178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604136+0000","last_change":"2024-04-20T21:31:26.693159+0000","last_active":"2024-04-20T21:32:31.604136+0000","last_peered":"2024-04-20T21:32:31.604136+0000","last_clean":"2024-04-20T21:32:31.604136+0000","last_became_active":"2024-04-20T21:31:26.692902+0000","last_became_peered":"2024-04-20T21:31:26.692902+0000","last_unstale":"2024-04-20T21:32:31.604136+0000","last_undegraded":"2024-04-20T21:32:31.604136+0000","last_fullsized":"2024-04-20T21:32:31.604136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:54.704343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610372+0000","last_change":"2024-04-20T21:31:27.541557+0000","last_active":"2024-04-20T21:32:31.610372+0000","last_peered":"2024-04-20T21:32:31.610372+0000","last_clean":"2024-04-20T21:32:31.610372+0000","last_became_active":"2024-04-20T21:31:27.541161+0000","last_became_peered":"2024-04-20T21:31:27.541161+0000","last_unstale":"2024-04-20T21:32:31.610372+0000","last_undegraded":"2024-04-20T21:32:31.610372+0000","last_fullsized":"2024-04-20T21:32:31.610372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:55:25.558918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606758+0000","last_change":"2024-04-20T21:31:26.688019+0000","last_active":"2024-04-20T21:32:31.606758+0000","last_peered":"2024-04-20T21:32:31.606758+0000","last_clean":"2024-04-20T21:32:31.606758+0000","last_became_active":"2024-04-20T21:31:26.686370+0000","last_became_peered":"2024-04-20T21:31:26.686370+0000","last_unstale":"2024-04-20T21:32:31.606758+0000","last_undegraded":"2024-04-20T21:32:31.606758+0000","last_fullsized":"2024-04-20T21:32:31.606758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:42:44.059365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610642+0000","last_change":"2024-04-20T21:31:27.511592+0000","last_active":"2024-04-20T21:32:31.610642+0000","last_peered":"2024-04-20T21:32:31.610642+0000","last_clean":"2024-04-20T21:32:31.610642+0000","last_became_active":"2024-04-20T21:31:27.511311+0000","last_became_peered":"2024-04-20T21:31:27.511311+0000","last_unstale":"2024-04-20T21:32:31.610642+0000","last_undegraded":"2024-04-20T21:32:31.610642+0000","last_fullsized":"2024-04-20T21:32:31.610642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:35:27.665578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605815+0000","last_change":"2024-04-20T21:31:27.511827+0000","last_active":"2024-04-20T21:32:31.605815+0000","last_peered":"2024-04-20T21:32:31.605815+0000","last_clean":"2024-04-20T21:32:31.605815+0000","last_became_active":"2024-04-20T21:31:27.511344+0000","last_became_peered":"2024-04-20T21:31:27.511344+0000","last_unstale":"2024-04-20T21:32:31.605815+0000","last_undegraded":"2024-04-20T21:32:31.605815+0000","last_fullsized":"2024-04-20T21:32:31.605815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:11:37.973050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609339+0000","last_change":"2024-04-20T21:31:27.510747+0000","last_active":"2024-04-20T21:32:31.609339+0000","last_peered":"2024-04-20T21:32:31.609339+0000","last_clean":"2024-04-20T21:32:31.609339+0000","last_became_active":"2024-04-20T21:31:27.510253+0000","last_became_peered":"2024-04-20T21:31:27.510253+0000","last_unstale":"2024-04-20T21:32:31.609339+0000","last_undegraded":"2024-04-20T21:32:31.609339+0000","last_fullsized":"2024-04-20T21:32:31.609339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:37:13.855840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609135+0000","last_change":"2024-04-20T21:31:27.541185+0000","last_active":"2024-04-20T21:32:31.609135+0000","last_peered":"2024-04-20T21:32:31.609135+0000","last_clean":"2024-04-20T21:32:31.609135+0000","last_became_active":"2024-04-20T21:31:27.540881+0000","last_became_peered":"2024-04-20T21:31:27.540881+0000","last_unstale":"2024-04-20T21:32:31.609135+0000","last_undegraded":"2024-04-20T21:32:31.609135+0000","last_fullsized":"2024-04-20T21:32:31.609135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:46:41.897081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609602+0000","last_change":"2024-04-20T21:31:27.527204+0000","last_active":"2024-04-20T21:32:31.609602+0000","last_peered":"2024-04-20T21:32:31.609602+0000","last_clean":"2024-04-20T21:32:31.609602+0000","last_became_active":"2024-04-20T21:31:27.526805+0000","last_became_peered":"2024-04-20T21:31:27.526805+0000","last_unstale":"2024-04-20T21:32:31.609602+0000","last_undegraded":"2024-04-20T21:32:31.609602+0000","last_fullsized":"2024-04-20T21:32:31.609602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:43.032609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605867+0000","last_change":"2024-04-20T21:31:27.525521+0000","last_active":"2024-04-20T21:32:31.605867+0000","last_peered":"2024-04-20T21:32:31.605867+0000","last_clean":"2024-04-20T21:32:31.605867+0000","last_became_active":"2024-04-20T21:31:27.521892+0000","last_became_peered":"2024-04-20T21:31:27.521892+0000","last_unstale":"2024-04-20T21:32:31.605867+0000","last_undegraded":"2024-04-20T21:32:31.605867+0000","last_fullsized":"2024-04-20T21:32:31.605867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:32:47.683115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605887+0000","last_change":"2024-04-20T21:31:27.509828+0000","last_active":"2024-04-20T21:32:31.605887+0000","last_peered":"2024-04-20T21:32:31.605887+0000","last_clean":"2024-04-20T21:32:31.605887+0000","last_became_active":"2024-04-20T21:31:27.509269+0000","last_became_peered":"2024-04-20T21:31:27.509269+0000","last_unstale":"2024-04-20T21:32:31.605887+0000","last_undegraded":"2024-04-20T21:32:31.605887+0000","last_fullsized":"2024-04-20T21:32:31.605887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:25:54.492533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610240+0000","last_change":"2024-04-20T21:31:27.512542+0000","last_active":"2024-04-20T21:32:31.610240+0000","last_peered":"2024-04-20T21:32:31.610240+0000","last_clean":"2024-04-20T21:32:31.610240+0000","last_became_active":"2024-04-20T21:31:27.511907+0000","last_became_peered":"2024-04-20T21:31:27.511907+0000","last_unstale":"2024-04-20T21:32:31.610240+0000","last_undegraded":"2024-04-20T21:32:31.610240+0000","last_fullsized":"2024-04-20T21:32:31.610240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:10:09.265471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.611218+0000","last_change":"2024-04-20T21:31:27.538697+0000","last_active":"2024-04-20T21:32:31.611218+0000","last_peered":"2024-04-20T21:32:31.611218+0000","last_clean":"2024-04-20T21:32:31.611218+0000","last_became_active":"2024-04-20T21:31:27.537972+0000","last_became_peered":"2024-04-20T21:31:27.537972+0000","last_unstale":"2024-04-20T21:32:31.611218+0000","last_undegraded":"2024-04-20T21:32:31.611218+0000","last_fullsized":"2024-04-20T21:32:31.611218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:51.974485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604729+0000","last_change":"2024-04-20T21:31:27.538092+0000","last_active":"2024-04-20T21:32:31.604729+0000","last_peered":"2024-04-20T21:32:31.604729+0000","last_clean":"2024-04-20T21:32:31.604729+0000","last_became_active":"2024-04-20T21:31:27.537821+0000","last_became_peered":"2024-04-20T21:31:27.537821+0000","last_unstale":"2024-04-20T21:32:31.604729+0000","last_undegraded":"2024-04-20T21:32:31.604729+0000","last_fullsized":"2024-04-20T21:32:31.604729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:52.773583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609957+0000","last_change":"2024-04-20T21:31:27.535498+0000","last_active":"2024-04-20T21:32:31.609957+0000","last_peered":"2024-04-20T21:32:31.609957+0000","last_clean":"2024-04-20T21:32:31.609957+0000","last_became_active":"2024-04-20T21:31:27.535162+0000","last_became_peered":"2024-04-20T21:31:27.535162+0000","last_unstale":"2024-04-20T21:32:31.609957+0000","last_undegraded":"2024-04-20T21:32:31.609957+0000","last_fullsized":"2024-04-20T21:32:31.609957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:46.638825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609120+0000","last_change":"2024-04-20T21:31:27.537776+0000","last_active":"2024-04-20T21:32:31.609120+0000","last_peered":"2024-04-20T21:32:31.609120+0000","last_clean":"2024-04-20T21:32:31.609120+0000","last_became_active":"2024-04-20T21:31:27.537428+0000","last_became_peered":"2024-04-20T21:31:27.537428+0000","last_unstale":"2024-04-20T21:32:31.609120+0000","last_undegraded":"2024-04-20T21:32:31.609120+0000","last_fullsized":"2024-04-20T21:32:31.609120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:15:03.344277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605983+0000","last_change":"2024-04-20T21:31:27.535943+0000","last_active":"2024-04-20T21:32:31.605983+0000","last_peered":"2024-04-20T21:32:31.605983+0000","last_clean":"2024-04-20T21:32:31.605983+0000","last_became_active":"2024-04-20T21:31:27.533694+0000","last_became_peered":"2024-04-20T21:31:27.533694+0000","last_unstale":"2024-04-20T21:32:31.605983+0000","last_undegraded":"2024-04-20T21:32:31.605983+0000","last_fullsized":"2024-04-20T21:32:31.605983+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:42.603232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606370+0000","last_change":"2024-04-20T21:31:27.511345+0000","last_active":"2024-04-20T21:32:31.606370+0000","last_peered":"2024-04-20T21:32:31.606370+0000","last_clean":"2024-04-20T21:32:31.606370+0000","last_became_active":"2024-04-20T21:31:27.510984+0000","last_became_peered":"2024-04-20T21:31:27.510984+0000","last_unstale":"2024-04-20T21:32:31.606370+0000","last_undegraded":"2024-04-20T21:32:31.606370+0000","last_fullsized":"2024-04-20T21:32:31.606370+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:19.965636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61844,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":170,"num_read_kb":147,"num_write":79,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":185532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285599,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":469132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547235,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1658880,"data_stored":1120857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710761,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":983040,"data_stored":469714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972401,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":469736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266740,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1060199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430268,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":469249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":530178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":469222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":530212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":469470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":469154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691905,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36896,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708256,"statfs":{"total":95995035648,"available":95957254144,"internally_reserved":0,"allocated":1589248,"data_stored":1059799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:32:53.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-20T21:32:53.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:52 smithi022 ceph-mon[16843]: pgmap v361: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:53.225 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:52 smithi022 ceph-mon[16843]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:32:53.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:52 smithi122 ceph-mon[22147]: pgmap v361: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:53.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:52 smithi122 ceph-mon[22147]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:32:53.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:52 smithi033 ceph-mon[21626]: pgmap v361: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:53.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:52 smithi033 ceph-mon[21626]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:32:53.611 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:55.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:54 smithi022 ceph-mon[16843]: pgmap v362: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:55.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:54 smithi122 ceph-mon[22147]: pgmap v362: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:55.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:54 smithi033 ceph-mon[21626]: pgmap v362: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:32:55.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.740+0000 7fc413400700 1 -- 172.21.15.22:0/258571242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc40c102aa0 msgr2=0x7fc40c102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:55.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.740+0000 7fc413400700 1 --2- 172.21.15.22:0/258571242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc40c102aa0 0x7fc40c102f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009990 tx=0x7fc3f8009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.740+0000 7fc413400700 1 -- 172.21.15.22:0/258571242 shutdown_connections 2024-04-20T21:32:55.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.740+0000 7fc413400700 1 --2- 172.21.15.22:0/258571242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40c103460 0x7fc40c10f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.740+0000 7fc413400700 1 --2- 172.21.15.22:0/258571242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc40c108a60 0x7fc40c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.740+0000 7fc413400700 1 --2- 172.21.15.22:0/258571242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc40c102aa0 0x7fc40c102f20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.740+0000 7fc413400700 1 -- 172.21.15.22:0/258571242 >> 172.21.15.22:0/258571242 conn(0x7fc40c079bb0 msgr2=0x7fc40c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:55.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.740+0000 7fc413400700 1 -- 172.21.15.22:0/258571242 shutdown_connections 2024-04-20T21:32:55.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.741+0000 7fc413400700 1 -- 172.21.15.22:0/258571242 wait complete. 2024-04-20T21:32:55.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.742+0000 7fc413400700 1 Processor -- start 2024-04-20T21:32:55.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.742+0000 7fc413400700 1 -- start start 2024-04-20T21:32:55.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.742+0000 7fc413400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40c102aa0 0x7fc40c19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.742+0000 7fc413400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc40c103460 0x7fc40c19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.742+0000 7fc413400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc40c108a60 0x7fc40c19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.742+0000 7fc413400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c112db0 con 0x7fc40c103460 2024-04-20T21:32:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.742+0000 7fc413400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c112c30 con 0x7fc40c108a60 2024-04-20T21:32:55.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.742+0000 7fc413400700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c112f30 con 0x7fc40c102aa0 2024-04-20T21:32:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.742+0000 7fc40be00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc40c103460 0x7fc40c19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.743+0000 7fc40be00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc40c103460 0x7fc40c19ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49338/0 (socket says 172.21.15.22:49338) 2024-04-20T21:32:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.743+0000 7fc40be00700 1 -- 172.21.15.22:0/189296336 learned_addr learned my addr 172.21.15.22:0/189296336 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.743+0000 7fc410a00700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40c102aa0 0x7fc40c19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.743+0000 7fc411400700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc40c108a60 0x7fc40c19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.743+0000 7fc40be00700 1 -- 172.21.15.22:0/189296336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40c102aa0 msgr2=0x7fc40c19b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.743+0000 7fc40be00700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40c102aa0 0x7fc40c19b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.743+0000 7fc40be00700 1 -- 172.21.15.22:0/189296336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc40c108a60 msgr2=0x7fc40c19fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.743+0000 7fc40be00700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc40c108a60 0x7fc40c19fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.743+0000 7fc40be00700 1 -- 172.21.15.22:0/189296336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c1a0510 con 0x7fc40c103460 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.743+0000 7fc410a00700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40c102aa0 0x7fc40c19b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.743+0000 7fc40be00700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc40c103460 0x7fc40c19ba40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8000c00 tx=0x7fc3f800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.744+0000 7fc409600700 1 -- 172.21.15.22:0/189296336 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8027070 con 0x7fc40c103460 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.744+0000 7fc409600700 1 -- 172.21.15.22:0/189296336 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f80056d0 con 0x7fc40c103460 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.744+0000 7fc413400700 1 -- 172.21.15.22:0/189296336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40c1a07a0 con 0x7fc40c103460 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.744+0000 7fc409600700 1 -- 172.21.15.22:0/189296336 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8022e10 con 0x7fc40c103460 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.744+0000 7fc413400700 1 -- 172.21.15.22:0/189296336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40c1a8080 con 0x7fc40c103460 2024-04-20T21:32:55.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.745+0000 7fc413400700 1 -- 172.21.15.22:0/189296336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40c069740 con 0x7fc40c103460 2024-04-20T21:32:55.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.747+0000 7fc409600700 1 -- 172.21.15.22:0/189296336 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc3f802a070 con 0x7fc40c103460 2024-04-20T21:32:55.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.748+0000 7fc409600700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc3f40a3d50 0x7fc3f40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:55.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.748+0000 7fc409600700 1 -- 172.21.15.22:0/189296336 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fc3f80d1230 con 0x7fc40c103460 2024-04-20T21:32:55.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.749+0000 7fc410a00700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc3f40a3d50 0x7fc3f40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:55.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.749+0000 7fc410a00700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc3f40a3d50 0x7fc3f40a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc400009fd0 tx=0x7fc400009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:55.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:55.753+0000 7fc409600700 1 -- 172.21.15.22:0/189296336 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc3f8013070 con 0x7fc40c103460 2024-04-20T21:32:56.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.006+0000 7fc413400700 1 -- 172.21.15.22:0/189296336 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc40c101c70 con 0x7fc3f40a3d50 2024-04-20T21:32:56.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.018+0000 7fc409600700 1 -- 172.21.15.22:0/189296336 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258641 (secure 0 0 0) 0x7fc40c101c70 con 0x7fc3f40a3d50 2024-04-20T21:32:56.021 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:32:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.024+0000 7fc413400700 1 -- 172.21.15.22:0/189296336 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc3f40a3d50 msgr2=0x7fc3f40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.024+0000 7fc413400700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc3f40a3d50 0x7fc3f40a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc400009fd0 tx=0x7fc400009260 comp rx=0 tx=0).stop 2024-04-20T21:32:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.024+0000 7fc413400700 1 -- 172.21.15.22:0/189296336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc40c103460 msgr2=0x7fc40c19ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.024+0000 7fc413400700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc40c103460 0x7fc40c19ba40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8000c00 tx=0x7fc3f800bab0 comp rx=0 tx=0).stop 2024-04-20T21:32:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.025+0000 7fc413400700 1 -- 172.21.15.22:0/189296336 shutdown_connections 2024-04-20T21:32:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.025+0000 7fc413400700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc3f40a3d50 0x7fc3f40a6210 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.025+0000 7fc413400700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc40c108a60 0x7fc40c19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.025+0000 7fc413400700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40c102aa0 0x7fc40c19b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:56.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.025+0000 7fc413400700 1 --2- 172.21.15.22:0/189296336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc40c103460 0x7fc40c19ba40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:56.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.025+0000 7fc413400700 1 -- 172.21.15.22:0/189296336 >> 172.21.15.22:0/189296336 conn(0x7fc40c079bb0 msgr2=0x7fc40c101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:56.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.025+0000 7fc413400700 1 -- 172.21.15.22:0/189296336 shutdown_connections 2024-04-20T21:32:56.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:56.025+0000 7fc413400700 1 -- 172.21.15.22:0/189296336 wait complete. 2024-04-20T21:32:56.026 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-20T21:32:56.829 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-04-20T21:32:55.691358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652446,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":316,"num_read_kb":271,"num_write":262,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379368,"kb_used_data":13500,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562456,"statfs":{"total":1151940427776,"available":1151551954944,"internally_reserved":0,"allocated":13824000,"data_stored":7586922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005397"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606348+0000","last_change":"2024-04-20T21:31:27.527151+0000","last_active":"2024-04-20T21:32:31.606348+0000","last_peered":"2024-04-20T21:32:31.606348+0000","last_clean":"2024-04-20T21:32:31.606348+0000","last_became_active":"2024-04-20T21:31:27.526739+0000","last_became_peered":"2024-04-20T21:31:27.526739+0000","last_unstale":"2024-04-20T21:32:31.606348+0000","last_undegraded":"2024-04-20T21:32:31.606348+0000","last_fullsized":"2024-04-20T21:32:31.606348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:16:39.204523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610559+0000","last_change":"2024-04-20T21:31:27.527437+0000","last_active":"2024-04-20T21:32:31.610559+0000","last_peered":"2024-04-20T21:32:31.610559+0000","last_clean":"2024-04-20T21:32:31.610559+0000","last_became_active":"2024-04-20T21:31:27.527040+0000","last_became_peered":"2024-04-20T21:31:27.527040+0000","last_unstale":"2024-04-20T21:32:31.610559+0000","last_undegraded":"2024-04-20T21:32:31.610559+0000","last_fullsized":"2024-04-20T21:32:31.610559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:53:39.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.612462+0000","last_change":"2024-04-20T21:31:27.531671+0000","last_active":"2024-04-20T21:32:31.612462+0000","last_peered":"2024-04-20T21:32:31.612462+0000","last_clean":"2024-04-20T21:32:31.612462+0000","last_became_active":"2024-04-20T21:31:27.531267+0000","last_became_peered":"2024-04-20T21:31:27.531267+0000","last_unstale":"2024-04-20T21:32:31.612462+0000","last_undegraded":"2024-04-20T21:32:31.612462+0000","last_fullsized":"2024-04-20T21:32:31.612462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:47.002481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.602598+0000","last_change":"2024-04-20T21:31:27.512033+0000","last_active":"2024-04-20T21:32:31.602598+0000","last_peered":"2024-04-20T21:32:31.602598+0000","last_clean":"2024-04-20T21:32:31.602598+0000","last_became_active":"2024-04-20T21:31:27.510801+0000","last_became_peered":"2024-04-20T21:31:27.510801+0000","last_unstale":"2024-04-20T21:32:31.602598+0000","last_undegraded":"2024-04-20T21:32:31.602598+0000","last_fullsized":"2024-04-20T21:32:31.602598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:43:25.407173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606987+0000","last_change":"2024-04-20T21:31:27.545862+0000","last_active":"2024-04-20T21:32:31.606987+0000","last_peered":"2024-04-20T21:32:31.606987+0000","last_clean":"2024-04-20T21:32:31.606987+0000","last_became_active":"2024-04-20T21:31:27.545500+0000","last_became_peered":"2024-04-20T21:31:27.545500+0000","last_unstale":"2024-04-20T21:32:31.606987+0000","last_undegraded":"2024-04-20T21:32:31.606987+0000","last_fullsized":"2024-04-20T21:32:31.606987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:37.640777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.995231+0000","last_change":"2024-04-20T21:31:27.526408+0000","last_active":"2024-04-20T21:32:31.995231+0000","last_peered":"2024-04-20T21:32:31.995231+0000","last_clean":"2024-04-20T21:32:31.995231+0000","last_became_active":"2024-04-20T21:31:27.526015+0000","last_became_peered":"2024-04-20T21:31:27.526015+0000","last_unstale":"2024-04-20T21:32:31.995231+0000","last_undegraded":"2024-04-20T21:32:31.995231+0000","last_fullsized":"2024-04-20T21:32:31.995231+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:56.904696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610098+0000","last_change":"2024-04-20T21:31:27.526272+0000","last_active":"2024-04-20T21:32:31.610098+0000","last_peered":"2024-04-20T21:32:31.610098+0000","last_clean":"2024-04-20T21:32:31.610098+0000","last_became_active":"2024-04-20T21:31:27.525852+0000","last_became_peered":"2024-04-20T21:31:27.525852+0000","last_unstale":"2024-04-20T21:32:31.610098+0000","last_undegraded":"2024-04-20T21:32:31.610098+0000","last_fullsized":"2024-04-20T21:32:31.610098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:33:57.811339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:32.000228+0000","last_change":"2024-04-20T21:32:31.998671+0000","last_active":"2024-04-20T21:32:32.000228+0000","last_peered":"2024-04-20T21:32:32.000228+0000","last_clean":"2024-04-20T21:32:32.000228+0000","last_became_active":"2024-04-20T21:32:31.996769+0000","last_became_peered":"2024-04-20T21:32:31.996769+0000","last_unstale":"2024-04-20T21:32:32.000228+0000","last_undegraded":"2024-04-20T21:32:32.000228+0000","last_fullsized":"2024-04-20T21:32:32.000228+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:03.135794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609584+0000","last_change":"2024-04-20T21:31:27.514810+0000","last_active":"2024-04-20T21:32:31.609584+0000","last_peered":"2024-04-20T21:32:31.609584+0000","last_clean":"2024-04-20T21:32:31.609584+0000","last_became_active":"2024-04-20T21:31:27.512548+0000","last_became_peered":"2024-04-20T21:31:27.512548+0000","last_unstale":"2024-04-20T21:32:31.609584+0000","last_undegraded":"2024-04-20T21:32:31.609584+0000","last_fullsized":"2024-04-20T21:32:31.609584+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:11:30.375294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.995577+0000","last_change":"2024-04-20T21:31:27.510303+0000","last_active":"2024-04-20T21:32:31.995577+0000","last_peered":"2024-04-20T21:32:31.995577+0000","last_clean":"2024-04-20T21:32:31.995577+0000","last_became_active":"2024-04-20T21:31:27.509907+0000","last_became_peered":"2024-04-20T21:31:27.509907+0000","last_unstale":"2024-04-20T21:32:31.995577+0000","last_undegraded":"2024-04-20T21:32:31.995577+0000","last_fullsized":"2024-04-20T21:32:31.995577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:31.160785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610593+0000","last_change":"2024-04-20T21:31:27.513749+0000","last_active":"2024-04-20T21:32:31.610593+0000","last_peered":"2024-04-20T21:32:31.610593+0000","last_clean":"2024-04-20T21:32:31.610593+0000","last_became_active":"2024-04-20T21:31:27.513404+0000","last_became_peered":"2024-04-20T21:31:27.513404+0000","last_unstale":"2024-04-20T21:32:31.610593+0000","last_undegraded":"2024-04-20T21:32:31.610593+0000","last_fullsized":"2024-04-20T21:32:31.610593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:26.279697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.602447+0000","last_change":"2024-04-20T21:31:27.507727+0000","last_active":"2024-04-20T21:32:31.602447+0000","last_peered":"2024-04-20T21:32:31.602447+0000","last_clean":"2024-04-20T21:32:31.602447+0000","last_became_active":"2024-04-20T21:31:27.507363+0000","last_became_peered":"2024-04-20T21:31:27.507363+0000","last_unstale":"2024-04-20T21:32:31.602447+0000","last_undegraded":"2024-04-20T21:32:31.602447+0000","last_fullsized":"2024-04-20T21:32:31.602447+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:01.139963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605691+0000","last_change":"2024-04-20T21:31:27.512941+0000","last_active":"2024-04-20T21:32:31.605691+0000","last_peered":"2024-04-20T21:32:31.605691+0000","last_clean":"2024-04-20T21:32:31.605691+0000","last_became_active":"2024-04-20T21:31:27.512571+0000","last_became_peered":"2024-04-20T21:31:27.512571+0000","last_unstale":"2024-04-20T21:32:31.605691+0000","last_undegraded":"2024-04-20T21:32:31.605691+0000","last_fullsized":"2024-04-20T21:32:31.605691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:30.977624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610949+0000","last_change":"2024-04-20T21:31:27.521439+0000","last_active":"2024-04-20T21:32:31.610949+0000","last_peered":"2024-04-20T21:32:31.610949+0000","last_clean":"2024-04-20T21:32:31.610949+0000","last_became_active":"2024-04-20T21:31:27.520245+0000","last_became_peered":"2024-04-20T21:31:27.520245+0000","last_unstale":"2024-04-20T21:32:31.610949+0000","last_undegraded":"2024-04-20T21:32:31.610949+0000","last_fullsized":"2024-04-20T21:32:31.610949+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:53:26.510845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.619209+0000","last_change":"2024-04-20T21:32:31.619209+0000","last_active":"2024-04-20T21:32:31.619209+0000","last_peered":"2024-04-20T21:32:31.619209+0000","last_clean":"2024-04-20T21:32:31.619209+0000","last_became_active":"2024-04-20T21:32:31.618527+0000","last_became_peered":"2024-04-20T21:32:31.618527+0000","last_unstale":"2024-04-20T21:32:31.619209+0000","last_undegraded":"2024-04-20T21:32:31.619209+0000","last_fullsized":"2024-04-20T21:32:31.619209+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:54:21.400896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.608731+0000","last_change":"2024-04-20T21:31:27.521533+0000","last_active":"2024-04-20T21:32:31.608731+0000","last_peered":"2024-04-20T21:32:31.608731+0000","last_clean":"2024-04-20T21:32:31.608731+0000","last_became_active":"2024-04-20T21:31:27.520920+0000","last_became_peered":"2024-04-20T21:31:27.520920+0000","last_unstale":"2024-04-20T21:32:31.608731+0000","last_undegraded":"2024-04-20T21:32:31.608731+0000","last_fullsized":"2024-04-20T21:32:31.608731+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:28:13.605238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606169+0000","last_change":"2024-04-20T21:31:27.527484+0000","last_active":"2024-04-20T21:32:31.606169+0000","last_peered":"2024-04-20T21:32:31.606169+0000","last_clean":"2024-04-20T21:32:31.606169+0000","last_became_active":"2024-04-20T21:31:27.527048+0000","last_became_peered":"2024-04-20T21:31:27.527048+0000","last_unstale":"2024-04-20T21:32:31.606169+0000","last_undegraded":"2024-04-20T21:32:31.606169+0000","last_fullsized":"2024-04-20T21:32:31.606169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:35:56.467661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.615976+0000","last_change":"2024-04-20T21:31:27.527870+0000","last_active":"2024-04-20T21:32:31.615976+0000","last_peered":"2024-04-20T21:32:31.615976+0000","last_clean":"2024-04-20T21:32:31.615976+0000","last_became_active":"2024-04-20T21:31:27.525918+0000","last_became_peered":"2024-04-20T21:31:27.525918+0000","last_unstale":"2024-04-20T21:32:31.615976+0000","last_undegraded":"2024-04-20T21:32:31.615976+0000","last_fullsized":"2024-04-20T21:32:31.615976+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:34.425370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610085+0000","last_change":"2024-04-20T21:31:27.544987+0000","last_active":"2024-04-20T21:32:31.610085+0000","last_peered":"2024-04-20T21:32:31.610085+0000","last_clean":"2024-04-20T21:32:31.610085+0000","last_became_active":"2024-04-20T21:31:27.544628+0000","last_became_peered":"2024-04-20T21:31:27.544628+0000","last_unstale":"2024-04-20T21:32:31.610085+0000","last_undegraded":"2024-04-20T21:32:31.610085+0000","last_fullsized":"2024-04-20T21:32:31.610085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:05:42.396775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.615021+0000","last_change":"2024-04-20T21:31:26.702803+0000","last_active":"2024-04-20T21:32:31.615021+0000","last_peered":"2024-04-20T21:32:31.615021+0000","last_clean":"2024-04-20T21:32:31.615021+0000","last_became_active":"2024-04-20T21:31:26.695781+0000","last_became_peered":"2024-04-20T21:31:26.695781+0000","last_unstale":"2024-04-20T21:32:31.615021+0000","last_undegraded":"2024-04-20T21:32:31.615021+0000","last_fullsized":"2024-04-20T21:32:31.615021+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:49:17.131832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.612344+0000","last_change":"2024-04-20T21:31:27.535264+0000","last_active":"2024-04-20T21:32:31.612344+0000","last_peered":"2024-04-20T21:32:31.612344+0000","last_clean":"2024-04-20T21:32:31.612344+0000","last_became_active":"2024-04-20T21:31:27.534888+0000","last_became_peered":"2024-04-20T21:31:27.534888+0000","last_unstale":"2024-04-20T21:32:31.612344+0000","last_undegraded":"2024-04-20T21:32:31.612344+0000","last_fullsized":"2024-04-20T21:32:31.612344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:04:20.951916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.612789+0000","last_change":"2024-04-20T21:31:26.700378+0000","last_active":"2024-04-20T21:32:31.612789+0000","last_peered":"2024-04-20T21:32:31.612789+0000","last_clean":"2024-04-20T21:32:31.612789+0000","last_became_active":"2024-04-20T21:31:26.700077+0000","last_became_peered":"2024-04-20T21:31:26.700077+0000","last_unstale":"2024-04-20T21:32:31.612789+0000","last_undegraded":"2024-04-20T21:32:31.612789+0000","last_fullsized":"2024-04-20T21:32:31.612789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:34:21.646402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.612348+0000","last_change":"2024-04-20T21:31:27.531069+0000","last_active":"2024-04-20T21:32:31.612348+0000","last_peered":"2024-04-20T21:32:31.612348+0000","last_clean":"2024-04-20T21:32:31.612348+0000","last_became_active":"2024-04-20T21:31:27.530457+0000","last_became_peered":"2024-04-20T21:31:27.530457+0000","last_unstale":"2024-04-20T21:32:31.612348+0000","last_undegraded":"2024-04-20T21:32:31.612348+0000","last_fullsized":"2024-04-20T21:32:31.612348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:46:59.829623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.607657+0000","last_change":"2024-04-20T21:31:26.684150+0000","last_active":"2024-04-20T21:32:31.607657+0000","last_peered":"2024-04-20T21:32:31.607657+0000","last_clean":"2024-04-20T21:32:31.607657+0000","last_became_active":"2024-04-20T21:31:26.683823+0000","last_became_peered":"2024-04-20T21:31:26.683823+0000","last_unstale":"2024-04-20T21:32:31.607657+0000","last_undegraded":"2024-04-20T21:32:31.607657+0000","last_fullsized":"2024-04-20T21:32:31.607657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:14.413606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609028+0000","last_change":"2024-04-20T21:31:27.507499+0000","last_active":"2024-04-20T21:32:31.609028+0000","last_peered":"2024-04-20T21:32:31.609028+0000","last_clean":"2024-04-20T21:32:31.609028+0000","last_became_active":"2024-04-20T21:31:27.506922+0000","last_became_peered":"2024-04-20T21:31:27.506922+0000","last_unstale":"2024-04-20T21:32:31.609028+0000","last_undegraded":"2024-04-20T21:32:31.609028+0000","last_fullsized":"2024-04-20T21:32:31.609028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:40.865232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.614078+0000","last_change":"2024-04-20T21:32:31.613577+0000","last_active":"2024-04-20T21:32:31.614078+0000","last_peered":"2024-04-20T21:32:31.614078+0000","last_clean":"2024-04-20T21:32:31.614078+0000","last_became_active":"2024-04-20T21:32:31.608943+0000","last_became_peered":"2024-04-20T21:32:31.608943+0000","last_unstale":"2024-04-20T21:32:31.614078+0000","last_undegraded":"2024-04-20T21:32:31.614078+0000","last_fullsized":"2024-04-20T21:32:31.614078+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:06.561157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604249+0000","last_change":"2024-04-20T21:31:27.517839+0000","last_active":"2024-04-20T21:32:31.604249+0000","last_peered":"2024-04-20T21:32:31.604249+0000","last_clean":"2024-04-20T21:32:31.604249+0000","last_became_active":"2024-04-20T21:31:27.517044+0000","last_became_peered":"2024-04-20T21:31:27.517044+0000","last_unstale":"2024-04-20T21:32:31.604249+0000","last_undegraded":"2024-04-20T21:32:31.604249+0000","last_fullsized":"2024-04-20T21:32:31.604249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:50.042665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604337+0000","last_change":"2024-04-20T21:31:26.686151+0000","last_active":"2024-04-20T21:32:31.604337+0000","last_peered":"2024-04-20T21:32:31.604337+0000","last_clean":"2024-04-20T21:32:31.604337+0000","last_became_active":"2024-04-20T21:31:26.685827+0000","last_became_peered":"2024-04-20T21:31:26.685827+0000","last_unstale":"2024-04-20T21:32:31.604337+0000","last_undegraded":"2024-04-20T21:32:31.604337+0000","last_fullsized":"2024-04-20T21:32:31.604337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:41:17.263420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.620563+0000","last_change":"2024-04-20T21:32:31.620563+0000","last_active":"2024-04-20T21:32:31.620563+0000","last_peered":"2024-04-20T21:32:31.620563+0000","last_clean":"2024-04-20T21:32:31.620563+0000","last_became_active":"2024-04-20T21:32:31.620145+0000","last_became_peered":"2024-04-20T21:32:31.620145+0000","last_unstale":"2024-04-20T21:32:31.620563+0000","last_undegraded":"2024-04-20T21:32:31.620563+0000","last_fullsized":"2024-04-20T21:32:31.620563+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:43.609490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.995117+0000","last_change":"2024-04-20T21:31:26.684573+0000","last_active":"2024-04-20T21:32:31.995117+0000","last_peered":"2024-04-20T21:32:31.995117+0000","last_clean":"2024-04-20T21:32:31.995117+0000","last_became_active":"2024-04-20T21:31:26.684163+0000","last_became_peered":"2024-04-20T21:31:26.684163+0000","last_unstale":"2024-04-20T21:32:31.995117+0000","last_undegraded":"2024-04-20T21:32:31.995117+0000","last_fullsized":"2024-04-20T21:32:31.995117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:39.445068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604500+0000","last_change":"2024-04-20T21:31:27.543680+0000","last_active":"2024-04-20T21:32:31.604500+0000","last_peered":"2024-04-20T21:32:31.604500+0000","last_clean":"2024-04-20T21:32:31.604500+0000","last_became_active":"2024-04-20T21:31:27.543379+0000","last_became_peered":"2024-04-20T21:31:27.543379+0000","last_unstale":"2024-04-20T21:32:31.604500+0000","last_undegraded":"2024-04-20T21:32:31.604500+0000","last_fullsized":"2024-04-20T21:32:31.604500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:58:17.168813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605053+0000","last_change":"2024-04-20T21:31:26.689665+0000","last_active":"2024-04-20T21:32:31.605053+0000","last_peered":"2024-04-20T21:32:31.605053+0000","last_clean":"2024-04-20T21:32:31.605053+0000","last_became_active":"2024-04-20T21:31:26.689175+0000","last_became_peered":"2024-04-20T21:31:26.689175+0000","last_unstale":"2024-04-20T21:32:31.605053+0000","last_undegraded":"2024-04-20T21:32:31.605053+0000","last_fullsized":"2024-04-20T21:32:31.605053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:21.342700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609476+0000","last_change":"2024-04-20T21:31:27.544137+0000","last_active":"2024-04-20T21:32:31.609476+0000","last_peered":"2024-04-20T21:32:31.609476+0000","last_clean":"2024-04-20T21:32:31.609476+0000","last_became_active":"2024-04-20T21:31:27.542841+0000","last_became_peered":"2024-04-20T21:31:27.542841+0000","last_unstale":"2024-04-20T21:32:31.609476+0000","last_undegraded":"2024-04-20T21:32:31.609476+0000","last_fullsized":"2024-04-20T21:32:31.609476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:04.610144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604714+0000","last_change":"2024-04-20T21:31:26.681937+0000","last_active":"2024-04-20T21:32:31.604714+0000","last_peered":"2024-04-20T21:32:31.604714+0000","last_clean":"2024-04-20T21:32:31.604714+0000","last_became_active":"2024-04-20T21:31:26.681621+0000","last_became_peered":"2024-04-20T21:31:26.681621+0000","last_unstale":"2024-04-20T21:32:31.604714+0000","last_undegraded":"2024-04-20T21:32:31.604714+0000","last_fullsized":"2024-04-20T21:32:31.604714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:06.641608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.612834+0000","last_change":"2024-04-20T21:31:27.540225+0000","last_active":"2024-04-20T21:32:31.612834+0000","last_peered":"2024-04-20T21:32:31.612834+0000","last_clean":"2024-04-20T21:32:31.612834+0000","last_became_active":"2024-04-20T21:31:27.539860+0000","last_became_peered":"2024-04-20T21:31:27.539860+0000","last_unstale":"2024-04-20T21:32:31.612834+0000","last_undegraded":"2024-04-20T21:32:31.612834+0000","last_fullsized":"2024-04-20T21:32:31.612834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:50.146761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.995600+0000","last_change":"2024-04-20T21:31:26.701816+0000","last_active":"2024-04-20T21:32:31.995600+0000","last_peered":"2024-04-20T21:32:31.995600+0000","last_clean":"2024-04-20T21:32:31.995600+0000","last_became_active":"2024-04-20T21:31:26.701530+0000","last_became_peered":"2024-04-20T21:31:26.701530+0000","last_unstale":"2024-04-20T21:32:31.995600+0000","last_undegraded":"2024-04-20T21:32:31.995600+0000","last_fullsized":"2024-04-20T21:32:31.995600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:31.175330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609579+0000","last_change":"2024-04-20T21:31:27.527222+0000","last_active":"2024-04-20T21:32:31.609579+0000","last_peered":"2024-04-20T21:32:31.609579+0000","last_clean":"2024-04-20T21:32:31.609579+0000","last_became_active":"2024-04-20T21:31:27.526779+0000","last_became_peered":"2024-04-20T21:31:27.526779+0000","last_unstale":"2024-04-20T21:32:31.609579+0000","last_undegraded":"2024-04-20T21:32:31.609579+0000","last_fullsized":"2024-04-20T21:32:31.609579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:13:54.500516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610604+0000","last_change":"2024-04-20T21:31:26.685969+0000","last_active":"2024-04-20T21:32:31.610604+0000","last_peered":"2024-04-20T21:32:31.610604+0000","last_clean":"2024-04-20T21:32:31.610604+0000","last_became_active":"2024-04-20T21:31:26.685580+0000","last_became_peered":"2024-04-20T21:31:26.685580+0000","last_unstale":"2024-04-20T21:32:31.610604+0000","last_undegraded":"2024-04-20T21:32:31.610604+0000","last_fullsized":"2024-04-20T21:32:31.610604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:22:50.558526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.612042+0000","last_change":"2024-04-20T21:31:27.525862+0000","last_active":"2024-04-20T21:32:31.612042+0000","last_peered":"2024-04-20T21:32:31.612042+0000","last_clean":"2024-04-20T21:32:31.612042+0000","last_became_active":"2024-04-20T21:31:27.525372+0000","last_became_peered":"2024-04-20T21:31:27.525372+0000","last_unstale":"2024-04-20T21:32:31.612042+0000","last_undegraded":"2024-04-20T21:32:31.612042+0000","last_fullsized":"2024-04-20T21:32:31.612042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:44:15.527137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604956+0000","last_change":"2024-04-20T21:31:26.691091+0000","last_active":"2024-04-20T21:32:31.604956+0000","last_peered":"2024-04-20T21:32:31.604956+0000","last_clean":"2024-04-20T21:32:31.604956+0000","last_became_active":"2024-04-20T21:31:26.690608+0000","last_became_peered":"2024-04-20T21:31:26.690608+0000","last_unstale":"2024-04-20T21:32:31.604956+0000","last_undegraded":"2024-04-20T21:32:31.604956+0000","last_fullsized":"2024-04-20T21:32:31.604956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:44:25.336193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609185+0000","last_change":"2024-04-20T21:31:27.507900+0000","last_active":"2024-04-20T21:32:31.609185+0000","last_peered":"2024-04-20T21:32:31.609185+0000","last_clean":"2024-04-20T21:32:31.609185+0000","last_became_active":"2024-04-20T21:31:27.507360+0000","last_became_peered":"2024-04-20T21:31:27.507360+0000","last_unstale":"2024-04-20T21:32:31.609185+0000","last_undegraded":"2024-04-20T21:32:31.609185+0000","last_fullsized":"2024-04-20T21:32:31.609185+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:27.110877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'26","reported_seq":219,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:53.165145+0000","last_change":"2024-04-20T21:31:26.685736+0000","last_active":"2024-04-20T21:32:53.165145+0000","last_peered":"2024-04-20T21:32:53.165145+0000","last_clean":"2024-04-20T21:32:53.165145+0000","last_became_active":"2024-04-20T21:31:26.685309+0000","last_became_peered":"2024-04-20T21:31:26.685309+0000","last_unstale":"2024-04-20T21:32:53.165145+0000","last_undegraded":"2024-04-20T21:32:53.165145+0000","last_fullsized":"2024-04-20T21:32:53.165145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:06.501753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":174,"num_read_kb":146,"num_write":26,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.607434+0000","last_change":"2024-04-20T21:31:27.507532+0000","last_active":"2024-04-20T21:32:31.607434+0000","last_peered":"2024-04-20T21:32:31.607434+0000","last_clean":"2024-04-20T21:32:31.607434+0000","last_became_active":"2024-04-20T21:31:27.507188+0000","last_became_peered":"2024-04-20T21:31:27.507188+0000","last_unstale":"2024-04-20T21:32:31.607434+0000","last_undegraded":"2024-04-20T21:32:31.607434+0000","last_fullsized":"2024-04-20T21:32:31.607434+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:58.814116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609522+0000","last_change":"2024-04-20T21:31:26.701361+0000","last_active":"2024-04-20T21:32:31.609522+0000","last_peered":"2024-04-20T21:32:31.609522+0000","last_clean":"2024-04-20T21:32:31.609522+0000","last_became_active":"2024-04-20T21:31:26.701068+0000","last_became_peered":"2024-04-20T21:31:26.701068+0000","last_unstale":"2024-04-20T21:32:31.609522+0000","last_undegraded":"2024-04-20T21:32:31.609522+0000","last_fullsized":"2024-04-20T21:32:31.609522+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:22:51.745294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604922+0000","last_change":"2024-04-20T21:31:27.542189+0000","last_active":"2024-04-20T21:32:31.604922+0000","last_peered":"2024-04-20T21:32:31.604922+0000","last_clean":"2024-04-20T21:32:31.604922+0000","last_became_active":"2024-04-20T21:31:27.541800+0000","last_became_peered":"2024-04-20T21:31:27.541800+0000","last_unstale":"2024-04-20T21:32:31.604922+0000","last_undegraded":"2024-04-20T21:32:31.604922+0000","last_fullsized":"2024-04-20T21:32:31.604922+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:31.898443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604003+0000","last_change":"2024-04-20T21:31:26.687652+0000","last_active":"2024-04-20T21:32:31.604003+0000","last_peered":"2024-04-20T21:32:31.604003+0000","last_clean":"2024-04-20T21:32:31.604003+0000","last_became_active":"2024-04-20T21:31:26.687333+0000","last_became_peered":"2024-04-20T21:31:26.687333+0000","last_unstale":"2024-04-20T21:32:31.604003+0000","last_undegraded":"2024-04-20T21:32:31.604003+0000","last_fullsized":"2024-04-20T21:32:31.604003+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:12.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610787+0000","last_change":"2024-04-20T21:31:27.522535+0000","last_active":"2024-04-20T21:32:31.610787+0000","last_peered":"2024-04-20T21:32:31.610787+0000","last_clean":"2024-04-20T21:32:31.610787+0000","last_became_active":"2024-04-20T21:31:27.521821+0000","last_became_peered":"2024-04-20T21:31:27.521821+0000","last_unstale":"2024-04-20T21:32:31.610787+0000","last_undegraded":"2024-04-20T21:32:31.610787+0000","last_fullsized":"2024-04-20T21:32:31.610787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:47:53.401129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.602154+0000","last_change":"2024-04-20T21:31:26.681758+0000","last_active":"2024-04-20T21:32:31.602154+0000","last_peered":"2024-04-20T21:32:31.602154+0000","last_clean":"2024-04-20T21:32:31.602154+0000","last_became_active":"2024-04-20T21:31:26.681395+0000","last_became_peered":"2024-04-20T21:31:26.681395+0000","last_unstale":"2024-04-20T21:32:31.602154+0000","last_undegraded":"2024-04-20T21:32:31.602154+0000","last_fullsized":"2024-04-20T21:32:31.602154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:33:00.924879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609948+0000","last_change":"2024-04-20T21:31:27.512873+0000","last_active":"2024-04-20T21:32:31.609948+0000","last_peered":"2024-04-20T21:32:31.609948+0000","last_clean":"2024-04-20T21:32:31.609948+0000","last_became_active":"2024-04-20T21:31:27.512554+0000","last_became_peered":"2024-04-20T21:31:27.512554+0000","last_unstale":"2024-04-20T21:32:31.609948+0000","last_undegraded":"2024-04-20T21:32:31.609948+0000","last_fullsized":"2024-04-20T21:32:31.609948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:26:06.670726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606898+0000","last_change":"2024-04-20T21:31:26.691210+0000","last_active":"2024-04-20T21:32:31.606898+0000","last_peered":"2024-04-20T21:32:31.606898+0000","last_clean":"2024-04-20T21:32:31.606898+0000","last_became_active":"2024-04-20T21:31:26.690859+0000","last_became_peered":"2024-04-20T21:31:26.690859+0000","last_unstale":"2024-04-20T21:32:31.606898+0000","last_undegraded":"2024-04-20T21:32:31.606898+0000","last_fullsized":"2024-04-20T21:32:31.606898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:36.789429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.616413+0000","last_change":"2024-04-20T21:32:31.616413+0000","last_active":"2024-04-20T21:32:31.616413+0000","last_peered":"2024-04-20T21:32:31.616413+0000","last_clean":"2024-04-20T21:32:31.616413+0000","last_became_active":"2024-04-20T21:32:31.616011+0000","last_became_peered":"2024-04-20T21:32:31.616011+0000","last_unstale":"2024-04-20T21:32:31.616413+0000","last_undegraded":"2024-04-20T21:32:31.616413+0000","last_fullsized":"2024-04-20T21:32:31.616413+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:11:05.630337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.607564+0000","last_change":"2024-04-20T21:31:26.682311+0000","last_active":"2024-04-20T21:32:31.607564+0000","last_peered":"2024-04-20T21:32:31.607564+0000","last_clean":"2024-04-20T21:32:31.607564+0000","last_became_active":"2024-04-20T21:31:26.681996+0000","last_became_peered":"2024-04-20T21:31:26.681996+0000","last_unstale":"2024-04-20T21:32:31.607564+0000","last_undegraded":"2024-04-20T21:32:31.607564+0000","last_fullsized":"2024-04-20T21:32:31.607564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:32:37.659946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605711+0000","last_change":"2024-04-20T21:31:27.510922+0000","last_active":"2024-04-20T21:32:31.605711+0000","last_peered":"2024-04-20T21:32:31.605711+0000","last_clean":"2024-04-20T21:32:31.605711+0000","last_became_active":"2024-04-20T21:31:27.510553+0000","last_became_peered":"2024-04-20T21:31:27.510553+0000","last_unstale":"2024-04-20T21:32:31.605711+0000","last_undegraded":"2024-04-20T21:32:31.605711+0000","last_fullsized":"2024-04-20T21:32:31.605711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:23.754768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.613686+0000","last_change":"2024-04-20T21:31:26.697010+0000","last_active":"2024-04-20T21:32:31.613686+0000","last_peered":"2024-04-20T21:32:31.613686+0000","last_clean":"2024-04-20T21:32:31.613686+0000","last_became_active":"2024-04-20T21:31:26.696112+0000","last_became_peered":"2024-04-20T21:31:26.696112+0000","last_unstale":"2024-04-20T21:32:31.613686+0000","last_undegraded":"2024-04-20T21:32:31.613686+0000","last_fullsized":"2024-04-20T21:32:31.613686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:57:02.715884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.995528+0000","last_change":"2024-04-20T21:31:27.545025+0000","last_active":"2024-04-20T21:32:31.995528+0000","last_peered":"2024-04-20T21:32:31.995528+0000","last_clean":"2024-04-20T21:32:31.995528+0000","last_became_active":"2024-04-20T21:31:27.544630+0000","last_became_peered":"2024-04-20T21:31:27.544630+0000","last_unstale":"2024-04-20T21:32:31.995528+0000","last_undegraded":"2024-04-20T21:32:31.995528+0000","last_fullsized":"2024-04-20T21:32:31.995528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:57.292481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610349+0000","last_change":"2024-04-20T21:31:26.682909+0000","last_active":"2024-04-20T21:32:31.610349+0000","last_peered":"2024-04-20T21:32:31.610349+0000","last_clean":"2024-04-20T21:32:31.610349+0000","last_became_active":"2024-04-20T21:31:26.682597+0000","last_became_peered":"2024-04-20T21:31:26.682597+0000","last_unstale":"2024-04-20T21:32:31.610349+0000","last_undegraded":"2024-04-20T21:32:31.610349+0000","last_fullsized":"2024-04-20T21:32:31.610349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:31.889513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.995498+0000","last_change":"2024-04-20T21:31:27.530217+0000","last_active":"2024-04-20T21:32:31.995498+0000","last_peered":"2024-04-20T21:32:31.995498+0000","last_clean":"2024-04-20T21:32:31.995498+0000","last_became_active":"2024-04-20T21:31:27.529780+0000","last_became_peered":"2024-04-20T21:31:27.529780+0000","last_unstale":"2024-04-20T21:32:31.995498+0000","last_undegraded":"2024-04-20T21:32:31.995498+0000","last_fullsized":"2024-04-20T21:32:31.995498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:21.613460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610727+0000","last_change":"2024-04-20T21:32:31.610727+0000","last_active":"2024-04-20T21:32:31.610727+0000","last_peered":"2024-04-20T21:32:31.610727+0000","last_clean":"2024-04-20T21:32:31.610727+0000","last_became_active":"2024-04-20T21:32:31.610321+0000","last_became_peered":"2024-04-20T21:32:31.610321+0000","last_unstale":"2024-04-20T21:32:31.610727+0000","last_undegraded":"2024-04-20T21:32:31.610727+0000","last_fullsized":"2024-04-20T21:32:31.610727+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:01:35.697137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604769+0000","last_change":"2024-04-20T21:31:26.692845+0000","last_active":"2024-04-20T21:32:31.604769+0000","last_peered":"2024-04-20T21:32:31.604769+0000","last_clean":"2024-04-20T21:32:31.604769+0000","last_became_active":"2024-04-20T21:31:26.691915+0000","last_became_peered":"2024-04-20T21:31:26.691915+0000","last_unstale":"2024-04-20T21:32:31.604769+0000","last_undegraded":"2024-04-20T21:32:31.604769+0000","last_fullsized":"2024-04-20T21:32:31.604769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:24:00.436955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610357+0000","last_change":"2024-04-20T21:31:27.541623+0000","last_active":"2024-04-20T21:32:31.610357+0000","last_peered":"2024-04-20T21:32:31.610357+0000","last_clean":"2024-04-20T21:32:31.610357+0000","last_became_active":"2024-04-20T21:31:27.540953+0000","last_became_peered":"2024-04-20T21:31:27.540953+0000","last_unstale":"2024-04-20T21:32:31.610357+0000","last_undegraded":"2024-04-20T21:32:31.610357+0000","last_fullsized":"2024-04-20T21:32:31.610357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:03:15.091886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.602251+0000","last_change":"2024-04-20T21:31:26.687373+0000","last_active":"2024-04-20T21:32:31.602251+0000","last_peered":"2024-04-20T21:32:31.602251+0000","last_clean":"2024-04-20T21:32:31.602251+0000","last_became_active":"2024-04-20T21:31:26.687009+0000","last_became_peered":"2024-04-20T21:31:26.687009+0000","last_unstale":"2024-04-20T21:32:31.602251+0000","last_undegraded":"2024-04-20T21:32:31.602251+0000","last_fullsized":"2024-04-20T21:32:31.602251+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:23:42.708129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610529+0000","last_change":"2024-04-20T21:30:06.995711+0000","last_active":"2024-04-20T21:32:31.610529+0000","last_peered":"2024-04-20T21:32:31.610529+0000","last_clean":"2024-04-20T21:32:31.610529+0000","last_became_active":"2024-04-20T21:30:06.995280+0000","last_became_peered":"2024-04-20T21:30:06.995280+0000","last_unstale":"2024-04-20T21:32:31.610529+0000","last_undegraded":"2024-04-20T21:32:31.610529+0000","last_fullsized":"2024-04-20T21:32:31.610529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_clean_scrub_stamp":"2024-04-20T21:26:16.477494+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:49:26.014145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604435+0000","last_change":"2024-04-20T21:31:27.514064+0000","last_active":"2024-04-20T21:32:31.604435+0000","last_peered":"2024-04-20T21:32:31.604435+0000","last_clean":"2024-04-20T21:32:31.604435+0000","last_became_active":"2024-04-20T21:31:27.513416+0000","last_became_peered":"2024-04-20T21:31:27.513416+0000","last_unstale":"2024-04-20T21:32:31.604435+0000","last_undegraded":"2024-04-20T21:32:31.604435+0000","last_fullsized":"2024-04-20T21:32:31.604435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:12.658265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610349+0000","last_change":"2024-04-20T21:31:26.680948+0000","last_active":"2024-04-20T21:32:31.610349+0000","last_peered":"2024-04-20T21:32:31.610349+0000","last_clean":"2024-04-20T21:32:31.610349+0000","last_became_active":"2024-04-20T21:31:26.680624+0000","last_became_peered":"2024-04-20T21:31:26.680624+0000","last_unstale":"2024-04-20T21:32:31.610349+0000","last_undegraded":"2024-04-20T21:32:31.610349+0000","last_fullsized":"2024-04-20T21:32:31.610349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:02.750166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.607783+0000","last_change":"2024-04-20T21:31:26.697632+0000","last_active":"2024-04-20T21:32:31.607783+0000","last_peered":"2024-04-20T21:32:31.607783+0000","last_clean":"2024-04-20T21:32:31.607783+0000","last_became_active":"2024-04-20T21:31:26.697393+0000","last_became_peered":"2024-04-20T21:31:26.697393+0000","last_unstale":"2024-04-20T21:32:31.607783+0000","last_undegraded":"2024-04-20T21:32:31.607783+0000","last_fullsized":"2024-04-20T21:32:31.607783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:08:39.856082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606331+0000","last_change":"2024-04-20T21:31:27.526613+0000","last_active":"2024-04-20T21:32:31.606331+0000","last_peered":"2024-04-20T21:32:31.606331+0000","last_clean":"2024-04-20T21:32:31.606331+0000","last_became_active":"2024-04-20T21:31:27.526234+0000","last_became_peered":"2024-04-20T21:31:27.526234+0000","last_unstale":"2024-04-20T21:32:31.606331+0000","last_undegraded":"2024-04-20T21:32:31.606331+0000","last_fullsized":"2024-04-20T21:32:31.606331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:06:44.114924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.625214+0000","last_change":"2024-04-20T21:32:31.625214+0000","last_active":"2024-04-20T21:32:31.625214+0000","last_peered":"2024-04-20T21:32:31.625214+0000","last_clean":"2024-04-20T21:32:31.625214+0000","last_became_active":"2024-04-20T21:32:31.619956+0000","last_became_peered":"2024-04-20T21:32:31.619956+0000","last_unstale":"2024-04-20T21:32:31.625214+0000","last_undegraded":"2024-04-20T21:32:31.625214+0000","last_fullsized":"2024-04-20T21:32:31.625214+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:19:41.620846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.611511+0000","last_change":"2024-04-20T21:31:27.538361+0000","last_active":"2024-04-20T21:32:31.611511+0000","last_peered":"2024-04-20T21:32:31.611511+0000","last_clean":"2024-04-20T21:32:31.611511+0000","last_became_active":"2024-04-20T21:31:27.537494+0000","last_became_peered":"2024-04-20T21:31:27.537494+0000","last_unstale":"2024-04-20T21:32:31.611511+0000","last_undegraded":"2024-04-20T21:32:31.611511+0000","last_fullsized":"2024-04-20T21:32:31.611511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:46.212759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.602516+0000","last_change":"2024-04-20T21:31:26.686960+0000","last_active":"2024-04-20T21:32:31.602516+0000","last_peered":"2024-04-20T21:32:31.602516+0000","last_clean":"2024-04-20T21:32:31.602516+0000","last_became_active":"2024-04-20T21:31:26.686565+0000","last_became_peered":"2024-04-20T21:31:26.686565+0000","last_unstale":"2024-04-20T21:32:31.602516+0000","last_undegraded":"2024-04-20T21:32:31.602516+0000","last_fullsized":"2024-04-20T21:32:31.602516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.608834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610858+0000","last_change":"2024-04-20T21:31:27.546978+0000","last_active":"2024-04-20T21:32:31.610858+0000","last_peered":"2024-04-20T21:32:31.610858+0000","last_clean":"2024-04-20T21:32:31.610858+0000","last_became_active":"2024-04-20T21:31:27.545941+0000","last_became_peered":"2024-04-20T21:31:27.545941+0000","last_unstale":"2024-04-20T21:32:31.610858+0000","last_undegraded":"2024-04-20T21:32:31.610858+0000","last_fullsized":"2024-04-20T21:32:31.610858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:08.363519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.608841+0000","last_change":"2024-04-20T21:31:26.690797+0000","last_active":"2024-04-20T21:32:31.608841+0000","last_peered":"2024-04-20T21:32:31.608841+0000","last_clean":"2024-04-20T21:32:31.608841+0000","last_became_active":"2024-04-20T21:31:26.690486+0000","last_became_peered":"2024-04-20T21:31:26.690486+0000","last_unstale":"2024-04-20T21:32:31.608841+0000","last_undegraded":"2024-04-20T21:32:31.608841+0000","last_fullsized":"2024-04-20T21:32:31.608841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:17:33.943212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606857+0000","last_change":"2024-04-20T21:31:27.507175+0000","last_active":"2024-04-20T21:32:31.606857+0000","last_peered":"2024-04-20T21:32:31.606857+0000","last_clean":"2024-04-20T21:32:31.606857+0000","last_became_active":"2024-04-20T21:31:27.506808+0000","last_became_peered":"2024-04-20T21:31:27.506808+0000","last_unstale":"2024-04-20T21:32:31.606857+0000","last_undegraded":"2024-04-20T21:32:31.606857+0000","last_fullsized":"2024-04-20T21:32:31.606857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:38:15.212880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610329+0000","last_change":"2024-04-20T21:31:26.698302+0000","last_active":"2024-04-20T21:32:31.610329+0000","last_peered":"2024-04-20T21:32:31.610329+0000","last_clean":"2024-04-20T21:32:31.610329+0000","last_became_active":"2024-04-20T21:31:26.696209+0000","last_became_peered":"2024-04-20T21:31:26.696209+0000","last_unstale":"2024-04-20T21:32:31.610329+0000","last_undegraded":"2024-04-20T21:32:31.610329+0000","last_fullsized":"2024-04-20T21:32:31.610329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:43.495880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605694+0000","last_change":"2024-04-20T21:31:27.521594+0000","last_active":"2024-04-20T21:32:31.605694+0000","last_peered":"2024-04-20T21:32:31.605694+0000","last_clean":"2024-04-20T21:32:31.605694+0000","last_became_active":"2024-04-20T21:31:27.521185+0000","last_became_peered":"2024-04-20T21:31:27.521185+0000","last_unstale":"2024-04-20T21:32:31.605694+0000","last_undegraded":"2024-04-20T21:32:31.605694+0000","last_fullsized":"2024-04-20T21:32:31.605694+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:22:08.647624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609271+0000","last_change":"2024-04-20T21:31:26.691536+0000","last_active":"2024-04-20T21:32:31.609271+0000","last_peered":"2024-04-20T21:32:31.609271+0000","last_clean":"2024-04-20T21:32:31.609271+0000","last_became_active":"2024-04-20T21:31:26.691202+0000","last_became_peered":"2024-04-20T21:31:26.691202+0000","last_unstale":"2024-04-20T21:32:31.609271+0000","last_undegraded":"2024-04-20T21:32:31.609271+0000","last_fullsized":"2024-04-20T21:32:31.609271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:58:59.331644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:32.000242+0000","last_change":"2024-04-20T21:32:31.998258+0000","last_active":"2024-04-20T21:32:32.000242+0000","last_peered":"2024-04-20T21:32:32.000242+0000","last_clean":"2024-04-20T21:32:32.000242+0000","last_became_active":"2024-04-20T21:32:31.997119+0000","last_became_peered":"2024-04-20T21:32:31.997119+0000","last_unstale":"2024-04-20T21:32:32.000242+0000","last_undegraded":"2024-04-20T21:32:32.000242+0000","last_fullsized":"2024-04-20T21:32:32.000242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:41:08.565434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.616245+0000","last_change":"2024-04-20T21:32:31.616245+0000","last_active":"2024-04-20T21:32:31.616245+0000","last_peered":"2024-04-20T21:32:31.616245+0000","last_clean":"2024-04-20T21:32:31.616245+0000","last_became_active":"2024-04-20T21:32:31.611535+0000","last_became_peered":"2024-04-20T21:32:31.611535+0000","last_unstale":"2024-04-20T21:32:31.616245+0000","last_undegraded":"2024-04-20T21:32:31.616245+0000","last_fullsized":"2024-04-20T21:32:31.616245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:28:17.609890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610597+0000","last_change":"2024-04-20T21:31:27.518346+0000","last_active":"2024-04-20T21:32:31.610597+0000","last_peered":"2024-04-20T21:32:31.610597+0000","last_clean":"2024-04-20T21:32:31.610597+0000","last_became_active":"2024-04-20T21:31:27.517782+0000","last_became_peered":"2024-04-20T21:31:27.517782+0000","last_unstale":"2024-04-20T21:32:31.610597+0000","last_undegraded":"2024-04-20T21:32:31.610597+0000","last_fullsized":"2024-04-20T21:32:31.610597+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:21.674663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.616197+0000","last_change":"2024-04-20T21:32:31.615626+0000","last_active":"2024-04-20T21:32:31.616197+0000","last_peered":"2024-04-20T21:32:31.616197+0000","last_clean":"2024-04-20T21:32:31.616197+0000","last_became_active":"2024-04-20T21:32:31.615181+0000","last_became_peered":"2024-04-20T21:32:31.615181+0000","last_unstale":"2024-04-20T21:32:31.616197+0000","last_undegraded":"2024-04-20T21:32:31.616197+0000","last_fullsized":"2024-04-20T21:32:31.616197+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:58:18.344583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610670+0000","last_change":"2024-04-20T21:31:27.510920+0000","last_active":"2024-04-20T21:32:31.610670+0000","last_peered":"2024-04-20T21:32:31.610670+0000","last_clean":"2024-04-20T21:32:31.610670+0000","last_became_active":"2024-04-20T21:31:27.510564+0000","last_became_peered":"2024-04-20T21:31:27.510564+0000","last_unstale":"2024-04-20T21:32:31.610670+0000","last_undegraded":"2024-04-20T21:32:31.610670+0000","last_fullsized":"2024-04-20T21:32:31.610670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:05.196178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604136+0000","last_change":"2024-04-20T21:31:26.693159+0000","last_active":"2024-04-20T21:32:31.604136+0000","last_peered":"2024-04-20T21:32:31.604136+0000","last_clean":"2024-04-20T21:32:31.604136+0000","last_became_active":"2024-04-20T21:31:26.692902+0000","last_became_peered":"2024-04-20T21:31:26.692902+0000","last_unstale":"2024-04-20T21:32:31.604136+0000","last_undegraded":"2024-04-20T21:32:31.604136+0000","last_fullsized":"2024-04-20T21:32:31.604136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:54.704343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610372+0000","last_change":"2024-04-20T21:31:27.541557+0000","last_active":"2024-04-20T21:32:31.610372+0000","last_peered":"2024-04-20T21:32:31.610372+0000","last_clean":"2024-04-20T21:32:31.610372+0000","last_became_active":"2024-04-20T21:31:27.541161+0000","last_became_peered":"2024-04-20T21:31:27.541161+0000","last_unstale":"2024-04-20T21:32:31.610372+0000","last_undegraded":"2024-04-20T21:32:31.610372+0000","last_fullsized":"2024-04-20T21:32:31.610372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:55:25.558918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606758+0000","last_change":"2024-04-20T21:31:26.688019+0000","last_active":"2024-04-20T21:32:31.606758+0000","last_peered":"2024-04-20T21:32:31.606758+0000","last_clean":"2024-04-20T21:32:31.606758+0000","last_became_active":"2024-04-20T21:31:26.686370+0000","last_became_peered":"2024-04-20T21:31:26.686370+0000","last_unstale":"2024-04-20T21:32:31.606758+0000","last_undegraded":"2024-04-20T21:32:31.606758+0000","last_fullsized":"2024-04-20T21:32:31.606758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:42:44.059365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610642+0000","last_change":"2024-04-20T21:31:27.511592+0000","last_active":"2024-04-20T21:32:31.610642+0000","last_peered":"2024-04-20T21:32:31.610642+0000","last_clean":"2024-04-20T21:32:31.610642+0000","last_became_active":"2024-04-20T21:31:27.511311+0000","last_became_peered":"2024-04-20T21:31:27.511311+0000","last_unstale":"2024-04-20T21:32:31.610642+0000","last_undegraded":"2024-04-20T21:32:31.610642+0000","last_fullsized":"2024-04-20T21:32:31.610642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:35:27.665578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605815+0000","last_change":"2024-04-20T21:31:27.511827+0000","last_active":"2024-04-20T21:32:31.605815+0000","last_peered":"2024-04-20T21:32:31.605815+0000","last_clean":"2024-04-20T21:32:31.605815+0000","last_became_active":"2024-04-20T21:31:27.511344+0000","last_became_peered":"2024-04-20T21:31:27.511344+0000","last_unstale":"2024-04-20T21:32:31.605815+0000","last_undegraded":"2024-04-20T21:32:31.605815+0000","last_fullsized":"2024-04-20T21:32:31.605815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:11:37.973050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609339+0000","last_change":"2024-04-20T21:31:27.510747+0000","last_active":"2024-04-20T21:32:31.609339+0000","last_peered":"2024-04-20T21:32:31.609339+0000","last_clean":"2024-04-20T21:32:31.609339+0000","last_became_active":"2024-04-20T21:31:27.510253+0000","last_became_peered":"2024-04-20T21:31:27.510253+0000","last_unstale":"2024-04-20T21:32:31.609339+0000","last_undegraded":"2024-04-20T21:32:31.609339+0000","last_fullsized":"2024-04-20T21:32:31.609339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:37:13.855840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609135+0000","last_change":"2024-04-20T21:31:27.541185+0000","last_active":"2024-04-20T21:32:31.609135+0000","last_peered":"2024-04-20T21:32:31.609135+0000","last_clean":"2024-04-20T21:32:31.609135+0000","last_became_active":"2024-04-20T21:31:27.540881+0000","last_became_peered":"2024-04-20T21:31:27.540881+0000","last_unstale":"2024-04-20T21:32:31.609135+0000","last_undegraded":"2024-04-20T21:32:31.609135+0000","last_fullsized":"2024-04-20T21:32:31.609135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:46:41.897081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609602+0000","last_change":"2024-04-20T21:31:27.527204+0000","last_active":"2024-04-20T21:32:31.609602+0000","last_peered":"2024-04-20T21:32:31.609602+0000","last_clean":"2024-04-20T21:32:31.609602+0000","last_became_active":"2024-04-20T21:31:27.526805+0000","last_became_peered":"2024-04-20T21:31:27.526805+0000","last_unstale":"2024-04-20T21:32:31.609602+0000","last_undegraded":"2024-04-20T21:32:31.609602+0000","last_fullsized":"2024-04-20T21:32:31.609602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:43.032609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605867+0000","last_change":"2024-04-20T21:31:27.525521+0000","last_active":"2024-04-20T21:32:31.605867+0000","last_peered":"2024-04-20T21:32:31.605867+0000","last_clean":"2024-04-20T21:32:31.605867+0000","last_became_active":"2024-04-20T21:31:27.521892+0000","last_became_peered":"2024-04-20T21:31:27.521892+0000","last_unstale":"2024-04-20T21:32:31.605867+0000","last_undegraded":"2024-04-20T21:32:31.605867+0000","last_fullsized":"2024-04-20T21:32:31.605867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:32:47.683115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605887+0000","last_change":"2024-04-20T21:31:27.509828+0000","last_active":"2024-04-20T21:32:31.605887+0000","last_peered":"2024-04-20T21:32:31.605887+0000","last_clean":"2024-04-20T21:32:31.605887+0000","last_became_active":"2024-04-20T21:31:27.509269+0000","last_became_peered":"2024-04-20T21:31:27.509269+0000","last_unstale":"2024-04-20T21:32:31.605887+0000","last_undegraded":"2024-04-20T21:32:31.605887+0000","last_fullsized":"2024-04-20T21:32:31.605887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:25:54.492533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.610240+0000","last_change":"2024-04-20T21:31:27.512542+0000","last_active":"2024-04-20T21:32:31.610240+0000","last_peered":"2024-04-20T21:32:31.610240+0000","last_clean":"2024-04-20T21:32:31.610240+0000","last_became_active":"2024-04-20T21:31:27.511907+0000","last_became_peered":"2024-04-20T21:31:27.511907+0000","last_unstale":"2024-04-20T21:32:31.610240+0000","last_undegraded":"2024-04-20T21:32:31.610240+0000","last_fullsized":"2024-04-20T21:32:31.610240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:10:09.265471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.611218+0000","last_change":"2024-04-20T21:31:27.538697+0000","last_active":"2024-04-20T21:32:31.611218+0000","last_peered":"2024-04-20T21:32:31.611218+0000","last_clean":"2024-04-20T21:32:31.611218+0000","last_became_active":"2024-04-20T21:31:27.537972+0000","last_became_peered":"2024-04-20T21:31:27.537972+0000","last_unstale":"2024-04-20T21:32:31.611218+0000","last_undegraded":"2024-04-20T21:32:31.611218+0000","last_fullsized":"2024-04-20T21:32:31.611218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:51.974485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.604729+0000","last_change":"2024-04-20T21:31:27.538092+0000","last_active":"2024-04-20T21:32:31.604729+0000","last_peered":"2024-04-20T21:32:31.604729+0000","last_clean":"2024-04-20T21:32:31.604729+0000","last_became_active":"2024-04-20T21:31:27.537821+0000","last_became_peered":"2024-04-20T21:31:27.537821+0000","last_unstale":"2024-04-20T21:32:31.604729+0000","last_undegraded":"2024-04-20T21:32:31.604729+0000","last_fullsized":"2024-04-20T21:32:31.604729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:52.773583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609957+0000","last_change":"2024-04-20T21:31:27.535498+0000","last_active":"2024-04-20T21:32:31.609957+0000","last_peered":"2024-04-20T21:32:31.609957+0000","last_clean":"2024-04-20T21:32:31.609957+0000","last_became_active":"2024-04-20T21:31:27.535162+0000","last_became_peered":"2024-04-20T21:31:27.535162+0000","last_unstale":"2024-04-20T21:32:31.609957+0000","last_undegraded":"2024-04-20T21:32:31.609957+0000","last_fullsized":"2024-04-20T21:32:31.609957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:46.638825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.609120+0000","last_change":"2024-04-20T21:31:27.537776+0000","last_active":"2024-04-20T21:32:31.609120+0000","last_peered":"2024-04-20T21:32:31.609120+0000","last_clean":"2024-04-20T21:32:31.609120+0000","last_became_active":"2024-04-20T21:31:27.537428+0000","last_became_peered":"2024-04-20T21:31:27.537428+0000","last_unstale":"2024-04-20T21:32:31.609120+0000","last_undegraded":"2024-04-20T21:32:31.609120+0000","last_fullsized":"2024-04-20T21:32:31.609120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:15:03.344277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.605983+0000","last_change":"2024-04-20T21:31:27.535943+0000","last_active":"2024-04-20T21:32:31.605983+0000","last_peered":"2024-04-20T21:32:31.605983+0000","last_clean":"2024-04-20T21:32:31.605983+0000","last_became_active":"2024-04-20T21:31:27.533694+0000","last_became_peered":"2024-04-20T21:31:27.533694+0000","last_unstale":"2024-04-20T21:32:31.605983+0000","last_undegraded":"2024-04-20T21:32:31.605983+0000","last_fullsized":"2024-04-20T21:32:31.605983+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:42.603232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-20T21:32:31.606370+0000","last_change":"2024-04-20T21:31:27.511345+0000","last_active":"2024-04-20T21:32:31.606370+0000","last_peered":"2024-04-20T21:32:31.606370+0000","last_clean":"2024-04-20T21:32:31.606370+0000","last_became_active":"2024-04-20T21:31:27.510984+0000","last_became_peered":"2024-04-20T21:31:27.510984+0000","last_unstale":"2024-04-20T21:32:31.606370+0000","last_undegraded":"2024-04-20T21:32:31.606370+0000","last_fullsized":"2024-04-20T21:32:31.606370+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:19.965636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61844,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":180,"num_read_kb":152,"num_write":79,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":185532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285600,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":469132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547236,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1658880,"data_stored":1120857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710762,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":983040,"data_stored":469714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972402,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":469736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266741,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1060199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430269,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":469249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":530178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":469222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":530212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":469470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":469154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691906,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36896,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708256,"statfs":{"total":95995035648,"available":95957254144,"internally_reserved":0,"allocated":1589248,"data_stored":1059799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:32:56.833 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-20T21:32:56.833 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T21:32:56.833 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-20T21:32:56.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph health --format=json 2024-04-20T21:32:57.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:56 smithi022 ceph-mon[16843]: pgmap v363: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:57.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:56 smithi022 ceph-mon[16843]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:32:57.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:56 smithi122 ceph-mon[22147]: pgmap v363: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:57.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:56 smithi122 ceph-mon[22147]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:32:57.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:56 smithi033 ceph-mon[21626]: pgmap v363: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:57.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:56 smithi033 ceph-mon[21626]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:32:57.333 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:59.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:58 smithi022 ceph-mon[16843]: pgmap v364: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:59.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:58 smithi122 ceph-mon[22147]: pgmap v364: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:59.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:58 smithi033 ceph-mon[21626]: pgmap v364: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:32:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.547+0000 7f9121800700 1 -- 172.21.15.22:0/1699543486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911c104de0 msgr2=0x7f911c1051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.547+0000 7f9121800700 1 --2- 172.21.15.22:0/1699543486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911c104de0 0x7f911c1051c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f910c0099c0 tx=0x7f910c009cd0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.547+0000 7f9121800700 1 -- 172.21.15.22:0/1699543486 shutdown_connections 2024-04-20T21:32:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.547+0000 7f9121800700 1 --2- 172.21.15.22:0/1699543486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f911c109ec0 0x7f911c111a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.547+0000 7f9121800700 1 --2- 172.21.15.22:0/1699543486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911c104de0 0x7f911c1051c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.547+0000 7f9121800700 1 --2- 172.21.15.22:0/1699543486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911c105790 0x7f911c1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.547+0000 7f9121800700 1 -- 172.21.15.22:0/1699543486 >> 172.21.15.22:0/1699543486 conn(0x7f911c100670 msgr2=0x7f911c102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.547+0000 7f9121800700 1 -- 172.21.15.22:0/1699543486 shutdown_connections 2024-04-20T21:32:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.547+0000 7f9121800700 1 -- 172.21.15.22:0/1699543486 wait complete. 2024-04-20T21:32:59.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.549+0000 7f9121800700 1 Processor -- start 2024-04-20T21:32:59.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.549+0000 7f9121800700 1 -- start start 2024-04-20T21:32:59.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.549+0000 7f9121800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f911c104de0 0x7f911c19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:59.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.549+0000 7f9121800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911c105790 0x7f911c19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:59.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.549+0000 7f9121800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911c109ec0 0x7f911c19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:59.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.549+0000 7f9121800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f911c114ec0 con 0x7f911c109ec0 2024-04-20T21:32:59.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.549+0000 7f9121800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f911c114d40 con 0x7f911c105790 2024-04-20T21:32:59.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.549+0000 7f9121800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f911c115040 con 0x7f911c104de0 2024-04-20T21:32:59.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.550+0000 7f911be00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f911c104de0 0x7f911c19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.550+0000 7f911be00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f911c104de0 0x7f911c19b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:54444/0 (socket says 172.21.15.22:54444) 2024-04-20T21:32:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.550+0000 7f911be00700 1 -- 172.21.15.22:0/221539922 learned_addr learned my addr 172.21.15.22:0/221539922 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:32:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.550+0000 7f9120e00700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911c109ec0 0x7f911c19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.550+0000 7f911b400700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911c105790 0x7f911c19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.550+0000 7f911be00700 1 -- 172.21.15.22:0/221539922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911c105790 msgr2=0x7f911c19b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.550+0000 7f911be00700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911c105790 0x7f911c19b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.550+0000 7f911be00700 1 -- 172.21.15.22:0/221539922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911c109ec0 msgr2=0x7f911c19fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.550+0000 7f911be00700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911c109ec0 0x7f911c19fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.550+0000 7f911be00700 1 -- 172.21.15.22:0/221539922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c1a02e0 con 0x7f911c104de0 2024-04-20T21:32:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.551+0000 7f911be00700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f911c104de0 0x7f911c19b2d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f910c006010 tx=0x7f910c003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:59.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.551+0000 7f9118c00700 1 -- 172.21.15.22:0/221539922 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c003680 con 0x7f911c104de0 2024-04-20T21:32:59.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.551+0000 7f9121800700 1 -- 172.21.15.22:0/221539922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911c1a0570 con 0x7f911c104de0 2024-04-20T21:32:59.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.551+0000 7f9118c00700 1 -- 172.21.15.22:0/221539922 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f910c003e50 con 0x7f911c104de0 2024-04-20T21:32:59.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.551+0000 7f9118c00700 1 -- 172.21.15.22:0/221539922 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c00fe10 con 0x7f911c104de0 2024-04-20T21:32:59.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.551+0000 7f9121800700 1 -- 172.21.15.22:0/221539922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911c1a7e50 con 0x7f911c104de0 2024-04-20T21:32:59.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.552+0000 7f9121800700 1 -- 172.21.15.22:0/221539922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911c069740 con 0x7f911c104de0 2024-04-20T21:32:59.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.554+0000 7f9118c00700 1 -- 172.21.15.22:0/221539922 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f910c00f460 con 0x7f911c104de0 2024-04-20T21:32:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.556+0000 7f9118c00700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91000a3ef0 0x7f91000a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.556+0000 7f9118c00700 1 -- 172.21.15.22:0/221539922 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f910c0d11d0 con 0x7f911c104de0 2024-04-20T21:32:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.556+0000 7f911b400700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91000a3ef0 0x7f91000a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.557+0000 7f911b400700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91000a3ef0 0x7f91000a63b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f911c10b460 tx=0x7f9108009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:59.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.561+0000 7f9118c00700 1 -- 172.21.15.22:0/221539922 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f910c013070 con 0x7f911c104de0 2024-04-20T21:32:59.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.877+0000 7f9121800700 1 -- 172.21.15.22:0/221539922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f911c02df90 con 0x7f911c104de0 2024-04-20T21:32:59.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.878+0000 7f9118c00700 1 -- 172.21.15.22:0/221539922 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f910c09da60 con 0x7f911c104de0 2024-04-20T21:32:59.878 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:32:59.878 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T21:32:59.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.882+0000 7f9121800700 1 -- 172.21.15.22:0/221539922 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91000a3ef0 msgr2=0x7f91000a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:59.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.882+0000 7f9121800700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91000a3ef0 0x7f91000a63b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f911c10b460 tx=0x7f9108009260 comp rx=0 tx=0).stop 2024-04-20T21:32:59.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.883+0000 7f9121800700 1 -- 172.21.15.22:0/221539922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f911c104de0 msgr2=0x7f911c19b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:59.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.883+0000 7f9121800700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f911c104de0 0x7f911c19b2d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f910c006010 tx=0x7f910c003a90 comp rx=0 tx=0).stop 2024-04-20T21:32:59.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.883+0000 7f9121800700 1 -- 172.21.15.22:0/221539922 shutdown_connections 2024-04-20T21:32:59.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.883+0000 7f9121800700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91000a3ef0 0x7f91000a63b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.883+0000 7f9121800700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911c109ec0 0x7f911c19fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.883+0000 7f9121800700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f911c104de0 0x7f911c19b2d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.883+0000 7f9121800700 1 --2- 172.21.15.22:0/221539922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911c105790 0x7f911c19b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.883+0000 7f9121800700 1 -- 172.21.15.22:0/221539922 >> 172.21.15.22:0/221539922 conn(0x7f911c100670 msgr2=0x7f911c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:59.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.883+0000 7f9121800700 1 -- 172.21.15.22:0/221539922 shutdown_connections 2024-04-20T21:32:59.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:32:59.883+0000 7f9121800700 1 -- 172.21.15.22:0/221539922 wait complete. 2024-04-20T21:33:00.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:32:59 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/221539922' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T21:33:00.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:32:59 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/221539922' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T21:33:00.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:32:59 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/221539922' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T21:33:00.788 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-20T21:33:00.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.018+0000 7f4a17200640 1 -- 172.21.15.22:0/3498419990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a181051d0 msgr2=0x7f4a18105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.018+0000 7f4a17200640 1 --2- 172.21.15.22:0/3498419990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a181051d0 0x7f4a18105650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009900 tx=0x7f4a0002bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.018+0000 7f4a17200640 1 -- 172.21.15.22:0/3498419990 shutdown_connections 2024-04-20T21:33:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.018+0000 7f4a17200640 1 --2- 172.21.15.22:0/3498419990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a18105b90 0x7f4a1810c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.018+0000 7f4a17200640 1 --2- 172.21.15.22:0/3498419990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a181051d0 0x7f4a18105650 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.018+0000 7f4a17200640 1 --2- 172.21.15.22:0/3498419990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18103fe0 0x7f4a18104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.018+0000 7f4a17200640 1 -- 172.21.15.22:0/3498419990 >> 172.21.15.22:0/3498419990 conn(0x7f4a180ff570 msgr2=0x7f4a18101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.019+0000 7f4a17200640 1 -- 172.21.15.22:0/3498419990 shutdown_connections 2024-04-20T21:33:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.019+0000 7f4a17200640 1 -- 172.21.15.22:0/3498419990 wait complete. 2024-04-20T21:33:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.020+0000 7f4a17200640 1 Processor -- start 2024-04-20T21:33:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.020+0000 7f4a17200640 1 -- start start 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.020+0000 7f4a17200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a18103fe0 0x7f4a1819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.020+0000 7f4a17200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a181051d0 0x7f4a1819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.020+0000 7f4a17200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18105b90 0x7f4a181a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.020+0000 7f4a17200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1810fcc0 con 0x7f4a18103fe0 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.020+0000 7f4a17200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1810fb40 con 0x7f4a18105b90 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.020+0000 7f4a17200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1810fe40 con 0x7f4a181051d0 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.020+0000 7f4a15e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a18103fe0 0x7f4a1819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.020+0000 7f4a15e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a18103fe0 0x7f4a1819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49366/0 (socket says 172.21.15.22:49366) 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.020+0000 7f4a15e00640 1 -- 172.21.15.22:0/1492659272 learned_addr learned my addr 172.21.15.22:0/1492659272 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a16800640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18105b90 0x7f4a181a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a0fe00640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a181051d0 0x7f4a1819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a15e00640 1 -- 172.21.15.22:0/1492659272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a181051d0 msgr2=0x7f4a1819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a15e00640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a181051d0 0x7f4a1819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a15e00640 1 -- 172.21.15.22:0/1492659272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18105b90 msgr2=0x7f4a181a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a15e00640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18105b90 0x7f4a181a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a15e00640 1 -- 172.21.15.22:0/1492659272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a181a1eb0 con 0x7f4a18103fe0 2024-04-20T21:33:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a0fe00640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a181051d0 0x7f4a1819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a15e00640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a18103fe0 0x7f4a1819a290 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800e9c0 tx=0x7f4a0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a0ea00640 1 -- 172.21.15.22:0/1492659272 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0800cd80 con 0x7f4a18103fe0 2024-04-20T21:33:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a0ea00640 1 -- 172.21.15.22:0/1492659272 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a080035f0 con 0x7f4a18103fe0 2024-04-20T21:33:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a0ea00640 1 -- 172.21.15.22:0/1492659272 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08010690 con 0x7f4a18103fe0 2024-04-20T21:33:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.021+0000 7f4a17200640 1 -- 172.21.15.22:0/1492659272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a181a21a0 con 0x7f4a18103fe0 2024-04-20T21:33:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.022+0000 7f4a17200640 1 -- 172.21.15.22:0/1492659272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a181a26c0 con 0x7f4a18103fe0 2024-04-20T21:33:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.023+0000 7f4a17200640 1 -- 172.21.15.22:0/1492659272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1810b300 con 0x7f4a18103fe0 2024-04-20T21:33:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.025+0000 7f4a0ea00640 1 -- 172.21.15.22:0/1492659272 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4a080037a0 con 0x7f4a18103fe0 2024-04-20T21:33:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.026+0000 7f4a0ea00640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f49fc0a3bb0 0x7f49fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.026+0000 7f4a0ea00640 1 -- 172.21.15.22:0/1492659272 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f4a080d0930 con 0x7f4a18103fe0 2024-04-20T21:33:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.026+0000 7f4a0fe00640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f49fc0a3bb0 0x7f49fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.027+0000 7f4a0fe00640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f49fc0a3bb0 0x7f49fc0a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4a0002c240 tx=0x7f4a0002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.031+0000 7f4a0ea00640 1 -- 172.21.15.22:0/1492659272 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4a0809d290 con 0x7f4a18103fe0 2024-04-20T21:33:01.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.242+0000 7f4a17200640 1 -- 172.21.15.22:0/1492659272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a18062b90 con 0x7f4a18103fe0 2024-04-20T21:33:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.246+0000 7f4a0ea00640 1 -- 172.21.15.22:0/1492659272 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22721 (secure 0 0 0) 0x7f4a0809cb90 con 0x7f4a18103fe0 2024-04-20T21:33:01.245 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:01.245 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":90,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:32:31.596064+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.249+0000 7f4a17200640 1 -- 172.21.15.22:0/1492659272 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f49fc0a3bb0 msgr2=0x7f49fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.249+0000 7f4a17200640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f49fc0a3bb0 0x7f49fc0a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4a0002c240 tx=0x7f4a0002df90 comp rx=0 tx=0).stop 2024-04-20T21:33:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.249+0000 7f4a17200640 1 -- 172.21.15.22:0/1492659272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a18103fe0 msgr2=0x7f4a1819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.249+0000 7f4a17200640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a18103fe0 0x7f4a1819a290 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800e9c0 tx=0x7f4a0800ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.249+0000 7f4a17200640 1 -- 172.21.15.22:0/1492659272 shutdown_connections 2024-04-20T21:33:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.249+0000 7f4a17200640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f49fc0a3bb0 0x7f49fc0a6070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.249+0000 7f4a17200640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18105b90 0x7f4a181a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.249+0000 7f4a17200640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a181051d0 0x7f4a1819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.249+0000 7f4a17200640 1 --2- 172.21.15.22:0/1492659272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a18103fe0 0x7f4a1819a290 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.249+0000 7f4a17200640 1 -- 172.21.15.22:0/1492659272 >> 172.21.15.22:0/1492659272 conn(0x7f4a180ff570 msgr2=0x7f4a181084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:01.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.249+0000 7f4a17200640 1 -- 172.21.15.22:0/1492659272 shutdown_connections 2024-04-20T21:33:01.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.249+0000 7f4a17200640 1 -- 172.21.15.22:0/1492659272 wait complete. 2024-04-20T21:33:01.265 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:00 smithi022 ceph-mon[16843]: pgmap v365: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:01.265 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:00 smithi022 ceph-mon[16843]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-20T21:33:01.269 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:26:15.688818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:31:25.506074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:31:26.126261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:33:01.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:33:01.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:00 smithi122 ceph-mon[22147]: pgmap v365: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:01.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:00 smithi122 ceph-mon[22147]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-20T21:33:01.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:00 smithi033 ceph-mon[21626]: pgmap v365: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:01.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:00 smithi033 ceph-mon[21626]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-20T21:33:01.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.486+0000 7f0191a00640 1 -- 172.21.15.22:0/1034651338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f018c1095b0 msgr2=0x7f018c109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.486+0000 7f0191a00640 1 --2- 172.21.15.22:0/1034651338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f018c1095b0 0x7f018c109a30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0174009900 tx=0x7f017402bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.486+0000 7f0191a00640 1 -- 172.21.15.22:0/1034651338 shutdown_connections 2024-04-20T21:33:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.486+0000 7f0191a00640 1 --2- 172.21.15.22:0/1034651338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f018c109f70 0x7f018c110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.486+0000 7f0191a00640 1 --2- 172.21.15.22:0/1034651338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f018c1095b0 0x7f018c109a30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.486+0000 7f0191a00640 1 --2- 172.21.15.22:0/1034651338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f018c1083c0 0x7f018c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.486+0000 7f0191a00640 1 -- 172.21.15.22:0/1034651338 >> 172.21.15.22:0/1034651338 conn(0x7f018c077360 msgr2=0x7f018c0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.486+0000 7f0191a00640 1 -- 172.21.15.22:0/1034651338 shutdown_connections 2024-04-20T21:33:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.486+0000 7f0191a00640 1 -- 172.21.15.22:0/1034651338 wait complete. 2024-04-20T21:33:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.487+0000 7f0191a00640 1 Processor -- start 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.487+0000 7f0191a00640 1 -- start start 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0191a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f018c1083c0 0x7f018c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0191a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f018c1095b0 0x7f018c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0191a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f018c109f70 0x7f018c1a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0191a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f018c1140a0 con 0x7f018c109f70 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0191a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f018c113f20 con 0x7f018c1083c0 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0191a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f018c114220 con 0x7f018c1095b0 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0191000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f018c109f70 0x7f018c1a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0183e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f018c1095b0 0x7f018c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0183e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f018c1095b0 0x7f018c19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:54474/0 (socket says 172.21.15.22:54474) 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0183e00640 1 -- 172.21.15.22:0/1510523192 learned_addr learned my addr 172.21.15.22:0/1510523192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f018be00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f018c1083c0 0x7f018c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0183e00640 1 -- 172.21.15.22:0/1510523192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f018c1083c0 msgr2=0x7f018c19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0183e00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f018c1083c0 0x7f018c19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0183e00640 1 -- 172.21.15.22:0/1510523192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f018c109f70 msgr2=0x7f018c1a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0183e00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f018c109f70 0x7f018c1a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f0183e00640 1 -- 172.21.15.22:0/1510523192 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018c1a6260 con 0x7f018c1095b0 2024-04-20T21:33:01.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.488+0000 7f018be00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f018c1083c0 0x7f018c19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:01.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.489+0000 7f0183e00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f018c1095b0 0x7f018c19eb80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f017402c8f0 tx=0x7f0174005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:01.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.489+0000 7f0189600640 1 -- 172.21.15.22:0/1510523192 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174041070 con 0x7f018c1095b0 2024-04-20T21:33:01.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.489+0000 7f0191a00640 1 -- 172.21.15.22:0/1510523192 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018c1a64f0 con 0x7f018c1095b0 2024-04-20T21:33:01.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.489+0000 7f0189600640 1 -- 172.21.15.22:0/1510523192 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01740044d0 con 0x7f018c1095b0 2024-04-20T21:33:01.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.489+0000 7f0189600640 1 -- 172.21.15.22:0/1510523192 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174034440 con 0x7f018c1095b0 2024-04-20T21:33:01.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.489+0000 7f0191a00640 1 -- 172.21.15.22:0/1510523192 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018c1a69e0 con 0x7f018c1095b0 2024-04-20T21:33:01.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.490+0000 7f0191a00640 1 -- 172.21.15.22:0/1510523192 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f018c10f6e0 con 0x7f018c1095b0 2024-04-20T21:33:01.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.493+0000 7f0189600640 1 -- 172.21.15.22:0/1510523192 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0174044070 con 0x7f018c1095b0 2024-04-20T21:33:01.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.494+0000 7f0189600640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f01700a3bb0 0x7f01700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:01.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.494+0000 7f0189600640 1 -- 172.21.15.22:0/1510523192 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f01740f20b0 con 0x7f018c1095b0 2024-04-20T21:33:01.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.494+0000 7f018be00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f01700a3bb0 0x7f01700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:01.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.495+0000 7f018be00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f01700a3bb0 0x7f01700a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f018c109410 tx=0x7f017c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:01.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.497+0000 7f0189600640 1 -- 172.21.15.22:0/1510523192 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f01740be420 con 0x7f018c1095b0 2024-04-20T21:33:01.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.701+0000 7f0191a00640 1 -- 172.21.15.22:0/1510523192 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f018c02d760 con 0x7f018c1095b0 2024-04-20T21:33:01.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.702+0000 7f0189600640 1 -- 172.21.15.22:0/1510523192 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f01740c2720 con 0x7f018c1095b0 2024-04-20T21:33:01.701 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-20T21:33:01.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.705+0000 7f0191a00640 1 -- 172.21.15.22:0/1510523192 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f01700a3bb0 msgr2=0x7f01700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.705+0000 7f0191a00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f01700a3bb0 0x7f01700a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f018c109410 tx=0x7f017c005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:01.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.705+0000 7f0191a00640 1 -- 172.21.15.22:0/1510523192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f018c1095b0 msgr2=0x7f018c19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.705+0000 7f0191a00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f018c1095b0 0x7f018c19eb80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f017402c8f0 tx=0x7f0174005d50 comp rx=0 tx=0).stop 2024-04-20T21:33:01.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.705+0000 7f0191a00640 1 -- 172.21.15.22:0/1510523192 shutdown_connections 2024-04-20T21:33:01.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.705+0000 7f0191a00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f01700a3bb0 0x7f01700a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.705+0000 7f0191a00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f018c109f70 0x7f018c1a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.705+0000 7f0191a00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f018c1095b0 0x7f018c19eb80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.705+0000 7f0191a00640 1 --2- 172.21.15.22:0/1510523192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f018c1083c0 0x7f018c19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.705+0000 7f0191a00640 1 -- 172.21.15.22:0/1510523192 >> 172.21.15.22:0/1510523192 conn(0x7f018c077360 msgr2=0x7f018c10c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:01.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.705+0000 7f0191a00640 1 -- 172.21.15.22:0/1510523192 shutdown_connections 2024-04-20T21:33:01.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.705+0000 7f0191a00640 1 -- 172.21.15.22:0/1510523192 wait complete. 2024-04-20T21:33:01.723 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:33:01.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.934+0000 7f8cf5200640 1 -- 172.21.15.22:0/1496714184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf01040a0 msgr2=0x7f8cf01044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.934+0000 7f8cf5200640 1 --2- 172.21.15.22:0/1496714184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf01040a0 0x7f8cf01044c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009900 tx=0x7f8ce002bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:01.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.934+0000 7f8cf5200640 1 -- 172.21.15.22:0/1496714184 shutdown_connections 2024-04-20T21:33:01.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.934+0000 7f8cf5200640 1 --2- 172.21.15.22:0/1496714184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf0105c50 0x7f8cf010c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.934+0000 7f8cf5200640 1 --2- 172.21.15.22:0/1496714184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cf0105290 0x7f8cf0105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.934+0000 7f8cf5200640 1 --2- 172.21.15.22:0/1496714184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf01040a0 0x7f8cf01044c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.934+0000 7f8cf5200640 1 -- 172.21.15.22:0/1496714184 >> 172.21.15.22:0/1496714184 conn(0x7f8cf00ff610 msgr2=0x7f8cf0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:01.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.934+0000 7f8cf5200640 1 -- 172.21.15.22:0/1496714184 shutdown_connections 2024-04-20T21:33:01.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.934+0000 7f8cf5200640 1 -- 172.21.15.22:0/1496714184 wait complete. 2024-04-20T21:33:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.935+0000 7f8cf5200640 1 Processor -- start 2024-04-20T21:33:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.935+0000 7f8cf5200640 1 -- start start 2024-04-20T21:33:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8cf5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cf01040a0 0x7f8cf019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8cf5200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf0105290 0x7f8cf019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8cf5200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf0105c50 0x7f8cf01a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8cf5200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf010fd30 con 0x7f8cf0105290 2024-04-20T21:33:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8cf5200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf010fbb0 con 0x7f8cf0105c50 2024-04-20T21:33:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8cf5200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf010feb0 con 0x7f8cf01040a0 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8ce6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf0105290 0x7f8cf019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8ce6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf0105290 0x7f8cf019a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49412/0 (socket says 172.21.15.22:49412) 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8ce6a00640 1 -- 172.21.15.22:0/2031431472 learned_addr learned my addr 172.21.15.22:0/2031431472 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8cefe00640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf0105c50 0x7f8cf01a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8cef400640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cf01040a0 0x7f8cf019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8ce6a00640 1 -- 172.21.15.22:0/2031431472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cf01040a0 msgr2=0x7f8cf019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8ce6a00640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cf01040a0 0x7f8cf019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8ce6a00640 1 -- 172.21.15.22:0/2031431472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf0105c50 msgr2=0x7f8cf01a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8ce6a00640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf0105c50 0x7f8cf01a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.936+0000 7f8ce6a00640 1 -- 172.21.15.22:0/2031431472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf01a2000 con 0x7f8cf0105290 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.937+0000 7f8ce6a00640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf0105290 0x7f8cf019a920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800e9f0 tx=0x7f8cd800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.937+0000 7f8cecc00640 1 -- 172.21.15.22:0/2031431472 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd800cd80 con 0x7f8cf0105290 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.937+0000 7f8cecc00640 1 -- 172.21.15.22:0/2031431472 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd8005500 con 0x7f8cf0105290 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.937+0000 7f8cecc00640 1 -- 172.21.15.22:0/2031431472 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8010690 con 0x7f8cf0105290 2024-04-20T21:33:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.937+0000 7f8cf5200640 1 -- 172.21.15.22:0/2031431472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf01a22f0 con 0x7f8cf0105290 2024-04-20T21:33:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.937+0000 7f8cf5200640 1 -- 172.21.15.22:0/2031431472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf01a27f0 con 0x7f8cf0105290 2024-04-20T21:33:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.938+0000 7f8cf5200640 1 -- 172.21.15.22:0/2031431472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf010b410 con 0x7f8cf0105290 2024-04-20T21:33:01.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.941+0000 7f8cecc00640 1 -- 172.21.15.22:0/2031431472 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8cd80046c0 con 0x7f8cf0105290 2024-04-20T21:33:01.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.942+0000 7f8cecc00640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8cd40a3c30 0x7f8cd40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:01.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.942+0000 7f8cecc00640 1 -- 172.21.15.22:0/2031431472 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f8cd80d0a00 con 0x7f8cf0105290 2024-04-20T21:33:01.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.942+0000 7f8cef400640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8cd40a3c30 0x7f8cd40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:01.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.943+0000 7f8cef400640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8cd40a3c30 0x7f8cd40a60f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0000c00 tx=0x7f8ce0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:01.945+0000 7f8cecc00640 1 -- 172.21.15.22:0/2031431472 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8cd8053ff0 con 0x7f8cf0105290 2024-04-20T21:33:02.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.142+0000 7f8cf5200640 1 -- 172.21.15.22:0/2031431472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8cf002d820 con 0x7f8cf0105290 2024-04-20T21:33:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.143+0000 7f8cecc00640 1 -- 172.21.15.22:0/2031431472 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f8cd80bbe20 con 0x7f8cf0105290 2024-04-20T21:33:02.142 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-20T21:33:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.146+0000 7f8cf5200640 1 -- 172.21.15.22:0/2031431472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8cd40a3c30 msgr2=0x7f8cd40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.146+0000 7f8cf5200640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8cd40a3c30 0x7f8cd40a60f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0000c00 tx=0x7f8ce0005fd0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.146+0000 7f8cf5200640 1 -- 172.21.15.22:0/2031431472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf0105290 msgr2=0x7f8cf019a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.146+0000 7f8cf5200640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf0105290 0x7f8cf019a920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800e9f0 tx=0x7f8cd800edf0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.146+0000 7f8cf5200640 1 -- 172.21.15.22:0/2031431472 shutdown_connections 2024-04-20T21:33:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.146+0000 7f8cf5200640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8cd40a3c30 0x7f8cd40a60f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.146+0000 7f8cf5200640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf0105c50 0x7f8cf01a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.146+0000 7f8cf5200640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf0105290 0x7f8cf019a920 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.146+0000 7f8cf5200640 1 --2- 172.21.15.22:0/2031431472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cf01040a0 0x7f8cf019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.146+0000 7f8cf5200640 1 -- 172.21.15.22:0/2031431472 >> 172.21.15.22:0/2031431472 conn(0x7f8cf00ff610 msgr2=0x7f8cf01085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.146+0000 7f8cf5200640 1 -- 172.21.15.22:0/2031431472 shutdown_connections 2024-04-20T21:33:02.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.146+0000 7f8cf5200640 1 -- 172.21.15.22:0/2031431472 wait complete. 2024-04-20T21:33:02.163 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:33:02.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:02 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1492659272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:02.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:02 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1510523192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:02.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:02 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1492659272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:02.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:02 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1510523192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:02.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.378+0000 7f0229200640 1 -- 172.21.15.22:0/1348251203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224103b40 msgr2=0x7f022410a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:02.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.378+0000 7f0229200640 1 --2- 172.21.15.22:0/1348251203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224103b40 0x7f022410a3d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f021c009be0 tx=0x7f021c02bef0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.378+0000 7f0229200640 1 -- 172.21.15.22:0/1348251203 shutdown_connections 2024-04-20T21:33:02.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.378+0000 7f0229200640 1 --2- 172.21.15.22:0/1348251203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224103b40 0x7f022410a3d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.378+0000 7f0229200640 1 --2- 172.21.15.22:0/1348251203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0224103180 0x7f0224103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.378+0000 7f0229200640 1 --2- 172.21.15.22:0/1348251203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0224101f90 0x7f02241023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.378+0000 7f0229200640 1 -- 172.21.15.22:0/1348251203 >> 172.21.15.22:0/1348251203 conn(0x7f02240fd520 msgr2=0x7f02240ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:02.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.378+0000 7f0229200640 1 -- 172.21.15.22:0/1348251203 shutdown_connections 2024-04-20T21:33:02.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.378+0000 7f0229200640 1 -- 172.21.15.22:0/1348251203 wait complete. 2024-04-20T21:33:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.379+0000 7f0229200640 1 Processor -- start 2024-04-20T21:33:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.380+0000 7f0229200640 1 -- start start 2024-04-20T21:33:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.380+0000 7f0229200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0224101f90 0x7f0224075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.380+0000 7f0229200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224103180 0x7f0224074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.380+0000 7f0229200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0224103b40 0x7f0224074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.380+0000 7f0229200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f022410dc30 con 0x7f0224101f90 2024-04-20T21:33:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.380+0000 7f0229200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f022410dab0 con 0x7f0224103180 2024-04-20T21:33:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.380+0000 7f0229200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f022410ddb0 con 0x7f0224103b40 2024-04-20T21:33:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.380+0000 7f0221e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0224101f90 0x7f0224075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.380+0000 7f0221e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0224101f90 0x7f0224075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47438/0 (socket says 172.21.15.22:47438) 2024-04-20T21:33:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.380+0000 7f0221e00640 1 -- 172.21.15.22:0/2285312681 learned_addr learned my addr 172.21.15.22:0/2285312681 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.380+0000 7f0222800640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0224103b40 0x7f0224074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.380+0000 7f021be00640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224103180 0x7f0224074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.381+0000 7f0221e00640 1 -- 172.21.15.22:0/2285312681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0224103b40 msgr2=0x7f0224074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.381+0000 7f0221e00640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0224103b40 0x7f0224074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.381+0000 7f0221e00640 1 -- 172.21.15.22:0/2285312681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224103180 msgr2=0x7f0224074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.381+0000 7f0221e00640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224103180 0x7f0224074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.381+0000 7f0221e00640 1 -- 172.21.15.22:0/2285312681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0224075120 con 0x7f0224101f90 2024-04-20T21:33:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.381+0000 7f0221e00640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0224101f90 0x7f0224075d10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f021400e9a0 tx=0x7f021400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.382+0000 7f021aa00640 1 -- 172.21.15.22:0/2285312681 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021400cd80 con 0x7f0224101f90 2024-04-20T21:33:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.382+0000 7f0229200640 1 -- 172.21.15.22:0/2285312681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02241a23b0 con 0x7f0224101f90 2024-04-20T21:33:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.382+0000 7f021aa00640 1 -- 172.21.15.22:0/2285312681 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0214010910 con 0x7f0224101f90 2024-04-20T21:33:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.382+0000 7f021aa00640 1 -- 172.21.15.22:0/2285312681 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214018a40 con 0x7f0224101f90 2024-04-20T21:33:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.382+0000 7f0229200640 1 -- 172.21.15.22:0/2285312681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02241a2900 con 0x7f0224101f90 2024-04-20T21:33:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.383+0000 7f0229200640 1 -- 172.21.15.22:0/2285312681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0224109250 con 0x7f0224101f90 2024-04-20T21:33:02.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.385+0000 7f021aa00640 1 -- 172.21.15.22:0/2285312681 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0214010ac0 con 0x7f0224101f90 2024-04-20T21:33:02.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.386+0000 7f021aa00640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f02080a3bb0 0x7f02080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:02.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.387+0000 7f021aa00640 1 -- 172.21.15.22:0/2285312681 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f02140d0e50 con 0x7f0224101f90 2024-04-20T21:33:02.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.387+0000 7f021be00640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f02080a3bb0 0x7f02080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:02.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.388+0000 7f021be00640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f02080a3bb0 0x7f02080a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0224075a80 tx=0x7f020c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:02.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.394+0000 7f021aa00640 1 -- 172.21.15.22:0/2285312681 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0214026070 con 0x7f0224101f90 2024-04-20T21:33:02.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:02 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1492659272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:02.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:02 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1510523192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:02.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.603+0000 7f0229200640 1 -- 172.21.15.22:0/2285312681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0224062b60 con 0x7f0224101f90 2024-04-20T21:33:02.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.604+0000 7f021aa00640 1 -- 172.21.15.22:0/2285312681 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f021409d810 con 0x7f0224101f90 2024-04-20T21:33:02.603 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-20T21:33:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.606+0000 7f0229200640 1 -- 172.21.15.22:0/2285312681 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f02080a3bb0 msgr2=0x7f02080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.606+0000 7f0229200640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f02080a3bb0 0x7f02080a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0224075a80 tx=0x7f020c005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.606+0000 7f0229200640 1 -- 172.21.15.22:0/2285312681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0224101f90 msgr2=0x7f0224075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.606+0000 7f0229200640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0224101f90 0x7f0224075d10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f021400e9a0 tx=0x7f021400ee60 comp rx=0 tx=0).stop 2024-04-20T21:33:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.607+0000 7f0229200640 1 -- 172.21.15.22:0/2285312681 shutdown_connections 2024-04-20T21:33:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.607+0000 7f0229200640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f02080a3bb0 0x7f02080a6070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.607+0000 7f0229200640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0224103b40 0x7f0224074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.607+0000 7f0229200640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224103180 0x7f0224074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.607+0000 7f0229200640 1 --2- 172.21.15.22:0/2285312681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0224101f90 0x7f0224075d10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.607+0000 7f0229200640 1 -- 172.21.15.22:0/2285312681 >> 172.21.15.22:0/2285312681 conn(0x7f02240fd520 msgr2=0x7f0224106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:02.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.607+0000 7f0229200640 1 -- 172.21.15.22:0/2285312681 shutdown_connections 2024-04-20T21:33:02.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.607+0000 7f0229200640 1 -- 172.21.15.22:0/2285312681 wait complete. 2024-04-20T21:33:02.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.819+0000 7f2727200640 1 -- 172.21.15.22:0/2568606155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2720074940 msgr2=0x7f2720074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.819+0000 7f2727200640 1 --2- 172.21.15.22:0/2568606155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2720074940 0x7f2720074da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f270c009860 tx=0x7f270c02bc00 comp rx=0 tx=0).stop 2024-04-20T21:33:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.819+0000 7f2727200640 1 -- 172.21.15.22:0/2568606155 shutdown_connections 2024-04-20T21:33:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.819+0000 7f2727200640 1 --2- 172.21.15.22:0/2568606155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2720109e60 0x7f272010c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.819+0000 7f2727200640 1 --2- 172.21.15.22:0/2568606155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2720074940 0x7f2720074da0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.819+0000 7f2727200640 1 --2- 172.21.15.22:0/2568606155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2720075f20 0x7f2720074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.819+0000 7f2727200640 1 -- 172.21.15.22:0/2568606155 >> 172.21.15.22:0/2568606155 conn(0x7f27200fd560 msgr2=0x7f27200ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.820+0000 7f2727200640 1 -- 172.21.15.22:0/2568606155 shutdown_connections 2024-04-20T21:33:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.820+0000 7f2727200640 1 -- 172.21.15.22:0/2568606155 wait complete. 2024-04-20T21:33:02.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f2727200640 1 Processor -- start 2024-04-20T21:33:02.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f2727200640 1 -- start start 2024-04-20T21:33:02.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f2727200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2720074940 0x7f272019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:02.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f2727200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2720075f20 0x7f272019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f2727200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2720109e60 0x7f27201a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f2727200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f272010fe50 con 0x7f2720074940 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f2727200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f272010fcd0 con 0x7f2720075f20 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f2727200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f272010ffd0 con 0x7f2720109e60 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f271fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2720074940 0x7f272019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f271fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2720074940 0x7f272019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47454/0 (socket says 172.21.15.22:47454) 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f271fe00640 1 -- 172.21.15.22:0/446108742 learned_addr learned my addr 172.21.15.22:0/446108742 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f271f400640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2720075f20 0x7f272019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.821+0000 7f2725200640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2720109e60 0x7f27201a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.822+0000 7f271fe00640 1 -- 172.21.15.22:0/446108742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2720109e60 msgr2=0x7f27201a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.822+0000 7f271fe00640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2720109e60 0x7f27201a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.822+0000 7f271fe00640 1 -- 172.21.15.22:0/446108742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2720075f20 msgr2=0x7f272019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.822+0000 7f271fe00640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2720075f20 0x7f272019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.822+0000 7f271fe00640 1 -- 172.21.15.22:0/446108742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27201a20a0 con 0x7f2720074940 2024-04-20T21:33:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.822+0000 7f2725200640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2720109e60 0x7f27201a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.822+0000 7f271f400640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2720075f20 0x7f272019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.822+0000 7f271fe00640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2720074940 0x7f272019a480 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f271000e9f0 tx=0x7f271000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.822+0000 7f271cc00640 1 -- 172.21.15.22:0/446108742 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271000cd80 con 0x7f2720074940 2024-04-20T21:33:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.822+0000 7f2727200640 1 -- 172.21.15.22:0/446108742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27201a2390 con 0x7f2720074940 2024-04-20T21:33:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.822+0000 7f271cc00640 1 -- 172.21.15.22:0/446108742 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2710005500 con 0x7f2720074940 2024-04-20T21:33:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.822+0000 7f271cc00640 1 -- 172.21.15.22:0/446108742 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710010690 con 0x7f2720074940 2024-04-20T21:33:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.823+0000 7f2727200640 1 -- 172.21.15.22:0/446108742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27201a2860 con 0x7f2720074940 2024-04-20T21:33:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.824+0000 7f2727200640 1 -- 172.21.15.22:0/446108742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2720108c20 con 0x7f2720074940 2024-04-20T21:33:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.826+0000 7f271cc00640 1 -- 172.21.15.22:0/446108742 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f27100046c0 con 0x7f2720074940 2024-04-20T21:33:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.828+0000 7f271cc00640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f27040a3bb0 0x7f27040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.828+0000 7f271cc00640 1 -- 172.21.15.22:0/446108742 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f27100d08d0 con 0x7f2720074940 2024-04-20T21:33:02.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.828+0000 7f271f400640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f27040a3bb0 0x7f27040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:02.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.829+0000 7f271f400640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f27040a3bb0 0x7f27040a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f270c009830 tx=0x7f270c034b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:02.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:02.832+0000 7f271cc00640 1 -- 172.21.15.22:0/446108742 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f271009d290 con 0x7f2720074940 2024-04-20T21:33:03.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.086+0000 7f2727200640 1 -- 172.21.15.22:0/446108742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f272002d5d0 con 0x7f2720074940 2024-04-20T21:33:03.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.087+0000 7f271cc00640 1 -- 172.21.15.22:0/446108742 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f271009cb90 con 0x7f2720074940 2024-04-20T21:33:03.086 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:03.086 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:33:03.087 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:33:03.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.089+0000 7f2727200640 1 -- 172.21.15.22:0/446108742 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f27040a3bb0 msgr2=0x7f27040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.089+0000 7f2727200640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f27040a3bb0 0x7f27040a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f270c009830 tx=0x7f270c034b50 comp rx=0 tx=0).stop 2024-04-20T21:33:03.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.089+0000 7f2727200640 1 -- 172.21.15.22:0/446108742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2720074940 msgr2=0x7f272019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.089+0000 7f2727200640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2720074940 0x7f272019a480 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f271000e9f0 tx=0x7f271000edf0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.090+0000 7f2727200640 1 -- 172.21.15.22:0/446108742 shutdown_connections 2024-04-20T21:33:03.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.090+0000 7f2727200640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f27040a3bb0 0x7f27040a6070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.090+0000 7f2727200640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2720109e60 0x7f27201a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.090+0000 7f2727200640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2720075f20 0x7f272019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.090+0000 7f2727200640 1 --2- 172.21.15.22:0/446108742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2720074940 0x7f272019a480 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.090+0000 7f2727200640 1 -- 172.21.15.22:0/446108742 >> 172.21.15.22:0/446108742 conn(0x7f27200fd560 msgr2=0x7f2720105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:03.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.090+0000 7f2727200640 1 -- 172.21.15.22:0/446108742 shutdown_connections 2024-04-20T21:33:03.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.090+0000 7f2727200640 1 -- 172.21.15.22:0/446108742 wait complete. 2024-04-20T21:33:03.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:03.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:03 smithi022 ceph-mon[16843]: pgmap v366: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:03.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:03 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2031431472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:03.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:03 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2285312681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:03.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:03 smithi122 ceph-mon[22147]: pgmap v366: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:03.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:03 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2031431472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:03.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:03 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2285312681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:03.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.299+0000 7fc511200640 1 -- 172.21.15.22:0/234686437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c107ac0 msgr2=0x7fc50c109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.299+0000 7fc511200640 1 --2- 172.21.15.22:0/234686437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c107ac0 0x7fc50c109f10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc504009ca0 tx=0x7fc50402c100 comp rx=0 tx=0).stop 2024-04-20T21:33:03.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.299+0000 7fc511200640 1 -- 172.21.15.22:0/234686437 shutdown_connections 2024-04-20T21:33:03.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.299+0000 7fc511200640 1 --2- 172.21.15.22:0/234686437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c107ac0 0x7fc50c109f10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.299+0000 7fc511200640 1 --2- 172.21.15.22:0/234686437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50c105190 0x7fc50c107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.299+0000 7fc511200640 1 --2- 172.21.15.22:0/234686437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c102860 0x7fc50c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.299+0000 7fc511200640 1 -- 172.21.15.22:0/234686437 >> 172.21.15.22:0/234686437 conn(0x7fc50c0fc230 msgr2=0x7fc50c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:03.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.300+0000 7fc511200640 1 -- 172.21.15.22:0/234686437 shutdown_connections 2024-04-20T21:33:03.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.300+0000 7fc511200640 1 -- 172.21.15.22:0/234686437 wait complete. 2024-04-20T21:33:03.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.300+0000 7fc511200640 1 Processor -- start 2024-04-20T21:33:03.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.301+0000 7fc511200640 1 -- start start 2024-04-20T21:33:03.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.301+0000 7fc511200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c102860 0x7fc50c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.301+0000 7fc511200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c105190 0x7fc50c19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.301+0000 7fc511200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50c107ac0 0x7fc50c1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.301+0000 7fc511200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c10da60 con 0x7fc50c102860 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.301+0000 7fc511200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c10d8e0 con 0x7fc50c107ac0 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.301+0000 7fc511200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c10dbe0 con 0x7fc50c105190 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.301+0000 7fc50b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c102860 0x7fc50c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.301+0000 7fc50b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c102860 0x7fc50c19a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47468/0 (socket says 172.21.15.22:47468) 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.301+0000 7fc50b400640 1 -- 172.21.15.22:0/2766251311 learned_addr learned my addr 172.21.15.22:0/2766251311 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.301+0000 7fc50aa00640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c105190 0x7fc50c19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.301+0000 7fc50be00640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50c107ac0 0x7fc50c1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.302+0000 7fc50b400640 1 -- 172.21.15.22:0/2766251311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c105190 msgr2=0x7fc50c19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.302+0000 7fc50b400640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c105190 0x7fc50c19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.302+0000 7fc50b400640 1 -- 172.21.15.22:0/2766251311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50c107ac0 msgr2=0x7fc50c1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.302+0000 7fc50b400640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50c107ac0 0x7fc50c1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.302+0000 7fc50b400640 1 -- 172.21.15.22:0/2766251311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c1a1ed0 con 0x7fc50c102860 2024-04-20T21:33:03.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.302+0000 7fc50b400640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c102860 0x7fc50c19a2b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00ea60 tx=0x7fc4fc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:03.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.302+0000 7fc503e00640 1 -- 172.21.15.22:0/2766251311 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc00cde0 con 0x7fc50c102860 2024-04-20T21:33:03.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.302+0000 7fc503e00640 1 -- 172.21.15.22:0/2766251311 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4fc0035f0 con 0x7fc50c102860 2024-04-20T21:33:03.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.302+0000 7fc503e00640 1 -- 172.21.15.22:0/2766251311 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc010670 con 0x7fc50c102860 2024-04-20T21:33:03.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.302+0000 7fc511200640 1 -- 172.21.15.22:0/2766251311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50c1a21c0 con 0x7fc50c102860 2024-04-20T21:33:03.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.302+0000 7fc511200640 1 -- 172.21.15.22:0/2766251311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc50c1a2710 con 0x7fc50c102860 2024-04-20T21:33:03.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:03 smithi033 ceph-mon[21626]: pgmap v366: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:03.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:03 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2031431472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:03.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:03 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2285312681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:03.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.304+0000 7fc511200640 1 -- 172.21.15.22:0/2766251311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc50c1095b0 con 0x7fc50c102860 2024-04-20T21:33:03.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.306+0000 7fc503e00640 1 -- 172.21.15.22:0/2766251311 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc4fc0037a0 con 0x7fc50c102860 2024-04-20T21:33:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.307+0000 7fc503e00640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc4f00a3bb0 0x7fc4f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.307+0000 7fc503e00640 1 -- 172.21.15.22:0/2766251311 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fc4fc0d07e0 con 0x7fc50c102860 2024-04-20T21:33:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.308+0000 7fc50aa00640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc4f00a3bb0 0x7fc4f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.308+0000 7fc50aa00640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc4f00a3bb0 0x7fc4f00a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80095f0 tx=0x7fc4f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:03.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.310+0000 7fc503e00640 1 -- 172.21.15.22:0/2766251311 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc4fc09d1d0 con 0x7fc50c102860 2024-04-20T21:33:03.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.509+0000 7fc511200640 1 -- 172.21.15.22:0/2766251311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc50c02d7c0 con 0x7fc50c102860 2024-04-20T21:33:03.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.512+0000 7fc503e00640 1 -- 172.21.15.22:0/2766251311 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22721 (secure 0 0 0) 0x7fc4fc09cad0 con 0x7fc50c102860 2024-04-20T21:33:03.511 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:03.511 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":90,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:32:31.596064+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.515+0000 7fc511200640 1 -- 172.21.15.22:0/2766251311 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc4f00a3bb0 msgr2=0x7fc4f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.515+0000 7fc511200640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc4f00a3bb0 0x7fc4f00a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80095f0 tx=0x7fc4f8005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.515+0000 7fc511200640 1 -- 172.21.15.22:0/2766251311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c102860 msgr2=0x7fc50c19a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.515+0000 7fc511200640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c102860 0x7fc50c19a2b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00ea60 tx=0x7fc4fc00ef20 comp rx=0 tx=0).stop 2024-04-20T21:33:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.515+0000 7fc511200640 1 -- 172.21.15.22:0/2766251311 shutdown_connections 2024-04-20T21:33:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.515+0000 7fc511200640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc4f00a3bb0 0x7fc4f00a6070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.515+0000 7fc511200640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50c107ac0 0x7fc50c1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.515+0000 7fc511200640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c105190 0x7fc50c19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.515+0000 7fc511200640 1 --2- 172.21.15.22:0/2766251311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c102860 0x7fc50c19a2b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.515+0000 7fc511200640 1 -- 172.21.15.22:0/2766251311 >> 172.21.15.22:0/2766251311 conn(0x7fc50c0fc230 msgr2=0x7fc50c101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.516+0000 7fc511200640 1 -- 172.21.15.22:0/2766251311 shutdown_connections 2024-04-20T21:33:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.516+0000 7fc511200640 1 -- 172.21.15.22:0/2766251311 wait complete. 2024-04-20T21:33:03.542 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:26:15.688818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:31:25.506074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:31:26.126261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:33:03.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:33:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.741+0000 7fda1a200640 1 -- 172.21.15.22:0/560295318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda14105130 msgr2=0x7fda14107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.741+0000 7fda1a200640 1 --2- 172.21.15.22:0/560295318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda14105130 0x7fda14107520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda00009900 tx=0x7fda0002bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:03.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.741+0000 7fda1a200640 1 -- 172.21.15.22:0/560295318 shutdown_connections 2024-04-20T21:33:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.741+0000 7fda1a200640 1 --2- 172.21.15.22:0/560295318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda14107a60 0x7fda14109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.741+0000 7fda1a200640 1 --2- 172.21.15.22:0/560295318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda14105130 0x7fda14107520 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.741+0000 7fda1a200640 1 --2- 172.21.15.22:0/560295318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14102800 0x7fda14104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.741+0000 7fda1a200640 1 -- 172.21.15.22:0/560295318 >> 172.21.15.22:0/560295318 conn(0x7fda140fc230 msgr2=0x7fda140fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.741+0000 7fda1a200640 1 -- 172.21.15.22:0/560295318 shutdown_connections 2024-04-20T21:33:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.741+0000 7fda1a200640 1 -- 172.21.15.22:0/560295318 wait complete. 2024-04-20T21:33:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.742+0000 7fda1a200640 1 Processor -- start 2024-04-20T21:33:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.742+0000 7fda1a200640 1 -- start start 2024-04-20T21:33:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda1a200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda14102800 0x7fda14102170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda1a200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda14105130 0x7fda141007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda1a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14107a60 0x7fda14100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda1a200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fda1410d9f0 con 0x7fda14107a60 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda1a200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fda1410d870 con 0x7fda14105130 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda1a200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fda1410db70 con 0x7fda14102800 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda18e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda14102800 0x7fda14102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda19800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14107a60 0x7fda14100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda18e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda14102800 0x7fda14102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33546/0 (socket says 172.21.15.22:33546) 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda18e00640 1 -- 172.21.15.22:0/4230757660 learned_addr learned my addr 172.21.15.22:0/4230757660 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda13e00640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda14105130 0x7fda141007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda18e00640 1 -- 172.21.15.22:0/4230757660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda14105130 msgr2=0x7fda141007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda18e00640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda14105130 0x7fda141007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda18e00640 1 -- 172.21.15.22:0/4230757660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14107a60 msgr2=0x7fda14100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda18e00640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14107a60 0x7fda14100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.743+0000 7fda18e00640 1 -- 172.21.15.22:0/4230757660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda14101580 con 0x7fda14102800 2024-04-20T21:33:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.744+0000 7fda18e00640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda14102800 0x7fda14102170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fda0800e9f0 tx=0x7fda0800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.744+0000 7fda11600640 1 -- 172.21.15.22:0/4230757660 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0800cd80 con 0x7fda14102800 2024-04-20T21:33:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.744+0000 7fda1a200640 1 -- 172.21.15.22:0/4230757660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda141a2180 con 0x7fda14102800 2024-04-20T21:33:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.744+0000 7fda11600640 1 -- 172.21.15.22:0/4230757660 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda08010910 con 0x7fda14102800 2024-04-20T21:33:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.744+0000 7fda11600640 1 -- 172.21.15.22:0/4230757660 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08018950 con 0x7fda14102800 2024-04-20T21:33:03.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.744+0000 7fda1a200640 1 -- 172.21.15.22:0/4230757660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda141a26f0 con 0x7fda14102800 2024-04-20T21:33:03.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.745+0000 7fda1a200640 1 -- 172.21.15.22:0/4230757660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda140fdd20 con 0x7fda14102800 2024-04-20T21:33:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.747+0000 7fda11600640 1 -- 172.21.15.22:0/4230757660 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fda08010ac0 con 0x7fda14102800 2024-04-20T21:33:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.749+0000 7fda11600640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd9fc0a3c30 0x7fd9fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.749+0000 7fda11600640 1 -- 172.21.15.22:0/4230757660 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fda080d0b30 con 0x7fda14102800 2024-04-20T21:33:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.749+0000 7fda13e00640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd9fc0a3c30 0x7fd9fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.750+0000 7fda13e00640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd9fc0a3c30 0x7fd9fc0a60f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fda00000c00 tx=0x7fda00005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.751+0000 7fda11600640 1 -- 172.21.15.22:0/4230757660 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fda0809ce40 con 0x7fda14102800 2024-04-20T21:33:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.954+0000 7fda1a200640 1 -- 172.21.15.22:0/4230757660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda14062b60 con 0x7fda14102800 2024-04-20T21:33:03.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.955+0000 7fda11600640 1 -- 172.21.15.22:0/4230757660 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fda080a1140 con 0x7fda14102800 2024-04-20T21:33:03.954 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-20T21:33:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.957+0000 7fda1a200640 1 -- 172.21.15.22:0/4230757660 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd9fc0a3c30 msgr2=0x7fd9fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.957+0000 7fda1a200640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd9fc0a3c30 0x7fd9fc0a60f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fda00000c00 tx=0x7fda00005fd0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.957+0000 7fda1a200640 1 -- 172.21.15.22:0/4230757660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda14102800 msgr2=0x7fda14102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.957+0000 7fda1a200640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda14102800 0x7fda14102170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fda0800e9f0 tx=0x7fda0800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.957+0000 7fda1a200640 1 -- 172.21.15.22:0/4230757660 shutdown_connections 2024-04-20T21:33:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.957+0000 7fda1a200640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd9fc0a3c30 0x7fd9fc0a60f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.957+0000 7fda1a200640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14107a60 0x7fda14100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.957+0000 7fda1a200640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda14105130 0x7fda141007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.957+0000 7fda1a200640 1 --2- 172.21.15.22:0/4230757660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda14102800 0x7fda14102170 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.957+0000 7fda1a200640 1 -- 172.21.15.22:0/4230757660 >> 172.21.15.22:0/4230757660 conn(0x7fda140fc230 msgr2=0x7fda14103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:03.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.958+0000 7fda1a200640 1 -- 172.21.15.22:0/4230757660 shutdown_connections 2024-04-20T21:33:03.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:03.958+0000 7fda1a200640 1 -- 172.21.15.22:0/4230757660 wait complete. 2024-04-20T21:33:03.974 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:33:04.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:04 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/446108742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:04.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:04 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2766251311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:04.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:04 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/4230757660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.177+0000 7f020a200640 1 -- 172.21.15.22:0/1096558477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0204109690 msgr2=0x7f0204109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.177+0000 7f020a200640 1 --2- 172.21.15.22:0/1096558477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0204109690 0x7f0204109b10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009900 tx=0x7f01f402bc50 comp rx=0 tx=0).stop 2024-04-20T21:33:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.177+0000 7f020a200640 1 -- 172.21.15.22:0/1096558477 shutdown_connections 2024-04-20T21:33:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.177+0000 7f020a200640 1 --2- 172.21.15.22:0/1096558477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f020410a050 0x7f02041108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.177+0000 7f020a200640 1 --2- 172.21.15.22:0/1096558477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0204109690 0x7f0204109b10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.177+0000 7f020a200640 1 --2- 172.21.15.22:0/1096558477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02041084a0 0x7f02041088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.177+0000 7f020a200640 1 -- 172.21.15.22:0/1096558477 >> 172.21.15.22:0/1096558477 conn(0x7f0204077430 msgr2=0x7f0204079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.177+0000 7f020a200640 1 -- 172.21.15.22:0/1096558477 shutdown_connections 2024-04-20T21:33:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.177+0000 7f020a200640 1 -- 172.21.15.22:0/1096558477 wait complete. 2024-04-20T21:33:04.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.178+0000 7f020a200640 1 Processor -- start 2024-04-20T21:33:04.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.179+0000 7f020a200640 1 -- start start 2024-04-20T21:33:04.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.179+0000 7f020a200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02041084a0 0x7f020419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.179+0000 7f020a200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204109690 0x7f020419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.179+0000 7f020a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f020410a050 0x7f02041a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.179+0000 7f020a200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0204114180 con 0x7f020410a050 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.179+0000 7f020a200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0204114000 con 0x7f0204109690 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.179+0000 7f020a200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0204114300 con 0x7f02041084a0 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.179+0000 7f0208e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02041084a0 0x7f020419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.179+0000 7f0208e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02041084a0 0x7f020419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33562/0 (socket says 172.21.15.22:33562) 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.179+0000 7f0208e00640 1 -- 172.21.15.22:0/3786500538 learned_addr learned my addr 172.21.15.22:0/3786500538 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.179+0000 7f0209800640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f020410a050 0x7f02041a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.179+0000 7f0203e00640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204109690 0x7f020419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.180+0000 7f0208e00640 1 -- 172.21.15.22:0/3786500538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204109690 msgr2=0x7f020419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.180+0000 7f0208e00640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204109690 0x7f020419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.180+0000 7f0208e00640 1 -- 172.21.15.22:0/3786500538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f020410a050 msgr2=0x7f02041a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.180+0000 7f0208e00640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f020410a050 0x7f02041a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.180+0000 7f0208e00640 1 -- 172.21.15.22:0/3786500538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02041a6390 con 0x7f02041084a0 2024-04-20T21:33:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.180+0000 7f0203e00640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204109690 0x7f020419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.180+0000 7f0208e00640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02041084a0 0x7f020419e770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f01f800e9c0 tx=0x7f01f800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.180+0000 7f0201600640 1 -- 172.21.15.22:0/3786500538 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f800cd80 con 0x7f02041084a0 2024-04-20T21:33:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.180+0000 7f020a200640 1 -- 172.21.15.22:0/3786500538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02041a6680 con 0x7f02041084a0 2024-04-20T21:33:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.180+0000 7f0201600640 1 -- 172.21.15.22:0/3786500538 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f80035f0 con 0x7f02041084a0 2024-04-20T21:33:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.180+0000 7f0201600640 1 -- 172.21.15.22:0/3786500538 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8010690 con 0x7f02041084a0 2024-04-20T21:33:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.180+0000 7f020a200640 1 -- 172.21.15.22:0/3786500538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02041a6b50 con 0x7f02041084a0 2024-04-20T21:33:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.182+0000 7f020a200640 1 -- 172.21.15.22:0/3786500538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020410f760 con 0x7f02041084a0 2024-04-20T21:33:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.183+0000 7f0201600640 1 -- 172.21.15.22:0/3786500538 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f01f80037a0 con 0x7f02041084a0 2024-04-20T21:33:04.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.185+0000 7f0201600640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f01ec0a3bb0 0x7f01ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:04.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.185+0000 7f0201600640 1 -- 172.21.15.22:0/3786500538 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f01f80d0890 con 0x7f02041084a0 2024-04-20T21:33:04.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.185+0000 7f0203e00640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f01ec0a3bb0 0x7f01ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:04.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.186+0000 7f0203e00640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f01ec0a3bb0 0x7f01ec0a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f01f402c260 tx=0x7f01f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:04.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.189+0000 7f0201600640 1 -- 172.21.15.22:0/3786500538 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f01f809d1f0 con 0x7f02041084a0 2024-04-20T21:33:04.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:04 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/446108742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:04.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:04 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2766251311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:04.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:04 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/4230757660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:04.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:04 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/446108742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:04.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:04 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2766251311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:04.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:04 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/4230757660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:04.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.392+0000 7f020a200640 1 -- 172.21.15.22:0/3786500538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0204062b60 con 0x7f02041084a0 2024-04-20T21:33:04.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.393+0000 7f0201600640 1 -- 172.21.15.22:0/3786500538 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f01f809caf0 con 0x7f02041084a0 2024-04-20T21:33:04.393 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-20T21:33:04.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.396+0000 7f020a200640 1 -- 172.21.15.22:0/3786500538 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f01ec0a3bb0 msgr2=0x7f01ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.396+0000 7f020a200640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f01ec0a3bb0 0x7f01ec0a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f01f402c260 tx=0x7f01f4005fd0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.396+0000 7f020a200640 1 -- 172.21.15.22:0/3786500538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02041084a0 msgr2=0x7f020419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.396+0000 7f020a200640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02041084a0 0x7f020419e770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f01f800e9c0 tx=0x7f01f800ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:04.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.397+0000 7f020a200640 1 -- 172.21.15.22:0/3786500538 shutdown_connections 2024-04-20T21:33:04.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.397+0000 7f020a200640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f01ec0a3bb0 0x7f01ec0a6070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.397+0000 7f020a200640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f020410a050 0x7f02041a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.397+0000 7f020a200640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204109690 0x7f020419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.397+0000 7f020a200640 1 --2- 172.21.15.22:0/3786500538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02041084a0 0x7f020419e770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.397+0000 7f020a200640 1 -- 172.21.15.22:0/3786500538 >> 172.21.15.22:0/3786500538 conn(0x7f0204077430 msgr2=0x7f020410c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:04.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.397+0000 7f020a200640 1 -- 172.21.15.22:0/3786500538 shutdown_connections 2024-04-20T21:33:04.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.397+0000 7f020a200640 1 -- 172.21.15.22:0/3786500538 wait complete. 2024-04-20T21:33:04.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:33:04.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.632+0000 7ff8bca00640 1 -- 172.21.15.22:0/2622374403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b8109f70 msgr2=0x7ff8b8110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.632+0000 7ff8bca00640 1 --2- 172.21.15.22:0/2622374403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b8109f70 0x7ff8b8110800 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009ca0 tx=0x7ff8b002c100 comp rx=0 tx=0).stop 2024-04-20T21:33:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.632+0000 7ff8bca00640 1 -- 172.21.15.22:0/2622374403 shutdown_connections 2024-04-20T21:33:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.632+0000 7ff8bca00640 1 --2- 172.21.15.22:0/2622374403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b8109f70 0x7ff8b8110800 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.632+0000 7ff8bca00640 1 --2- 172.21.15.22:0/2622374403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8b81095b0 0x7ff8b8109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.632+0000 7ff8bca00640 1 --2- 172.21.15.22:0/2622374403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b81083c0 0x7ff8b81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.632+0000 7ff8bca00640 1 -- 172.21.15.22:0/2622374403 >> 172.21.15.22:0/2622374403 conn(0x7ff8b8077360 msgr2=0x7ff8b80797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.632+0000 7ff8bca00640 1 -- 172.21.15.22:0/2622374403 shutdown_connections 2024-04-20T21:33:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.632+0000 7ff8bca00640 1 -- 172.21.15.22:0/2622374403 wait complete. 2024-04-20T21:33:04.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.633+0000 7ff8bca00640 1 Processor -- start 2024-04-20T21:33:04.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.633+0000 7ff8bca00640 1 -- start start 2024-04-20T21:33:04.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8bca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b81083c0 0x7ff8b819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:04.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8bca00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8b81095b0 0x7ff8b819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:04.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8bca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b8109f70 0x7ff8b81a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8bca00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8114050 con 0x7ff8b81083c0 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8bca00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8113ed0 con 0x7ff8b81095b0 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8bca00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b81141d0 con 0x7ff8b8109f70 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8b6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b81083c0 0x7ff8b819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8b6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b81083c0 0x7ff8b819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47524/0 (socket says 172.21.15.22:47524) 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8b6800640 1 -- 172.21.15.22:0/870988564 learned_addr learned my addr 172.21.15.22:0/870988564 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8afe00640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8b81095b0 0x7ff8b819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8b7200640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b8109f70 0x7ff8b81a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8b6800640 1 -- 172.21.15.22:0/870988564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b8109f70 msgr2=0x7ff8b81a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8b6800640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b8109f70 0x7ff8b81a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8b6800640 1 -- 172.21.15.22:0/870988564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8b81095b0 msgr2=0x7ff8b819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8b6800640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8b81095b0 0x7ff8b819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.634+0000 7ff8b6800640 1 -- 172.21.15.22:0/870988564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b81a6260 con 0x7ff8b81083c0 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.635+0000 7ff8b6800640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b81083c0 0x7ff8b819e640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800e9c0 tx=0x7ff8a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:04.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.635+0000 7ff8af400640 1 -- 172.21.15.22:0/870988564 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800cd80 con 0x7ff8b81083c0 2024-04-20T21:33:04.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.635+0000 7ff8af400640 1 -- 172.21.15.22:0/870988564 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a80035f0 con 0x7ff8b81083c0 2024-04-20T21:33:04.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.635+0000 7ff8af400640 1 -- 172.21.15.22:0/870988564 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8010690 con 0x7ff8b81083c0 2024-04-20T21:33:04.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.635+0000 7ff8bca00640 1 -- 172.21.15.22:0/870988564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b81a6550 con 0x7ff8b81083c0 2024-04-20T21:33:04.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.635+0000 7ff8bca00640 1 -- 172.21.15.22:0/870988564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b81a6a50 con 0x7ff8b81083c0 2024-04-20T21:33:04.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.636+0000 7ff8bca00640 1 -- 172.21.15.22:0/870988564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b810f680 con 0x7ff8b81083c0 2024-04-20T21:33:04.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.639+0000 7ff8af400640 1 -- 172.21.15.22:0/870988564 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff8a80037a0 con 0x7ff8b81083c0 2024-04-20T21:33:04.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.641+0000 7ff8af400640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff89c0a3c30 0x7ff89c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:04.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.641+0000 7ff8af400640 1 -- 172.21.15.22:0/870988564 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7ff8a80d08d0 con 0x7ff8b81083c0 2024-04-20T21:33:04.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.641+0000 7ff8afe00640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff89c0a3c30 0x7ff89c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:04.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.642+0000 7ff8afe00640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff89c0a3c30 0x7ff89c0a60f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff8b819fb50 tx=0x7ff8a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:04.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.644+0000 7ff8af400640 1 -- 172.21.15.22:0/870988564 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff8a809d230 con 0x7ff8b81083c0 2024-04-20T21:33:04.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.851+0000 7ff8bca00640 1 -- 172.21.15.22:0/870988564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff8b8062c00 con 0x7ff8b81083c0 2024-04-20T21:33:04.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.852+0000 7ff8af400640 1 -- 172.21.15.22:0/870988564 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7ff8a809cb30 con 0x7ff8b81083c0 2024-04-20T21:33:04.851 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-20T21:33:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.854+0000 7ff8bca00640 1 -- 172.21.15.22:0/870988564 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff89c0a3c30 msgr2=0x7ff89c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.854+0000 7ff8bca00640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff89c0a3c30 0x7ff89c0a60f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff8b819fb50 tx=0x7ff8a0005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.854+0000 7ff8bca00640 1 -- 172.21.15.22:0/870988564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b81083c0 msgr2=0x7ff8b819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.854+0000 7ff8bca00640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b81083c0 0x7ff8b819e640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800e9c0 tx=0x7ff8a800ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.855+0000 7ff8bca00640 1 -- 172.21.15.22:0/870988564 shutdown_connections 2024-04-20T21:33:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.855+0000 7ff8bca00640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff89c0a3c30 0x7ff89c0a60f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.855+0000 7ff8bca00640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b8109f70 0x7ff8b81a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.855+0000 7ff8bca00640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8b81095b0 0x7ff8b819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.855+0000 7ff8bca00640 1 --2- 172.21.15.22:0/870988564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b81083c0 0x7ff8b819e640 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.855+0000 7ff8bca00640 1 -- 172.21.15.22:0/870988564 >> 172.21.15.22:0/870988564 conn(0x7ff8b8077360 msgr2=0x7ff8b810c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:04.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.855+0000 7ff8bca00640 1 -- 172.21.15.22:0/870988564 shutdown_connections 2024-04-20T21:33:04.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:04.855+0000 7ff8bca00640 1 -- 172.21.15.22:0/870988564 wait complete. 2024-04-20T21:33:04.873 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:05.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.077+0000 7f59cde00640 1 -- 172.21.15.22:0/813713405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59c80749c0 msgr2=0x7f59c8074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.077+0000 7f59cde00640 1 --2- 172.21.15.22:0/813713405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59c80749c0 0x7f59c8074e40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009930 tx=0x7f59b402bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:05.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.077+0000 7f59cde00640 1 -- 172.21.15.22:0/813713405 shutdown_connections 2024-04-20T21:33:05.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.077+0000 7f59cde00640 1 --2- 172.21.15.22:0/813713405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59c810e3d0 0x7f59c81107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.077+0000 7f59cde00640 1 --2- 172.21.15.22:0/813713405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59c80749c0 0x7f59c8074e40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.077+0000 7f59cde00640 1 --2- 172.21.15.22:0/813713405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c8075fa0 0x7f59c80743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.077+0000 7f59cde00640 1 -- 172.21.15.22:0/813713405 >> 172.21.15.22:0/813713405 conn(0x7f59c80fd370 msgr2=0x7f59c80ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:05.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.077+0000 7f59cde00640 1 -- 172.21.15.22:0/813713405 shutdown_connections 2024-04-20T21:33:05.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.077+0000 7f59cde00640 1 -- 172.21.15.22:0/813713405 wait complete. 2024-04-20T21:33:05.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.078+0000 7f59cde00640 1 Processor -- start 2024-04-20T21:33:05.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.078+0000 7f59cde00640 1 -- start start 2024-04-20T21:33:05.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.079+0000 7f59cde00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c80749c0 0x7f59c819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.079+0000 7f59cde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59c8075fa0 0x7f59c819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.079+0000 7f59cde00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59c810e3d0 0x7f59c81a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.079+0000 7f59cde00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f59c8114050 con 0x7f59c8075fa0 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.079+0000 7f59cde00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59c8113ed0 con 0x7f59c80749c0 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.079+0000 7f59cde00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81141d0 con 0x7f59c810e3d0 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.079+0000 7f59c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59c8075fa0 0x7f59c819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.079+0000 7f59c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59c8075fa0 0x7f59c819ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47556/0 (socket says 172.21.15.22:47556) 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.079+0000 7f59c7e00640 1 -- 172.21.15.22:0/628410841 learned_addr learned my addr 172.21.15.22:0/628410841 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.079+0000 7f59cd400640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59c810e3d0 0x7f59c81a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.079+0000 7f59cca00640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c80749c0 0x7f59c819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.080+0000 7f59c7e00640 1 -- 172.21.15.22:0/628410841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59c810e3d0 msgr2=0x7f59c81a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.080+0000 7f59c7e00640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59c810e3d0 0x7f59c81a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.080+0000 7f59c7e00640 1 -- 172.21.15.22:0/628410841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c80749c0 msgr2=0x7f59c819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.080+0000 7f59c7e00640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c80749c0 0x7f59c819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.080+0000 7f59c7e00640 1 -- 172.21.15.22:0/628410841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c81a6370 con 0x7f59c8075fa0 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.080+0000 7f59c7e00640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59c8075fa0 0x7f59c819ec90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f59b4000c00 tx=0x7f59b4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.080+0000 7f59c5600640 1 -- 172.21.15.22:0/628410841 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4041070 con 0x7f59c8075fa0 2024-04-20T21:33:05.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.080+0000 7f59c5600640 1 -- 172.21.15.22:0/628410841 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b40044d0 con 0x7f59c8075fa0 2024-04-20T21:33:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.080+0000 7f59c5600640 1 -- 172.21.15.22:0/628410841 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4034440 con 0x7f59c8075fa0 2024-04-20T21:33:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.080+0000 7f59cde00640 1 -- 172.21.15.22:0/628410841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c81a6600 con 0x7f59c8075fa0 2024-04-20T21:33:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.080+0000 7f59cde00640 1 -- 172.21.15.22:0/628410841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c81a6b40 con 0x7f59c8075fa0 2024-04-20T21:33:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.082+0000 7f59cde00640 1 -- 172.21.15.22:0/628410841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c81045a0 con 0x7f59c8075fa0 2024-04-20T21:33:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.084+0000 7f59c5600640 1 -- 172.21.15.22:0/628410841 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f59b4044070 con 0x7f59c8075fa0 2024-04-20T21:33:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.085+0000 7f59c5600640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f59ac0a3c30 0x7f59ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.086+0000 7f59c5600640 1 -- 172.21.15.22:0/628410841 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f59b40f1ea0 con 0x7f59c8075fa0 2024-04-20T21:33:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.086+0000 7f59cca00640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f59ac0a3c30 0x7f59ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.086+0000 7f59cca00640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f59ac0a3c30 0x7f59ac0a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f59b8005fd0 tx=0x7f59b8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.090+0000 7f59c5600640 1 -- 172.21.15.22:0/628410841 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f59b40be210 con 0x7f59c8075fa0 2024-04-20T21:33:05.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:05 smithi022 ceph-mon[16843]: pgmap v367: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:05.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:05 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3786500538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:05.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:05 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/870988564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:05.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:05 smithi122 ceph-mon[22147]: pgmap v367: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:05.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:05 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3786500538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:05.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:05 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/870988564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:05.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:05 smithi033 ceph-mon[21626]: pgmap v367: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:05.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:05 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3786500538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:05.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:05 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/870988564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:05.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.337+0000 7f59cde00640 1 -- 172.21.15.22:0/628410841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59c8062b60 con 0x7f59c8075fa0 2024-04-20T21:33:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.339+0000 7f59c5600640 1 -- 172.21.15.22:0/628410841 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f59b40c2510 con 0x7f59c8075fa0 2024-04-20T21:33:05.338 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:05.338 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:33:05.338 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:33:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.341+0000 7f59cde00640 1 -- 172.21.15.22:0/628410841 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f59ac0a3c30 msgr2=0x7f59ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.341+0000 7f59cde00640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f59ac0a3c30 0x7f59ac0a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f59b8005fd0 tx=0x7f59b8009500 comp rx=0 tx=0).stop 2024-04-20T21:33:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.341+0000 7f59cde00640 1 -- 172.21.15.22:0/628410841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59c8075fa0 msgr2=0x7f59c819ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.341+0000 7f59cde00640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59c8075fa0 0x7f59c819ec90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f59b4000c00 tx=0x7f59b4005d50 comp rx=0 tx=0).stop 2024-04-20T21:33:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.341+0000 7f59cde00640 1 -- 172.21.15.22:0/628410841 shutdown_connections 2024-04-20T21:33:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.341+0000 7f59cde00640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f59ac0a3c30 0x7f59ac0a60f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.341+0000 7f59cde00640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59c810e3d0 0x7f59c81a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.341+0000 7f59cde00640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59c8075fa0 0x7f59c819ec90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.341+0000 7f59cde00640 1 --2- 172.21.15.22:0/628410841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c80749c0 0x7f59c819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.341+0000 7f59cde00640 1 -- 172.21.15.22:0/628410841 >> 172.21.15.22:0/628410841 conn(0x7f59c80fd370 msgr2=0x7f59c8101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:05.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.341+0000 7f59cde00640 1 -- 172.21.15.22:0/628410841 shutdown_connections 2024-04-20T21:33:05.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.341+0000 7f59cde00640 1 -- 172.21.15.22:0/628410841 wait complete. 2024-04-20T21:33:05.359 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:05.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.569+0000 7fe99c000640 1 -- 172.21.15.22:0/4093216792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe994077b60 msgr2=0x7fe994077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.569+0000 7fe99c000640 1 --2- 172.21.15.22:0/4093216792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe994077b60 0x7fe994077fc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe980009900 tx=0x7fe98002bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:05.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.569+0000 7fe99c000640 1 -- 172.21.15.22:0/4093216792 shutdown_connections 2024-04-20T21:33:05.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.569+0000 7fe99c000640 1 --2- 172.21.15.22:0/4093216792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe994078500 0x7fe994078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.569+0000 7fe99c000640 1 --2- 172.21.15.22:0/4093216792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe994077b60 0x7fe994077fc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.569+0000 7fe99c000640 1 --2- 172.21.15.22:0/4093216792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994076920 0x7fe994076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.569+0000 7fe99c000640 1 -- 172.21.15.22:0/4093216792 >> 172.21.15.22:0/4093216792 conn(0x7fe9940ff750 msgr2=0x7fe994101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:05.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.569+0000 7fe99c000640 1 -- 172.21.15.22:0/4093216792 shutdown_connections 2024-04-20T21:33:05.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.569+0000 7fe99c000640 1 -- 172.21.15.22:0/4093216792 wait complete. 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.570+0000 7fe99c000640 1 Processor -- start 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.571+0000 7fe99c000640 1 -- start start 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.571+0000 7fe99c000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994076920 0x7fe99419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.571+0000 7fe99c000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe994077b60 0x7fe99419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.571+0000 7fe99c000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe994078500 0x7fe9941a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.571+0000 7fe99c000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe994114290 con 0x7fe994077b60 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.571+0000 7fe99c000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe994114110 con 0x7fe994076920 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.571+0000 7fe99c000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe994114410 con 0x7fe994078500 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.571+0000 7fe998c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe994077b60 0x7fe99419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.571+0000 7fe998c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe994077b60 0x7fe99419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47562/0 (socket says 172.21.15.22:47562) 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.571+0000 7fe998c00640 1 -- 172.21.15.22:0/3959090445 learned_addr learned my addr 172.21.15.22:0/3959090445 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.571+0000 7fe999600640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994076920 0x7fe99419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.571+0000 7fe99a000640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe994078500 0x7fe9941a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.572+0000 7fe999600640 1 -- 172.21.15.22:0/3959090445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe994078500 msgr2=0x7fe9941a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.572+0000 7fe999600640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe994078500 0x7fe9941a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.572+0000 7fe999600640 1 -- 172.21.15.22:0/3959090445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe994077b60 msgr2=0x7fe99419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.572+0000 7fe999600640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe994077b60 0x7fe99419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.572+0000 7fe999600640 1 -- 172.21.15.22:0/3959090445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9941a64e0 con 0x7fe994076920 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.572+0000 7fe998c00640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe994077b60 0x7fe99419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.572+0000 7fe999600640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994076920 0x7fe99419e8c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00eb00 tx=0x7fe98c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:05.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.572+0000 7fe992000640 1 -- 172.21.15.22:0/3959090445 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c010430 con 0x7fe994076920 2024-04-20T21:33:05.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.572+0000 7fe99c000640 1 -- 172.21.15.22:0/3959090445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9941a67d0 con 0x7fe994076920 2024-04-20T21:33:05.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.572+0000 7fe992000640 1 -- 172.21.15.22:0/3959090445 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c010ab0 con 0x7fe994076920 2024-04-20T21:33:05.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.572+0000 7fe992000640 1 -- 172.21.15.22:0/3959090445 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c017bb0 con 0x7fe994076920 2024-04-20T21:33:05.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.572+0000 7fe99c000640 1 -- 172.21.15.22:0/3959090445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9941a6d40 con 0x7fe994076920 2024-04-20T21:33:05.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.573+0000 7fe99c000640 1 -- 172.21.15.22:0/3959090445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe994110c60 con 0x7fe994076920 2024-04-20T21:33:05.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.575+0000 7fe992000640 1 -- 172.21.15.22:0/3959090445 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe98c010c60 con 0x7fe994076920 2024-04-20T21:33:05.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.577+0000 7fe992000640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe9780a3bb0 0x7fe9780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.577+0000 7fe992000640 1 -- 172.21.15.22:0/3959090445 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fe98c0d0e20 con 0x7fe994076920 2024-04-20T21:33:05.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.577+0000 7fe998c00640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe9780a3bb0 0x7fe9780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.578+0000 7fe998c00640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe9780a3bb0 0x7fe9780a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe98002c8e0 tx=0x7fe98002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:05.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.582+0000 7fe992000640 1 -- 172.21.15.22:0/3959090445 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe98c09d780 con 0x7fe994076920 2024-04-20T21:33:05.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.790+0000 7fe99c000640 1 -- 172.21.15.22:0/3959090445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe994062b60 con 0x7fe994076920 2024-04-20T21:33:05.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.792+0000 7fe992000640 1 -- 172.21.15.22:0/3959090445 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22721 (secure 0 0 0) 0x7fe98c09d080 con 0x7fe994076920 2024-04-20T21:33:05.794 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:05.794 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":90,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:32:31.596064+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.795+0000 7fe99c000640 1 -- 172.21.15.22:0/3959090445 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe9780a3bb0 msgr2=0x7fe9780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.795+0000 7fe99c000640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe9780a3bb0 0x7fe9780a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe98002c8e0 tx=0x7fe98002df90 comp rx=0 tx=0).stop 2024-04-20T21:33:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.795+0000 7fe99c000640 1 -- 172.21.15.22:0/3959090445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994076920 msgr2=0x7fe99419e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.795+0000 7fe99c000640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994076920 0x7fe99419e8c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00eb00 tx=0x7fe98c00c2d0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.795+0000 7fe99c000640 1 -- 172.21.15.22:0/3959090445 shutdown_connections 2024-04-20T21:33:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.795+0000 7fe99c000640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe9780a3bb0 0x7fe9780a6070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.795+0000 7fe99c000640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe994078500 0x7fe9941a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.795+0000 7fe99c000640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe994077b60 0x7fe99419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.795+0000 7fe99c000640 1 --2- 172.21.15.22:0/3959090445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe994076920 0x7fe99419e8c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.795+0000 7fe99c000640 1 -- 172.21.15.22:0/3959090445 >> 172.21.15.22:0/3959090445 conn(0x7fe9940ff750 msgr2=0x7fe994100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.795+0000 7fe99c000640 1 -- 172.21.15.22:0/3959090445 shutdown_connections 2024-04-20T21:33:05.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:05.795+0000 7fe99c000640 1 -- 172.21.15.22:0/3959090445 wait complete. 2024-04-20T21:33:05.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:06.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.031+0000 7f0e2be00640 1 -- 172.21.15.22:0/2639013510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2c076920 msgr2=0x7f0e2c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.031+0000 7f0e2be00640 1 --2- 172.21.15.22:0/2639013510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2c076920 0x7f0e2c076d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009ca0 tx=0x7f0e1c02c100 comp rx=0 tx=0).stop 2024-04-20T21:33:06.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.031+0000 7f0e2be00640 1 -- 172.21.15.22:0/2639013510 shutdown_connections 2024-04-20T21:33:06.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.031+0000 7f0e2be00640 1 --2- 172.21.15.22:0/2639013510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e2c078500 0x7f0e2c078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.031+0000 7f0e2be00640 1 --2- 172.21.15.22:0/2639013510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2c077b60 0x7f0e2c077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.031+0000 7f0e2be00640 1 --2- 172.21.15.22:0/2639013510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2c076920 0x7f0e2c076d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.031+0000 7f0e2be00640 1 -- 172.21.15.22:0/2639013510 >> 172.21.15.22:0/2639013510 conn(0x7f0e2c0ff570 msgr2=0x7f0e2c1019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.032+0000 7f0e2be00640 1 -- 172.21.15.22:0/2639013510 shutdown_connections 2024-04-20T21:33:06.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.032+0000 7f0e2be00640 1 -- 172.21.15.22:0/2639013510 wait complete. 2024-04-20T21:33:06.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.033+0000 7f0e2be00640 1 Processor -- start 2024-04-20T21:33:06.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.033+0000 7f0e2be00640 1 -- start start 2024-04-20T21:33:06.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.033+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2c076920 0x7f0e2c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.033+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2c077b60 0x7f0e2c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.033+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e2c078500 0x7f0e2c1a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.033+0000 7f0e2be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c114040 con 0x7f0e2c076920 2024-04-20T21:33:06.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.033+0000 7f0e2be00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c113ec0 con 0x7f0e2c078500 2024-04-20T21:33:06.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.033+0000 7f0e2be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c1141c0 con 0x7f0e2c077b60 2024-04-20T21:33:06.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.034+0000 7f0e2aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2c076920 0x7f0e2c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.034+0000 7f0e22000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2c077b60 0x7f0e2c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.034+0000 7f0e22000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2c077b60 0x7f0e2c19ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33648/0 (socket says 172.21.15.22:33648) 2024-04-20T21:33:06.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.034+0000 7f0e22000640 1 -- 172.21.15.22:0/2649521472 learned_addr learned my addr 172.21.15.22:0/2649521472 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:06.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.034+0000 7f0e2b400640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e2c078500 0x7f0e2c1a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.034+0000 7f0e22000640 1 -- 172.21.15.22:0/2649521472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e2c078500 msgr2=0x7f0e2c1a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.034+0000 7f0e22000640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e2c078500 0x7f0e2c1a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.034+0000 7f0e22000640 1 -- 172.21.15.22:0/2649521472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2c076920 msgr2=0x7f0e2c19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.034+0000 7f0e22000640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2c076920 0x7f0e2c19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.034+0000 7f0e22000640 1 -- 172.21.15.22:0/2649521472 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2c1a62e0 con 0x7f0e2c077b60 2024-04-20T21:33:06.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.034+0000 7f0e2aa00640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2c076920 0x7f0e2c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:06.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.034+0000 7f0e22000640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2c077b60 0x7f0e2c19ec00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400e9f0 tx=0x7f0e1400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.035+0000 7f0e23e00640 1 -- 172.21.15.22:0/2649521472 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400cd80 con 0x7f0e2c077b60 2024-04-20T21:33:06.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.035+0000 7f0e23e00640 1 -- 172.21.15.22:0/2649521472 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e14010910 con 0x7f0e2c077b60 2024-04-20T21:33:06.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.035+0000 7f0e23e00640 1 -- 172.21.15.22:0/2649521472 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14018990 con 0x7f0e2c077b60 2024-04-20T21:33:06.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.035+0000 7f0e2be00640 1 -- 172.21.15.22:0/2649521472 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2c1a65d0 con 0x7f0e2c077b60 2024-04-20T21:33:06.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.035+0000 7f0e2be00640 1 -- 172.21.15.22:0/2649521472 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2c1a6b20 con 0x7f0e2c077b60 2024-04-20T21:33:06.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.036+0000 7f0e2be00640 1 -- 172.21.15.22:0/2649521472 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e2c110970 con 0x7f0e2c077b60 2024-04-20T21:33:06.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.038+0000 7f0e23e00640 1 -- 172.21.15.22:0/2649521472 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0e14010ac0 con 0x7f0e2c077b60 2024-04-20T21:33:06.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.039+0000 7f0e23e00640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f0e100a3bb0 0x7f0e100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.039+0000 7f0e23e00640 1 -- 172.21.15.22:0/2649521472 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f0e140d07a0 con 0x7f0e2c077b60 2024-04-20T21:33:06.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.040+0000 7f0e2aa00640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f0e100a3bb0 0x7f0e100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.040+0000 7f0e2aa00640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f0e100a3bb0 0x7f0e100a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c02c710 tx=0x7f0e1c005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.044+0000 7f0e23e00640 1 -- 172.21.15.22:0/2649521472 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0e1409cab0 con 0x7f0e2c077b60 2024-04-20T21:33:06.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:06 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/628410841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:06.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:06 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3959090445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:06.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.299+0000 7f0e2be00640 1 -- 172.21.15.22:0/2649521472 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e2c062b60 con 0x7f0e2c077b60 2024-04-20T21:33:06.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.300+0000 7f0e23e00640 1 -- 172.21.15.22:0/2649521472 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f0e140a0db0 con 0x7f0e2c077b60 2024-04-20T21:33:06.300 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:06.300 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:33:06.300 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:33:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.303+0000 7f0e2be00640 1 -- 172.21.15.22:0/2649521472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f0e100a3bb0 msgr2=0x7f0e100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.303+0000 7f0e2be00640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f0e100a3bb0 0x7f0e100a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c02c710 tx=0x7f0e1c005ef0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.303+0000 7f0e2be00640 1 -- 172.21.15.22:0/2649521472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2c077b60 msgr2=0x7f0e2c19ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.303+0000 7f0e2be00640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2c077b60 0x7f0e2c19ec00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400e9f0 tx=0x7f0e1400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.303+0000 7f0e2be00640 1 -- 172.21.15.22:0/2649521472 shutdown_connections 2024-04-20T21:33:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.303+0000 7f0e2be00640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f0e100a3bb0 0x7f0e100a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.303+0000 7f0e2be00640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e2c078500 0x7f0e2c1a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.303+0000 7f0e2be00640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2c077b60 0x7f0e2c19ec00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.303+0000 7f0e2be00640 1 --2- 172.21.15.22:0/2649521472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2c076920 0x7f0e2c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.303+0000 7f0e2be00640 1 -- 172.21.15.22:0/2649521472 >> 172.21.15.22:0/2649521472 conn(0x7f0e2c0ff570 msgr2=0x7f0e2c101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.303+0000 7f0e2be00640 1 -- 172.21.15.22:0/2649521472 shutdown_connections 2024-04-20T21:33:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.303+0000 7f0e2be00640 1 -- 172.21.15.22:0/2649521472 wait complete. 2024-04-20T21:33:06.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:06 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/628410841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:06.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:06 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3959090445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:06.323 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-20T21:31:27.176318+0000', 'modified': '2024-04-20T21:31:31.525301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24581}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24568': {'gid': 24568, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6839/2642727200', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 2642727200}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 2642727200}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6839/300232041', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6838', 'nonce': 300232041}, {'type': 'v1', 'addr': '172.21.15.33:6839', 'nonce': 300232041}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24581]} 2024-04-20T21:33:06.323 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:33:06.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T21:33:06.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:06 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/628410841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:06.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:06 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3959090445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:06.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.531+0000 7f35fd600640 1 -- 172.21.15.22:0/2891013101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35f81085c0 msgr2=0x7f35f81089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.531+0000 7f35fd600640 1 --2- 172.21.15.22:0/2891013101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35f81085c0 0x7f35f81089e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009900 tx=0x7f35e802bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:06.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.531+0000 7f35fd600640 1 -- 172.21.15.22:0/2891013101 shutdown_connections 2024-04-20T21:33:06.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.531+0000 7f35fd600640 1 --2- 172.21.15.22:0/2891013101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f810a170 0x7f35f8110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.531+0000 7f35fd600640 1 --2- 172.21.15.22:0/2891013101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35f81097b0 0x7f35f8109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.531+0000 7f35fd600640 1 --2- 172.21.15.22:0/2891013101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35f81085c0 0x7f35f81089e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.531+0000 7f35fd600640 1 -- 172.21.15.22:0/2891013101 >> 172.21.15.22:0/2891013101 conn(0x7f35f8077560 msgr2=0x7f35f80799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.532+0000 7f35fd600640 1 -- 172.21.15.22:0/2891013101 shutdown_connections 2024-04-20T21:33:06.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.532+0000 7f35fd600640 1 -- 172.21.15.22:0/2891013101 wait complete. 2024-04-20T21:33:06.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.533+0000 7f35fd600640 1 Processor -- start 2024-04-20T21:33:06.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.533+0000 7f35fd600640 1 -- start start 2024-04-20T21:33:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.533+0000 7f35fd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f81085c0 0x7f35f819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.533+0000 7f35fd600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35f81097b0 0x7f35f819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.533+0000 7f35fd600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35f810a170 0x7f35f81a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.533+0000 7f35fd600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f35f8114290 con 0x7f35f81085c0 2024-04-20T21:33:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.533+0000 7f35fd600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35f8114110 con 0x7f35f81097b0 2024-04-20T21:33:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.533+0000 7f35fd600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f35f8114410 con 0x7f35f810a170 2024-04-20T21:33:06.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.534+0000 7f35f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f81085c0 0x7f35f819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.534+0000 7f35f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f81085c0 0x7f35f819e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47620/0 (socket says 172.21.15.22:47620) 2024-04-20T21:33:06.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.534+0000 7f35f6a00640 1 -- 172.21.15.22:0/680939801 learned_addr learned my addr 172.21.15.22:0/680939801 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:06.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.534+0000 7f35f7400640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35f810a170 0x7f35f81a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.534+0000 7f35efe00640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35f81097b0 0x7f35f819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.534+0000 7f35f6a00640 1 -- 172.21.15.22:0/680939801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35f810a170 msgr2=0x7f35f81a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.534+0000 7f35f6a00640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35f810a170 0x7f35f81a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.534+0000 7f35f6a00640 1 -- 172.21.15.22:0/680939801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35f81097b0 msgr2=0x7f35f819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.534+0000 7f35f6a00640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35f81097b0 0x7f35f819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.534+0000 7f35f6a00640 1 -- 172.21.15.22:0/680939801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f81a6560 con 0x7f35f81085c0 2024-04-20T21:33:06.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.535+0000 7f35f6a00640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f81085c0 0x7f35f819e940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f35e8000c00 tx=0x7f35e802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.535+0000 7f35ef400640 1 -- 172.21.15.22:0/680939801 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8043020 con 0x7f35f81085c0 2024-04-20T21:33:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.535+0000 7f35ef400640 1 -- 172.21.15.22:0/680939801 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e80054e0 con 0x7f35f81085c0 2024-04-20T21:33:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.535+0000 7f35fd600640 1 -- 172.21.15.22:0/680939801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f81a67f0 con 0x7f35f81085c0 2024-04-20T21:33:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.535+0000 7f35ef400640 1 -- 172.21.15.22:0/680939801 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8003c40 con 0x7f35f81085c0 2024-04-20T21:33:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.535+0000 7f35fd600640 1 -- 172.21.15.22:0/680939801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f81a6d40 con 0x7f35f81085c0 2024-04-20T21:33:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.538+0000 7f35ef400640 1 -- 172.21.15.22:0/680939801 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f35e8046020 con 0x7f35f81085c0 2024-04-20T21:33:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.538+0000 7f36003dd740 1 Processor -- start 2024-04-20T21:33:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.539+0000 7f36003dd740 1 -- start start 2024-04-20T21:33:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.539+0000 7f36003dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff075930 0x5564ff075d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.539+0000 7f36003dd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 0x5564ff07ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.539+0000 7f36003dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff07b490 0x5564ff07d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.539+0000 7f36003dd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5564fefdc9d0 con 0x5564ff076320 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.539+0000 7f36003dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5564fef83270 con 0x5564ff075930 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.539+0000 7f36003dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5564fefd8990 con 0x5564ff07b490 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.539+0000 7f35f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 0x5564ff07ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.539+0000 7f35f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 0x5564ff07ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47622/0 (socket says 172.21.15.22:47622) 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.539+0000 7f35f6a00640 1 -- 172.21.15.22:0/1053750991 learned_addr learned my addr 172.21.15.22:0/1053750991 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.539+0000 7f35f7400640 1 --2- 172.21.15.22:0/1053750991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff075930 0x5564ff075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.539+0000 7f35efe00640 1 --2- 172.21.15.22:0/1053750991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff07b490 0x5564ff07d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.540+0000 7f35f7400640 1 -- 172.21.15.22:0/1053750991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff07b490 msgr2=0x5564ff07d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.540+0000 7f35f7400640 1 --2- 172.21.15.22:0/1053750991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff07b490 0x5564ff07d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.540+0000 7f35f7400640 1 -- 172.21.15.22:0/1053750991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 msgr2=0x5564ff07ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.540+0000 7f35f7400640 1 --2- 172.21.15.22:0/1053750991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 0x5564ff07ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.540+0000 7f35f7400640 1 -- 172.21.15.22:0/1053750991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564fecef2d0 con 0x5564ff075930 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.540+0000 7f35f7400640 1 --2- 172.21.15.22:0/1053750991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff075930 0x5564ff075d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009f00 tx=0x7f35f0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c5b2791f7d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.540+0000 7f35ef400640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f35dc0a3bb0 0x7f35dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.540+0000 7f35e7400640 1 -- 172.21.15.22:0/1053750991 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0004d20 con 0x5564ff075930 2024-04-20T21:33:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.540+0000 7f35e7400640 1 -- 172.21.15.22:0/1053750991 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f0004030 con 0x5564ff075930 2024-04-20T21:33:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.540+0000 7f35e7400640 1 -- 172.21.15.22:0/1053750991 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0005260 con 0x5564ff075930 2024-04-20T21:33:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.540+0000 7f35ef400640 1 -- 172.21.15.22:0/680939801 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f35e80f2420 con 0x7f35f81085c0 2024-04-20T21:33:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.540+0000 7f35f6a00640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f35dc0a3bb0 0x7f35dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.541+0000 7f36003dd740 1 -- 172.21.15.22:0/1053750991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff075930 msgr2=0x5564ff075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.541+0000 7f36003dd740 1 --2- 172.21.15.22:0/1053750991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff075930 0x5564ff075d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009f00 tx=0x7f35f0011810 comp rx=0 tx=0).stop 2024-04-20T21:33:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.541+0000 7f36003dd740 1 -- 172.21.15.22:0/1053750991 shutdown_connections 2024-04-20T21:33:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.541+0000 7f36003dd740 1 --2- 172.21.15.22:0/1053750991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff07b490 0x5564ff07d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.541+0000 7f36003dd740 1 --2- 172.21.15.22:0/1053750991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 0x5564ff07ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.541+0000 7f36003dd740 1 --2- 172.21.15.22:0/1053750991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff075930 0x5564ff075d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.541+0000 7f36003dd740 1 -- 172.21.15.22:0/1053750991 >> 172.21.15.22:0/1053750991 conn(0x5564ff070e50 msgr2=0x5564ff071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.541+0000 7f36003dd740 1 -- 172.21.15.22:0/1053750991 shutdown_connections 2024-04-20T21:33:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.541+0000 7f35f6a00640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f35dc0a3bb0 0x7f35dc0a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f35e802ceb0 tx=0x7f35e80f5e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.541+0000 7f36003dd740 1 -- 172.21.15.22:0/1053750991 wait complete. 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.542+0000 7f36003dd740 1 -- start start 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.542+0000 7f36003dd740 1 Processor -- start 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.542+0000 7f36003dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff075930 0x5564ff07ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.542+0000 7f36003dd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 0x5564ff080440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.542+0000 7f36003dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff07b490 0x5564ff087480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f36003dd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5564ff07e440 con 0x5564ff076320 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f36003dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5564fef965e0 con 0x5564ff07b490 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f36003dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5564ff07e5c0 con 0x5564ff075930 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f36003dd740 10 client.0 _ll_register_callbacks cb 0x5564feceeac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f35efe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 0x5564ff080440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f35f7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff075930 0x5564ff07ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f35efe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 0x5564ff080440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47632/0 (socket says 172.21.15.22:47632) 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f35efe00640 1 -- 172.21.15.22:0/3294795315 learned_addr learned my addr 172.21.15.22:0/3294795315 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f35f7400640 1 --2- 172.21.15.22:0/3294795315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff07b490 0x5564ff087480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f35efe00640 1 -- 172.21.15.22:0/3294795315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff075930 msgr2=0x5564ff07ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f35efe00640 1 --2- 172.21.15.22:0/3294795315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff075930 0x5564ff07ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f35efe00640 1 -- 172.21.15.22:0/3294795315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff07b490 msgr2=0x5564ff087480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f35efe00640 1 --2- 172.21.15.22:0/3294795315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff07b490 0x5564ff087480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f35efe00640 1 -- 172.21.15.22:0/3294795315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564fedb96b0 con 0x5564ff076320 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.543+0000 7f35efe00640 1 --2- 172.21.15.22:0/3294795315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 0x5564ff080440 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f35e00126e0 tx=0x7f35e0012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89db48d10fa8585d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.544+0000 7f35dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.544+0000 7f35dbe00640 1 -- 172.21.15.22:0/3294795315 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e000f8c0 con 0x5564ff076320 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.544+0000 7f35dbe00640 1 -- 172.21.15.22:0/3294795315 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f35e000fa60 con 0x5564ff076320 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.544+0000 7f36003dd740 1 -- 172.21.15.22:0/3294795315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f0004030 con 0x5564ff076320 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.544+0000 7f35dbe00640 1 -- 172.21.15.22:0/3294795315 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f35e000fc60 con 0x5564ff076320 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.544+0000 7f36003dd740 10 client.14874 fetch_fsmap learned FSMap version 20 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.544+0000 7f36003dd740 1 -- 172.21.15.22:0/3294795315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5564ff08a940 con 0x5564ff076320 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.544+0000 7f35dbe00640 1 -- 172.21.15.22:0/3294795315 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f0004030 con 0x5564ff076320 2024-04-20T21:33:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.545+0000 7f35dbe00640 1 -- 172.21.15.22:0/3294795315 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f35e00264b0 con 0x5564ff076320 2024-04-20T21:33:06.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.545+0000 7f36003dd740 10 client.14874 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:33:06.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.545+0000 7f36003dd740 10 client.14874 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:33:06.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.545+0000 7f36003dd740 20 client.14874 populate_metadata read hostname 'smithi022' 2024-04-20T21:33:06.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.545+0000 7f36003dd740 1 --2- 172.21.15.22:0/3294795315 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5564ff08d040 0x5564ff08e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.545+0000 7f36003dd740 4 client.14874 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:33:06.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.545+0000 7f36003dd740 1 -- 172.21.15.22:0/3294795315 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564fefea3f0 con 0x5564ff08d040 2024-04-20T21:33:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.545+0000 7f35f7400640 1 --2- 172.21.15.22:0/3294795315 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5564ff08d040 0x5564ff08e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.546+0000 7f35f7400640 1 --2- 172.21.15.22:0/3294795315 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5564ff08d040 0x5564ff08e4b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf119cc73e578ed server_cookie=434a4333f3dc54a7 in_seq=0 out_seq=0 2024-04-20T21:33:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.546+0000 7f35dbe00640 10 client.14874 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.547+0000 7f35dbe00640 1 -- 172.21.15.22:0/3294795315 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564fefea3f0 con 0x5564ff08d040 2024-04-20T21:33:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.547+0000 7f35dbe00640 10 client.14874 handle_command_reply: tid=0 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.547+0000 7f36003dd740 1 client.14874 shutdown 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.547+0000 7f36003dd740 1 -- 172.21.15.22:0/3294795315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 msgr2=0x5564ff080440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.547+0000 7f36003dd740 1 --2- 172.21.15.22:0/3294795315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 0x5564ff080440 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f35e00126e0 tx=0x7f35e0012ba0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.547+0000 7f36003dd740 1 -- 172.21.15.22:0/3294795315 shutdown_connections 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.547+0000 7f36003dd740 1 --2- 172.21.15.22:0/3294795315 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5564ff08d040 0x5564ff08e4b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.547+0000 7f36003dd740 1 --2- 172.21.15.22:0/3294795315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff07b490 0x5564ff087480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.547+0000 7f36003dd740 1 --2- 172.21.15.22:0/3294795315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff076320 0x5564ff080440 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.547+0000 7f36003dd740 1 --2- 172.21.15.22:0/3294795315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff075930 0x5564ff07ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.547+0000 7f36003dd740 1 -- 172.21.15.22:0/3294795315 >> 172.21.15.22:0/3294795315 conn(0x5564ff070e50 msgr2=0x5564ff0700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.547+0000 7f35dbe00640 0 client.14874 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.548+0000 7f36003dd740 1 -- 172.21.15.22:0/3294795315 shutdown_connections 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.548+0000 7f36003dd740 1 -- 172.21.15.22:0/3294795315 wait complete. 2024-04-20T21:33:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.548+0000 7f36003dd740 20 client.14874 trim_cache size 0 max 16384 2024-04-20T21:33:06.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.567+0000 7f36003dd740 1 Processor -- start 2024-04-20T21:33:06.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.568+0000 7f36003dd740 1 -- start start 2024-04-20T21:33:06.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.568+0000 7f36003dd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff075930 0x5564ff06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.568+0000 7f36003dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff076320 0x5564ff087440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.568+0000 7f36003dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff07b490 0x5564ff06fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.568+0000 7f36003dd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5564ff08c7f0 con 0x5564ff075930 2024-04-20T21:33:06.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.568+0000 7f36003dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5564ff08cec0 con 0x5564ff07b490 2024-04-20T21:33:06.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.568+0000 7f36003dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5564ff08d590 con 0x5564ff076320 2024-04-20T21:33:06.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.568+0000 7f35efe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff076320 0x5564ff087440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.568+0000 7f35f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff075930 0x5564ff06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.568+0000 7f35efe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff076320 0x5564ff087440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33700/0 (socket says 172.21.15.22:33700) 2024-04-20T21:33:06.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.568+0000 7f35efe00640 1 -- 172.21.15.22:0/2723060248 learned_addr learned my addr 172.21.15.22:0/2723060248 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.569+0000 7f35f7400640 1 --2- 172.21.15.22:0/2723060248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff07b490 0x5564ff06fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.569+0000 7f35efe00640 1 -- 172.21.15.22:0/2723060248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff07b490 msgr2=0x5564ff06fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.569+0000 7f35efe00640 1 --2- 172.21.15.22:0/2723060248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff07b490 0x5564ff06fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.569+0000 7f35efe00640 1 -- 172.21.15.22:0/2723060248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff075930 msgr2=0x5564ff06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.569+0000 7f35efe00640 1 --2- 172.21.15.22:0/2723060248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff075930 0x5564ff06f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.569+0000 7f35efe00640 1 -- 172.21.15.22:0/2723060248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ff071c10 con 0x5564ff076320 2024-04-20T21:33:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.569+0000 7f35f7400640 1 --2- 172.21.15.22:0/2723060248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff07b490 0x5564ff06fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:33:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.569+0000 7f35efe00640 1 --2- 172.21.15.22:0/2723060248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff076320 0x5564ff087440 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f35e000ff90 tx=0x7f35e0026b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c08034d4c01b58df server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.569+0000 7f35e6a00640 1 -- 172.21.15.22:0/2723060248 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00264e0 con 0x5564ff076320 2024-04-20T21:33:06.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.569+0000 7f35e6a00640 1 -- 172.21.15.22:0/2723060248 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e0026680 con 0x5564ff076320 2024-04-20T21:33:06.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.569+0000 7f35e6a00640 1 -- 172.21.15.22:0/2723060248 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0004640 con 0x5564ff076320 2024-04-20T21:33:06.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.570+0000 7f36003dd740 1 -- 172.21.15.22:0/2723060248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff076320 msgr2=0x5564ff087440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.570+0000 7f36003dd740 1 --2- 172.21.15.22:0/2723060248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff076320 0x5564ff087440 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f35e000ff90 tx=0x7f35e0026b90 comp rx=0 tx=0).stop 2024-04-20T21:33:06.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.570+0000 7f36003dd740 1 -- 172.21.15.22:0/2723060248 shutdown_connections 2024-04-20T21:33:06.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.570+0000 7f36003dd740 1 --2- 172.21.15.22:0/2723060248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff07b490 0x5564ff06fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.570+0000 7f36003dd740 1 --2- 172.21.15.22:0/2723060248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff076320 0x5564ff087440 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.570+0000 7f36003dd740 1 --2- 172.21.15.22:0/2723060248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff075930 0x5564ff06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.570+0000 7f36003dd740 1 -- 172.21.15.22:0/2723060248 >> 172.21.15.22:0/2723060248 conn(0x5564ff070e50 msgr2=0x5564ff077820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.570+0000 7f36003dd740 1 -- 172.21.15.22:0/2723060248 shutdown_connections 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.570+0000 7f36003dd740 1 -- 172.21.15.22:0/2723060248 wait complete. 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.571+0000 7f36003dd740 1 -- start start 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.571+0000 7f36003dd740 1 Processor -- start 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.571+0000 7f36003dd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff075930 0x5564ff082e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.571+0000 7f36003dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff07b490 0x5564ff083380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.571+0000 7f36003dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff086350 0x5564ff0867d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.571+0000 7f36003dd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5564ff08e330 con 0x5564ff075930 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.571+0000 7f36003dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5564ff08dc60 con 0x5564ff086350 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.571+0000 7f36003dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5564ff08ea00 con 0x5564ff07b490 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.571+0000 7f36003dd740 10 client.0 _ll_register_callbacks cb 0x7f35e0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35efe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff07b490 0x5564ff083380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff075930 0x5564ff082e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35efe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff07b490 0x5564ff083380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33708/0 (socket says 172.21.15.22:33708) 2024-04-20T21:33:06.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35efe00640 1 -- 172.21.15.22:0/3773751178 learned_addr learned my addr 172.21.15.22:0/3773751178 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35f7400640 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff086350 0x5564ff0867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35efe00640 1 -- 172.21.15.22:0/3773751178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff086350 msgr2=0x5564ff0867d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35efe00640 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff086350 0x5564ff0867d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35efe00640 1 -- 172.21.15.22:0/3773751178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff075930 msgr2=0x5564ff082e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35efe00640 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff075930 0x5564ff082e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35efe00640 1 -- 172.21.15.22:0/3773751178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ff086d10 con 0x5564ff07b490 2024-04-20T21:33:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35f7400640 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff086350 0x5564ff0867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:33:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35efe00640 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff07b490 0x5564ff083380 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f35e0000c00 tx=0x7f35e0003cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c269db7f5deec79e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.572+0000 7f35dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:33:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.573+0000 7f35dbe00640 1 -- 172.21.15.22:0/3773751178 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0035020 con 0x5564ff07b490 2024-04-20T21:33:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.573+0000 7f36003dd740 1 -- 172.21.15.22:0/3773751178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e0026680 con 0x5564ff07b490 2024-04-20T21:33:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.573+0000 7f35dbe00640 1 -- 172.21.15.22:0/3773751178 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f35e0009290 con 0x5564ff07b490 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.573+0000 7f35dbe00640 1 -- 172.21.15.22:0/3773751178 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f35e0040050 con 0x5564ff07b490 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.573+0000 7f36003dd740 10 client.25021 fetch_fsmap learned FSMap version 20 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.573+0000 7f36003dd740 1 -- 172.21.15.22:0/3773751178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5564ff0f35a0 con 0x5564ff07b490 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.573+0000 7f35dbe00640 1 -- 172.21.15.22:0/3773751178 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e0026680 con 0x5564ff07b490 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.574+0000 7f35dbe00640 1 -- 172.21.15.22:0/3773751178 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f35e0030460 con 0x5564ff07b490 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.574+0000 7f36003dd740 10 client.25021 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.574+0000 7f36003dd740 10 client.25021 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.574+0000 7f36003dd740 20 client.25021 populate_metadata read hostname 'smithi022' 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.574+0000 7f36003dd740 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5564ff0f7fb0 0x5564ff0f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.574+0000 7f36003dd740 4 client.25021 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.574+0000 7f36003dd740 1 -- 172.21.15.22:0/3773751178 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "status"}) -- 0x5564fefe4260 con 0x5564ff0f7fb0 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.574+0000 7f35f7400640 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5564ff0f7fb0 0x5564ff0f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f35f7400640 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5564ff0f7fb0 0x5564ff0f6cc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dce4454ea9908dc server_cookie=2d8d8237948abc91 in_seq=0 out_seq=0 2024-04-20T21:33:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f35dbe00640 10 client.25021 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f35dbe00640 1 -- 172.21.15.22:0/3773751178 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5564fefe4260 con 0x5564ff0f7fb0 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f35dbe00640 10 client.25021 handle_command_reply: tid=0 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f36003dd740 1 client.25021 shutdown 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f36003dd740 1 -- 172.21.15.22:0/3773751178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff07b490 msgr2=0x5564ff083380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f36003dd740 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff07b490 0x5564ff083380 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f35e0000c00 tx=0x7f35e0003cc0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f36003dd740 1 -- 172.21.15.22:0/3773751178 shutdown_connections 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f36003dd740 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5564ff0f7fb0 0x5564ff0f6cc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f36003dd740 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ff086350 0x5564ff0867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f36003dd740 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564ff07b490 0x5564ff083380 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f36003dd740 1 --2- 172.21.15.22:0/3773751178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5564ff075930 0x5564ff082e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f35dbe00640 0 client.25021 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.575+0000 7f36003dd740 1 -- 172.21.15.22:0/3773751178 >> 172.21.15.22:0/3773751178 conn(0x5564ff070e50 msgr2=0x5564ff074100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f36003dd740 1 -- 172.21.15.22:0/3773751178 shutdown_connections 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f36003dd740 1 -- 172.21.15.22:0/3773751178 wait complete. 2024-04-20T21:33:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f36003dd740 20 client.25021 trim_cache size 0 max 16384 2024-04-20T21:33:06.578 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:33:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "92c49c5c-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:33:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-04-20T21:33:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24581, 2024-04-20T21:33:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-04-20T21:33:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-04-20T21:33:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-04-20T21:33:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 98.903482878000005, 2024-04-20T21:33:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 20, 2024-04-20T21:33:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 90, 2024-04-20T21:33:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 88, 2024-04-20T21:33:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 105.372363296 2024-04-20T21:33:06.579 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:33:06.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f35fd600640 1 -- 172.21.15.22:0/680939801 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f35dc0a3bb0 msgr2=0x7f35dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f35fd600640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f35dc0a3bb0 0x7f35dc0a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f35e802ceb0 tx=0x7f35e80f5e60 comp rx=0 tx=0).stop 2024-04-20T21:33:06.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f35fd600640 1 -- 172.21.15.22:0/680939801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f81085c0 msgr2=0x7f35f819e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f35fd600640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f81085c0 0x7f35f819e940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f35e8000c00 tx=0x7f35e802c650 comp rx=0 tx=0).stop 2024-04-20T21:33:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f35fd600640 1 -- 172.21.15.22:0/680939801 shutdown_connections 2024-04-20T21:33:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f35fd600640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f35dc0a3bb0 0x7f35dc0a6070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f35fd600640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35f810a170 0x7f35f81a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f35fd600640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35f81097b0 0x7f35f819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f35fd600640 1 --2- 172.21.15.22:0/680939801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f81085c0 0x7f35f819e940 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.576+0000 7f35fd600640 1 -- 172.21.15.22:0/680939801 >> 172.21.15.22:0/680939801 conn(0x7f35f8077560 msgr2=0x7f35f810cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.577+0000 7f35fd600640 1 -- 172.21.15.22:0/680939801 shutdown_connections 2024-04-20T21:33:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:06.577+0000 7f35fd600640 1 -- 172.21.15.22:0/680939801 wait complete. 2024-04-20T21:33:06.593 INFO:tasks.cephadm:Setup complete, yielding 2024-04-20T21:33:06.594 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T21:33:06.603 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-04-20T21:33:06.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-20T21:33:07.076 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:33:07.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:07 smithi033 ceph-mon[21626]: pgmap v368: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:07.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:07 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2649521472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:07.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:07 smithi022 ceph-mon[16843]: pgmap v368: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:07.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:07 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2649521472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:07.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:07 smithi122 ceph-mon[22147]: pgmap v368: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:07.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:07 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2649521472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.266+0000 7f3277c00700 1 -- 172.21.15.22:0/1717547187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f327010a990 msgr2=0x7f327010ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.266+0000 7f3277c00700 1 --2- 172.21.15.22:0/1717547187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f327010a990 0x7f327010ce20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3268009f90 tx=0x7f326800b330 comp rx=0 tx=0).stop 2024-04-20T21:33:09.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.266+0000 7f3277c00700 1 -- 172.21.15.22:0/1717547187 shutdown_connections 2024-04-20T21:33:09.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.266+0000 7f3277c00700 1 --2- 172.21.15.22:0/1717547187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f327010a990 0x7f327010ce20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.266+0000 7f3277c00700 1 --2- 172.21.15.22:0/1717547187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3270076a80 0x7f3270074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.266+0000 7f3277c00700 1 --2- 172.21.15.22:0/1717547187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32700754b0 0x7f3270075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.266+0000 7f3277c00700 1 -- 172.21.15.22:0/1717547187 >> 172.21.15.22:0/1717547187 conn(0x7f32700fe020 msgr2=0x7f32701004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:09.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.266+0000 7f3277c00700 1 -- 172.21.15.22:0/1717547187 shutdown_connections 2024-04-20T21:33:09.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.268+0000 7f3277c00700 1 -- 172.21.15.22:0/1717547187 wait complete. 2024-04-20T21:33:09.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.269+0000 7f3277c00700 1 Processor -- start 2024-04-20T21:33:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.269+0000 7f3277c00700 1 -- start start 2024-04-20T21:33:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.269+0000 7f3277c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32700754b0 0x7f327019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.269+0000 7f3277c00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3270076a80 0x7f327019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.269+0000 7f3277c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f327010a990 0x7f32701a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.269+0000 7f3277c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f32701108a0 con 0x7f327010a990 2024-04-20T21:33:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.269+0000 7f3277c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3270110720 con 0x7f32700754b0 2024-04-20T21:33:09.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.269+0000 7f3277c00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3270110a20 con 0x7f3270076a80 2024-04-20T21:33:09.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.270+0000 7f3275c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f327010a990 0x7f32701a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:09.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.270+0000 7f326fe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3270076a80 0x7f327019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:09.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.270+0000 7f3275200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32700754b0 0x7f327019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:09.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.270+0000 7f3275c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f327010a990 0x7f32701a2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47676/0 (socket says 172.21.15.22:47676) 2024-04-20T21:33:09.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.270+0000 7f326fe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3270076a80 0x7f327019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33734/0 (socket says 172.21.15.22:33734) 2024-04-20T21:33:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.270+0000 7f3275c00700 1 -- 172.21.15.22:0/2806008320 learned_addr learned my addr 172.21.15.22:0/2806008320 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.270+0000 7f3275200700 1 -- 172.21.15.22:0/2806008320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3270076a80 msgr2=0x7f327019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.270+0000 7f3275200700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3270076a80 0x7f327019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.270+0000 7f3275200700 1 -- 172.21.15.22:0/2806008320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f327010a990 msgr2=0x7f32701a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.270+0000 7f3275200700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f327010a990 0x7f32701a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.270+0000 7f3275200700 1 -- 172.21.15.22:0/2806008320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32701a2ab0 con 0x7f32700754b0 2024-04-20T21:33:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.270+0000 7f326fe00700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3270076a80 0x7f327019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.271+0000 7f3275200700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32700754b0 0x7f327019ae50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f326400e9c0 tx=0x7f326400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:09.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.271+0000 7f326d600700 1 -- 172.21.15.22:0/2806008320 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326400c9c0 con 0x7f32700754b0 2024-04-20T21:33:09.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.271+0000 7f3277c00700 1 -- 172.21.15.22:0/2806008320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32701a2da0 con 0x7f32700754b0 2024-04-20T21:33:09.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.271+0000 7f326d600700 1 -- 172.21.15.22:0/2806008320 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f326400cb60 con 0x7f32700754b0 2024-04-20T21:33:09.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.271+0000 7f326d600700 1 -- 172.21.15.22:0/2806008320 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264018680 con 0x7f32700754b0 2024-04-20T21:33:09.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.271+0000 7f3277c00700 1 -- 172.21.15.22:0/2806008320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32701a31f0 con 0x7f32700754b0 2024-04-20T21:33:09.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.272+0000 7f3277c00700 1 -- 172.21.15.22:0/2806008320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327002df90 con 0x7f32700754b0 2024-04-20T21:33:09.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.274+0000 7f326d600700 1 -- 172.21.15.22:0/2806008320 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3264010430 con 0x7f32700754b0 2024-04-20T21:33:09.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.275+0000 7f326d600700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f32580a3c40 0x7f32580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:09.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.275+0000 7f326d600700 1 -- 172.21.15.22:0/2806008320 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f32640d0540 con 0x7f32700754b0 2024-04-20T21:33:09.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.276+0000 7f326fe00700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f32580a3c40 0x7f32580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:09.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.276+0000 7f326fe00700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f32580a3c40 0x7f32580a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f325c009750 tx=0x7f325c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:09.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.279+0000 7f326d600700 1 -- 172.21.15.22:0/2806008320 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f326409cea0 con 0x7f32700754b0 2024-04-20T21:33:09.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:09 smithi033 ceph-mon[21626]: pgmap v369: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:09.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:09 smithi022 ceph-mon[16843]: pgmap v369: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:09.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:09 smithi122 ceph-mon[22147]: pgmap v369: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:09.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.534+0000 7f3277c00700 1 -- 172.21.15.22:0/2806008320 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3270062e90 con 0x7f32580a3c40 2024-04-20T21:33:09.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.537+0000 7f326d600700 1 -- 172.21.15.22:0/2806008320 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3270062e90 con 0x7f32580a3c40 2024-04-20T21:33:09.536 INFO:teuthology.orchestra.run.smithi022.stdout:Backend: cephadm 2024-04-20T21:33:09.536 INFO:teuthology.orchestra.run.smithi022.stdout:Available: Yes 2024-04-20T21:33:09.536 INFO:teuthology.orchestra.run.smithi022.stdout:Paused: No 2024-04-20T21:33:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.540+0000 7f3277c00700 1 -- 172.21.15.22:0/2806008320 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f32580a3c40 msgr2=0x7f32580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.540+0000 7f3277c00700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f32580a3c40 0x7f32580a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f325c009750 tx=0x7f325c009310 comp rx=0 tx=0).stop 2024-04-20T21:33:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.540+0000 7f3277c00700 1 -- 172.21.15.22:0/2806008320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32700754b0 msgr2=0x7f327019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.540+0000 7f3277c00700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32700754b0 0x7f327019ae50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f326400e9c0 tx=0x7f326400ecd0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.540+0000 7f3277c00700 1 -- 172.21.15.22:0/2806008320 shutdown_connections 2024-04-20T21:33:09.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.540+0000 7f3277c00700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f32580a3c40 0x7f32580a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.540+0000 7f3277c00700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f327010a990 0x7f32701a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.540+0000 7f3277c00700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32700754b0 0x7f327019ae50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.540+0000 7f3277c00700 1 --2- 172.21.15.22:0/2806008320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3270076a80 0x7f327019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.540+0000 7f3277c00700 1 -- 172.21.15.22:0/2806008320 >> 172.21.15.22:0/2806008320 conn(0x7f32700fe020 msgr2=0x7f3270106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:09.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.540+0000 7f3277c00700 1 -- 172.21.15.22:0/2806008320 shutdown_connections 2024-04-20T21:33:09.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:09.540+0000 7f3277c00700 1 -- 172.21.15.22:0/2806008320 wait complete. 2024-04-20T21:33:10.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:10 smithi022 ceph-mon[16843]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:10.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:10 smithi033 ceph-mon[21626]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:10.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-20T21:33:10.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:10 smithi122 ceph-mon[22147]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:10.849 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:33:11.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:11 smithi033 ceph-mon[21626]: pgmap v370: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:11.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:11 smithi022 ceph-mon[16843]: pgmap v370: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:11.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:11 smithi122 ceph-mon[22147]: pgmap v370: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.106+0000 7f194ae00700 1 -- 172.21.15.22:0/1846046467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194410a1e0 msgr2=0x7f194410a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.106+0000 7f194ae00700 1 --2- 172.21.15.22:0/1846046467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194410a1e0 0x7f194410a640 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f19300099c0 tx=0x7f1930009cd0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.106+0000 7f194ae00700 1 -- 172.21.15.22:0/1846046467 shutdown_connections 2024-04-20T21:33:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.106+0000 7f194ae00700 1 --2- 172.21.15.22:0/1846046467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194410ab80 0x7f1944111430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.106+0000 7f194ae00700 1 --2- 172.21.15.22:0/1846046467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1944108fe0 0x7f1944109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.106+0000 7f194ae00700 1 --2- 172.21.15.22:0/1846046467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194410a1e0 0x7f194410a640 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.106+0000 7f194ae00700 1 -- 172.21.15.22:0/1846046467 >> 172.21.15.22:0/1846046467 conn(0x7f1944077fb0 msgr2=0x7f194407a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:13.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.107+0000 7f194ae00700 1 -- 172.21.15.22:0/1846046467 shutdown_connections 2024-04-20T21:33:13.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.107+0000 7f194ae00700 1 -- 172.21.15.22:0/1846046467 wait complete. 2024-04-20T21:33:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.108+0000 7f194ae00700 1 Processor -- start 2024-04-20T21:33:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.108+0000 7f194ae00700 1 -- start start 2024-04-20T21:33:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.108+0000 7f194ae00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1944108fe0 0x7f194419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.108+0000 7f194ae00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194410a1e0 0x7f194419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.108+0000 7f194ae00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194410ab80 0x7f19441a6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.108+0000 7f194ae00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1944114ba0 con 0x7f1944108fe0 2024-04-20T21:33:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.108+0000 7f194ae00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1944114a20 con 0x7f194410a1e0 2024-04-20T21:33:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.108+0000 7f194ae00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1944114d20 con 0x7f194410ab80 2024-04-20T21:33:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.108+0000 7f1949a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1944108fe0 0x7f194419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.109+0000 7f1949000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194410a1e0 0x7f194419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.109+0000 7f1949a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1944108fe0 0x7f194419f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53546/0 (socket says 172.21.15.22:53546) 2024-04-20T21:33:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.109+0000 7f1949a00700 1 -- 172.21.15.22:0/2518738972 learned_addr learned my addr 172.21.15.22:0/2518738972 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.109+0000 7f194a400700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194410ab80 0x7f19441a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.109+0000 7f1949000700 1 -- 172.21.15.22:0/2518738972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194410ab80 msgr2=0x7f19441a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:13.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.109+0000 7f1949000700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194410ab80 0x7f19441a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.109+0000 7f1949000700 1 -- 172.21.15.22:0/2518738972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1944108fe0 msgr2=0x7f194419f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:13.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.109+0000 7f1949000700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1944108fe0 0x7f194419f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.109+0000 7f1949000700 1 -- 172.21.15.22:0/2518738972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19441a6de0 con 0x7f194410a1e0 2024-04-20T21:33:13.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.109+0000 7f1949000700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194410a1e0 0x7f194419f6c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1930003790 tx=0x7f1930009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:13.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.110+0000 7f1942000700 1 -- 172.21.15.22:0/2518738972 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930009e60 con 0x7f194410a1e0 2024-04-20T21:33:13.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.110+0000 7f194ae00700 1 -- 172.21.15.22:0/2518738972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19441a7070 con 0x7f194410a1e0 2024-04-20T21:33:13.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.110+0000 7f1942000700 1 -- 172.21.15.22:0/2518738972 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1930003960 con 0x7f194410a1e0 2024-04-20T21:33:13.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.110+0000 7f1942000700 1 -- 172.21.15.22:0/2518738972 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930005560 con 0x7f194410a1e0 2024-04-20T21:33:13.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.110+0000 7f194ae00700 1 -- 172.21.15.22:0/2518738972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19441a7550 con 0x7f194410a1e0 2024-04-20T21:33:13.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.111+0000 7f194ae00700 1 -- 172.21.15.22:0/2518738972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1944069740 con 0x7f194410a1e0 2024-04-20T21:33:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.113+0000 7f1942000700 1 -- 172.21.15.22:0/2518738972 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1930003b10 con 0x7f194410a1e0 2024-04-20T21:33:13.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.114+0000 7f1942000700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f192c0a3c40 0x7f192c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:13.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.114+0000 7f1942000700 1 -- 172.21.15.22:0/2518738972 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f19300d11a0 con 0x7f194410a1e0 2024-04-20T21:33:13.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.114+0000 7f1949a00700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f192c0a3c40 0x7f192c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:13.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.115+0000 7f1949a00700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f192c0a3c40 0x7f192c0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1938005950 tx=0x7f1938009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:13.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.119+0000 7f1942000700 1 -- 172.21.15.22:0/2518738972 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1930013070 con 0x7f194410a1e0 2024-04-20T21:33:13.246 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:13 smithi022 ceph-mon[16843]: pgmap v371: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:13.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:13 smithi122 ceph-mon[22147]: pgmap v371: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.366+0000 7f194ae00700 1 -- 172.21.15.22:0/2518738972 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f1944062e90 con 0x7f192c0a3c40 2024-04-20T21:33:13.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.409+0000 7f1942000700 1 -- 172.21.15.22:0/2518738972 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f1944062e90 con 0x7f192c0a3c40 2024-04-20T21:33:13.408 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T21:33:13.408 INFO:teuthology.orchestra.run.smithi022.stdout:mds.a smithi022 running (115s) 23s ago 114s 12.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 9c4294888507 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mds.b smithi033 running (112s) 43s ago 112s 28.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 ac595150a420 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mds.c smithi122 running (2m) 43s ago 2m 15.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 d6f9c5f0e01c 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mds.d smithi022 running (2m) 23s ago 2m 12.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 5952b92e908c 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mds.e smithi033 running (2m) 43s ago 2m 13.2M - 19.0.0-3133-ga8c84f59 76bc6b91b726 bb6003d28292 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mds.f smithi122 running (2m) 43s ago 2m 16.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 669031ef3cd0 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mds.g smithi022 running (119s) 23s ago 119s 12.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 84dd5dfa8f03 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mds.h smithi033 running (2m) 43s ago 2m 13.9M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fd7697cbf577 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mds.i smithi122 running (2m) 43s ago 2m 19.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 18510cab3c10 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mds.j smithi022 running (2m) 23s ago 2m 12.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 e1fde6631cfc 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mds.k smithi033 running (2m) 43s ago 2m 16.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 7f796241d1be 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mds.l smithi122 running (2m) 43s ago 2m 15.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 f62252f8aaf2 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.x smithi022 *:9283,8765 running (10m) 23s ago 10m 515M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3f9d41b7577e 2024-04-20T21:33:13.409 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.y smithi033 *:8443,8765 running (8m) 43s ago 8m 454M - 19.0.0-3133-ga8c84f59 76bc6b91b726 f64fc78265e2 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.z smithi122 *:8443,8765 running (8m) 43s ago 8m 454M - 19.0.0-3133-ga8c84f59 76bc6b91b726 9a11fad32bee 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:mon.a smithi022 running (10m) 23s ago 10m 47.0M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 4f99bfab2067 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:mon.b smithi033 running (8m) 43s ago 8m 39.9M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 2ea070e847cc 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:mon.c smithi122 running (8m) 43s ago 8m 42.0M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 b8ff97a8380d 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (7m) 23s ago 7m 46.6M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 c38a89d36d8e 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi033 running (7m) 43s ago 7m 54.7M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 459d5fa72b8e 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi122 running (7m) 43s ago 7m 49.5M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 6a89ee3089fe 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi022 running (6m) 23s ago 6m 48.7M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 92af2d4bf416 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi033 running (6m) 43s ago 6m 53.4M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 2091bf9e0c08 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi122 running (5m) 43s ago 5m 57.5M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 bee52ee40c05 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 smithi022 running (5m) 23s ago 5m 49.2M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 3a1633f88c5c 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 smithi033 running (4m) 43s ago 4m 49.2M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 8f5b29468d5d 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.8 smithi122 running (4m) 43s ago 4m 56.0M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 c1ef21c85fb7 2024-04-20T21:33:13.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.9 smithi022 running (3m) 23s ago 3m 49.5M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 a29e5ee3302c 2024-04-20T21:33:13.411 INFO:teuthology.orchestra.run.smithi022.stdout:osd.10 smithi033 running (3m) 43s ago 3m 46.4M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 27bb3d6239cb 2024-04-20T21:33:13.411 INFO:teuthology.orchestra.run.smithi022.stdout:osd.11 smithi122 running (2m) 43s ago 2m 43.8M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 fc4581786c9f 2024-04-20T21:33:13.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.413+0000 7f194ae00700 1 -- 172.21.15.22:0/2518738972 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f192c0a3c40 msgr2=0x7f192c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:13.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.413+0000 7f194ae00700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f192c0a3c40 0x7f192c0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1938005950 tx=0x7f1938009260 comp rx=0 tx=0).stop 2024-04-20T21:33:13.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.413+0000 7f194ae00700 1 -- 172.21.15.22:0/2518738972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194410a1e0 msgr2=0x7f194419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.413+0000 7f194ae00700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194410a1e0 0x7f194419f6c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1930003790 tx=0x7f1930009580 comp rx=0 tx=0).stop 2024-04-20T21:33:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.413+0000 7f194ae00700 1 -- 172.21.15.22:0/2518738972 shutdown_connections 2024-04-20T21:33:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.413+0000 7f194ae00700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f192c0a3c40 0x7f192c0a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.413+0000 7f194ae00700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194410ab80 0x7f19441a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.413+0000 7f194ae00700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1944108fe0 0x7f194419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.413+0000 7f194ae00700 1 --2- 172.21.15.22:0/2518738972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194410a1e0 0x7f194419f6c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.413+0000 7f194ae00700 1 -- 172.21.15.22:0/2518738972 >> 172.21.15.22:0/2518738972 conn(0x7f1944077fb0 msgr2=0x7f194410d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.413+0000 7f194ae00700 1 -- 172.21.15.22:0/2518738972 shutdown_connections 2024-04-20T21:33:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:13.413+0000 7f194ae00700 1 -- 172.21.15.22:0/2518738972 wait complete. 2024-04-20T21:33:13.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:13 smithi033 ceph-mon[21626]: pgmap v371: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:14.169 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-20T21:33:14.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:14 smithi022 ceph-mon[16843]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:14.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:14 smithi122 ceph-mon[22147]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:14.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:14 smithi033 ceph-mon[21626]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:14.603 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:33:15.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:15 smithi022 ceph-mon[16843]: pgmap v372: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:15.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:15 smithi122 ceph-mon[22147]: pgmap v372: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:15.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:15 smithi033 ceph-mon[21626]: pgmap v372: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T21:33:16.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.803+0000 7f8befa00700 1 -- 172.21.15.22:0/142819012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be8106890 msgr2=0x7f8be810d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:16.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.803+0000 7f8befa00700 1 --2- 172.21.15.22:0/142819012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be8106890 0x7f8be810d140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8be000b0d0 tx=0x7f8be000b3e0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.803+0000 7f8befa00700 1 -- 172.21.15.22:0/142819012 shutdown_connections 2024-04-20T21:33:16.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.803+0000 7f8befa00700 1 --2- 172.21.15.22:0/142819012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be8106890 0x7f8be810d140 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.803+0000 7f8befa00700 1 --2- 172.21.15.22:0/142819012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8be8104cf0 0x7f8be8105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.803+0000 7f8befa00700 1 --2- 172.21.15.22:0/142819012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8be8105ef0 0x7f8be8106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.803+0000 7f8befa00700 1 -- 172.21.15.22:0/142819012 >> 172.21.15.22:0/142819012 conn(0x7f8be81002b0 msgr2=0x7f8be81026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:16.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.803+0000 7f8befa00700 1 -- 172.21.15.22:0/142819012 shutdown_connections 2024-04-20T21:33:16.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.803+0000 7f8befa00700 1 -- 172.21.15.22:0/142819012 wait complete. 2024-04-20T21:33:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.804+0000 7f8befa00700 1 Processor -- start 2024-04-20T21:33:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.804+0000 7f8befa00700 1 -- start start 2024-04-20T21:33:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.804+0000 7f8befa00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8be8104cf0 0x7f8be819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.804+0000 7f8befa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be8105ef0 0x7f8be819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.804+0000 7f8befa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8be8106890 0x7f8be81a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.804+0000 7f8befa00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8be8110930 con 0x7f8be8105ef0 2024-04-20T21:33:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.804+0000 7f8befa00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8be81107b0 con 0x7f8be8106890 2024-04-20T21:33:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.804+0000 7f8befa00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8be8110ab0 con 0x7f8be8104cf0 2024-04-20T21:33:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.805+0000 7f8bed000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8be8104cf0 0x7f8be819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.805+0000 7f8bed000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8be8104cf0 0x7f8be819ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:48094/0 (socket says 172.21.15.22:48094) 2024-04-20T21:33:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.805+0000 7f8bed000700 1 -- 172.21.15.22:0/2242411853 learned_addr learned my addr 172.21.15.22:0/2242411853 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.805+0000 7f8be7e00700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be8105ef0 0x7f8be819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.805+0000 7f8beda00700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8be8106890 0x7f8be81a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.805+0000 7f8bed000700 1 -- 172.21.15.22:0/2242411853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8be8106890 msgr2=0x7f8be81a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.805+0000 7f8bed000700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8be8106890 0x7f8be81a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.805+0000 7f8bed000700 1 -- 172.21.15.22:0/2242411853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be8105ef0 msgr2=0x7f8be819b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.805+0000 7f8bed000700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be8105ef0 0x7f8be819b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.805+0000 7f8bed000700 1 -- 172.21.15.22:0/2242411853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be81a2ab0 con 0x7f8be8104cf0 2024-04-20T21:33:16.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.806+0000 7f8bed000700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8be8104cf0 0x7f8be819ae50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00e980 tx=0x7f8bdc00ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:16.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.806+0000 7f8be5600700 1 -- 172.21.15.22:0/2242411853 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc004230 con 0x7f8be8104cf0 2024-04-20T21:33:16.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.806+0000 7f8befa00700 1 -- 172.21.15.22:0/2242411853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be81a2da0 con 0x7f8be8104cf0 2024-04-20T21:33:16.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.806+0000 7f8be5600700 1 -- 172.21.15.22:0/2242411853 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bdc0043d0 con 0x7f8be8104cf0 2024-04-20T21:33:16.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.806+0000 7f8be5600700 1 -- 172.21.15.22:0/2242411853 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc003dc0 con 0x7f8be8104cf0 2024-04-20T21:33:16.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.806+0000 7f8befa00700 1 -- 172.21.15.22:0/2242411853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be81a31a0 con 0x7f8be8104cf0 2024-04-20T21:33:16.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.807+0000 7f8befa00700 1 -- 172.21.15.22:0/2242411853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be802df90 con 0x7f8be8104cf0 2024-04-20T21:33:16.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.809+0000 7f8be5600700 1 -- 172.21.15.22:0/2242411853 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8bdc00c810 con 0x7f8be8104cf0 2024-04-20T21:33:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.810+0000 7f8be5600700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8bd00a3c40 0x7f8bd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.811+0000 7f8be5600700 1 -- 172.21.15.22:0/2242411853 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f8bdc0d0690 con 0x7f8be8104cf0 2024-04-20T21:33:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.811+0000 7f8be7e00700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8bd00a3c40 0x7f8bd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.811+0000 7f8be7e00700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8bd00a3c40 0x7f8bd00a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009c60 tx=0x7f8bd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:16.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:16.813+0000 7f8be5600700 1 -- 172.21.15.22:0/2242411853 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8bdc09cff0 con 0x7f8be8104cf0 2024-04-20T21:33:17.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.052+0000 7f8befa00700 1 -- 172.21.15.22:0/2242411853 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f8be8062e90 con 0x7f8bd00a3c40 2024-04-20T21:33:17.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.059+0000 7f8be5600700 1 -- 172.21.15.22:0/2242411853 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f8be8062e90 con 0x7f8bd00a3c40 2024-04-20T21:33:17.059 INFO:teuthology.orchestra.run.smithi022.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-20T21:33:17.059 INFO:teuthology.orchestra.run.smithi022.stdout:mds.all 12/12 47s ago 2m smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi033=b;smithi033=e;smithi033=h;smithi033=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-04-20T21:33:17.059 INFO:teuthology.orchestra.run.smithi022.stdout:mgr 3/3 47s ago 8m smithi022=x;smithi033=y;smithi122=z;count:3 2024-04-20T21:33:17.059 INFO:teuthology.orchestra.run.smithi022.stdout:mon 3/3 47s ago 8m smithi022:172.21.15.22=a;smithi033:172.21.15.33=b;smithi122:172.21.15.122=c;count:3 2024-04-20T21:33:17.059 INFO:teuthology.orchestra.run.smithi022.stdout:osd 12 47s ago - 2024-04-20T21:33:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.062+0000 7f8befa00700 1 -- 172.21.15.22:0/2242411853 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8bd00a3c40 msgr2=0x7f8bd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.062+0000 7f8befa00700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8bd00a3c40 0x7f8bd00a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009c60 tx=0x7f8bd4009260 comp rx=0 tx=0).stop 2024-04-20T21:33:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.062+0000 7f8befa00700 1 -- 172.21.15.22:0/2242411853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8be8104cf0 msgr2=0x7f8be819ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.062+0000 7f8befa00700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8be8104cf0 0x7f8be819ae50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00e980 tx=0x7f8bdc00ec90 comp rx=0 tx=0).stop 2024-04-20T21:33:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.063+0000 7f8befa00700 1 -- 172.21.15.22:0/2242411853 shutdown_connections 2024-04-20T21:33:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.063+0000 7f8befa00700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f8bd00a3c40 0x7f8bd00a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.063+0000 7f8befa00700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8be8106890 0x7f8be81a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.063+0000 7f8befa00700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8be8104cf0 0x7f8be819ae50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.063+0000 7f8befa00700 1 --2- 172.21.15.22:0/2242411853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be8105ef0 0x7f8be819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.063+0000 7f8befa00700 1 -- 172.21.15.22:0/2242411853 >> 172.21.15.22:0/2242411853 conn(0x7f8be81002b0 msgr2=0x7f8be8109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:17.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.063+0000 7f8befa00700 1 -- 172.21.15.22:0/2242411853 shutdown_connections 2024-04-20T21:33:17.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:17.063+0000 7f8befa00700 1 -- 172.21.15.22:0/2242411853 wait complete. 2024-04-20T21:33:17.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:17 smithi022 ceph-mon[16843]: pgmap v373: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:17.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:17 smithi122 ceph-mon[22147]: pgmap v373: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:17.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:17 smithi033 ceph-mon[21626]: pgmap v373: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:17.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-20T21:33:18.331 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:33:18.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:18 smithi022 ceph-mon[16843]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:18.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:18 smithi122 ceph-mon[22147]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:18.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:18 smithi033 ceph-mon[21626]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:19.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:19 smithi022 ceph-mon[16843]: pgmap v374: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:19.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:19 smithi122 ceph-mon[22147]: pgmap v374: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:19.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:19 smithi033 ceph-mon[21626]: pgmap v374: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.560+0000 7fe57e800700 1 -- 172.21.15.22:0/979897250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe578105ef0 msgr2=0x7fe578106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.560+0000 7fe57e800700 1 --2- 172.21.15.22:0/979897250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe578105ef0 0x7fe578106350 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe5640099c0 tx=0x7fe564009cd0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.560+0000 7fe57e800700 1 -- 172.21.15.22:0/979897250 shutdown_connections 2024-04-20T21:33:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.560+0000 7fe57e800700 1 --2- 172.21.15.22:0/979897250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe578106890 0x7fe57810d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.560+0000 7fe57e800700 1 --2- 172.21.15.22:0/979897250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578104cf0 0x7fe578105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.560+0000 7fe57e800700 1 --2- 172.21.15.22:0/979897250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe578105ef0 0x7fe578106350 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.560+0000 7fe57e800700 1 -- 172.21.15.22:0/979897250 >> 172.21.15.22:0/979897250 conn(0x7fe5781002b0 msgr2=0x7fe5781026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.560+0000 7fe57e800700 1 -- 172.21.15.22:0/979897250 shutdown_connections 2024-04-20T21:33:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.560+0000 7fe57e800700 1 -- 172.21.15.22:0/979897250 wait complete. 2024-04-20T21:33:20.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.561+0000 7fe57e800700 1 Processor -- start 2024-04-20T21:33:20.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.562+0000 7fe57e800700 1 -- start start 2024-04-20T21:33:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.562+0000 7fe57e800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe578104cf0 0x7fe57819aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.562+0000 7fe57e800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe578105ef0 0x7fe57819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.562+0000 7fe57e800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578106890 0x7fe5781a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.562+0000 7fe57e800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe578110900 con 0x7fe578105ef0 2024-04-20T21:33:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.562+0000 7fe57e800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe578110780 con 0x7fe578104cf0 2024-04-20T21:33:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.562+0000 7fe57e800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe578110a80 con 0x7fe578106890 2024-04-20T21:33:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.563+0000 7fe576a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe578105ef0 0x7fe57819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.563+0000 7fe577400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe578104cf0 0x7fe57819aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.563+0000 7fe576a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe578105ef0 0x7fe57819b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53572/0 (socket says 172.21.15.22:53572) 2024-04-20T21:33:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.563+0000 7fe577e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578106890 0x7fe5781a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:20.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.563+0000 7fe576a00700 1 -- 172.21.15.22:0/1703097090 learned_addr learned my addr 172.21.15.22:0/1703097090 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:20.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.563+0000 7fe577e00700 1 -- 172.21.15.22:0/1703097090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe578104cf0 msgr2=0x7fe57819aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.563+0000 7fe577e00700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe578104cf0 0x7fe57819aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.563+0000 7fe577e00700 1 -- 172.21.15.22:0/1703097090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe578105ef0 msgr2=0x7fe57819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.563+0000 7fe577e00700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe578105ef0 0x7fe57819b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.563+0000 7fe577e00700 1 -- 172.21.15.22:0/1703097090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5781a2b40 con 0x7fe578106890 2024-04-20T21:33:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.563+0000 7fe576a00700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe578105ef0 0x7fe57819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.564+0000 7fe577e00700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578106890 0x7fe5781a24a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe57000c0c0 tx=0x7fe57000c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.564+0000 7fe56fe00700 1 -- 172.21.15.22:0/1703097090 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe570003a00 con 0x7fe578106890 2024-04-20T21:33:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.564+0000 7fe56fe00700 1 -- 172.21.15.22:0/1703097090 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe570003ba0 con 0x7fe578106890 2024-04-20T21:33:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.564+0000 7fe56fe00700 1 -- 172.21.15.22:0/1703097090 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe570004840 con 0x7fe578106890 2024-04-20T21:33:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.564+0000 7fe57e800700 1 -- 172.21.15.22:0/1703097090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5781a2e30 con 0x7fe578106890 2024-04-20T21:33:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.564+0000 7fe57e800700 1 -- 172.21.15.22:0/1703097090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5781a3290 con 0x7fe578106890 2024-04-20T21:33:20.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.565+0000 7fe57e800700 1 -- 172.21.15.22:0/1703097090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe578069780 con 0x7fe578106890 2024-04-20T21:33:20.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.568+0000 7fe56fe00700 1 -- 172.21.15.22:0/1703097090 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe570012050 con 0x7fe578106890 2024-04-20T21:33:20.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.569+0000 7fe56fe00700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe55c0a3c40 0x7fe55c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:20.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.569+0000 7fe56fe00700 1 -- 172.21.15.22:0/1703097090 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fe5700cfee0 con 0x7fe578106890 2024-04-20T21:33:20.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.569+0000 7fe577400700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe55c0a3c40 0x7fe55c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:20.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.570+0000 7fe577400700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe55c0a3c40 0x7fe55c0a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe578105d50 tx=0x7fe568009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:20.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.571+0000 7fe56fe00700 1 -- 172.21.15.22:0/1703097090 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe570022060 con 0x7fe578106890 2024-04-20T21:33:20.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.809+0000 7fe57e800700 1 -- 172.21.15.22:0/1703097090 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fe578062e90 con 0x7fe55c0a3c40 2024-04-20T21:33:20.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.811+0000 7fe56fe00700 1 -- 172.21.15.22:0/1703097090 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fe578062e90 con 0x7fe55c0a3c40 2024-04-20T21:33:20.811 INFO:teuthology.orchestra.run.smithi022.stdout:HOST ADDR LABELS STATUS 2024-04-20T21:33:20.811 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 172.21.15.22 2024-04-20T21:33:20.811 INFO:teuthology.orchestra.run.smithi022.stdout:smithi033 172.21.15.33 2024-04-20T21:33:20.811 INFO:teuthology.orchestra.run.smithi022.stdout:smithi122 172.21.15.122 2024-04-20T21:33:20.811 INFO:teuthology.orchestra.run.smithi022.stdout:3 hosts in cluster 2024-04-20T21:33:20.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.814+0000 7fe57e800700 1 -- 172.21.15.22:0/1703097090 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe55c0a3c40 msgr2=0x7fe55c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:20.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.814+0000 7fe57e800700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe55c0a3c40 0x7fe55c0a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe578105d50 tx=0x7fe568009260 comp rx=0 tx=0).stop 2024-04-20T21:33:20.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.814+0000 7fe57e800700 1 -- 172.21.15.22:0/1703097090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578106890 msgr2=0x7fe5781a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:20.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.814+0000 7fe57e800700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578106890 0x7fe5781a24a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe57000c0c0 tx=0x7fe57000c3d0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.814+0000 7fe57e800700 1 -- 172.21.15.22:0/1703097090 shutdown_connections 2024-04-20T21:33:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.814+0000 7fe57e800700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe55c0a3c40 0x7fe55c0a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.814+0000 7fe57e800700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578106890 0x7fe5781a24a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.814+0000 7fe57e800700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe578104cf0 0x7fe57819aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.814+0000 7fe57e800700 1 --2- 172.21.15.22:0/1703097090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe578105ef0 0x7fe57819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.814+0000 7fe57e800700 1 -- 172.21.15.22:0/1703097090 >> 172.21.15.22:0/1703097090 conn(0x7fe5781002b0 msgr2=0x7fe5781091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.815+0000 7fe57e800700 1 -- 172.21.15.22:0/1703097090 shutdown_connections 2024-04-20T21:33:20.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:20.815+0000 7fe57e800700 1 -- 172.21.15.22:0/1703097090 wait complete. 2024-04-20T21:33:21.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:21 smithi022 ceph-mon[16843]: pgmap v375: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:21.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:21 smithi122 ceph-mon[22147]: pgmap v375: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:21.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:21 smithi033 ceph-mon[21626]: pgmap v375: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:21.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-20T21:33:22.044 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:33:22.354 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:22 smithi022 ceph-mon[16843]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:22.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:22 smithi122 ceph-mon[22147]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:22.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:22 smithi033 ceph-mon[21626]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:23.299 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:23 smithi022 ceph-mon[16843]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:23.454 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:23 smithi033 ceph-mon[21626]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:23.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:23 smithi122 ceph-mon[22147]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:24.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.158+0000 7fe595400700 1 -- 172.21.15.22:0/3726930370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe590075430 msgr2=0x7fe5900758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.158+0000 7fe595400700 1 --2- 172.21.15.22:0/3726930370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe590075430 0x7fe5900758b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009990 tx=0x7fe57c009ca0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.159+0000 7fe595400700 1 -- 172.21.15.22:0/3726930370 shutdown_connections 2024-04-20T21:33:24.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.159+0000 7fe595400700 1 --2- 172.21.15.22:0/3726930370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe59010a9b0 0x7fe59010ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.159+0000 7fe595400700 1 --2- 172.21.15.22:0/3726930370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe590076a00 0x7fe590074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.159+0000 7fe595400700 1 --2- 172.21.15.22:0/3726930370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe590075430 0x7fe5900758b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.159+0000 7fe595400700 1 -- 172.21.15.22:0/3726930370 >> 172.21.15.22:0/3726930370 conn(0x7fe5900fe050 msgr2=0x7fe590100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.159+0000 7fe595400700 1 -- 172.21.15.22:0/3726930370 shutdown_connections 2024-04-20T21:33:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.159+0000 7fe595400700 1 -- 172.21.15.22:0/3726930370 wait complete. 2024-04-20T21:33:24.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.160+0000 7fe595400700 1 Processor -- start 2024-04-20T21:33:24.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.160+0000 7fe595400700 1 -- start start 2024-04-20T21:33:24.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.160+0000 7fe595400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe590075430 0x7fe59019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.160+0000 7fe595400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe590076a00 0x7fe59019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.160+0000 7fe595400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe59010a9b0 0x7fe5901a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.160+0000 7fe595400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901108b0 con 0x7fe59010a9b0 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.160+0000 7fe595400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe590110730 con 0x7fe590075430 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.160+0000 7fe595400700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe590110a30 con 0x7fe590076a00 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.161+0000 7fe58de00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe590076a00 0x7fe59019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.161+0000 7fe58de00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe590076a00 0x7fe59019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:45474/0 (socket says 172.21.15.22:45474) 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.161+0000 7fe58e800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe590075430 0x7fe59019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.161+0000 7fe58de00700 1 -- 172.21.15.22:0/1477951013 learned_addr learned my addr 172.21.15.22:0/1477951013 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.161+0000 7fe58f200700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe59010a9b0 0x7fe5901a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.161+0000 7fe58de00700 1 -- 172.21.15.22:0/1477951013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe590075430 msgr2=0x7fe59019aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.161+0000 7fe58de00700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe590075430 0x7fe59019aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.161+0000 7fe58de00700 1 -- 172.21.15.22:0/1477951013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe59010a9b0 msgr2=0x7fe5901a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.161+0000 7fe58de00700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe59010a9b0 0x7fe5901a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.161+0000 7fe58de00700 1 -- 172.21.15.22:0/1477951013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5901a2b40 con 0x7fe590076a00 2024-04-20T21:33:24.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.161+0000 7fe58de00700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe590076a00 0x7fe59019b420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0095c0 tx=0x7fe57c00bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:24.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.162+0000 7fe587400700 1 -- 172.21.15.22:0/1477951013 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c026070 con 0x7fe590076a00 2024-04-20T21:33:24.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.162+0000 7fe595400700 1 -- 172.21.15.22:0/1477951013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5901a2dd0 con 0x7fe590076a00 2024-04-20T21:33:24.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.162+0000 7fe587400700 1 -- 172.21.15.22:0/1477951013 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe57c003d70 con 0x7fe590076a00 2024-04-20T21:33:24.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.162+0000 7fe587400700 1 -- 172.21.15.22:0/1477951013 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c00fe60 con 0x7fe590076a00 2024-04-20T21:33:24.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.162+0000 7fe595400700 1 -- 172.21.15.22:0/1477951013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5901a31d0 con 0x7fe590076a00 2024-04-20T21:33:24.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.163+0000 7fe595400700 1 -- 172.21.15.22:0/1477951013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe590069740 con 0x7fe590076a00 2024-04-20T21:33:24.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.165+0000 7fe587400700 1 -- 172.21.15.22:0/1477951013 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe57c029070 con 0x7fe590076a00 2024-04-20T21:33:24.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.166+0000 7fe587400700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe5740a3c40 0x7fe5740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:24.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.167+0000 7fe587400700 1 -- 172.21.15.22:0/1477951013 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fe57c0d12f0 con 0x7fe590076a00 2024-04-20T21:33:24.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.167+0000 7fe58e800700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe5740a3c40 0x7fe5740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:24.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.167+0000 7fe58e800700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe5740a3c40 0x7fe5740a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe580009c60 tx=0x7fe580009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:24.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.171+0000 7fe587400700 1 -- 172.21.15.22:0/1477951013 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe57c05dff0 con 0x7fe590076a00 2024-04-20T21:33:24.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.416+0000 7fe595400700 1 -- 172.21.15.22:0/1477951013 --> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe590062e90 con 0x7fe5740a3c40 2024-04-20T21:33:24.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.420+0000 7fe587400700 1 -- 172.21.15.22:0/1477951013 <== mgr.14152 v2:172.21.15.22:6800/1911146506 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe590062e90 con 0x7fe5740a3c40 2024-04-20T21:33:24.420 INFO:teuthology.orchestra.run.smithi022.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-20T21:33:24.420 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402EC480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-20T21:33:24.420 INFO:teuthology.orchestra.run.smithi022.stdout:smithi033 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-20T21:33:24.420 INFO:teuthology.orchestra.run.smithi022.stdout:smithi122 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010U400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-20T21:33:24.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.424+0000 7fe595400700 1 -- 172.21.15.22:0/1477951013 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe5740a3c40 msgr2=0x7fe5740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.424+0000 7fe595400700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe5740a3c40 0x7fe5740a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe580009c60 tx=0x7fe580009260 comp rx=0 tx=0).stop 2024-04-20T21:33:24.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.424+0000 7fe595400700 1 -- 172.21.15.22:0/1477951013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe590076a00 msgr2=0x7fe59019b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.424+0000 7fe595400700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe590076a00 0x7fe59019b420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0095c0 tx=0x7fe57c00bb90 comp rx=0 tx=0).stop 2024-04-20T21:33:24.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.424+0000 7fe595400700 1 -- 172.21.15.22:0/1477951013 shutdown_connections 2024-04-20T21:33:24.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.424+0000 7fe595400700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe5740a3c40 0x7fe5740a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.424+0000 7fe595400700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe59010a9b0 0x7fe5901a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.424+0000 7fe595400700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe590075430 0x7fe59019aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.424+0000 7fe595400700 1 --2- 172.21.15.22:0/1477951013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe590076a00 0x7fe59019b420 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.424+0000 7fe595400700 1 -- 172.21.15.22:0/1477951013 >> 172.21.15.22:0/1477951013 conn(0x7fe5900fe050 msgr2=0x7fe590106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:24.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.424+0000 7fe595400700 1 -- 172.21.15.22:0/1477951013 shutdown_connections 2024-04-20T21:33:24.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:24.424+0000 7fe595400700 1 -- 172.21.15.22:0/1477951013 wait complete. 2024-04-20T21:33:25.179 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T21:33:25.188 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-04-20T21:33:25.188 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-20T21:33:25.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:25 smithi022 ceph-mon[16843]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:25.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:25 smithi022 ceph-mon[16843]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:25.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:25 smithi122 ceph-mon[22147]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:25.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:25 smithi122 ceph-mon[22147]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:25.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:25 smithi033 ceph-mon[21626]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:25.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:25 smithi033 ceph-mon[21626]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:25.634 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:33:27.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:27 smithi022 ceph-mon[16843]: pgmap v378: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:27.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:27 smithi122 ceph-mon[22147]: pgmap v378: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:27.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:27 smithi033 ceph-mon[21626]: pgmap v378: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.781+0000 7f2f85800700 1 -- 172.21.15.22:0/3959696982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f80103540 msgr2=0x7f2f80105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.781+0000 7f2f85800700 1 --2- 172.21.15.22:0/3959696982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f80103540 0x7f2f80105930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009990 tx=0x7f2f70009ca0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.781+0000 7f2f85800700 1 -- 172.21.15.22:0/3959696982 shutdown_connections 2024-04-20T21:33:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.781+0000 7f2f85800700 1 --2- 172.21.15.22:0/3959696982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f801087a0 0x7f2f8010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.781+0000 7f2f85800700 1 --2- 172.21.15.22:0/3959696982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f80103540 0x7f2f80105930 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.781+0000 7f2f85800700 1 --2- 172.21.15.22:0/3959696982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f80105e70 0x7f2f80108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.781+0000 7f2f85800700 1 -- 172.21.15.22:0/3959696982 >> 172.21.15.22:0/3959696982 conn(0x7f2f800fcf50 msgr2=0x7f2f800ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.781+0000 7f2f85800700 1 -- 172.21.15.22:0/3959696982 shutdown_connections 2024-04-20T21:33:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.781+0000 7f2f85800700 1 -- 172.21.15.22:0/3959696982 wait complete. 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.782+0000 7f2f85800700 1 Processor -- start 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.783+0000 7f2f85800700 1 -- start start 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.783+0000 7f2f85800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f80103540 0x7f2f80102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.783+0000 7f2f85800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f80105e70 0x7f2f801014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.783+0000 7f2f85800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f801087a0 0x7f2f80101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.783+0000 7f2f85800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8010e680 con 0x7f2f80103540 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.783+0000 7f2f85800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8010e500 con 0x7f2f801087a0 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.783+0000 7f2f85800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8010e800 con 0x7f2f80105e70 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.783+0000 7f2f7ea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f80103540 0x7f2f80102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.783+0000 7f2f7f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f801087a0 0x7f2f80101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.783+0000 7f2f7ea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f80103540 0x7f2f80102ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39276/0 (socket says 172.21.15.22:39276) 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.783+0000 7f2f7ea00700 1 -- 172.21.15.22:0/3461732303 learned_addr learned my addr 172.21.15.22:0/3461732303 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.783+0000 7f2f7e000700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f80105e70 0x7f2f801014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.784+0000 7f2f7ea00700 1 -- 172.21.15.22:0/3461732303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f80105e70 msgr2=0x7f2f801014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.784+0000 7f2f7ea00700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f80105e70 0x7f2f801014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.784+0000 7f2f7ea00700 1 -- 172.21.15.22:0/3461732303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f801087a0 msgr2=0x7f2f80101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.784+0000 7f2f7ea00700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f801087a0 0x7f2f80101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.784+0000 7f2f7ea00700 1 -- 172.21.15.22:0/3461732303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f801022e0 con 0x7f2f80103540 2024-04-20T21:33:27.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.784+0000 7f2f7f400700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f801087a0 0x7f2f80101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:27.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.784+0000 7f2f7ea00700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f80103540 0x7f2f80102ea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000bbb0 tx=0x7f2f7000bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:27.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.784+0000 7f2f77400700 1 -- 172.21.15.22:0/3461732303 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70026070 con 0x7f2f80103540 2024-04-20T21:33:27.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.784+0000 7f2f77400700 1 -- 172.21.15.22:0/3461732303 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f7000f460 con 0x7f2f80103540 2024-04-20T21:33:27.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.784+0000 7f2f77400700 1 -- 172.21.15.22:0/3461732303 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70021650 con 0x7f2f80103540 2024-04-20T21:33:27.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.785+0000 7f2f85800700 1 -- 172.21.15.22:0/3461732303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f801a2e50 con 0x7f2f80103540 2024-04-20T21:33:27.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.785+0000 7f2f85800700 1 -- 172.21.15.22:0/3461732303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f801a31f0 con 0x7f2f80103540 2024-04-20T21:33:27.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.786+0000 7f2f85800700 1 -- 172.21.15.22:0/3461732303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f8002df90 con 0x7f2f80103540 2024-04-20T21:33:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.788+0000 7f2f77400700 1 -- 172.21.15.22:0/3461732303 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2f70029070 con 0x7f2f80103540 2024-04-20T21:33:27.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.789+0000 7f2f77400700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2f640a3cc0 0x7f2f640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:27.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.789+0000 7f2f77400700 1 -- 172.21.15.22:0/3461732303 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f2f700d1220 con 0x7f2f80103540 2024-04-20T21:33:27.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.790+0000 7f2f7e000700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2f640a3cc0 0x7f2f640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:27.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.790+0000 7f2f7e000700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2f640a3cc0 0x7f2f640a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009c60 tx=0x7f2f6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:27.794+0000 7f2f77400700 1 -- 172.21.15.22:0/3461732303 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2f7005dff0 con 0x7f2f80103540 2024-04-20T21:33:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.076+0000 7f2f85800700 1 -- 172.21.15.22:0/3461732303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2f8002e1b0 con 0x7f2f80103540 2024-04-20T21:33:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.077+0000 7f2f77400700 1 -- 172.21.15.22:0/3461732303 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+3019 (secure 0 0 0) 0x7f2f70013070 con 0x7f2f80103540 2024-04-20T21:33:28.078 INFO:teuthology.orchestra.run.smithi022.stdout:e20 2024-04-20T21:33:28.078 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T21:33:28.078 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T21:33:28.078 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-04-20T21:33:28.078 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:28.078 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-04-20T21:33:28.078 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-04-20T21:33:28.078 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 20 2024-04-20T21:33:28.078 INFO:teuthology.orchestra.run.smithi022.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-04-20T21:31:27.176318+0000 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-04-20T21:31:31.525301+0000 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 300 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 0 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:snap_rstat 1 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 1 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:in 0 2024-04-20T21:33:28.079 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=24581} 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data disabled 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 0 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:qdb_cluster leader: 24581 members: 24581 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b{0:24581} state up:active seq 3 addr [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.i{0:24568} state up:standby-replay seq 1 addr [v2:172.21.15.122:6838/2642727200,v1:172.21.15.122:6839/2642727200] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:28.080 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-04-20T21:33:28.081 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:28.081 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.j{-1:14382} state up:standby seq 1 addr [v2:172.21.15.22:6834/2822771951,v1:172.21.15.22:6835/2822771951] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:33:28.081 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d{-1:14400} state up:standby seq 1 addr [v2:172.21.15.22:6836/2696291995,v1:172.21.15.22:6837/2696291995] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:33:28.081 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.g{-1:14448} state up:standby seq 1 addr [v2:172.21.15.22:6838/4122889060,v1:172.21.15.22:6839/4122889060] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:33:28.081 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a{-1:14466} state up:standby seq 1 addr [v2:172.21.15.22:6840/2604771291,v1:172.21.15.22:6841/2604771291] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:33:28.081 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.k{-1:24446} state up:standby seq 1 addr [v2:172.21.15.33:6832/721989242,v1:172.21.15.33:6833/721989242] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:33:28.081 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.l{-1:24463} state up:standby seq 1 addr [v2:172.21.15.122:6832/1883458757,v1:172.21.15.122:6833/1883458757] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:33:28.081 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.e{-1:24494} state up:standby seq 1 addr [v2:172.21.15.33:6834/1440214109,v1:172.21.15.33:6835/1440214109] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:33:28.081 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.h{-1:24527} state up:standby seq 1 addr [v2:172.21.15.33:6836/1181941460,v1:172.21.15.33:6837/1181941460] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:33:28.081 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.f{-1:24535} state up:standby seq 1 addr [v2:172.21.15.122:6834/3637338060,v1:172.21.15.122:6835/3637338060] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:33:28.081 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c{-1:24547} state up:standby seq 1 addr [v2:172.21.15.122:6836/3716866957,v1:172.21.15.122:6837/3716866957] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:33:28.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.081+0000 7f2f85800700 1 -- 172.21.15.22:0/3461732303 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2f640a3cc0 msgr2=0x7f2f640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.081+0000 7f2f85800700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2f640a3cc0 0x7f2f640a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009c60 tx=0x7f2f6c009260 comp rx=0 tx=0).stop 2024-04-20T21:33:28.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.081+0000 7f2f85800700 1 -- 172.21.15.22:0/3461732303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f80103540 msgr2=0x7f2f80102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.081+0000 7f2f85800700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f80103540 0x7f2f80102ea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000bbb0 tx=0x7f2f7000bbe0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.081+0000 7f2f85800700 1 -- 172.21.15.22:0/3461732303 shutdown_connections 2024-04-20T21:33:28.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.081+0000 7f2f85800700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2f640a3cc0 0x7f2f640a6180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.081+0000 7f2f85800700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f801087a0 0x7f2f80101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.081+0000 7f2f85800700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f80103540 0x7f2f80102ea0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.081+0000 7f2f85800700 1 --2- 172.21.15.22:0/3461732303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f80105e70 0x7f2f801014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.081+0000 7f2f85800700 1 -- 172.21.15.22:0/3461732303 >> 172.21.15.22:0/3461732303 conn(0x7f2f800fcf50 msgr2=0x7f2f80103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:28.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.081+0000 7f2f85800700 1 -- 172.21.15.22:0/3461732303 shutdown_connections 2024-04-20T21:33:28.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:28.081+0000 7f2f85800700 1 -- 172.21.15.22:0/3461732303 wait complete. 2024-04-20T21:33:28.083 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:33:28.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:28 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3461732303' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T21:33:28.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:28 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3461732303' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T21:33:28.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:28 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3461732303' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T21:33:28.880 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-20T21:33:29.352 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:33:29.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:29 smithi022 ceph-mon[16843]: pgmap v379: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:29.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:29 smithi122 ceph-mon[22147]: pgmap v379: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:29.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:29 smithi033 ceph-mon[21626]: pgmap v379: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:33:31.306 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:31 smithi022 ceph-mon[16843]: pgmap v380: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:31.306 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:31 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-20T21:33:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.482+0000 7fb3d3e00700 1 -- 172.21.15.22:0/1649396161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d41086d0 msgr2=0x7fb3d410ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.482+0000 7fb3d3e00700 1 --2- 172.21.15.22:0/1649396161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d41086d0 0x7fb3d410ab40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00b0a0 tx=0x7fb3cc00b3b0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.482+0000 7fb3d3e00700 1 -- 172.21.15.22:0/1649396161 shutdown_connections 2024-04-20T21:33:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.482+0000 7fb3d3e00700 1 --2- 172.21.15.22:0/1649396161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d41086d0 0x7fb3d410ab40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.482+0000 7fb3d3e00700 1 --2- 172.21.15.22:0/1649396161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3d4103470 0x7fb3d4105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.482+0000 7fb3d3e00700 1 --2- 172.21.15.22:0/1649396161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d4105da0 0x7fb3d4108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.482+0000 7fb3d3e00700 1 -- 172.21.15.22:0/1649396161 >> 172.21.15.22:0/1649396161 conn(0x7fb3d40fce60 msgr2=0x7fb3d40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:31.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.483+0000 7fb3d3e00700 1 -- 172.21.15.22:0/1649396161 shutdown_connections 2024-04-20T21:33:31.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.483+0000 7fb3d3e00700 1 -- 172.21.15.22:0/1649396161 wait complete. 2024-04-20T21:33:31.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.484+0000 7fb3d3e00700 1 Processor -- start 2024-04-20T21:33:31.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.484+0000 7fb3d3e00700 1 -- start start 2024-04-20T21:33:31.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.485+0000 7fb3d3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d4103470 0x7fb3d419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.485+0000 7fb3d3e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d4105da0 0x7fb3d419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.485+0000 7fb3d3e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3d41086d0 0x7fb3d41a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.485+0000 7fb3d3e00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d410e600 con 0x7fb3d4103470 2024-04-20T21:33:31.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.485+0000 7fb3d3e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d410e480 con 0x7fb3d4105da0 2024-04-20T21:33:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.485+0000 7fb3d3e00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d410e780 con 0x7fb3d41086d0 2024-04-20T21:33:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.485+0000 7fb3d2a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d4103470 0x7fb3d419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.485+0000 7fb3d2a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d4103470 0x7fb3d419ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39294/0 (socket says 172.21.15.22:39294) 2024-04-20T21:33:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.485+0000 7fb3d2a00700 1 -- 172.21.15.22:0/174335708 learned_addr learned my addr 172.21.15.22:0/174335708 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.485+0000 7fb3d3400700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3d41086d0 0x7fb3d41a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.485+0000 7fb3d2000700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d4105da0 0x7fb3d419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.486+0000 7fb3d2a00700 1 -- 172.21.15.22:0/174335708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3d41086d0 msgr2=0x7fb3d41a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.486+0000 7fb3d2a00700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3d41086d0 0x7fb3d41a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.486+0000 7fb3d2a00700 1 -- 172.21.15.22:0/174335708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d4105da0 msgr2=0x7fb3d419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.486+0000 7fb3d2a00700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d4105da0 0x7fb3d419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.486+0000 7fb3d2a00700 1 -- 172.21.15.22:0/174335708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d41a29f0 con 0x7fb3d4103470 2024-04-20T21:33:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.486+0000 7fb3d2a00700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d4103470 0x7fb3d419ada0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e9c0 tx=0x7fb3c400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.486+0000 7fb3cb400700 1 -- 172.21.15.22:0/174335708 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400cbd0 con 0x7fb3d4103470 2024-04-20T21:33:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.487+0000 7fb3d3e00700 1 -- 172.21.15.22:0/174335708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d41a2ce0 con 0x7fb3d4103470 2024-04-20T21:33:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.487+0000 7fb3cb400700 1 -- 172.21.15.22:0/174335708 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c400ee60 con 0x7fb3d4103470 2024-04-20T21:33:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.487+0000 7fb3cb400700 1 -- 172.21.15.22:0/174335708 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4005630 con 0x7fb3d4103470 2024-04-20T21:33:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.487+0000 7fb3d3e00700 1 -- 172.21.15.22:0/174335708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d41a3170 con 0x7fb3d4103470 2024-04-20T21:33:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.488+0000 7fb3d3e00700 1 -- 172.21.15.22:0/174335708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d4069740 con 0x7fb3d4103470 2024-04-20T21:33:31.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.490+0000 7fb3cb400700 1 -- 172.21.15.22:0/174335708 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb3c4004750 con 0x7fb3d4103470 2024-04-20T21:33:31.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.492+0000 7fb3cb400700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb3b80a3cc0 0x7fb3b80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.492+0000 7fb3cb400700 1 -- 172.21.15.22:0/174335708 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8298+0+0 (secure 0 0 0) 0x7fb3c40d0890 con 0x7fb3d4103470 2024-04-20T21:33:31.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.492+0000 7fb3d2000700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb3b80a3cc0 0x7fb3b80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.493+0000 7fb3d2000700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb3b80a3cc0 0x7fb3b80a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009c60 tx=0x7fb3c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:31.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.495+0000 7fb3cb400700 1 -- 172.21.15.22:0/174335708 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb3c4053ff0 con 0x7fb3d4103470 2024-04-20T21:33:31.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:31 smithi122 ceph-mon[22147]: pgmap v380: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:31.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:31 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-20T21:33:31.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:31 smithi033 ceph-mon[21626]: pgmap v380: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:31.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:31 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-20T21:33:31.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.730+0000 7fb3d3e00700 1 -- 172.21.15.22:0/174335708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fb3d402e060 con 0x7fb3d4103470 2024-04-20T21:33:31.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.731+0000 7fb3cb400700 1 -- 172.21.15.22:0/174335708 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5352 (secure 0 0 0) 0x7fb3c409d1d0 con 0x7fb3d4103470 2024-04-20T21:33:31.731 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 91 2024-04-20T21:33:31.731 INFO:teuthology.orchestra.run.smithi022.stdout:fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:33:31.731 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-04-20T21:22:14.364120+0000 2024-04-20T21:33:31.731 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-04-20T21:33:31.122950+0000 2024-04-20T21:33:31.731 INFO:teuthology.orchestra.run.smithi022.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-20T21:33:31.731 INFO:teuthology.orchestra.run.smithi022.stdout:crush_version 26 2024-04-20T21:33:31.731 INFO:teuthology.orchestra.run.smithi022.stdout:full_ratio 0.95 2024-04-20T21:33:31.731 INFO:teuthology.orchestra.run.smithi022.stdout:backfillfull_ratio 0.9 2024-04-20T21:33:31.731 INFO:teuthology.orchestra.run.smithi022.stdout:nearfull_ratio 0.85 2024-04-20T21:33:31.731 INFO:teuthology.orchestra.run.smithi022.stdout:require_min_compat_client luminous 2024-04-20T21:33:31.731 INFO:teuthology.orchestra.run.smithi022.stdout:min_compat_client luminous 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:require_osd_release squid 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:stretch_mode_enabled false 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:max_osd 12 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6802/2391966324,v1:172.21.15.22:6803/2391966324] [v2:172.21.15.22:6804/2391966324,v1:172.21.15.22:6805/2391966324] exists,up 67f06b5e-0fdc-4d2a-9355-fc7aa718d77e 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6800/2822429441,v1:172.21.15.33:6801/2822429441] [v2:172.21.15.33:6802/2822429441,v1:172.21.15.33:6803/2822429441] exists,up 728fe9c7-163a-4b08-b7fe-5cd2e76aaa99 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6800/4001148534,v1:172.21.15.122:6801/4001148534] [v2:172.21.15.122:6802/4001148534,v1:172.21.15.122:6803/4001148534] exists,up cdddd7e5-5a5e-4d68-9849-420d9f891c98 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 up in weight 1 up_from 26 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6810/1762015045,v1:172.21.15.22:6811/1762015045] [v2:172.21.15.22:6812/1762015045,v1:172.21.15.22:6813/1762015045] exists,up 9989d732-245e-417b-9a43-83c77374159d 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 up in weight 1 up_from 34 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6808/1935934998,v1:172.21.15.33:6809/1935934998] [v2:172.21.15.33:6810/1935934998,v1:172.21.15.33:6811/1935934998] exists,up 12ff6d17-2779-4462-bb8b-1b50e117c78c 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 up in weight 1 up_from 40 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6808/1738459470,v1:172.21.15.122:6809/1738459470] [v2:172.21.15.122:6810/1738459470,v1:172.21.15.122:6811/1738459470] exists,up 70a6256a-db56-4eab-bd2c-3f7a499cb166 2024-04-20T21:33:31.732 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 up in weight 1 up_from 48 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6818/1631721286,v1:172.21.15.22:6819/1631721286] [v2:172.21.15.22:6820/1631721286,v1:172.21.15.22:6821/1631721286] exists,up 283c2314-3dbb-4f48-964d-127ae967d38f 2024-04-20T21:33:31.733 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 up in weight 1 up_from 53 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6816/539722982,v1:172.21.15.33:6817/539722982] [v2:172.21.15.33:6818/539722982,v1:172.21.15.33:6819/539722982] exists,up 986f4852-193c-413d-8ffb-2faba217f4ce 2024-04-20T21:33:31.733 INFO:teuthology.orchestra.run.smithi022.stdout:osd.8 up in weight 1 up_from 62 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6816/2876209556,v1:172.21.15.122:6817/2876209556] [v2:172.21.15.122:6818/2876209556,v1:172.21.15.122:6819/2876209556] exists,up 4a6f0da8-369c-4576-9105-13a02e09b03d 2024-04-20T21:33:31.733 INFO:teuthology.orchestra.run.smithi022.stdout:osd.9 up in weight 1 up_from 70 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6826/1246646711,v1:172.21.15.22:6827/1246646711] [v2:172.21.15.22:6828/1246646711,v1:172.21.15.22:6829/1246646711] exists,up c35e9ebb-ebc5-4547-9490-09c2944b3e9c 2024-04-20T21:33:31.733 INFO:teuthology.orchestra.run.smithi022.stdout:osd.10 up in weight 1 up_from 75 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6824/1408592244,v1:172.21.15.33:6825/1408592244] [v2:172.21.15.33:6826/1408592244,v1:172.21.15.33:6827/1408592244] exists,up bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c 2024-04-20T21:33:31.733 INFO:teuthology.orchestra.run.smithi022.stdout:osd.11 up in weight 1 up_from 83 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6824/3622105204,v1:172.21.15.122:6825/3622105204] [v2:172.21.15.122:6826/3622105204,v1:172.21.15.122:6827/3622105204] exists,up e35bd98e-f771-40fb-b428-3c182f1c59c1 2024-04-20T21:33:31.733 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.6 [8,11] 2024-04-20T21:33:31.733 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.b [2,1] 2024-04-20T21:33:31.733 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.c [2,4] 2024-04-20T21:33:31.733 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.1c [8,11] 2024-04-20T21:33:31.733 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.0 [4,8] 2024-04-20T21:33:31.733 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.b [3,9] 2024-04-20T21:33:31.733 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.11 [4,6] 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.1a [5,7] 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.25 [5,9] 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.31 [4,8] 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.38 [3,9] 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/250835174 expires 2024-04-21T21:23:29.510166+0000 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6801/2098569388 expires 2024-04-21T21:23:29.510166+0000 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/2098569388 expires 2024-04-21T21:23:29.510166+0000 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/2009335712 expires 2024-04-21T21:23:29.510166+0000 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/1022821242 expires 2024-04-21T21:22:55.485151+0000 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/3042114972 expires 2024-04-21T21:22:55.485151+0000 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/3430200041 expires 2024-04-21T21:22:55.485151+0000 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/1146611493 expires 2024-04-21T21:22:55.485151+0000 2024-04-20T21:33:31.734 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6801/3430200041 expires 2024-04-21T21:22:55.485151+0000 2024-04-20T21:33:31.735 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/1810226571 expires 2024-04-21T21:23:29.510166+0000 2024-04-20T21:33:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.734+0000 7fb3d3e00700 1 -- 172.21.15.22:0/174335708 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb3b80a3cc0 msgr2=0x7fb3b80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.734+0000 7fb3d3e00700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb3b80a3cc0 0x7fb3b80a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009c60 tx=0x7fb3c0009260 comp rx=0 tx=0).stop 2024-04-20T21:33:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.734+0000 7fb3d3e00700 1 -- 172.21.15.22:0/174335708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d4103470 msgr2=0x7fb3d419ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.734+0000 7fb3d3e00700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d4103470 0x7fb3d419ada0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e9c0 tx=0x7fb3c400ecd0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.734+0000 7fb3d3e00700 1 -- 172.21.15.22:0/174335708 shutdown_connections 2024-04-20T21:33:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.734+0000 7fb3d3e00700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fb3b80a3cc0 0x7fb3b80a6180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.734+0000 7fb3d3e00700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3d41086d0 0x7fb3d41a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.734+0000 7fb3d3e00700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d4103470 0x7fb3d419ada0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.734+0000 7fb3d3e00700 1 --2- 172.21.15.22:0/174335708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d4105da0 0x7fb3d419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.734+0000 7fb3d3e00700 1 -- 172.21.15.22:0/174335708 >> 172.21.15.22:0/174335708 conn(0x7fb3d40fce60 msgr2=0x7fb3d4102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.734+0000 7fb3d3e00700 1 -- 172.21.15.22:0/174335708 shutdown_connections 2024-04-20T21:33:31.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:31.734+0000 7fb3d3e00700 1 -- 172.21.15.22:0/174335708 wait complete. 2024-04-20T21:33:32.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:32 smithi022 ceph-mon[16843]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-20T21:33:32.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:32 smithi022 ceph-mon[16843]: osdmap e91: 12 total, 12 up, 12 in 2024-04-20T21:33:32.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:32 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/174335708' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-20T21:33:32.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:32 smithi122 ceph-mon[22147]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-20T21:33:32.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:32 smithi122 ceph-mon[22147]: osdmap e91: 12 total, 12 up, 12 in 2024-04-20T21:33:32.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:32 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/174335708' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-20T21:33:32.529 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-20T21:33:32.541 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:32 smithi033 ceph-mon[21626]: from='mgr.14152 172.21.15.22:0/5194087' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-20T21:33:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:32 smithi033 ceph-mon[21626]: osdmap e91: 12 total, 12 up, 12 in 2024-04-20T21:33:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:32 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/174335708' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-20T21:33:32.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.767+0000 7f920f200640 1 -- 172.21.15.22:0/1066790588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208076920 msgr2=0x7f9208076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:32.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.767+0000 7f920f200640 1 --2- 172.21.15.22:0/1066790588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208076920 0x7f9208076d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009ca0 tx=0x7f91f802c0f0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.768+0000 7f920f200640 1 -- 172.21.15.22:0/1066790588 shutdown_connections 2024-04-20T21:33:32.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.768+0000 7f920f200640 1 --2- 172.21.15.22:0/1066790588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208078500 0x7f9208078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.768+0000 7f920f200640 1 --2- 172.21.15.22:0/1066790588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9208077b60 0x7f9208077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.768+0000 7f920f200640 1 --2- 172.21.15.22:0/1066790588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208076920 0x7f9208076d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.768+0000 7f920f200640 1 -- 172.21.15.22:0/1066790588 >> 172.21.15.22:0/1066790588 conn(0x7f92080ff750 msgr2=0x7f9208101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:32.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.768+0000 7f920f200640 1 -- 172.21.15.22:0/1066790588 shutdown_connections 2024-04-20T21:33:32.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.768+0000 7f920f200640 1 -- 172.21.15.22:0/1066790588 wait complete. 2024-04-20T21:33:32.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.769+0000 7f920f200640 1 Processor -- start 2024-04-20T21:33:32.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.769+0000 7f920f200640 1 -- start start 2024-04-20T21:33:32.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.769+0000 7f920f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208076920 0x7f920819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:32.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.769+0000 7f920f200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208077b60 0x7f920819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:32.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f920f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9208078500 0x7f92081a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:32.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f920f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9208114250 con 0x7f9208076920 2024-04-20T21:33:32.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f920f200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f92081140d0 con 0x7f9208077b60 2024-04-20T21:33:32.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f920f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f92081143d0 con 0x7f9208078500 2024-04-20T21:33:32.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f9207400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208077b60 0x7f920819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:32.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f9207e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208076920 0x7f920819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:32.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f9207400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208077b60 0x7f920819ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:47164/0 (socket says 172.21.15.22:47164) 2024-04-20T21:33:32.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f9207e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208076920 0x7f920819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55296/0 (socket says 172.21.15.22:55296) 2024-04-20T21:33:32.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f9207400640 1 -- 172.21.15.22:0/3448051268 learned_addr learned my addr 172.21.15.22:0/3448051268 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:32.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f920d200640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9208078500 0x7f92081a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:32.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f9207e00640 1 -- 172.21.15.22:0/3448051268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9208078500 msgr2=0x7f92081a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:32.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f9207e00640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9208078500 0x7f92081a5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f9207e00640 1 -- 172.21.15.22:0/3448051268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208077b60 msgr2=0x7f920819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:32.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f9207e00640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208077b60 0x7f920819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.770+0000 7f9207e00640 1 -- 172.21.15.22:0/3448051268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92081a6550 con 0x7f9208076920 2024-04-20T21:33:32.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.771+0000 7f9207e00640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208076920 0x7f920819e930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009bf0 tx=0x7f91f8032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:32.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.771+0000 7f9204c00640 1 -- 172.21.15.22:0/3448051268 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8041020 con 0x7f9208076920 2024-04-20T21:33:32.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.771+0000 7f9204c00640 1 -- 172.21.15.22:0/3448051268 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f8034cd0 con 0x7f9208076920 2024-04-20T21:33:32.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.771+0000 7f920f200640 1 -- 172.21.15.22:0/3448051268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92081a67e0 con 0x7f9208076920 2024-04-20T21:33:32.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.771+0000 7f9204c00640 1 -- 172.21.15.22:0/3448051268 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8039040 con 0x7f9208076920 2024-04-20T21:33:32.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.772+0000 7f920f200640 1 -- 172.21.15.22:0/3448051268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92081a6ca0 con 0x7f9208076920 2024-04-20T21:33:32.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.773+0000 7f920f200640 1 -- 172.21.15.22:0/3448051268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9208110d00 con 0x7f9208076920 2024-04-20T21:33:32.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.775+0000 7f9204c00640 1 -- 172.21.15.22:0/3448051268 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f91f8044020 con 0x7f9208076920 2024-04-20T21:33:32.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.777+0000 7f9204c00640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91ec0a3bb0 0x7f91ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:32.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.778+0000 7f9204c00640 1 -- 172.21.15.22:0/3448051268 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7f91f80f2560 con 0x7f9208076920 2024-04-20T21:33:32.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.778+0000 7f9207400640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91ec0a3bb0 0x7f91ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:32.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.779+0000 7f9207400640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91ec0a3bb0 0x7f91ec0a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f920819fe40 tx=0x7f91f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:32.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.780+0000 7f9204c00640 1 -- 172.21.15.22:0/3448051268 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f91f804b020 con 0x7f9208076920 2024-04-20T21:33:32.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.988+0000 7f920f200640 1 -- 172.21.15.22:0/3448051268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f920802d7a0 con 0x7f9208076920 2024-04-20T21:33:32.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.992+0000 7f9204c00640 1 -- 172.21.15.22:0/3448051268 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22768 (secure 0 0 0) 0x7f91f80bf050 con 0x7f9208076920 2024-04-20T21:33:32.992 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:32.992 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:33:32.135126+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.995+0000 7f920f200640 1 -- 172.21.15.22:0/3448051268 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91ec0a3bb0 msgr2=0x7f91ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.995+0000 7f920f200640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91ec0a3bb0 0x7f91ec0a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f920819fe40 tx=0x7f91f0005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.995+0000 7f920f200640 1 -- 172.21.15.22:0/3448051268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208076920 msgr2=0x7f920819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.995+0000 7f920f200640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208076920 0x7f920819e930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009bf0 tx=0x7f91f8032ec0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.995+0000 7f920f200640 1 -- 172.21.15.22:0/3448051268 shutdown_connections 2024-04-20T21:33:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.995+0000 7f920f200640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f91ec0a3bb0 0x7f91ec0a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.995+0000 7f920f200640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9208078500 0x7f92081a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.995+0000 7f920f200640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208077b60 0x7f920819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.995+0000 7f920f200640 1 --2- 172.21.15.22:0/3448051268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9208076920 0x7f920819e930 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.995+0000 7f920f200640 1 -- 172.21.15.22:0/3448051268 >> 172.21.15.22:0/3448051268 conn(0x7f92080ff750 msgr2=0x7f9208100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.995+0000 7f920f200640 1 -- 172.21.15.22:0/3448051268 shutdown_connections 2024-04-20T21:33:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:32.995+0000 7f920f200640 1 -- 172.21.15.22:0/3448051268 wait complete. 2024-04-20T21:33:33.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:26:15.688818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:31:25.506074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:31:26.126261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:33:33.017 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:33:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.228+0000 7ff8d1800640 1 -- 172.21.15.22:0/1309178670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0749c0 msgr2=0x7ff8cc074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.228+0000 7ff8d1800640 1 --2- 172.21.15.22:0/1309178670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0749c0 0x7ff8cc074e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009890 tx=0x7ff8b402bb70 comp rx=0 tx=0).stop 2024-04-20T21:33:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.228+0000 7ff8d1800640 1 -- 172.21.15.22:0/1309178670 shutdown_connections 2024-04-20T21:33:33.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.228+0000 7ff8d1800640 1 --2- 172.21.15.22:0/1309178670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8cc10e5d0 0x7ff8cc1109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.228+0000 7ff8d1800640 1 --2- 172.21.15.22:0/1309178670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0749c0 0x7ff8cc074e40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.228+0000 7ff8d1800640 1 --2- 172.21.15.22:0/1309178670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8cc075fa0 0x7ff8cc0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.228+0000 7ff8d1800640 1 -- 172.21.15.22:0/1309178670 >> 172.21.15.22:0/1309178670 conn(0x7ff8cc0fd550 msgr2=0x7ff8cc0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:33.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.228+0000 7ff8d1800640 1 -- 172.21.15.22:0/1309178670 shutdown_connections 2024-04-20T21:33:33.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.228+0000 7ff8d1800640 1 -- 172.21.15.22:0/1309178670 wait complete. 2024-04-20T21:33:33.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.229+0000 7ff8d1800640 1 Processor -- start 2024-04-20T21:33:33.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.229+0000 7ff8d1800640 1 -- start start 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8d1800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0749c0 0x7ff8cc19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8d1800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8cc075fa0 0x7ff8cc19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8d1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8cc10e5d0 0x7ff8cc1a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8d1800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc114220 con 0x7ff8cc10e5d0 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8d1800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc1140a0 con 0x7ff8cc0749c0 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8d1800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc1143a0 con 0x7ff8cc075fa0 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8ca000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8cc075fa0 0x7ff8cc19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8cb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8cc10e5d0 0x7ff8cc1a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8ca000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8cc075fa0 0x7ff8cc19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:35772/0 (socket says 172.21.15.22:35772) 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8ca000640 1 -- 172.21.15.22:0/477993676 learned_addr learned my addr 172.21.15.22:0/477993676 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8caa00640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0749c0 0x7ff8cc19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8ca000640 1 -- 172.21.15.22:0/477993676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0749c0 msgr2=0x7ff8cc19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8ca000640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0749c0 0x7ff8cc19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8ca000640 1 -- 172.21.15.22:0/477993676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8cc10e5d0 msgr2=0x7ff8cc1a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8ca000640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8cc10e5d0 0x7ff8cc1a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.230+0000 7ff8ca000640 1 -- 172.21.15.22:0/477993676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc1a6510 con 0x7ff8cc075fa0 2024-04-20T21:33:33.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.231+0000 7ff8ca000640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8cc075fa0 0x7ff8cc19ee30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40099b0 tx=0x7ff8b4034760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:33.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.231+0000 7ff8c3400640 1 -- 172.21.15.22:0/477993676 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4039040 con 0x7ff8cc075fa0 2024-04-20T21:33:33.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.231+0000 7ff8c3400640 1 -- 172.21.15.22:0/477993676 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8b4042070 con 0x7ff8cc075fa0 2024-04-20T21:33:33.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.231+0000 7ff8d1800640 1 -- 172.21.15.22:0/477993676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc1a67a0 con 0x7ff8cc075fa0 2024-04-20T21:33:33.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.231+0000 7ff8c3400640 1 -- 172.21.15.22:0/477993676 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4032c40 con 0x7ff8cc075fa0 2024-04-20T21:33:33.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.231+0000 7ff8d1800640 1 -- 172.21.15.22:0/477993676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc1a6ce0 con 0x7ff8cc075fa0 2024-04-20T21:33:33.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.232+0000 7ff8d1800640 1 -- 172.21.15.22:0/477993676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8cc104690 con 0x7ff8cc075fa0 2024-04-20T21:33:33.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.235+0000 7ff8c3400640 1 -- 172.21.15.22:0/477993676 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff8b402c510 con 0x7ff8cc075fa0 2024-04-20T21:33:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.236+0000 7ff8c3400640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff8b00a3c30 0x7ff8b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.236+0000 7ff8c3400640 1 -- 172.21.15.22:0/477993676 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7ff8b40f1d70 con 0x7ff8cc075fa0 2024-04-20T21:33:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.236+0000 7ff8caa00640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff8b00a3c30 0x7ff8b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.237+0000 7ff8caa00640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff8b00a3c30 0x7ff8b00a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc005fd0 tx=0x7ff8bc010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:33.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.238+0000 7ff8c3400640 1 -- 172.21.15.22:0/477993676 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff8b40be6b0 con 0x7ff8cc075fa0 2024-04-20T21:33:33.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:33 smithi022 ceph-mon[16843]: pgmap v382: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:33:33.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:33 smithi022 ceph-mon[16843]: osdmap e92: 12 total, 12 up, 12 in 2024-04-20T21:33:33.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:33 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3448051268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:33.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.448+0000 7ff8d1800640 1 -- 172.21.15.22:0/477993676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff8cc02d820 con 0x7ff8cc075fa0 2024-04-20T21:33:33.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.448+0000 7ff8c3400640 1 -- 172.21.15.22:0/477993676 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7ff8b40bdfb0 con 0x7ff8cc075fa0 2024-04-20T21:33:33.448 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-20T21:33:33.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.452+0000 7ff8d1800640 1 -- 172.21.15.22:0/477993676 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff8b00a3c30 msgr2=0x7ff8b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.452+0000 7ff8d1800640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff8b00a3c30 0x7ff8b00a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc005fd0 tx=0x7ff8bc010040 comp rx=0 tx=0).stop 2024-04-20T21:33:33.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.452+0000 7ff8d1800640 1 -- 172.21.15.22:0/477993676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8cc075fa0 msgr2=0x7ff8cc19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.452+0000 7ff8d1800640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8cc075fa0 0x7ff8cc19ee30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40099b0 tx=0x7ff8b4034760 comp rx=0 tx=0).stop 2024-04-20T21:33:33.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.452+0000 7ff8d1800640 1 -- 172.21.15.22:0/477993676 shutdown_connections 2024-04-20T21:33:33.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.452+0000 7ff8d1800640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff8b00a3c30 0x7ff8b00a60f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.452+0000 7ff8d1800640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8cc10e5d0 0x7ff8cc1a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.452+0000 7ff8d1800640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8cc075fa0 0x7ff8cc19ee30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.452+0000 7ff8d1800640 1 --2- 172.21.15.22:0/477993676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0749c0 0x7ff8cc19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.452+0000 7ff8d1800640 1 -- 172.21.15.22:0/477993676 >> 172.21.15.22:0/477993676 conn(0x7ff8cc0fd550 msgr2=0x7ff8cc101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:33.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.452+0000 7ff8d1800640 1 -- 172.21.15.22:0/477993676 shutdown_connections 2024-04-20T21:33:33.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.452+0000 7ff8d1800640 1 -- 172.21.15.22:0/477993676 wait complete. 2024-04-20T21:33:33.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:33:33.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:33 smithi122 ceph-mon[22147]: pgmap v382: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:33:33.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:33 smithi122 ceph-mon[22147]: osdmap e92: 12 total, 12 up, 12 in 2024-04-20T21:33:33.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:33 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3448051268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:33.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:33 smithi033 ceph-mon[21626]: pgmap v382: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:33:33.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:33 smithi033 ceph-mon[21626]: osdmap e92: 12 total, 12 up, 12 in 2024-04-20T21:33:33.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:33 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3448051268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:33.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.675+0000 7f53e3200640 1 -- 172.21.15.22:0/626255508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4102fe0 msgr2=0x7f53e4103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.675+0000 7f53e3200640 1 --2- 172.21.15.22:0/626255508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4102fe0 0x7f53e4103460 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009900 tx=0x7f53cc02bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:33.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.675+0000 7f53e3200640 1 -- 172.21.15.22:0/626255508 shutdown_connections 2024-04-20T21:33:33.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.675+0000 7f53e3200640 1 --2- 172.21.15.22:0/626255508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41039a0 0x7f53e410a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.675+0000 7f53e3200640 1 --2- 172.21.15.22:0/626255508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4102fe0 0x7f53e4103460 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.675+0000 7f53e3200640 1 --2- 172.21.15.22:0/626255508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e4101e90 0x7f53e41022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.675+0000 7f53e3200640 1 -- 172.21.15.22:0/626255508 >> 172.21.15.22:0/626255508 conn(0x7f53e40fd400 msgr2=0x7f53e40ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:33.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.675+0000 7f53e3200640 1 -- 172.21.15.22:0/626255508 shutdown_connections 2024-04-20T21:33:33.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.675+0000 7f53e3200640 1 -- 172.21.15.22:0/626255508 wait complete. 2024-04-20T21:33:33.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.676+0000 7f53e3200640 1 Processor -- start 2024-04-20T21:33:33.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.677+0000 7f53e3200640 1 -- start start 2024-04-20T21:33:33.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.677+0000 7f53e3200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e4101e90 0x7f53e419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.677+0000 7f53e3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4102fe0 0x7f53e419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.677+0000 7f53e3200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41039a0 0x7f53e41a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.677+0000 7f53e3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53e410db70 con 0x7f53e4102fe0 2024-04-20T21:33:33.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.677+0000 7f53e3200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f53e410d9f0 con 0x7f53e4101e90 2024-04-20T21:33:33.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.677+0000 7f53e3200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53e410dcf0 con 0x7f53e41039a0 2024-04-20T21:33:33.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.677+0000 7f53dbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4102fe0 0x7f53e419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.677+0000 7f53dbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4102fe0 0x7f53e419a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55334/0 (socket says 172.21.15.22:55334) 2024-04-20T21:33:33.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.677+0000 7f53e1e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e4101e90 0x7f53e419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.677+0000 7f53dbe00640 1 -- 172.21.15.22:0/784094130 learned_addr learned my addr 172.21.15.22:0/784094130 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:33.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.678+0000 7f53e2800640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41039a0 0x7f53e41a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.678+0000 7f53dbe00640 1 -- 172.21.15.22:0/784094130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41039a0 msgr2=0x7f53e41a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.678+0000 7f53dbe00640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41039a0 0x7f53e41a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.678+0000 7f53dbe00640 1 -- 172.21.15.22:0/784094130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e4101e90 msgr2=0x7f53e419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.678+0000 7f53dbe00640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e4101e90 0x7f53e419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.678+0000 7f53dbe00640 1 -- 172.21.15.22:0/784094130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e41a1f90 con 0x7f53e4102fe0 2024-04-20T21:33:33.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.678+0000 7f53e2800640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41039a0 0x7f53e41a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:33.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.678+0000 7f53dbe00640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4102fe0 0x7f53e419a8b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f53cc000c00 tx=0x7f53cc02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:33.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.679+0000 7f53daa00640 1 -- 172.21.15.22:0/784094130 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc043020 con 0x7f53e4102fe0 2024-04-20T21:33:33.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.679+0000 7f53daa00640 1 -- 172.21.15.22:0/784094130 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53cc0054e0 con 0x7f53e4102fe0 2024-04-20T21:33:33.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.679+0000 7f53e3200640 1 -- 172.21.15.22:0/784094130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e41a2220 con 0x7f53e4102fe0 2024-04-20T21:33:33.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.679+0000 7f53daa00640 1 -- 172.21.15.22:0/784094130 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc003ce0 con 0x7f53e4102fe0 2024-04-20T21:33:33.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.679+0000 7f53e3200640 1 -- 172.21.15.22:0/784094130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e41a2740 con 0x7f53e4102fe0 2024-04-20T21:33:33.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.680+0000 7f53e3200640 1 -- 172.21.15.22:0/784094130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e41a2940 con 0x7f53e4102fe0 2024-04-20T21:33:33.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.682+0000 7f53daa00640 1 -- 172.21.15.22:0/784094130 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f53cc046020 con 0x7f53e4102fe0 2024-04-20T21:33:33.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.685+0000 7f53daa00640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f53c80a3bb0 0x7f53c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:33.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.685+0000 7f53daa00640 1 -- 172.21.15.22:0/784094130 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7f53cc0f2a20 con 0x7f53e4102fe0 2024-04-20T21:33:33.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.685+0000 7f53e1e00640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f53c80a3bb0 0x7f53c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:33.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.686+0000 7f53e1e00640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f53c80a3bb0 0x7f53c80a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009bd0 tx=0x7f53d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:33.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.686+0000 7f53daa00640 1 -- 172.21.15.22:0/784094130 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f53cc04d020 con 0x7f53e4102fe0 2024-04-20T21:33:33.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.898+0000 7f53e3200640 1 -- 172.21.15.22:0/784094130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f53e41a2b10 con 0x7f53e4102fe0 2024-04-20T21:33:33.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.899+0000 7f53daa00640 1 -- 172.21.15.22:0/784094130 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f53cc0dde20 con 0x7f53e4102fe0 2024-04-20T21:33:33.898 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-20T21:33:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.901+0000 7f53e3200640 1 -- 172.21.15.22:0/784094130 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f53c80a3bb0 msgr2=0x7f53c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.901+0000 7f53e3200640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f53c80a3bb0 0x7f53c80a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009bd0 tx=0x7f53d4005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.901+0000 7f53e3200640 1 -- 172.21.15.22:0/784094130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4102fe0 msgr2=0x7f53e419a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.901+0000 7f53e3200640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4102fe0 0x7f53e419a8b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f53cc000c00 tx=0x7f53cc02c6c0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.902+0000 7f53e3200640 1 -- 172.21.15.22:0/784094130 shutdown_connections 2024-04-20T21:33:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.902+0000 7f53e3200640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f53c80a3bb0 0x7f53c80a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.902+0000 7f53e3200640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41039a0 0x7f53e41a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.902+0000 7f53e3200640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4102fe0 0x7f53e419a8b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.902+0000 7f53e3200640 1 --2- 172.21.15.22:0/784094130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e4101e90 0x7f53e419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.902+0000 7f53e3200640 1 -- 172.21.15.22:0/784094130 >> 172.21.15.22:0/784094130 conn(0x7f53e40fd400 msgr2=0x7f53e4074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.902+0000 7f53e3200640 1 -- 172.21.15.22:0/784094130 shutdown_connections 2024-04-20T21:33:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:33.902+0000 7f53e3200640 1 -- 172.21.15.22:0/784094130 wait complete. 2024-04-20T21:33:33.920 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:33:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.125+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3703284860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7f4107a60 msgr2=0x7fe7f4109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.125+0000 7fe7f3e00640 1 --2- 172.21.15.22:0/3703284860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7f4107a60 0x7fe7f4109eb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009ca0 tx=0x7fe7ec02c0f0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.125+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3703284860 shutdown_connections 2024-04-20T21:33:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.125+0000 7fe7f3e00640 1 --2- 172.21.15.22:0/3703284860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7f4107a60 0x7fe7f4109eb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.125+0000 7fe7f3e00640 1 --2- 172.21.15.22:0/3703284860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f4105130 0x7fe7f4107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.125+0000 7fe7f3e00640 1 --2- 172.21.15.22:0/3703284860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7f4102800 0x7fe7f4104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.125+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3703284860 >> 172.21.15.22:0/3703284860 conn(0x7fe7f40fc230 msgr2=0x7fe7f40fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.125+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3703284860 shutdown_connections 2024-04-20T21:33:34.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.125+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3703284860 wait complete. 2024-04-20T21:33:34.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.126+0000 7fe7f3e00640 1 Processor -- start 2024-04-20T21:33:34.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.126+0000 7fe7f3e00640 1 -- start start 2024-04-20T21:33:34.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.127+0000 7fe7f3e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7f4102800 0x7fe7f419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:34.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.127+0000 7fe7f3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7f4105130 0x7fe7f419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.127+0000 7fe7f3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f4107a60 0x7fe7f41a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.127+0000 7fe7f3e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f410da00 con 0x7fe7f4105130 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.127+0000 7fe7f3e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f410d880 con 0x7fe7f4102800 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.127+0000 7fe7f3e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f410db80 con 0x7fe7f4107a60 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.127+0000 7fe7f2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7f4105130 0x7fe7f419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.127+0000 7fe7f2a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7f4102800 0x7fe7f419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.127+0000 7fe7f2a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7f4102800 0x7fe7f419a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:47240/0 (socket says 172.21.15.22:47240) 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.127+0000 7fe7f2a00640 1 -- 172.21.15.22:0/3239758204 learned_addr learned my addr 172.21.15.22:0/3239758204 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.127+0000 7fe7f3400640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f4107a60 0x7fe7f41a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.127+0000 7fe7f2a00640 1 -- 172.21.15.22:0/3239758204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f4107a60 msgr2=0x7fe7f41a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.128+0000 7fe7f2a00640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f4107a60 0x7fe7f41a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.128+0000 7fe7f2a00640 1 -- 172.21.15.22:0/3239758204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7f4105130 msgr2=0x7fe7f419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.128+0000 7fe7f2a00640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7f4105130 0x7fe7f419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.128+0000 7fe7f2a00640 1 -- 172.21.15.22:0/3239758204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f41a1e90 con 0x7fe7f4102800 2024-04-20T21:33:34.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.128+0000 7fe7f2000640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7f4105130 0x7fe7f419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:34.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.128+0000 7fe7f3400640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f4107a60 0x7fe7f41a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:34.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.128+0000 7fe7f2a00640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7f4102800 0x7fe7f419a270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400ea60 tx=0x7fe7e400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:34.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.128+0000 7fe7eb400640 1 -- 172.21.15.22:0/3239758204 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e400cde0 con 0x7fe7f4102800 2024-04-20T21:33:34.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.128+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3239758204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f41a2180 con 0x7fe7f4102800 2024-04-20T21:33:34.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.128+0000 7fe7eb400640 1 -- 172.21.15.22:0/3239758204 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e40035f0 con 0x7fe7f4102800 2024-04-20T21:33:34.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.128+0000 7fe7eb400640 1 -- 172.21.15.22:0/3239758204 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4010670 con 0x7fe7f4102800 2024-04-20T21:33:34.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.128+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3239758204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f41a2720 con 0x7fe7f4102800 2024-04-20T21:33:34.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.129+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3239758204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f4109550 con 0x7fe7f4102800 2024-04-20T21:33:34.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.133+0000 7fe7eb400640 1 -- 172.21.15.22:0/3239758204 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe7e40037a0 con 0x7fe7f4102800 2024-04-20T21:33:34.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.134+0000 7fe7eb400640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe7d80a3c30 0x7fe7d80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:34.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.134+0000 7fe7eb400640 1 -- 172.21.15.22:0/3239758204 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7fe7e40d07b0 con 0x7fe7f4102800 2024-04-20T21:33:34.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.135+0000 7fe7f2000640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe7d80a3c30 0x7fe7d80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:34.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.135+0000 7fe7eb400640 1 -- 172.21.15.22:0/3239758204 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe7e409d180 con 0x7fe7f4102800 2024-04-20T21:33:34.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.135+0000 7fe7f2000640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe7d80a3c30 0x7fe7d80a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe7f419b780 tx=0x7fe7e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:34.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.337+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3239758204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe7f402d5d0 con 0x7fe7f4102800 2024-04-20T21:33:34.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.338+0000 7fe7eb400640 1 -- 172.21.15.22:0/3239758204 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fe7e409ca80 con 0x7fe7f4102800 2024-04-20T21:33:34.337 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-20T21:33:34.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.340+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3239758204 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe7d80a3c30 msgr2=0x7fe7d80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.340+0000 7fe7f3e00640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe7d80a3c30 0x7fe7d80a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe7f419b780 tx=0x7fe7e0005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.340+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3239758204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7f4102800 msgr2=0x7fe7f419a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.340+0000 7fe7f3e00640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7f4102800 0x7fe7f419a270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400ea60 tx=0x7fe7e400ef20 comp rx=0 tx=0).stop 2024-04-20T21:33:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.341+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3239758204 shutdown_connections 2024-04-20T21:33:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.341+0000 7fe7f3e00640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fe7d80a3c30 0x7fe7d80a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.341+0000 7fe7f3e00640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f4107a60 0x7fe7f41a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.341+0000 7fe7f3e00640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7f4105130 0x7fe7f419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.341+0000 7fe7f3e00640 1 --2- 172.21.15.22:0/3239758204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7f4102800 0x7fe7f419a270 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.341+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3239758204 >> 172.21.15.22:0/3239758204 conn(0x7fe7f40fc230 msgr2=0x7fe7f4101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.341+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3239758204 shutdown_connections 2024-04-20T21:33:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.341+0000 7fe7f3e00640 1 -- 172.21.15.22:0/3239758204 wait complete. 2024-04-20T21:33:34.355 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:34 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/477993676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:34.355 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:34 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/784094130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:34.359 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:34.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:34 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/477993676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:34.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:34 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/784094130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.544+0000 7f0955200640 1 -- 172.21.15.22:0/3480989678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0950101ed0 msgr2=0x7f0950102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:34.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.544+0000 7f0955200640 1 --2- 172.21.15.22:0/3480989678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0950101ed0 0x7f0950102350 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f093c009900 tx=0x7f093c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:34.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.544+0000 7f0955200640 1 -- 172.21.15.22:0/3480989678 shutdown_connections 2024-04-20T21:33:34.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.544+0000 7f0955200640 1 --2- 172.21.15.22:0/3480989678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0950102890 0x7f0950109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.544+0000 7f0955200640 1 --2- 172.21.15.22:0/3480989678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0950101ed0 0x7f0950102350 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.544+0000 7f0955200640 1 --2- 172.21.15.22:0/3480989678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0950100ce0 0x7f0950101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.544+0000 7f0955200640 1 -- 172.21.15.22:0/3480989678 >> 172.21.15.22:0/3480989678 conn(0x7f09500fc230 msgr2=0x7f09500fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:34.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.544+0000 7f0955200640 1 -- 172.21.15.22:0/3480989678 shutdown_connections 2024-04-20T21:33:34.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.544+0000 7f0955200640 1 -- 172.21.15.22:0/3480989678 wait complete. 2024-04-20T21:33:34.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.545+0000 7f0955200640 1 Processor -- start 2024-04-20T21:33:34.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.545+0000 7f0955200640 1 -- start start 2024-04-20T21:33:34.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f0955200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0950100ce0 0x7f095019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:34.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f0955200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0950101ed0 0x7f095019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:34.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f0955200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0950102890 0x7f09501a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f0955200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f095010c990 con 0x7f0950100ce0 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f0955200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f095010c810 con 0x7f0950102890 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f0955200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f095010cb10 con 0x7f0950101ed0 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f094f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0950100ce0 0x7f095019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f094f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0950100ce0 0x7f095019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55372/0 (socket says 172.21.15.22:55372) 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f094f400640 1 -- 172.21.15.22:0/2438491522 learned_addr learned my addr 172.21.15.22:0/2438491522 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f094ea00640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0950101ed0 0x7f095019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f094fe00640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0950102890 0x7f09501a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f094f400640 1 -- 172.21.15.22:0/2438491522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0950101ed0 msgr2=0x7f095019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f094f400640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0950101ed0 0x7f095019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f094f400640 1 -- 172.21.15.22:0/2438491522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0950102890 msgr2=0x7f09501a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f094f400640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0950102890 0x7f09501a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f094f400640 1 -- 172.21.15.22:0/2438491522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09501a1f80 con 0x7f0950100ce0 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.546+0000 7f094fe00640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0950102890 0x7f09501a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.547+0000 7f094f400640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0950100ce0 0x7f095019a360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f094000e9c0 tx=0x7f094000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:34.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.547+0000 7f0947e00640 1 -- 172.21.15.22:0/2438491522 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094000cde0 con 0x7f0950100ce0 2024-04-20T21:33:34.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.547+0000 7f0955200640 1 -- 172.21.15.22:0/2438491522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09501a2270 con 0x7f0950100ce0 2024-04-20T21:33:34.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.547+0000 7f0947e00640 1 -- 172.21.15.22:0/2438491522 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0940010910 con 0x7f0950100ce0 2024-04-20T21:33:34.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.547+0000 7f0947e00640 1 -- 172.21.15.22:0/2438491522 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0940018a10 con 0x7f0950100ce0 2024-04-20T21:33:34.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.547+0000 7f0955200640 1 -- 172.21.15.22:0/2438491522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09501a27e0 con 0x7f0950100ce0 2024-04-20T21:33:34.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.548+0000 7f0955200640 1 -- 172.21.15.22:0/2438491522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09501a2a40 con 0x7f0950100ce0 2024-04-20T21:33:34.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.550+0000 7f0947e00640 1 -- 172.21.15.22:0/2438491522 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f094001f020 con 0x7f0950100ce0 2024-04-20T21:33:34.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.551+0000 7f0947e00640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f09340a3bb0 0x7f09340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:34.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.552+0000 7f0947e00640 1 -- 172.21.15.22:0/2438491522 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7f09400d0740 con 0x7f0950100ce0 2024-04-20T21:33:34.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.552+0000 7f094ea00640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f09340a3bb0 0x7f09340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:34.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.553+0000 7f094ea00640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f09340a3bb0 0x7f09340a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f093c02c240 tx=0x7f093c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:34 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/477993676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:34 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/784094130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:34.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.556+0000 7f0947e00640 1 -- 172.21.15.22:0/2438491522 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f094009d080 con 0x7f0950100ce0 2024-04-20T21:33:34.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.805+0000 7f0955200640 1 -- 172.21.15.22:0/2438491522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09501a2dd0 con 0x7f0950100ce0 2024-04-20T21:33:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.806+0000 7f0947e00640 1 -- 172.21.15.22:0/2438491522 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f094009c980 con 0x7f0950100ce0 2024-04-20T21:33:34.806 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:34.806 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:33:34.806 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:33:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.809+0000 7f0955200640 1 -- 172.21.15.22:0/2438491522 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f09340a3bb0 msgr2=0x7f09340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.809+0000 7f0955200640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f09340a3bb0 0x7f09340a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f093c02c240 tx=0x7f093c02df90 comp rx=0 tx=0).stop 2024-04-20T21:33:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.809+0000 7f0955200640 1 -- 172.21.15.22:0/2438491522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0950100ce0 msgr2=0x7f095019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.809+0000 7f0955200640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0950100ce0 0x7f095019a360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f094000e9c0 tx=0x7f094000ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.809+0000 7f0955200640 1 -- 172.21.15.22:0/2438491522 shutdown_connections 2024-04-20T21:33:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.809+0000 7f0955200640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f09340a3bb0 0x7f09340a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.809+0000 7f0955200640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0950102890 0x7f09501a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.809+0000 7f0955200640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0950101ed0 0x7f095019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.809+0000 7f0955200640 1 --2- 172.21.15.22:0/2438491522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0950100ce0 0x7f095019a360 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.809+0000 7f0955200640 1 -- 172.21.15.22:0/2438491522 >> 172.21.15.22:0/2438491522 conn(0x7f09500fc230 msgr2=0x7f09500fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.809+0000 7f0955200640 1 -- 172.21.15.22:0/2438491522 shutdown_connections 2024-04-20T21:33:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:34.809+0000 7f0955200640 1 -- 172.21.15.22:0/2438491522 wait complete. 2024-04-20T21:33:34.826 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:35.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.020+0000 7fd09da00640 1 -- 172.21.15.22:0/92956864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd098105b90 msgr2=0x7fd09810c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.020+0000 7fd09da00640 1 --2- 172.21.15.22:0/92956864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd098105b90 0x7fd09810c420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd090009ca0 tx=0x7fd09002c100 comp rx=0 tx=0).stop 2024-04-20T21:33:35.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.021+0000 7fd09da00640 1 -- 172.21.15.22:0/92956864 shutdown_connections 2024-04-20T21:33:35.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.021+0000 7fd09da00640 1 --2- 172.21.15.22:0/92956864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd098105b90 0x7fd09810c420 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.021+0000 7fd09da00640 1 --2- 172.21.15.22:0/92956864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0981051d0 0x7fd098105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.021+0000 7fd09da00640 1 --2- 172.21.15.22:0/92956864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd098103fe0 0x7fd098104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.021+0000 7fd09da00640 1 -- 172.21.15.22:0/92956864 >> 172.21.15.22:0/92956864 conn(0x7fd0980ff570 msgr2=0x7fd098101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:35.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.021+0000 7fd09da00640 1 -- 172.21.15.22:0/92956864 shutdown_connections 2024-04-20T21:33:35.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.021+0000 7fd09da00640 1 -- 172.21.15.22:0/92956864 wait complete. 2024-04-20T21:33:35.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd09da00640 1 Processor -- start 2024-04-20T21:33:35.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd09da00640 1 -- start start 2024-04-20T21:33:35.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd09da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd098103fe0 0x7fd09819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:35.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd09da00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0981051d0 0x7fd09819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:35.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd09da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd098105b90 0x7fd0981a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:35.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd09da00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd09810fc70 con 0x7fd098103fe0 2024-04-20T21:33:35.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd09da00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd09810faf0 con 0x7fd0981051d0 2024-04-20T21:33:35.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd09da00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd09810fdf0 con 0x7fd098105b90 2024-04-20T21:33:35.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd097e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd098103fe0 0x7fd09819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:35.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd097e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd098103fe0 0x7fd09819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55380/0 (socket says 172.21.15.22:55380) 2024-04-20T21:33:35.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd097e00640 1 -- 172.21.15.22:0/3520429985 learned_addr learned my addr 172.21.15.22:0/3520429985 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:35.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd097400640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0981051d0 0x7fd09819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:35.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.022+0000 7fd09d000640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd098105b90 0x7fd0981a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:35.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.023+0000 7fd097e00640 1 -- 172.21.15.22:0/3520429985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd098105b90 msgr2=0x7fd0981a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.023+0000 7fd097e00640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd098105b90 0x7fd0981a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.023+0000 7fd097e00640 1 -- 172.21.15.22:0/3520429985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0981051d0 msgr2=0x7fd09819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.023+0000 7fd097e00640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0981051d0 0x7fd09819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.023+0000 7fd097e00640 1 -- 172.21.15.22:0/3520429985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0981a1eb0 con 0x7fd098103fe0 2024-04-20T21:33:35.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.023+0000 7fd097e00640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd098103fe0 0x7fd09819a290 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd08800d950 tx=0x7fd08800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:35.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.023+0000 7fd094c00640 1 -- 172.21.15.22:0/3520429985 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088017020 con 0x7fd098103fe0 2024-04-20T21:33:35.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.023+0000 7fd09da00640 1 -- 172.21.15.22:0/3520429985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0981a21a0 con 0x7fd098103fe0 2024-04-20T21:33:35.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.023+0000 7fd094c00640 1 -- 172.21.15.22:0/3520429985 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd088009c30 con 0x7fd098103fe0 2024-04-20T21:33:35.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.023+0000 7fd094c00640 1 -- 172.21.15.22:0/3520429985 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08800f740 con 0x7fd098103fe0 2024-04-20T21:33:35.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.023+0000 7fd09da00640 1 -- 172.21.15.22:0/3520429985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0981a2740 con 0x7fd098103fe0 2024-04-20T21:33:35.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.025+0000 7fd09da00640 1 -- 172.21.15.22:0/3520429985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd09810b2a0 con 0x7fd098103fe0 2024-04-20T21:33:35.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.027+0000 7fd094c00640 1 -- 172.21.15.22:0/3520429985 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd08801a020 con 0x7fd098103fe0 2024-04-20T21:33:35.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.028+0000 7fd094c00640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd07c0a3bb0 0x7fd07c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:35.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.029+0000 7fd094c00640 1 -- 172.21.15.22:0/3520429985 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7fd0880d0160 con 0x7fd098103fe0 2024-04-20T21:33:35.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.029+0000 7fd097400640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd07c0a3bb0 0x7fd07c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:35.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.029+0000 7fd097400640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd07c0a3bb0 0x7fd07c0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd09819b7a0 tx=0x7fd084005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:35.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.031+0000 7fd094c00640 1 -- 172.21.15.22:0/3520429985 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd088021070 con 0x7fd098103fe0 2024-04-20T21:33:35.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.241+0000 7fd09da00640 1 -- 172.21.15.22:0/3520429985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd098062b60 con 0x7fd098103fe0 2024-04-20T21:33:35.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.245+0000 7fd094c00640 1 -- 172.21.15.22:0/3520429985 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22768 (secure 0 0 0) 0x7fd088009de0 con 0x7fd098103fe0 2024-04-20T21:33:35.244 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:35.245 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:33:32.135126+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:35.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.248+0000 7fd09da00640 1 -- 172.21.15.22:0/3520429985 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd07c0a3bb0 msgr2=0x7fd07c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.248+0000 7fd09da00640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd07c0a3bb0 0x7fd07c0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd09819b7a0 tx=0x7fd084005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:35.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.248+0000 7fd09da00640 1 -- 172.21.15.22:0/3520429985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd098103fe0 msgr2=0x7fd09819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.248+0000 7fd09da00640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd098103fe0 0x7fd09819a290 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd08800d950 tx=0x7fd08800de10 comp rx=0 tx=0).stop 2024-04-20T21:33:35.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.248+0000 7fd09da00640 1 -- 172.21.15.22:0/3520429985 shutdown_connections 2024-04-20T21:33:35.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.248+0000 7fd09da00640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fd07c0a3bb0 0x7fd07c0a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.248+0000 7fd09da00640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd098105b90 0x7fd0981a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.248+0000 7fd09da00640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0981051d0 0x7fd09819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.248+0000 7fd09da00640 1 --2- 172.21.15.22:0/3520429985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd098103fe0 0x7fd09819a290 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.248+0000 7fd09da00640 1 -- 172.21.15.22:0/3520429985 >> 172.21.15.22:0/3520429985 conn(0x7fd0980ff570 msgr2=0x7fd098108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:35.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.249+0000 7fd09da00640 1 -- 172.21.15.22:0/3520429985 shutdown_connections 2024-04-20T21:33:35.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.249+0000 7fd09da00640 1 -- 172.21.15.22:0/3520429985 wait complete. 2024-04-20T21:33:35.266 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:26:15.688818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:31:25.506074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:31:26.126261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:33:35.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:33:35.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:35 smithi022 ceph-mon[16843]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:35.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:35 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3239758204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:35.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:35 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2438491522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:35.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.478+0000 7f9af3000640 1 -- 172.21.15.22:0/1950784621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9aec1095b0 msgr2=0x7f9aec109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.478+0000 7f9af3000640 1 --2- 172.21.15.22:0/1950784621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9aec1095b0 0x7f9aec109a30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009900 tx=0x7f9adc02bc50 comp rx=0 tx=0).stop 2024-04-20T21:33:35.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.478+0000 7f9af3000640 1 -- 172.21.15.22:0/1950784621 shutdown_connections 2024-04-20T21:33:35.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.478+0000 7f9af3000640 1 --2- 172.21.15.22:0/1950784621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aec109f70 0x7f9aec110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.478+0000 7f9af3000640 1 --2- 172.21.15.22:0/1950784621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9aec1095b0 0x7f9aec109a30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.478+0000 7f9af3000640 1 --2- 172.21.15.22:0/1950784621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aec1083c0 0x7f9aec1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.478+0000 7f9af3000640 1 -- 172.21.15.22:0/1950784621 >> 172.21.15.22:0/1950784621 conn(0x7f9aec077360 msgr2=0x7f9aec0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:35.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.478+0000 7f9af3000640 1 -- 172.21.15.22:0/1950784621 shutdown_connections 2024-04-20T21:33:35.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.478+0000 7f9af3000640 1 -- 172.21.15.22:0/1950784621 wait complete. 2024-04-20T21:33:35.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.479+0000 7f9af3000640 1 Processor -- start 2024-04-20T21:33:35.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.479+0000 7f9af3000640 1 -- start start 2024-04-20T21:33:35.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.479+0000 7f9af3000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9aec1083c0 0x7f9aec19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:35.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.479+0000 7f9af3000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aec1095b0 0x7f9aec19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:35.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.479+0000 7f9af3000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aec109f70 0x7f9aec1a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:35.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.479+0000 7f9af3000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec1140a0 con 0x7f9aec1083c0 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.479+0000 7f9af3000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec113f20 con 0x7f9aec109f70 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.479+0000 7f9af3000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec114220 con 0x7f9aec1095b0 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.479+0000 7f9af1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9aec1083c0 0x7f9aec19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.480+0000 7f9af1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9aec1083c0 0x7f9aec19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55410/0 (socket says 172.21.15.22:55410) 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.480+0000 7f9af1c00640 1 -- 172.21.15.22:0/1260454559 learned_addr learned my addr 172.21.15.22:0/1260454559 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.480+0000 7f9af1200640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aec1095b0 0x7f9aec19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.480+0000 7f9af2600640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aec109f70 0x7f9aec1a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.480+0000 7f9af1c00640 1 -- 172.21.15.22:0/1260454559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aec1095b0 msgr2=0x7f9aec19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.480+0000 7f9af1c00640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aec1095b0 0x7f9aec19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.480+0000 7f9af1c00640 1 -- 172.21.15.22:0/1260454559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aec109f70 msgr2=0x7f9aec1a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.480+0000 7f9af1c00640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aec109f70 0x7f9aec1a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.480+0000 7f9af1c00640 1 -- 172.21.15.22:0/1260454559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec1a62b0 con 0x7f9aec1083c0 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.480+0000 7f9af1200640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aec1095b0 0x7f9aec19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:35.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.480+0000 7f9af2600640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aec109f70 0x7f9aec1a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:35.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.480+0000 7f9af1c00640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9aec1083c0 0x7f9aec19e690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000e9c0 tx=0x7f9ae000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:35.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.481+0000 7f9aea000640 1 -- 172.21.15.22:0/1260454559 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000cd80 con 0x7f9aec1083c0 2024-04-20T21:33:35.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.481+0000 7f9aea000640 1 -- 172.21.15.22:0/1260454559 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae00035f0 con 0x7f9aec1083c0 2024-04-20T21:33:35.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.481+0000 7f9af3000640 1 -- 172.21.15.22:0/1260454559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aec1a65a0 con 0x7f9aec1083c0 2024-04-20T21:33:35.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.481+0000 7f9aea000640 1 -- 172.21.15.22:0/1260454559 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0010690 con 0x7f9aec1083c0 2024-04-20T21:33:35.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.481+0000 7f9af3000640 1 -- 172.21.15.22:0/1260454559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aec1a6ac0 con 0x7f9aec1083c0 2024-04-20T21:33:35.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.482+0000 7f9af3000640 1 -- 172.21.15.22:0/1260454559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aec10f680 con 0x7f9aec1083c0 2024-04-20T21:33:35.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.484+0000 7f9aea000640 1 -- 172.21.15.22:0/1260454559 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9ae00037a0 con 0x7f9aec1083c0 2024-04-20T21:33:35.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.486+0000 7f9aea000640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f9ad40a3bb0 0x7f9ad40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:35.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.486+0000 7f9aea000640 1 -- 172.21.15.22:0/1260454559 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7f9ae00d0840 con 0x7f9aec1083c0 2024-04-20T21:33:35.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.486+0000 7f9af1200640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f9ad40a3bb0 0x7f9ad40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:35.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.487+0000 7f9af1200640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f9ad40a3bb0 0x7f9ad40a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9adc02c260 tx=0x7f9adc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:35.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.491+0000 7f9aea000640 1 -- 172.21.15.22:0/1260454559 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9ae009d1e0 con 0x7f9aec1083c0 2024-04-20T21:33:35.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:35 smithi122 ceph-mon[22147]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:35 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3239758204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:35.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:35 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2438491522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:35.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:35 smithi033 ceph-mon[21626]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:35.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:35 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3239758204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:35.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:35 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2438491522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:35.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.694+0000 7f9af3000640 1 -- 172.21.15.22:0/1260454559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9aec062b90 con 0x7f9aec1083c0 2024-04-20T21:33:35.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.695+0000 7f9aea000640 1 -- 172.21.15.22:0/1260454559 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f9ae009cae0 con 0x7f9aec1083c0 2024-04-20T21:33:35.694 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-20T21:33:35.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.697+0000 7f9af3000640 1 -- 172.21.15.22:0/1260454559 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f9ad40a3bb0 msgr2=0x7f9ad40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.697+0000 7f9af3000640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f9ad40a3bb0 0x7f9ad40a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9adc02c260 tx=0x7f9adc000bc0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.698+0000 7f9af3000640 1 -- 172.21.15.22:0/1260454559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9aec1083c0 msgr2=0x7f9aec19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.698+0000 7f9af3000640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9aec1083c0 0x7f9aec19e690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000e9c0 tx=0x7f9ae000ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:35.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.698+0000 7f9af3000640 1 -- 172.21.15.22:0/1260454559 shutdown_connections 2024-04-20T21:33:35.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.698+0000 7f9af3000640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f9ad40a3bb0 0x7f9ad40a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.698+0000 7f9af3000640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aec109f70 0x7f9aec1a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.698+0000 7f9af3000640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aec1095b0 0x7f9aec19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.698+0000 7f9af3000640 1 --2- 172.21.15.22:0/1260454559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9aec1083c0 0x7f9aec19e690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.698+0000 7f9af3000640 1 -- 172.21.15.22:0/1260454559 >> 172.21.15.22:0/1260454559 conn(0x7f9aec077360 msgr2=0x7f9aec10c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:35.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.698+0000 7f9af3000640 1 -- 172.21.15.22:0/1260454559 shutdown_connections 2024-04-20T21:33:35.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.698+0000 7f9af3000640 1 -- 172.21.15.22:0/1260454559 wait complete. 2024-04-20T21:33:35.715 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:33:35.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.920+0000 7f5cbd600640 1 -- 172.21.15.22:0/780284370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb81040a0 msgr2=0x7f5cb81044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.920+0000 7f5cbd600640 1 --2- 172.21.15.22:0/780284370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb81040a0 0x7f5cb81044c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009900 tx=0x7f5ca802bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:35.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.921+0000 7f5cbd600640 1 -- 172.21.15.22:0/780284370 shutdown_connections 2024-04-20T21:33:35.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.921+0000 7f5cbd600640 1 --2- 172.21.15.22:0/780284370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cb8105c50 0x7f5cb810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.921+0000 7f5cbd600640 1 --2- 172.21.15.22:0/780284370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cb8105290 0x7f5cb8105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.921+0000 7f5cbd600640 1 --2- 172.21.15.22:0/780284370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb81040a0 0x7f5cb81044c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.921+0000 7f5cbd600640 1 -- 172.21.15.22:0/780284370 >> 172.21.15.22:0/780284370 conn(0x7f5cb80ff610 msgr2=0x7f5cb8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:35.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.921+0000 7f5cbd600640 1 -- 172.21.15.22:0/780284370 shutdown_connections 2024-04-20T21:33:35.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.921+0000 7f5cbd600640 1 -- 172.21.15.22:0/780284370 wait complete. 2024-04-20T21:33:35.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.922+0000 7f5cbd600640 1 Processor -- start 2024-04-20T21:33:35.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.922+0000 7f5cbd600640 1 -- start start 2024-04-20T21:33:35.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.922+0000 7f5cbd600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cb81040a0 0x7f5cb819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:35.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.922+0000 7f5cbd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb8105290 0x7f5cb819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:35.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.922+0000 7f5cbd600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cb8105c50 0x7f5cb81a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:35.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.922+0000 7f5cbd600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb810fd30 con 0x7f5cb8105290 2024-04-20T21:33:35.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.922+0000 7f5cbd600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb810fbb0 con 0x7f5cb81040a0 2024-04-20T21:33:35.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.922+0000 7f5cbd600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb810feb0 con 0x7f5cb8105c50 2024-04-20T21:33:35.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.922+0000 7f5cb7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb8105290 0x7f5cb819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:35.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.922+0000 7f5cb7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb8105290 0x7f5cb819a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55422/0 (socket says 172.21.15.22:55422) 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.923+0000 7f5cb7400640 1 -- 172.21.15.22:0/2618048886 learned_addr learned my addr 172.21.15.22:0/2618048886 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.923+0000 7f5cb7e00640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cb81040a0 0x7f5cb819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.923+0000 7f5cbcc00640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cb8105c50 0x7f5cb81a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.923+0000 7f5cb7400640 1 -- 172.21.15.22:0/2618048886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cb8105c50 msgr2=0x7f5cb81a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.923+0000 7f5cb7400640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cb8105c50 0x7f5cb81a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.923+0000 7f5cb7400640 1 -- 172.21.15.22:0/2618048886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cb81040a0 msgr2=0x7f5cb819a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.923+0000 7f5cb7400640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cb81040a0 0x7f5cb819a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.923+0000 7f5cb7400640 1 -- 172.21.15.22:0/2618048886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb81a1f90 con 0x7f5cb8105290 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.923+0000 7f5cbcc00640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cb8105c50 0x7f5cb81a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.923+0000 7f5cb7400640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb8105290 0x7f5cb819a8b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400d950 tx=0x7f5ca400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.924+0000 7f5cb4c00640 1 -- 172.21.15.22:0/2618048886 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4017070 con 0x7f5cb8105290 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.924+0000 7f5cb4c00640 1 -- 172.21.15.22:0/2618048886 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ca4009c30 con 0x7f5cb8105290 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.924+0000 7f5cbd600640 1 -- 172.21.15.22:0/2618048886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb81a2280 con 0x7f5cb8105290 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.924+0000 7f5cb4c00640 1 -- 172.21.15.22:0/2618048886 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400f6b0 con 0x7f5cb8105290 2024-04-20T21:33:35.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.924+0000 7f5cbd600640 1 -- 172.21.15.22:0/2618048886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb81a2820 con 0x7f5cb8105290 2024-04-20T21:33:35.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.925+0000 7f5cbd600640 1 -- 172.21.15.22:0/2618048886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb810b490 con 0x7f5cb8105290 2024-04-20T21:33:35.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.927+0000 7f5cb4c00640 1 -- 172.21.15.22:0/2618048886 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5ca40109d0 con 0x7f5cb8105290 2024-04-20T21:33:35.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.928+0000 7f5cb4c00640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f5c9c0a3c30 0x7f5c9c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:35.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.929+0000 7f5cb4c00640 1 -- 172.21.15.22:0/2618048886 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7f5ca40cff90 con 0x7f5cb8105290 2024-04-20T21:33:35.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.929+0000 7f5cb7e00640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f5c9c0a3c30 0x7f5c9c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:35.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.929+0000 7f5cb7e00640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f5c9c0a3c30 0x7f5c9c0a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8000c00 tx=0x7f5ca8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:35.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:35.932+0000 7f5cb4c00640 1 -- 172.21.15.22:0/2618048886 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5ca4009de0 con 0x7f5cb8105290 2024-04-20T21:33:36.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.138+0000 7f5cbd600640 1 -- 172.21.15.22:0/2618048886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5cb8062b60 con 0x7f5cb8105290 2024-04-20T21:33:36.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.139+0000 7f5cb4c00640 1 -- 172.21.15.22:0/2618048886 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f5ca409c280 con 0x7f5cb8105290 2024-04-20T21:33:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-20T21:33:36.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.141+0000 7f5cbd600640 1 -- 172.21.15.22:0/2618048886 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f5c9c0a3c30 msgr2=0x7f5c9c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.141+0000 7f5cbd600640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f5c9c0a3c30 0x7f5c9c0a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8000c00 tx=0x7f5ca8005fd0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.141+0000 7f5cbd600640 1 -- 172.21.15.22:0/2618048886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb8105290 msgr2=0x7f5cb819a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.141+0000 7f5cbd600640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb8105290 0x7f5cb819a8b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400d950 tx=0x7f5ca400de10 comp rx=0 tx=0).stop 2024-04-20T21:33:36.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.142+0000 7f5cbd600640 1 -- 172.21.15.22:0/2618048886 shutdown_connections 2024-04-20T21:33:36.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.142+0000 7f5cbd600640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f5c9c0a3c30 0x7f5c9c0a60f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.142+0000 7f5cbd600640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cb8105c50 0x7f5cb81a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.142+0000 7f5cbd600640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb8105290 0x7f5cb819a8b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.142+0000 7f5cbd600640 1 --2- 172.21.15.22:0/2618048886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cb81040a0 0x7f5cb819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.142+0000 7f5cbd600640 1 -- 172.21.15.22:0/2618048886 >> 172.21.15.22:0/2618048886 conn(0x7f5cb80ff610 msgr2=0x7f5cb8108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:36.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.142+0000 7f5cbd600640 1 -- 172.21.15.22:0/2618048886 shutdown_connections 2024-04-20T21:33:36.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.142+0000 7f5cbd600640 1 -- 172.21.15.22:0/2618048886 wait complete. 2024-04-20T21:33:36.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:33:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.367+0000 7f26c9800640 1 -- 172.21.15.22:0/1158116930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26c41083c0 msgr2=0x7f26c41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.367+0000 7f26c9800640 1 --2- 172.21.15.22:0/1158116930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26c41083c0 0x7f26c41087e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009900 tx=0x7f26b402bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.367+0000 7f26c9800640 1 -- 172.21.15.22:0/1158116930 shutdown_connections 2024-04-20T21:33:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.367+0000 7f26c9800640 1 --2- 172.21.15.22:0/1158116930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c4109f70 0x7f26c4110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.367+0000 7f26c9800640 1 --2- 172.21.15.22:0/1158116930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26c41095b0 0x7f26c4109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.367+0000 7f26c9800640 1 --2- 172.21.15.22:0/1158116930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26c41083c0 0x7f26c41087e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.367+0000 7f26c9800640 1 -- 172.21.15.22:0/1158116930 >> 172.21.15.22:0/1158116930 conn(0x7f26c4077360 msgr2=0x7f26c40797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.367+0000 7f26c9800640 1 -- 172.21.15.22:0/1158116930 shutdown_connections 2024-04-20T21:33:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.367+0000 7f26c9800640 1 -- 172.21.15.22:0/1158116930 wait complete. 2024-04-20T21:33:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.368+0000 7f26c9800640 1 Processor -- start 2024-04-20T21:33:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.368+0000 7f26c9800640 1 -- start start 2024-04-20T21:33:36.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.369+0000 7f26c9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26c41083c0 0x7f26c419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.369+0000 7f26c9800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c41095b0 0x7f26c419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.369+0000 7f26c9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26c4109f70 0x7f26c41a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.369+0000 7f26c9800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f26c41140a0 con 0x7f26c41083c0 2024-04-20T21:33:36.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.369+0000 7f26c9800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4113f20 con 0x7f26c41095b0 2024-04-20T21:33:36.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.369+0000 7f26c9800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4114220 con 0x7f26c4109f70 2024-04-20T21:33:36.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.369+0000 7f26c3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26c41083c0 0x7f26c419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.369+0000 7f26c3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26c41083c0 0x7f26c419e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55442/0 (socket says 172.21.15.22:55442) 2024-04-20T21:33:36.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.369+0000 7f26c3e00640 1 -- 172.21.15.22:0/3121793183 learned_addr learned my addr 172.21.15.22:0/3121793183 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:36.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.369+0000 7f26c3400640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c41095b0 0x7f26c419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.369+0000 7f26c8e00640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26c4109f70 0x7f26c41a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.370+0000 7f26c3e00640 1 -- 172.21.15.22:0/3121793183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26c4109f70 msgr2=0x7f26c41a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.370+0000 7f26c3e00640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26c4109f70 0x7f26c41a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.370+0000 7f26c3e00640 1 -- 172.21.15.22:0/3121793183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c41095b0 msgr2=0x7f26c419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.370+0000 7f26c3e00640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c41095b0 0x7f26c419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.370+0000 7f26c3e00640 1 -- 172.21.15.22:0/3121793183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c41a62e0 con 0x7f26c41083c0 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.370+0000 7f26c3400640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c41095b0 0x7f26c419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.370+0000 7f26c8e00640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26c4109f70 0x7f26c41a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.370+0000 7f26c3e00640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26c41083c0 0x7f26c419e6c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26b4000c00 tx=0x7f26b402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.370+0000 7f26c0c00640 1 -- 172.21.15.22:0/3121793183 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4043020 con 0x7f26c41083c0 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.370+0000 7f26c0c00640 1 -- 172.21.15.22:0/3121793183 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b40054e0 con 0x7f26c41083c0 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.370+0000 7f26c9800640 1 -- 172.21.15.22:0/3121793183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c41a6570 con 0x7f26c41083c0 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.370+0000 7f26c0c00640 1 -- 172.21.15.22:0/3121793183 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4003ce0 con 0x7f26c41083c0 2024-04-20T21:33:36.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.371+0000 7f26c9800640 1 -- 172.21.15.22:0/3121793183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c41a6a90 con 0x7f26c41083c0 2024-04-20T21:33:36.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.372+0000 7f26c9800640 1 -- 172.21.15.22:0/3121793183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c410f6e0 con 0x7f26c41083c0 2024-04-20T21:33:36.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.374+0000 7f26c0c00640 1 -- 172.21.15.22:0/3121793183 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f26b4046020 con 0x7f26c41083c0 2024-04-20T21:33:36.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.375+0000 7f26c0c00640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f26a80a3bb0 0x7f26a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.375+0000 7f26c0c00640 1 -- 172.21.15.22:0/3121793183 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7f26b40f2530 con 0x7f26c41083c0 2024-04-20T21:33:36.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.376+0000 7f26c3400640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f26a80a3bb0 0x7f26a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.376+0000 7f26c3400640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f26a80a3bb0 0x7f26a80a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f26c419fbd0 tx=0x7f26b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:36.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.380+0000 7f26c0c00640 1 -- 172.21.15.22:0/3121793183 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f26b404d020 con 0x7f26c41083c0 2024-04-20T21:33:36.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:36 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3520429985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:36.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:36 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1260454559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:36.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:36 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2618048886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:36 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3520429985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:36 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1260454559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:36 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2618048886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:36.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.590+0000 7f26c9800640 1 -- 172.21.15.22:0/3121793183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f26c402d7f0 con 0x7f26c41083c0 2024-04-20T21:33:36.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.591+0000 7f26c0c00640 1 -- 172.21.15.22:0/3121793183 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f26b40bee70 con 0x7f26c41083c0 2024-04-20T21:33:36.590 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-20T21:33:36.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.593+0000 7f26c9800640 1 -- 172.21.15.22:0/3121793183 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f26a80a3bb0 msgr2=0x7f26a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.593+0000 7f26c9800640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f26a80a3bb0 0x7f26a80a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f26c419fbd0 tx=0x7f26b000b390 comp rx=0 tx=0).stop 2024-04-20T21:33:36.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.593+0000 7f26c9800640 1 -- 172.21.15.22:0/3121793183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26c41083c0 msgr2=0x7f26c419e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.593+0000 7f26c9800640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26c41083c0 0x7f26c419e6c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26b4000c00 tx=0x7f26b402c6c0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.593+0000 7f26c9800640 1 -- 172.21.15.22:0/3121793183 shutdown_connections 2024-04-20T21:33:36.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.593+0000 7f26c9800640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f26a80a3bb0 0x7f26a80a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.593+0000 7f26c9800640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26c4109f70 0x7f26c41a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.593+0000 7f26c9800640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c41095b0 0x7f26c419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.593+0000 7f26c9800640 1 --2- 172.21.15.22:0/3121793183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26c41083c0 0x7f26c419e6c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.593+0000 7f26c9800640 1 -- 172.21.15.22:0/3121793183 >> 172.21.15.22:0/3121793183 conn(0x7f26c4077360 msgr2=0x7f26c410c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:36.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.594+0000 7f26c9800640 1 -- 172.21.15.22:0/3121793183 shutdown_connections 2024-04-20T21:33:36.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.594+0000 7f26c9800640 1 -- 172.21.15.22:0/3121793183 wait complete. 2024-04-20T21:33:36.611 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:36 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3520429985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:36.611 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:36 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1260454559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:36.611 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:36 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2618048886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:36.614 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:36.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.833+0000 7f2d2e600640 1 -- 172.21.15.22:0/3849786001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d28078500 msgr2=0x7f2d28078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.833+0000 7f2d2e600640 1 --2- 172.21.15.22:0/3849786001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d28078500 0x7f2d28078da0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009900 tx=0x7f2d1802bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:36.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.833+0000 7f2d2e600640 1 -- 172.21.15.22:0/3849786001 shutdown_connections 2024-04-20T21:33:36.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.833+0000 7f2d2e600640 1 --2- 172.21.15.22:0/3849786001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d28078500 0x7f2d28078da0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.833+0000 7f2d2e600640 1 --2- 172.21.15.22:0/3849786001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d28077b60 0x7f2d28077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.833+0000 7f2d2e600640 1 --2- 172.21.15.22:0/3849786001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d28076920 0x7f2d28076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.833+0000 7f2d2e600640 1 -- 172.21.15.22:0/3849786001 >> 172.21.15.22:0/3849786001 conn(0x7f2d280ff750 msgr2=0x7f2d28101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:36.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.834+0000 7f2d2e600640 1 -- 172.21.15.22:0/3849786001 shutdown_connections 2024-04-20T21:33:36.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.834+0000 7f2d2e600640 1 -- 172.21.15.22:0/3849786001 wait complete. 2024-04-20T21:33:36.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.834+0000 7f2d2e600640 1 Processor -- start 2024-04-20T21:33:36.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.835+0000 7f2d2e600640 1 -- start start 2024-04-20T21:33:36.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.835+0000 7f2d2e600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d28076920 0x7f2d2819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.835+0000 7f2d2e600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d28077b60 0x7f2d2819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.835+0000 7f2d2e600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d28078500 0x7f2d281a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.835+0000 7f2d2e600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28114240 con 0x7f2d28078500 2024-04-20T21:33:36.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.835+0000 7f2d2e600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281140c0 con 0x7f2d28076920 2024-04-20T21:33:36.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.835+0000 7f2d2e600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281143c0 con 0x7f2d28077b60 2024-04-20T21:33:36.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.835+0000 7f2d27e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d28078500 0x7f2d281a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.835+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d28077b60 0x7f2d2819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.835+0000 7f2d27400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d28076920 0x7f2d2819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.835+0000 7f2d27e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d28078500 0x7f2d281a5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55468/0 (socket says 172.21.15.22:55468) 2024-04-20T21:33:36.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.835+0000 7f2d27e00640 1 -- 172.21.15.22:0/2911719328 learned_addr learned my addr 172.21.15.22:0/2911719328 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:36.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.836+0000 7f2d27400640 1 -- 172.21.15.22:0/2911719328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d28077b60 msgr2=0x7f2d2819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.836+0000 7f2d27400640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d28077b60 0x7f2d2819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.836+0000 7f2d27400640 1 -- 172.21.15.22:0/2911719328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d28078500 msgr2=0x7f2d281a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:36.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.836+0000 7f2d27400640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d28078500 0x7f2d281a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:36.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.836+0000 7f2d27400640 1 -- 172.21.15.22:0/2911719328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d281a6460 con 0x7f2d28076920 2024-04-20T21:33:36.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.836+0000 7f2d27e00640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d28078500 0x7f2d281a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:36.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.836+0000 7f2d27400640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d28076920 0x7f2d2819e840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000e9f0 tx=0x7f2d2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:36.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.837+0000 7f2d1fe00640 1 -- 172.21.15.22:0/2911719328 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000ccc0 con 0x7f2d28076920 2024-04-20T21:33:36.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.837+0000 7f2d1fe00640 1 -- 172.21.15.22:0/2911719328 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d20010910 con 0x7f2d28076920 2024-04-20T21:33:36.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.837+0000 7f2d1fe00640 1 -- 172.21.15.22:0/2911719328 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20018920 con 0x7f2d28076920 2024-04-20T21:33:36.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.837+0000 7f2d2e600640 1 -- 172.21.15.22:0/2911719328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d281a6750 con 0x7f2d28076920 2024-04-20T21:33:36.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.837+0000 7f2d2e600640 1 -- 172.21.15.22:0/2911719328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d281a6cf0 con 0x7f2d28076920 2024-04-20T21:33:36.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.838+0000 7f2d2e600640 1 -- 172.21.15.22:0/2911719328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d28110c90 con 0x7f2d28076920 2024-04-20T21:33:36.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.840+0000 7f2d1fe00640 1 -- 172.21.15.22:0/2911719328 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2d20010ac0 con 0x7f2d28076920 2024-04-20T21:33:36.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.842+0000 7f2d1fe00640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2d0c0a3c80 0x7f2d0c0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.842+0000 7f2d1fe00640 1 -- 172.21.15.22:0/2911719328 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7f2d200d0a10 con 0x7f2d28076920 2024-04-20T21:33:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.842+0000 7f2d26a00640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2d0c0a3c80 0x7f2d0c0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.843+0000 7f2d26a00640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2d0c0a3c80 0x7f2d0c0a6140 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d100099a0 tx=0x7f2d10009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:36.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:36.844+0000 7f2d1fe00640 1 -- 172.21.15.22:0/2911719328 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2d2009cd00 con 0x7f2d28076920 2024-04-20T21:33:37.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.091+0000 7f2d2e600640 1 -- 172.21.15.22:0/2911719328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d2802d5d0 con 0x7f2d28076920 2024-04-20T21:33:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.092+0000 7f2d1fe00640 1 -- 172.21.15.22:0/2911719328 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f2d200a1000 con 0x7f2d28076920 2024-04-20T21:33:37.091 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:37.091 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:33:37.092 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:33:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.095+0000 7f2d2e600640 1 -- 172.21.15.22:0/2911719328 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2d0c0a3c80 msgr2=0x7f2d0c0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.095+0000 7f2d2e600640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2d0c0a3c80 0x7f2d0c0a6140 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d100099a0 tx=0x7f2d10009500 comp rx=0 tx=0).stop 2024-04-20T21:33:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.095+0000 7f2d2e600640 1 -- 172.21.15.22:0/2911719328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d28076920 msgr2=0x7f2d2819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.095+0000 7f2d2e600640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d28076920 0x7f2d2819e840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000e9f0 tx=0x7f2d2000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.095+0000 7f2d2e600640 1 -- 172.21.15.22:0/2911719328 shutdown_connections 2024-04-20T21:33:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.095+0000 7f2d2e600640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f2d0c0a3c80 0x7f2d0c0a6140 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.095+0000 7f2d2e600640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d28078500 0x7f2d281a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.095+0000 7f2d2e600640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d28077b60 0x7f2d2819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.095+0000 7f2d2e600640 1 --2- 172.21.15.22:0/2911719328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d28076920 0x7f2d2819e840 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.095+0000 7f2d2e600640 1 -- 172.21.15.22:0/2911719328 >> 172.21.15.22:0/2911719328 conn(0x7f2d280ff750 msgr2=0x7f2d28100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.095+0000 7f2d2e600640 1 -- 172.21.15.22:0/2911719328 shutdown_connections 2024-04-20T21:33:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.095+0000 7f2d2e600640 1 -- 172.21.15.22:0/2911719328 wait complete. 2024-04-20T21:33:37.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:37.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.329+0000 7f2901800640 1 -- 172.21.15.22:0/1701575481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28fc101e00 msgr2=0x7f28fc102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.329+0000 7f2901800640 1 --2- 172.21.15.22:0/1701575481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28fc101e00 0x7f28fc102220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009900 tx=0x7f28ec02bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:37.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.329+0000 7f2901800640 1 -- 172.21.15.22:0/1701575481 shutdown_connections 2024-04-20T21:33:37.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.329+0000 7f2901800640 1 --2- 172.21.15.22:0/1701575481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28fc1039b0 0x7f28fc10a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.329+0000 7f2901800640 1 --2- 172.21.15.22:0/1701575481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28fc102ff0 0x7f28fc103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.329+0000 7f2901800640 1 --2- 172.21.15.22:0/1701575481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28fc101e00 0x7f28fc102220 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.329+0000 7f2901800640 1 -- 172.21.15.22:0/1701575481 >> 172.21.15.22:0/1701575481 conn(0x7f28fc0fd370 msgr2=0x7f28fc0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:37.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.329+0000 7f2901800640 1 -- 172.21.15.22:0/1701575481 shutdown_connections 2024-04-20T21:33:37.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.329+0000 7f2901800640 1 -- 172.21.15.22:0/1701575481 wait complete. 2024-04-20T21:33:37.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.330+0000 7f2901800640 1 Processor -- start 2024-04-20T21:33:37.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.330+0000 7f2901800640 1 -- start start 2024-04-20T21:33:37.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.331+0000 7f2901800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28fc101e00 0x7f28fc19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.331+0000 7f2901800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28fc102ff0 0x7f28fc19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.331+0000 7f2901800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28fc1039b0 0x7f28fc1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.331+0000 7f2901800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc10dac0 con 0x7f28fc102ff0 2024-04-20T21:33:37.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.331+0000 7f2901800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc10d940 con 0x7f28fc101e00 2024-04-20T21:33:37.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.331+0000 7f2901800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc10dc40 con 0x7f28fc1039b0 2024-04-20T21:33:37.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.331+0000 7f28fb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28fc102ff0 0x7f28fc19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.331+0000 7f28fb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28fc102ff0 0x7f28fc19a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55490/0 (socket says 172.21.15.22:55490) 2024-04-20T21:33:37.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.331+0000 7f28fb400640 1 -- 172.21.15.22:0/2108182840 learned_addr learned my addr 172.21.15.22:0/2108182840 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:37.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.331+0000 7f2900e00640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28fc1039b0 0x7f28fc1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.331+0000 7f28fbe00640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28fc101e00 0x7f28fc19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.332+0000 7f28fb400640 1 -- 172.21.15.22:0/2108182840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28fc1039b0 msgr2=0x7f28fc1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.332+0000 7f28fb400640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28fc1039b0 0x7f28fc1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.332+0000 7f28fb400640 1 -- 172.21.15.22:0/2108182840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28fc101e00 msgr2=0x7f28fc19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.332+0000 7f28fb400640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28fc101e00 0x7f28fc19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.332+0000 7f28fb400640 1 -- 172.21.15.22:0/2108182840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28fc1a1f70 con 0x7f28fc102ff0 2024-04-20T21:33:37.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.332+0000 7f28fb400640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28fc102ff0 0x7f28fc19a890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f28e800eb90 tx=0x7f28e800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:37.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.332+0000 7f28f8c00640 1 -- 172.21.15.22:0/2108182840 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8004d20 con 0x7f28fc102ff0 2024-04-20T21:33:37.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.332+0000 7f2901800640 1 -- 172.21.15.22:0/2108182840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28fc1a2260 con 0x7f28fc102ff0 2024-04-20T21:33:37.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.332+0000 7f28f8c00640 1 -- 172.21.15.22:0/2108182840 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e800ee90 con 0x7f28fc102ff0 2024-04-20T21:33:37.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.332+0000 7f28f8c00640 1 -- 172.21.15.22:0/2108182840 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8010830 con 0x7f28fc102ff0 2024-04-20T21:33:37.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.332+0000 7f2901800640 1 -- 172.21.15.22:0/2108182840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28fc1a27b0 con 0x7f28fc102ff0 2024-04-20T21:33:37.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.333+0000 7f2901800640 1 -- 172.21.15.22:0/2108182840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28fc1a2a70 con 0x7f28fc102ff0 2024-04-20T21:33:37.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.336+0000 7f28f8c00640 1 -- 172.21.15.22:0/2108182840 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f28e8005020 con 0x7f28fc102ff0 2024-04-20T21:33:37.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.338+0000 7f28f8c00640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f28e00a3bb0 0x7f28e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.338+0000 7f28f8c00640 1 -- 172.21.15.22:0/2108182840 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7f28e80d0b00 con 0x7f28fc102ff0 2024-04-20T21:33:37.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.338+0000 7f28fbe00640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f28e00a3bb0 0x7f28e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.339+0000 7f28fbe00640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f28e00a3bb0 0x7f28e00a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f28ec02c170 tx=0x7f28ec02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:37.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.342+0000 7f28f8c00640 1 -- 172.21.15.22:0/2108182840 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f28e809d440 con 0x7f28fc102ff0 2024-04-20T21:33:37.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:37 smithi022 ceph-mon[16843]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:37.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:37 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3121793183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:37.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:37 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2911719328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:37.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:37 smithi122 ceph-mon[22147]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:37.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:37 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3121793183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:37.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:37 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2911719328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:37.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.551+0000 7f2901800640 1 -- 172.21.15.22:0/2108182840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28fc1a2c40 con 0x7f28fc102ff0 2024-04-20T21:33:37.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:37 smithi033 ceph-mon[21626]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:33:37.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:37 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3121793183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:37.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:37 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2911719328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:37.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.555+0000 7f28f8c00640 1 -- 172.21.15.22:0/2108182840 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22768 (secure 0 0 0) 0x7f28e809cd40 con 0x7f28fc102ff0 2024-04-20T21:33:37.554 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:37.554 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:33:32.135126+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.557+0000 7f2901800640 1 -- 172.21.15.22:0/2108182840 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f28e00a3bb0 msgr2=0x7f28e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.557+0000 7f2901800640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f28e00a3bb0 0x7f28e00a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f28ec02c170 tx=0x7f28ec02df90 comp rx=0 tx=0).stop 2024-04-20T21:33:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.557+0000 7f2901800640 1 -- 172.21.15.22:0/2108182840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28fc102ff0 msgr2=0x7f28fc19a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.557+0000 7f2901800640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28fc102ff0 0x7f28fc19a890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f28e800eb90 tx=0x7f28e800c6a0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.557+0000 7f2901800640 1 -- 172.21.15.22:0/2108182840 shutdown_connections 2024-04-20T21:33:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.557+0000 7f2901800640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f28e00a3bb0 0x7f28e00a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.557+0000 7f2901800640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28fc1039b0 0x7f28fc1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.557+0000 7f2901800640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28fc102ff0 0x7f28fc19a890 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.557+0000 7f2901800640 1 --2- 172.21.15.22:0/2108182840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28fc101e00 0x7f28fc19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.557+0000 7f2901800640 1 -- 172.21.15.22:0/2108182840 >> 172.21.15.22:0/2108182840 conn(0x7f28fc0fd370 msgr2=0x7f28fc074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.558+0000 7f2901800640 1 -- 172.21.15.22:0/2108182840 shutdown_connections 2024-04-20T21:33:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.558+0000 7f2901800640 1 -- 172.21.15.22:0/2108182840 wait complete. 2024-04-20T21:33:37.577 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-20T21:31:27.176318+0000', 'modified': '2024-04-20T21:31:31.525301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24581}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24568': {'gid': 24568, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6839/2642727200', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 2642727200}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 2642727200}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6839/300232041', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6838', 'nonce': 300232041}, {'type': 'v1', 'addr': '172.21.15.33:6839', 'nonce': 300232041}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24581]} 2024-04-20T21:33:37.577 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:33:37.577 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T21:33:37.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.785+0000 7fc1fd200640 1 -- 172.21.15.22:0/1993944817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8075f40 msgr2=0x7fc1f8074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.785+0000 7fc1fd200640 1 --2- 172.21.15.22:0/1993944817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8075f40 0x7fc1f8074390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009ca0 tx=0x7fc1e802c190 comp rx=0 tx=0).stop 2024-04-20T21:33:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.785+0000 7fc1fd200640 1 -- 172.21.15.22:0/1993944817 shutdown_connections 2024-04-20T21:33:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.785+0000 7fc1fd200640 1 --2- 172.21.15.22:0/1993944817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f8109d20 0x7fc1f810c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.785+0000 7fc1fd200640 1 --2- 172.21.15.22:0/1993944817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f8074960 0x7fc1f8074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.785+0000 7fc1fd200640 1 --2- 172.21.15.22:0/1993944817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8075f40 0x7fc1f8074390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.785+0000 7fc1fd200640 1 -- 172.21.15.22:0/1993944817 >> 172.21.15.22:0/1993944817 conn(0x7fc1f80fd400 msgr2=0x7fc1f80ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.785+0000 7fc1fd200640 1 -- 172.21.15.22:0/1993944817 shutdown_connections 2024-04-20T21:33:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.785+0000 7fc1fd200640 1 -- 172.21.15.22:0/1993944817 wait complete. 2024-04-20T21:33:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.786+0000 7fc1fd200640 1 Processor -- start 2024-04-20T21:33:37.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.787+0000 7fc1fd200640 1 -- start start 2024-04-20T21:33:37.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.787+0000 7fc1fd200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f8074960 0x7fc1f819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.787+0000 7fc1fd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f8075f40 0x7fc1f819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.787+0000 7fc1fd200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8109d20 0x7fc1f81a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.787+0000 7fc1fd200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f810fd20 con 0x7fc1f8075f40 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.787+0000 7fc1fd200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f810fba0 con 0x7fc1f8074960 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.787+0000 7fc1fd200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f810fea0 con 0x7fc1f8109d20 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.787+0000 7fc1f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f8075f40 0x7fc1f819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.787+0000 7fc1f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f8075f40 0x7fc1f819a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55504/0 (socket says 172.21.15.22:55504) 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.787+0000 7fc1f6a00640 1 -- 172.21.15.22:0/3337880512 learned_addr learned my addr 172.21.15.22:0/3337880512 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.787+0000 7fc1f7400640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f8074960 0x7fc1f819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.787+0000 7fc1f7e00640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8109d20 0x7fc1f81a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.788+0000 7fc1f6a00640 1 -- 172.21.15.22:0/3337880512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8109d20 msgr2=0x7fc1f81a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.788+0000 7fc1f6a00640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8109d20 0x7fc1f81a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.788+0000 7fc1f6a00640 1 -- 172.21.15.22:0/3337880512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f8074960 msgr2=0x7fc1f819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.788+0000 7fc1f6a00640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f8074960 0x7fc1f819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.788+0000 7fc1f6a00640 1 -- 172.21.15.22:0/3337880512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f81a1fb0 con 0x7fc1f8075f40 2024-04-20T21:33:37.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.788+0000 7fc1f6a00640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f8075f40 0x7fc1f819a8d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000e9f0 tx=0x7fc1e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.788+0000 7fc1efe00640 1 -- 172.21.15.22:0/3337880512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000ccc0 con 0x7fc1f8075f40 2024-04-20T21:33:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.788+0000 7fc1efe00640 1 -- 172.21.15.22:0/3337880512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1e000ce60 con 0x7fc1f8075f40 2024-04-20T21:33:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.788+0000 7fc1efe00640 1 -- 172.21.15.22:0/3337880512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0010690 con 0x7fc1f8075f40 2024-04-20T21:33:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.788+0000 7fc1fd200640 1 -- 172.21.15.22:0/3337880512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f81a22a0 con 0x7fc1f8075f40 2024-04-20T21:33:37.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.788+0000 7fc1fd200640 1 -- 172.21.15.22:0/3337880512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f81a27a0 con 0x7fc1f8075f40 2024-04-20T21:33:37.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.792+0000 7fc2015f7740 1 Processor -- start 2024-04-20T21:33:37.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.792+0000 7fc2015f7740 1 -- start start 2024-04-20T21:33:37.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.792+0000 7fc1efe00640 1 -- 172.21.15.22:0/3337880512 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc1e00046c0 con 0x7fc1f8075f40 2024-04-20T21:33:37.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.792+0000 7fc2015f7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c8ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.792+0000 7fc2015f7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3c95030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.792+0000 7fc2015f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3c97b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.792+0000 7fc2015f7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3bf69d0 con 0x55a5b3c90530 2024-04-20T21:33:37.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.792+0000 7fc2015f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3b9d270 con 0x55a5b3c956a0 2024-04-20T21:33:37.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.792+0000 7fc2015f7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3bf2990 con 0x55a5b3c8fb40 2024-04-20T21:33:37.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.793+0000 7fc1f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3c95030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.793+0000 7fc1f6a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3c97b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.793+0000 7fc1f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3c95030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55516/0 (socket says 172.21.15.22:55516) 2024-04-20T21:33:37.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.793+0000 7fc1f7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c8ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.793+0000 7fc1f7400640 1 -- 172.21.15.22:0/1765993184 learned_addr learned my addr 172.21.15.22:0/1765993184 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.793+0000 7fc1f7400640 1 -- 172.21.15.22:0/1765993184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 msgr2=0x55a5b3c8ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.793+0000 7fc1f7400640 1 --2- 172.21.15.22:0/1765993184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c8ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.793+0000 7fc1f7400640 1 -- 172.21.15.22:0/1765993184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 msgr2=0x55a5b3c97b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.793+0000 7fc1f7400640 1 --2- 172.21.15.22:0/1765993184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3c97b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.793+0000 7fc1f7400640 1 -- 172.21.15.22:0/1765993184 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b39092d0 con 0x55a5b3c90530 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.793+0000 7fc1f7400640 1 --2- 172.21.15.22:0/1765993184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3c95030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005e30 tx=0x7fc1e8032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0aeee2696a6625 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.794+0000 7fc1efe00640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc1dc0a3bb0 0x7fc1dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.794+0000 7fc1ecc00640 1 -- 172.21.15.22:0/1765993184 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8043070 con 0x55a5b3c90530 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.794+0000 7fc1ecc00640 1 -- 172.21.15.22:0/1765993184 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1e8003e50 con 0x55a5b3c90530 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.794+0000 7fc1ecc00640 1 -- 172.21.15.22:0/1765993184 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8039040 con 0x55a5b3c90530 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.794+0000 7fc1efe00640 1 -- 172.21.15.22:0/3337880512 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7fc1e00d0800 con 0x7fc1f8075f40 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.794+0000 7fc1f7e00640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc1dc0a3bb0 0x7fc1dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.795+0000 7fc1f7e00640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc1dc0a3bb0 0x7fc1dc0a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55a5b3c90d90 tx=0x7fc1f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.795+0000 7fc2015f7740 1 -- 172.21.15.22:0/1765993184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 msgr2=0x55a5b3c95030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.795+0000 7fc2015f7740 1 --2- 172.21.15.22:0/1765993184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3c95030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005e30 tx=0x7fc1e8032340 comp rx=0 tx=0).stop 2024-04-20T21:33:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.795+0000 7fc2015f7740 1 -- 172.21.15.22:0/1765993184 shutdown_connections 2024-04-20T21:33:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.795+0000 7fc2015f7740 1 --2- 172.21.15.22:0/1765993184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3c97b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.795+0000 7fc2015f7740 1 --2- 172.21.15.22:0/1765993184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3c95030 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.795+0000 7fc2015f7740 1 --2- 172.21.15.22:0/1765993184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c8ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.795+0000 7fc2015f7740 1 -- 172.21.15.22:0/1765993184 >> 172.21.15.22:0/1765993184 conn(0x55a5b3c8b0b0 msgr2=0x55a5b3c8b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.795+0000 7fc2015f7740 1 -- 172.21.15.22:0/1765993184 shutdown_connections 2024-04-20T21:33:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.795+0000 7fc2015f7740 1 -- 172.21.15.22:0/1765993184 wait complete. 2024-04-20T21:33:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.795+0000 7fc2015f7740 1 -- start start 2024-04-20T21:33:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.796+0000 7fc2015f7740 1 Processor -- start 2024-04-20T21:33:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.796+0000 7fc2015f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c9a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.796+0000 7fc2015f7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c90530 0x55a5b3c9a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.796+0000 7fc2015f7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c956a0 0x55a5b3ca15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.796+0000 7fc2015f7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3c985f0 con 0x55a5b3c956a0 2024-04-20T21:33:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.796+0000 7fc2015f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3bb05e0 con 0x55a5b3c8fb40 2024-04-20T21:33:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.796+0000 7fc2015f7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3c98770 con 0x55a5b3c90530 2024-04-20T21:33:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.796+0000 7fc2015f7740 10 client.0 _ll_register_callbacks cb 0x55a5b3908ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.796+0000 7fc1f7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c956a0 0x55a5b3ca15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.796+0000 7fc1f7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c956a0 0x55a5b3ca15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55520/0 (socket says 172.21.15.22:55520) 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.796+0000 7fc1f7e00640 1 -- 172.21.15.22:0/980267813 learned_addr learned my addr 172.21.15.22:0/980267813 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.796+0000 7fc1f6a00640 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c90530 0x55a5b3c9a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.797+0000 7fc1f7400640 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c9a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.797+0000 7fc1f7e00640 1 -- 172.21.15.22:0/980267813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c90530 msgr2=0x55a5b3c9a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.797+0000 7fc1f7e00640 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c90530 0x55a5b3c9a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.797+0000 7fc1f7e00640 1 -- 172.21.15.22:0/980267813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c8fb40 msgr2=0x55a5b3c9a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.797+0000 7fc1f7e00640 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c9a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.797+0000 7fc1f7e00640 1 -- 172.21.15.22:0/980267813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b39d36b0 con 0x55a5b3c956a0 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.797+0000 7fc1f6a00640 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c90530 0x55a5b3c9a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.797+0000 7fc1f7400640 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c9a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.797+0000 7fc1f7e00640 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c956a0 0x55a5b3ca15d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc1f00045e0 tx=0x7fc1f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8775d004f18306f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.797+0000 7fc1e5600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.798+0000 7fc1e5600640 1 -- 172.21.15.22:0/980267813 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f001e470 con 0x55a5b3c956a0 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.798+0000 7fc1e5600640 1 -- 172.21.15.22:0/980267813 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7fc1f001e610 con 0x55a5b3c956a0 2024-04-20T21:33:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.798+0000 7fc2015f7740 1 -- 172.21.15.22:0/980267813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1e8003e50 con 0x55a5b3c956a0 2024-04-20T21:33:37.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.798+0000 7fc1e5600640 1 -- 172.21.15.22:0/980267813 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc1f001e850 con 0x55a5b3c956a0 2024-04-20T21:33:37.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.798+0000 7fc2015f7740 10 client.15000 fetch_fsmap learned FSMap version 20 2024-04-20T21:33:37.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.798+0000 7fc2015f7740 1 -- 172.21.15.22:0/980267813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a5b3ca4a10 con 0x55a5b3c956a0 2024-04-20T21:33:37.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.798+0000 7fc1e5600640 1 -- 172.21.15.22:0/980267813 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1e8003e50 con 0x55a5b3c956a0 2024-04-20T21:33:37.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.798+0000 7fc1e5600640 1 -- 172.21.15.22:0/980267813 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fc1f0027070 con 0x55a5b3c956a0 2024-04-20T21:33:37.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.798+0000 7fc2015f7740 10 client.15000 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:33:37.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.799+0000 7fc2015f7740 10 client.15000 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:33:37.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.799+0000 7fc2015f7740 20 client.15000 populate_metadata read hostname 'smithi022' 2024-04-20T21:33:37.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.799+0000 7fc2015f7740 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a5b3ca70e0 0x55a5b3ca8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.799+0000 7fc2015f7740 4 client.15000 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:33:37.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.799+0000 7fc2015f7740 1 -- 172.21.15.22:0/980267813 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5b3c043f0 con 0x55a5b3ca70e0 2024-04-20T21:33:37.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.799+0000 7fc1f7400640 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a5b3ca70e0 0x55a5b3ca8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.799+0000 7fc1f7400640 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a5b3ca70e0 0x55a5b3ca8550 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e94393d0022feb88 server_cookie=9940575db950c3ea in_seq=0 out_seq=0 2024-04-20T21:33:37.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.799+0000 7fc1e5600640 10 client.15000 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:37.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.800+0000 7fc1e5600640 1 -- 172.21.15.22:0/980267813 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5b3c043f0 con 0x55a5b3ca70e0 2024-04-20T21:33:37.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.800+0000 7fc1e5600640 10 client.15000 handle_command_reply: tid=0 2024-04-20T21:33:37.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.800+0000 7fc2015f7740 1 client.15000 shutdown 2024-04-20T21:33:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.801+0000 7fc2015f7740 1 -- 172.21.15.22:0/980267813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c956a0 msgr2=0x55a5b3ca15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.801+0000 7fc2015f7740 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c956a0 0x55a5b3ca15d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc1f00045e0 tx=0x7fc1f0004730 comp rx=0 tx=0).stop 2024-04-20T21:33:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.801+0000 7fc2015f7740 1 -- 172.21.15.22:0/980267813 shutdown_connections 2024-04-20T21:33:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.801+0000 7fc2015f7740 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c956a0 0x55a5b3ca15d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.801+0000 7fc2015f7740 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c90530 0x55a5b3c9a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.801+0000 7fc2015f7740 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a5b3ca70e0 0x55a5b3ca8550 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.801+0000 7fc2015f7740 1 --2- 172.21.15.22:0/980267813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c9a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.801+0000 7fc2015f7740 1 -- 172.21.15.22:0/980267813 >> 172.21.15.22:0/980267813 conn(0x55a5b3c8b0b0 msgr2=0x55a5b3c8a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.801+0000 7fc1e5600640 0 client.15000 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.801+0000 7fc2015f7740 1 -- 172.21.15.22:0/980267813 shutdown_connections 2024-04-20T21:33:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.801+0000 7fc2015f7740 1 -- 172.21.15.22:0/980267813 wait complete. 2024-04-20T21:33:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.801+0000 7fc2015f7740 20 client.15000 trim_cache size 0 max 16384 2024-04-20T21:33:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.821+0000 7fc2015f7740 1 Processor -- start 2024-04-20T21:33:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.821+0000 7fc2015f7740 1 -- start start 2024-04-20T21:33:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.821+0000 7fc2015f7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c86440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.821+0000 7fc2015f7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3ca14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.821+0000 7fc2015f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3c98b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.821+0000 7fc2015f7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3ca6890 con 0x55a5b3c90530 2024-04-20T21:33:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.821+0000 7fc2015f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3ca6f60 con 0x55a5b3c956a0 2024-04-20T21:33:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.821+0000 7fc2015f7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3ca7630 con 0x55a5b3c8fb40 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.822+0000 7fc1f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3ca14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.822+0000 7fc1f7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c86440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.822+0000 7fc1f7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3c98b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.822+0000 7fc1f7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c86440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:35986/0 (socket says 172.21.15.22:35986) 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.822+0000 7fc1f7400640 1 -- 172.21.15.22:0/741323705 learned_addr learned my addr 172.21.15.22:0/741323705 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.822+0000 7fc1f7e00640 1 -- 172.21.15.22:0/741323705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 msgr2=0x55a5b3c86440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.822+0000 7fc1f7e00640 1 --2- 172.21.15.22:0/741323705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c86440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.822+0000 7fc1f7e00640 1 -- 172.21.15.22:0/741323705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 msgr2=0x55a5b3ca14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.822+0000 7fc1f7e00640 1 --2- 172.21.15.22:0/741323705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3ca14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.822+0000 7fc1f7e00640 1 -- 172.21.15.22:0/741323705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b3c90dc0 con 0x55a5b3c956a0 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.822+0000 7fc1f6a00640 1 --2- 172.21.15.22:0/741323705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3ca14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.822+0000 7fc1f7e00640 1 --2- 172.21.15.22:0/741323705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3c98b10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0033e80 tx=0x7fc1f0005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9c4c007a1270a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.823+0000 7fc1e7e00640 1 -- 172.21.15.22:0/741323705 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00294c0 con 0x55a5b3c956a0 2024-04-20T21:33:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.823+0000 7fc1e7e00640 1 -- 172.21.15.22:0/741323705 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1f0029660 con 0x55a5b3c956a0 2024-04-20T21:33:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.823+0000 7fc1e7e00640 1 -- 172.21.15.22:0/741323705 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0029d00 con 0x55a5b3c956a0 2024-04-20T21:33:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.823+0000 7fc2015f7740 1 -- 172.21.15.22:0/741323705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 msgr2=0x55a5b3c98b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.823+0000 7fc2015f7740 1 --2- 172.21.15.22:0/741323705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3c98b10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0033e80 tx=0x7fc1f0005db0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.823+0000 7fc2015f7740 1 -- 172.21.15.22:0/741323705 shutdown_connections 2024-04-20T21:33:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.824+0000 7fc2015f7740 1 --2- 172.21.15.22:0/741323705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3c98b10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.824+0000 7fc2015f7740 1 --2- 172.21.15.22:0/741323705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3ca14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.824+0000 7fc2015f7740 1 --2- 172.21.15.22:0/741323705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c86440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.824+0000 7fc2015f7740 1 -- 172.21.15.22:0/741323705 >> 172.21.15.22:0/741323705 conn(0x55a5b3c8b0b0 msgr2=0x55a5b3c90fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:37.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.824+0000 7fc2015f7740 1 -- 172.21.15.22:0/741323705 shutdown_connections 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.824+0000 7fc2015f7740 1 -- 172.21.15.22:0/741323705 wait complete. 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.824+0000 7fc2015f7740 1 -- start start 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.824+0000 7fc2015f7740 1 Processor -- start 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.825+0000 7fc2015f7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c9c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.825+0000 7fc2015f7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3c9c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.825+0000 7fc2015f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3d13890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.825+0000 7fc2015f7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3ca83d0 con 0x55a5b3c90530 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.825+0000 7fc2015f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3ca7d00 con 0x55a5b3c956a0 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.825+0000 7fc2015f7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3ca8aa0 con 0x55a5b3c8fb40 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.825+0000 7fc2015f7740 10 client.0 _ll_register_callbacks cb 0x7fc1f0009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.825+0000 7fc1f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3c9c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.825+0000 7fc1f7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3d13890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.825+0000 7fc1f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3c9c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55548/0 (socket says 172.21.15.22:55548) 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.825+0000 7fc1f7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c9c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.825+0000 7fc1f6a00640 1 -- 172.21.15.22:0/2457581319 learned_addr learned my addr 172.21.15.22:0/2457581319 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:37.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.826+0000 7fc1f7400640 1 -- 172.21.15.22:0/2457581319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 msgr2=0x55a5b3d13890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:37.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.826+0000 7fc1f7400640 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3d13890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.826+0000 7fc1f7400640 1 -- 172.21.15.22:0/2457581319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 msgr2=0x55a5b3c9c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:37.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.826+0000 7fc1f7400640 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3c9c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.826+0000 7fc1f7400640 1 -- 172.21.15.22:0/2457581319 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b3ca1840 con 0x55a5b3c8fb40 2024-04-20T21:33:37.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.826+0000 7fc1f7e00640 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3d13890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:37.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.826+0000 7fc1f6a00640 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3c9c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:37.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.826+0000 7fc1f7400640 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c9c220 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80325a0 tx=0x7fc1e8009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eebeef0f37898e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:37.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.826+0000 7fc1e5600640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:33:37.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.826+0000 7fc1e5600640 1 -- 172.21.15.22:0/2457581319 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8057060 con 0x55a5b3c8fb40 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.826+0000 7fc2015f7740 1 -- 172.21.15.22:0/2457581319 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f0029660 con 0x55a5b3c8fb40 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.826+0000 7fc1e5600640 1 -- 172.21.15.22:0/2457581319 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7fc1e8051640 con 0x55a5b3c8fb40 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.827+0000 7fc1e5600640 1 -- 172.21.15.22:0/2457581319 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc1e8039040 con 0x55a5b3c8fb40 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.827+0000 7fc2015f7740 10 client.25147 fetch_fsmap learned FSMap version 20 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.827+0000 7fc2015f7740 1 -- 172.21.15.22:0/2457581319 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a5b3d14d50 con 0x55a5b3c8fb40 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.827+0000 7fc1e5600640 1 -- 172.21.15.22:0/2457581319 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1f0029660 con 0x55a5b3c8fb40 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.827+0000 7fc1e5600640 1 -- 172.21.15.22:0/2457581319 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fc1e8048610 con 0x55a5b3c8fb40 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.827+0000 7fc2015f7740 10 client.25147 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.827+0000 7fc2015f7740 10 client.25147 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.827+0000 7fc2015f7740 20 client.25147 populate_metadata read hostname 'smithi022' 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.827+0000 7fc2015f7740 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a5b3d19530 0x55a5b3d18240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.827+0000 7fc2015f7740 4 client.25147 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.827+0000 7fc2015f7740 1 -- 172.21.15.22:0/2457581319 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "status"}) -- 0x55a5b3bfe260 con 0x55a5b3d19530 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.828+0000 7fc1f6a00640 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a5b3d19530 0x55a5b3d18240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.828+0000 7fc1f6a00640 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a5b3d19530 0x55a5b3d18240 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f0ef1df0022e2c server_cookie=5cbe3e1ba8c7add3 in_seq=0 out_seq=0 2024-04-20T21:33:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.828+0000 7fc1e5600640 10 client.25147 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.828+0000 7fc1e5600640 1 -- 172.21.15.22:0/2457581319 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a5b3bfe260 con 0x55a5b3d19530 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.828+0000 7fc1e5600640 10 client.25147 handle_command_reply: tid=0 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.828+0000 7fc2015f7740 1 client.25147 shutdown 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.829+0000 7fc2015f7740 1 -- 172.21.15.22:0/2457581319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 msgr2=0x55a5b3c9c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.829+0000 7fc2015f7740 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c9c220 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80325a0 tx=0x7fc1e8009360 comp rx=0 tx=0).stop 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.829+0000 7fc2015f7740 1 -- 172.21.15.22:0/2457581319 shutdown_connections 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.829+0000 7fc2015f7740 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a5b3d19530 0x55a5b3d18240 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.829+0000 7fc2015f7740 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5b3c956a0 0x55a5b3d13890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.829+0000 7fc2015f7740 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a5b3c90530 0x55a5b3c9c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.829+0000 7fc2015f7740 1 --2- 172.21.15.22:0/2457581319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5b3c8fb40 0x55a5b3c9c220 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.829+0000 7fc1e5600640 0 client.25147 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.829+0000 7fc2015f7740 1 -- 172.21.15.22:0/2457581319 >> 172.21.15.22:0/2457581319 conn(0x55a5b3c8b0b0 msgr2=0x55a5b3c90fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.829+0000 7fc2015f7740 1 -- 172.21.15.22:0/2457581319 shutdown_connections 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.829+0000 7fc2015f7740 1 -- 172.21.15.22:0/2457581319 wait complete. 2024-04-20T21:33:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.829+0000 7fc2015f7740 20 client.25147 trim_cache size 0 max 16384 2024-04-20T21:33:37.831 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:33:37.831 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "92c49c5c-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:33:37.831 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-04-20T21:33:37.831 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24581, 2024-04-20T21:33:37.831 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-04-20T21:33:37.831 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-04-20T21:33:37.831 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-04-20T21:33:37.831 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 130.15659052800001, 2024-04-20T21:33:37.831 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 20, 2024-04-20T21:33:37.831 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 92, 2024-04-20T21:33:37.831 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 88, 2024-04-20T21:33:37.831 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 136.62547097800001 2024-04-20T21:33:37.832 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:33:37.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.830+0000 7fc1fd200640 1 -- 172.21.15.22:0/3337880512 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc1dc0a3bb0 msgr2=0x7fc1dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.830+0000 7fc1fd200640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc1dc0a3bb0 0x7fc1dc0a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55a5b3c90d90 tx=0x7fc1f0005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:37.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.830+0000 7fc1fd200640 1 -- 172.21.15.22:0/3337880512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f8075f40 msgr2=0x7fc1f819a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:37.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.830+0000 7fc1fd200640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f8075f40 0x7fc1f819a8d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000e9f0 tx=0x7fc1e000edf0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.830+0000 7fc1fd200640 1 -- 172.21.15.22:0/3337880512 shutdown_connections 2024-04-20T21:33:37.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.830+0000 7fc1fd200640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fc1dc0a3bb0 0x7fc1dc0a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.830+0000 7fc1fd200640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8109d20 0x7fc1f81a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.830+0000 7fc1fd200640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f8075f40 0x7fc1f819a8d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.830+0000 7fc1fd200640 1 --2- 172.21.15.22:0/3337880512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f8074960 0x7fc1f819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.830+0000 7fc1fd200640 1 -- 172.21.15.22:0/3337880512 >> 172.21.15.22:0/3337880512 conn(0x7fc1f80fd400 msgr2=0x7fc1f8105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.830+0000 7fc1fd200640 1 -- 172.21.15.22:0/3337880512 shutdown_connections 2024-04-20T21:33:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:37.830+0000 7fc1fd200640 1 -- 172.21.15.22:0/3337880512 wait complete. 2024-04-20T21:33:37.848 INFO:teuthology.run_tasks:Running task sequential... 2024-04-20T21:33:37.858 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-20T21:33:37.858 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-20T21:33:37.858 INFO:teuthology.task.print:Enabling mgr modules 2024-04-20T21:33:37.858 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-20T21:33:37.859 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T21:33:37.859 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-04-20T21:33:37.859 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-20T21:33:38.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.071+0000 7f3487e00640 1 -- 172.21.15.22:0/3891066378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3488076920 msgr2=0x7f3488076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:38.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.071+0000 7f3487e00640 1 --2- 172.21.15.22:0/3891066378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3488076920 0x7f3488076d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3478009900 tx=0x7f347802bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:38.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.071+0000 7f3487e00640 1 -- 172.21.15.22:0/3891066378 shutdown_connections 2024-04-20T21:33:38.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.071+0000 7f3487e00640 1 --2- 172.21.15.22:0/3891066378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3488078500 0x7f3488078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:38.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.071+0000 7f3487e00640 1 --2- 172.21.15.22:0/3891066378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3488077b60 0x7f3488077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:38.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.071+0000 7f3487e00640 1 --2- 172.21.15.22:0/3891066378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3488076920 0x7f3488076d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:38.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.071+0000 7f3487e00640 1 -- 172.21.15.22:0/3891066378 >> 172.21.15.22:0/3891066378 conn(0x7f34880ff570 msgr2=0x7f34881019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:38.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.071+0000 7f3487e00640 1 -- 172.21.15.22:0/3891066378 shutdown_connections 2024-04-20T21:33:38.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.071+0000 7f3487e00640 1 -- 172.21.15.22:0/3891066378 wait complete. 2024-04-20T21:33:38.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.072+0000 7f3487e00640 1 Processor -- start 2024-04-20T21:33:38.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.072+0000 7f3487e00640 1 -- start start 2024-04-20T21:33:38.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f3487e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3488076920 0x7f348819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f3487e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3488077b60 0x7f348819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f3487e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3488078500 0x7f34881a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f3487e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34881140d0 con 0x7f3488077b60 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f3487e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3488113f50 con 0x7f3488076920 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f3487e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3488114250 con 0x7f3488078500 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f347e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3488077b60 0x7f348819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f347e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3488077b60 0x7f348819ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55564/0 (socket says 172.21.15.22:55564) 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f347e000640 1 -- 172.21.15.22:0/190671398 learned_addr learned my addr 172.21.15.22:0/190671398 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f3486a00640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3488076920 0x7f348819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f3487400640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3488078500 0x7f34881a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f347e000640 1 -- 172.21.15.22:0/190671398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3488078500 msgr2=0x7f34881a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f347e000640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3488078500 0x7f34881a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f347e000640 1 -- 172.21.15.22:0/190671398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3488076920 msgr2=0x7f348819e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f347e000640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3488076920 0x7f348819e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:38.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.073+0000 7f347e000640 1 -- 172.21.15.22:0/190671398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34881a62f0 con 0x7f3488077b60 2024-04-20T21:33:38.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.074+0000 7f347e000640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3488077b60 0x7f348819ec10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f347000eb90 tx=0x7f347000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:38.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.074+0000 7f347fe00640 1 -- 172.21.15.22:0/190671398 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470004d20 con 0x7f3488077b60 2024-04-20T21:33:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.074+0000 7f3487e00640 1 -- 172.21.15.22:0/190671398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34881a65e0 con 0x7f3488077b60 2024-04-20T21:33:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.074+0000 7f347fe00640 1 -- 172.21.15.22:0/190671398 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f347000ee90 con 0x7f3488077b60 2024-04-20T21:33:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.074+0000 7f347fe00640 1 -- 172.21.15.22:0/190671398 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470010830 con 0x7f3488077b60 2024-04-20T21:33:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.074+0000 7f3487e00640 1 -- 172.21.15.22:0/190671398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34881a6b30 con 0x7f3488077b60 2024-04-20T21:33:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.075+0000 7f3487e00640 1 -- 172.21.15.22:0/190671398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3488110970 con 0x7f3488077b60 2024-04-20T21:33:38.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.077+0000 7f347fe00640 1 -- 172.21.15.22:0/190671398 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3470005020 con 0x7f3488077b60 2024-04-20T21:33:38.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.079+0000 7f347fe00640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f346c0a3ba0 0x7f346c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:38.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.079+0000 7f347fe00640 1 -- 172.21.15.22:0/190671398 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7f34700d0b00 con 0x7f3488077b60 2024-04-20T21:33:38.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.079+0000 7f3486a00640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f346c0a3ba0 0x7f346c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:38.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.080+0000 7f3486a00640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f346c0a3ba0 0x7f346c0a6060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f347802c250 tx=0x7f347802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:38.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.083+0000 7f347fe00640 1 -- 172.21.15.22:0/190671398 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f347009d440 con 0x7f3488077b60 2024-04-20T21:33:38.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:38.347+0000 7f3487e00640 1 -- 172.21.15.22:0/190671398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f3488062b60 con 0x7f3488077b60 2024-04-20T21:33:38.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:38 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2108182840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:38 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2108182840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:38.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:38 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2108182840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:39.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.195+0000 7f347fe00640 1 -- 172.21.15.22:0/190671398 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f347009cd40 con 0x7f3488077b60 2024-04-20T21:33:39.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.199+0000 7f3487e00640 1 -- 172.21.15.22:0/190671398 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f346c0a3ba0 msgr2=0x7f346c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:39.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.199+0000 7f3487e00640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f346c0a3ba0 0x7f346c0a6060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f347802c250 tx=0x7f347802cf90 comp rx=0 tx=0).stop 2024-04-20T21:33:39.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.199+0000 7f3487e00640 1 -- 172.21.15.22:0/190671398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3488077b60 msgr2=0x7f348819ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:39.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.199+0000 7f3487e00640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3488077b60 0x7f348819ec10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f347000eb90 tx=0x7f347000c6a0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.200+0000 7f3487e00640 1 -- 172.21.15.22:0/190671398 shutdown_connections 2024-04-20T21:33:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.200+0000 7f3487e00640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f346c0a3ba0 0x7f346c0a6060 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.200+0000 7f3487e00640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3488078500 0x7f34881a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.200+0000 7f3487e00640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3488077b60 0x7f348819ec10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.200+0000 7f3487e00640 1 --2- 172.21.15.22:0/190671398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3488076920 0x7f348819e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.200+0000 7f3487e00640 1 -- 172.21.15.22:0/190671398 >> 172.21.15.22:0/190671398 conn(0x7f34880ff570 msgr2=0x7f3488101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.201+0000 7f3487e00640 1 -- 172.21.15.22:0/190671398 shutdown_connections 2024-04-20T21:33:39.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.201+0000 7f3487e00640 1 -- 172.21.15.22:0/190671398 wait complete. 2024-04-20T21:33:39.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-20T21:33:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.418+0000 7fba4ea00640 1 -- 172.21.15.22:0/709606155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba481041f0 msgr2=0x7fba48104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.418+0000 7fba4ea00640 1 --2- 172.21.15.22:0/709606155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba481041f0 0x7fba48104610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fba40009ca0 tx=0x7fba4002c0f0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.418+0000 7fba4ea00640 1 -- 172.21.15.22:0/709606155 shutdown_connections 2024-04-20T21:33:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.418+0000 7fba4ea00640 1 --2- 172.21.15.22:0/709606155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48105da0 0x7fba4810c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.418+0000 7fba4ea00640 1 --2- 172.21.15.22:0/709606155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba481053e0 0x7fba48105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.418+0000 7fba4ea00640 1 --2- 172.21.15.22:0/709606155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba481041f0 0x7fba48104610 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.418+0000 7fba4ea00640 1 -- 172.21.15.22:0/709606155 >> 172.21.15.22:0/709606155 conn(0x7fba480ff780 msgr2=0x7fba48101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:39.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.419+0000 7fba4ea00640 1 -- 172.21.15.22:0/709606155 shutdown_connections 2024-04-20T21:33:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.419+0000 7fba4ea00640 1 -- 172.21.15.22:0/709606155 wait complete. 2024-04-20T21:33:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.419+0000 7fba4ea00640 1 Processor -- start 2024-04-20T21:33:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.420+0000 7fba4ea00640 1 -- start start 2024-04-20T21:33:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.420+0000 7fba4ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba481041f0 0x7fba4819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.420+0000 7fba4ea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba481053e0 0x7fba4819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.420+0000 7fba4ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48105da0 0x7fba481a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.420+0000 7fba4ea00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba4810fe50 con 0x7fba481041f0 2024-04-20T21:33:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.420+0000 7fba4ea00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba4810fcd0 con 0x7fba481053e0 2024-04-20T21:33:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.420+0000 7fba4ea00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba4810ffd0 con 0x7fba48105da0 2024-04-20T21:33:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.420+0000 7fba47e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba481041f0 0x7fba4819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.420+0000 7fba47400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba481053e0 0x7fba4819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.420+0000 7fba47e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba481041f0 0x7fba4819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55608/0 (socket says 172.21.15.22:55608) 2024-04-20T21:33:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.420+0000 7fba47e00640 1 -- 172.21.15.22:0/1770171775 learned_addr learned my addr 172.21.15.22:0/1770171775 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.420+0000 7fba4ca00640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48105da0 0x7fba481a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.421+0000 7fba47400640 1 -- 172.21.15.22:0/1770171775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48105da0 msgr2=0x7fba481a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.421+0000 7fba47400640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48105da0 0x7fba481a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.421+0000 7fba47400640 1 -- 172.21.15.22:0/1770171775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba481041f0 msgr2=0x7fba4819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.421+0000 7fba47400640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba481041f0 0x7fba4819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.421+0000 7fba47400640 1 -- 172.21.15.22:0/1770171775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba481a2050 con 0x7fba481053e0 2024-04-20T21:33:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.421+0000 7fba47400640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba481053e0 0x7fba4819a970 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fba3000e9f0 tx=0x7fba3000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:39.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.421+0000 7fba44c00640 1 -- 172.21.15.22:0/1770171775 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3000ccc0 con 0x7fba481053e0 2024-04-20T21:33:39.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.421+0000 7fba44c00640 1 -- 172.21.15.22:0/1770171775 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba30010910 con 0x7fba481053e0 2024-04-20T21:33:39.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.421+0000 7fba44c00640 1 -- 172.21.15.22:0/1770171775 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30018960 con 0x7fba481053e0 2024-04-20T21:33:39.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.421+0000 7fba4ea00640 1 -- 172.21.15.22:0/1770171775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba481a2340 con 0x7fba481053e0 2024-04-20T21:33:39.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.421+0000 7fba4ea00640 1 -- 172.21.15.22:0/1770171775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba481a2840 con 0x7fba481053e0 2024-04-20T21:33:39.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.423+0000 7fba4ea00640 1 -- 172.21.15.22:0/1770171775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba4810b440 con 0x7fba481053e0 2024-04-20T21:33:39.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.424+0000 7fba44c00640 1 -- 172.21.15.22:0/1770171775 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fba30010ac0 con 0x7fba481053e0 2024-04-20T21:33:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.426+0000 7fba44c00640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d20 0x7fba2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.426+0000 7fba47e00640 1 -- 172.21.15.22:0/1770171775 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d20 msgr2=0x7fba2c0a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1911146506 2024-04-20T21:33:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.426+0000 7fba44c00640 1 -- 172.21.15.22:0/1770171775 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7fba300d0800 con 0x7fba481053e0 2024-04-20T21:33:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.426+0000 7fba47e00640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d20 0x7fba2c0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:33:39.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.430+0000 7fba44c00640 1 -- 172.21.15.22:0/1770171775 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fba3009caf0 con 0x7fba481053e0 2024-04-20T21:33:39.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:39 smithi022 ceph-mon[16843]: pgmap v386: 97 pgs: 97 active+clean; 637 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-20T21:33:39.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:39 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/190671398' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T21:33:39.522 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: ignoring --setuser ceph since I am not root 2024-04-20T21:33:39.522 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: ignoring --setgroup ceph since I am not root 2024-04-20T21:33:39.522 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:39.521+0000 7f2439076200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:33:39.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:39 smithi122 ceph-mon[22147]: pgmap v386: 97 pgs: 97 active+clean; 637 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-20T21:33:39.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:39 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/190671398' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T21:33:39.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:39 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: ignoring --setuser ceph since I am not root 2024-04-20T21:33:39.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:39 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: ignoring --setgroup ceph since I am not root 2024-04-20T21:33:39.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:39 smithi122 ceph-mgr[24024]: -- 172.21.15.122:0/232563049 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56009856e5a0 con 0x560098543000 2024-04-20T21:33:39.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:39 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:39.378+0000 7fe29b37d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:33:39.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:39 smithi033 ceph-mon[21626]: pgmap v386: 97 pgs: 97 active+clean; 637 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-20T21:33:39.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:39 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/190671398' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T21:33:39.553 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:39 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: ignoring --setuser ceph since I am not root 2024-04-20T21:33:39.553 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:39 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: ignoring --setgroup ceph since I am not root 2024-04-20T21:33:39.553 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:39 smithi033 ceph-mgr[23007]: -- 172.21.15.33:0/2257708731 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c5c5d25a0 con 0x564c5c5a6800 2024-04-20T21:33:39.554 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:39 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:39.384+0000 7f55e3a4d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:33:39.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.627+0000 7fba47e00640 1 -- 172.21.15.22:0/1770171775 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d20 msgr2=0x7fba2c0a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1911146506 2024-04-20T21:33:39.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.627+0000 7fba47e00640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d20 0x7fba2c0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:33:39.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.637+0000 7fba4ea00640 1 -- 172.21.15.22:0/1770171775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fba48062b90 con 0x7fba481053e0 2024-04-20T21:33:39.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.643+0000 7fba44c00640 1 -- 172.21.15.22:0/1770171775 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7fba300a0df0 con 0x7fba481053e0 2024-04-20T21:33:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.645+0000 7fba4ea00640 1 -- 172.21.15.22:0/1770171775 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d20 msgr2=0x7fba2c0a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:33:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.645+0000 7fba4ea00640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d20 0x7fba2c0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.645+0000 7fba4ea00640 1 -- 172.21.15.22:0/1770171775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba481053e0 msgr2=0x7fba4819a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.645+0000 7fba4ea00640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba481053e0 0x7fba4819a970 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fba3000e9f0 tx=0x7fba3000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.645+0000 7fba4ea00640 1 -- 172.21.15.22:0/1770171775 shutdown_connections 2024-04-20T21:33:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.645+0000 7fba4ea00640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7fba2c0a3d20 0x7fba2c0a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.645+0000 7fba4ea00640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba48105da0 0x7fba481a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.645+0000 7fba4ea00640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba481053e0 0x7fba4819a970 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.645+0000 7fba4ea00640 1 --2- 172.21.15.22:0/1770171775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba481041f0 0x7fba4819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.645+0000 7fba4ea00640 1 -- 172.21.15.22:0/1770171775 >> 172.21.15.22:0/1770171775 conn(0x7fba480ff780 msgr2=0x7fba48108610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.646+0000 7fba4ea00640 1 -- 172.21.15.22:0/1770171775 shutdown_connections 2024-04-20T21:33:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.646+0000 7fba4ea00640 1 -- 172.21.15.22:0/1770171775 wait complete. 2024-04-20T21:33:39.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-20T21:33:39.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.852+0000 7f87ae400640 1 -- 172.21.15.22:0/3645035645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a8103990 msgr2=0x7f87a810a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:39.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.852+0000 7f87ae400640 1 --2- 172.21.15.22:0/3645035645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a8103990 0x7f87a810a2b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009ca0 tx=0x7f87a002c100 comp rx=0 tx=0).stop 2024-04-20T21:33:39.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.852+0000 7f87ae400640 1 -- 172.21.15.22:0/3645035645 shutdown_connections 2024-04-20T21:33:39.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.852+0000 7f87ae400640 1 --2- 172.21.15.22:0/3645035645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a8103990 0x7f87a810a2b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.852+0000 7f87ae400640 1 --2- 172.21.15.22:0/3645035645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a8102fd0 0x7f87a8103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.852+0000 7f87ae400640 1 --2- 172.21.15.22:0/3645035645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8101e80 0x7f87a81022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.852+0000 7f87ae400640 1 -- 172.21.15.22:0/3645035645 >> 172.21.15.22:0/3645035645 conn(0x7f87a80fd3f0 msgr2=0x7f87a80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:39.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.852+0000 7f87ae400640 1 -- 172.21.15.22:0/3645035645 shutdown_connections 2024-04-20T21:33:39.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.852+0000 7f87ae400640 1 -- 172.21.15.22:0/3645035645 wait complete. 2024-04-20T21:33:39.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.853+0000 7f87ae400640 1 Processor -- start 2024-04-20T21:33:39.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.854+0000 7f87ae400640 1 -- start start 2024-04-20T21:33:39.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.854+0000 7f87ae400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a8101e80 0x7f87a8198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:39.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.854+0000 7f87ae400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a8102fd0 0x7f87a8198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:39.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.854+0000 7f87ae400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8103990 0x7f87a819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:39.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.854+0000 7f87ae400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810da90 con 0x7f87a8101e80 2024-04-20T21:33:39.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.854+0000 7f87ae400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810d910 con 0x7f87a8103990 2024-04-20T21:33:39.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.854+0000 7f87ae400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810dc10 con 0x7f87a8102fd0 2024-04-20T21:33:39.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.854+0000 7f87ad000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a8101e80 0x7f87a8198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:39.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.854+0000 7f87a7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a8102fd0 0x7f87a8198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:39.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.855+0000 7f87ada00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8103990 0x7f87a819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:39.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.855+0000 7f87a7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a8102fd0 0x7f87a8198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36032/0 (socket says 172.21.15.22:36032) 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.855+0000 7f87ada00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8103990 0x7f87a819f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:47512/0 (socket says 172.21.15.22:47512) 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.855+0000 7f87ad000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a8101e80 0x7f87a8198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55620/0 (socket says 172.21.15.22:55620) 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.855+0000 7f87a7e00640 1 -- 172.21.15.22:0/3095372432 learned_addr learned my addr 172.21.15.22:0/3095372432 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.855+0000 7f87ada00640 1 -- 172.21.15.22:0/3095372432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a8102fd0 msgr2=0x7f87a8198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.855+0000 7f87ada00640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a8102fd0 0x7f87a8198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.855+0000 7f87ada00640 1 -- 172.21.15.22:0/3095372432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a8101e80 msgr2=0x7f87a8198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.855+0000 7f87ada00640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a8101e80 0x7f87a8198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.855+0000 7f87ada00640 1 -- 172.21.15.22:0/3095372432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a819fc60 con 0x7f87a8103990 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.855+0000 7f87a7e00640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a8102fd0 0x7f87a8198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.855+0000 7f87ada00640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8103990 0x7f87a819f5f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f87a0037040 tx=0x7f87a0035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.856+0000 7f87a5600640 1 -- 172.21.15.22:0/3095372432 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0003970 con 0x7f87a8103990 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.856+0000 7f87a5600640 1 -- 172.21.15.22:0/3095372432 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87a0004040 con 0x7f87a8103990 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.856+0000 7f87ae400640 1 -- 172.21.15.22:0/3095372432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a8074190 con 0x7f87a8103990 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.856+0000 7f87a5600640 1 -- 172.21.15.22:0/3095372432 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a003c7e0 con 0x7f87a8103990 2024-04-20T21:33:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.856+0000 7f87ae400640 1 -- 172.21.15.22:0/3095372432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a80745f0 con 0x7f87a8103990 2024-04-20T21:33:39.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.857+0000 7f87ae400640 1 -- 172.21.15.22:0/3095372432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a8109130 con 0x7f87a8103990 2024-04-20T21:33:39.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.859+0000 7f87a5600640 1 -- 172.21.15.22:0/3095372432 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f87a004d060 con 0x7f87a8103990 2024-04-20T21:33:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.860+0000 7f87a5600640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f87900a3bf0 0x7f87900a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.860+0000 7f87ad000640 1 -- 172.21.15.22:0/3095372432 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f87900a3bf0 msgr2=0x7f87900a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1911146506 2024-04-20T21:33:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.860+0000 7f87a5600640 1 -- 172.21.15.22:0/3095372432 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7f87a00f6510 con 0x7f87a8103990 2024-04-20T21:33:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.860+0000 7f87ad000640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f87900a3bf0 0x7f87900a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:33:39.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:39.864+0000 7f87a5600640 1 -- 172.21.15.22:0/3095372432 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f87a0041070 con 0x7f87a8103990 2024-04-20T21:33:39.906 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:39.771+0000 7f2439076200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:33:39.970 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:39 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:39.562+0000 7fe29b37d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:33:39.970 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:39 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:39.664+0000 7fe29b37d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:33:39.999 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:39 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:39.576+0000 7f55e3a4d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:33:39.999 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:39 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:39.681+0000 7f55e3a4d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:33:39.999 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:40 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:40.000+0000 7f55e3a4d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:33:40.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.061+0000 7f87ad000640 1 -- 172.21.15.22:0/3095372432 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f87900a3bf0 msgr2=0x7f87900a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1911146506 2024-04-20T21:33:40.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.061+0000 7f87ad000640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f87900a3bf0 0x7f87900a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:33:40.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.068+0000 7f87ae400640 1 -- 172.21.15.22:0/3095372432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f87a802c450 con 0x7f87a8103990 2024-04-20T21:33:40.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.074+0000 7f87a5600640 1 -- 172.21.15.22:0/3095372432 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f87a00c2e50 con 0x7f87a8103990 2024-04-20T21:33:40.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.077+0000 7f87ae400640 1 -- 172.21.15.22:0/3095372432 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f87900a3bf0 msgr2=0x7f87900a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:33:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.077+0000 7f87ae400640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f87900a3bf0 0x7f87900a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.077+0000 7f87ae400640 1 -- 172.21.15.22:0/3095372432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8103990 msgr2=0x7f87a819f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.077+0000 7f87ae400640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8103990 0x7f87a819f5f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f87a0037040 tx=0x7f87a0035d10 comp rx=0 tx=0).stop 2024-04-20T21:33:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.077+0000 7f87ae400640 1 -- 172.21.15.22:0/3095372432 shutdown_connections 2024-04-20T21:33:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.077+0000 7f87ae400640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f87900a3bf0 0x7f87900a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.077+0000 7f87ae400640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87a8103990 0x7f87a819f5f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.077+0000 7f87ae400640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87a8102fd0 0x7f87a8198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.077+0000 7f87ae400640 1 --2- 172.21.15.22:0/3095372432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87a8101e80 0x7f87a8198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.077+0000 7f87ae400640 1 -- 172.21.15.22:0/3095372432 >> 172.21.15.22:0/3095372432 conn(0x7f87a80fd3f0 msgr2=0x7f87a8106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.077+0000 7f87ae400640 1 -- 172.21.15.22:0/3095372432 shutdown_connections 2024-04-20T21:33:40.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:40.078+0000 7f87ae400640 1 -- 172.21.15.22:0/3095372432 wait complete. 2024-04-20T21:33:40.093 INFO:teuthology.run_tasks:Running task kclient... 2024-04-20T21:33:40.117 INFO:tasks.kclient:Mounting kernel clients... 2024-04-20T21:33:40.118 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-20T21:33:40.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:40.118 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-04-20T21:33:40.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:40.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-04-20T21:33:40.195 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:39.909+0000 7f2439076200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:33:40.245 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2024-04-20T21:33:40.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:33:40.249 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-20T21:33:40.249 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T21:33:40.249 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:40.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph fs ls 2024-04-20T21:33:40.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:40 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/190671398' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-20T21:33:40.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:40 smithi122 ceph-mon[22147]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-20T21:33:40.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:40 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' 2024-04-20T21:33:40.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:40 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' 2024-04-20T21:33:40.275 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:39 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:39.971+0000 7fe29b37d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:33:40.275 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:40 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:40.055+0000 7fe29b37d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:33:40.303 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:40 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:40.089+0000 7f55e3a4d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:33:40.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:40 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/190671398' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-20T21:33:40.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:40 smithi033 ceph-mon[21626]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-20T21:33:40.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:40 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' 2024-04-20T21:33:40.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:40 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' 2024-04-20T21:33:40.446 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:40 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/190671398' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-20T21:33:40.446 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:40 smithi022 ceph-mon[16843]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-20T21:33:40.446 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:40 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' 2024-04-20T21:33:40.447 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:40 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' 2024-04-20T21:33:40.447 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:40 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:40.326+0000 7f2439076200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:33:40.756 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:33:40.906 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:40 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:40.446+0000 7f2439076200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:33:41.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:41.224+0000 7fe29b37d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:33:41.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:41.393+0000 7fe29b37d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:33:41.685 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:41 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:41.334+0000 7f55e3a4d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:33:41.685 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:41 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:41.510+0000 7f55e3a4d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:33:41.844 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:41.561+0000 7fe29b37d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:33:41.984 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:41 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:41.686+0000 7f55e3a4d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:33:42.135 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:41.844+0000 7fe29b37d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:33:42.135 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:41.936+0000 7fe29b37d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:33:42.135 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:42 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:42.135+0000 7fe29b37d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:33:42.289 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:41 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:41.985+0000 7f55e3a4d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:33:42.289 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:42 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:42.081+0000 7f55e3a4d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:33:42.359 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:42 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:42.098+0000 7f2439076200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:33:42.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:42 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:42.327+0000 7fe29b37d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:33:42.554 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:42 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:42.290+0000 7f55e3a4d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:33:42.555 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:42 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:42.491+0000 7f55e3a4d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:33:42.621 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:42 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:42.359+0000 7f2439076200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:33:42.621 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:42 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:42.602+0000 7f2439076200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:33:42.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.892+0000 7f4891400700 1 -- 172.21.15.22:0/2422356834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488c076f80 msgr2=0x7f488c077360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:42.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.892+0000 7f4891400700 1 --2- 172.21.15.22:0/2422356834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488c076f80 0x7f488c077360 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f487c00b0a0 tx=0x7f487c00b3b0 comp rx=0 tx=0).stop 2024-04-20T21:33:42.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.892+0000 7f4891400700 1 -- 172.21.15.22:0/2422356834 shutdown_connections 2024-04-20T21:33:42.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.892+0000 7f4891400700 1 --2- 172.21.15.22:0/2422356834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f488c113800 0x7f488c115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:42.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.892+0000 7f4891400700 1 --2- 172.21.15.22:0/2422356834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488c076f80 0x7f488c077360 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:42.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.892+0000 7f4891400700 1 --2- 172.21.15.22:0/2422356834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488c0778a0 0x7f488c1132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:42.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.892+0000 7f4891400700 1 -- 172.21.15.22:0/2422356834 >> 172.21.15.22:0/2422356834 conn(0x7f488c100630 msgr2=0x7f488c102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:42.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.892+0000 7f4891400700 1 -- 172.21.15.22:0/2422356834 shutdown_connections 2024-04-20T21:33:42.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.892+0000 7f4891400700 1 -- 172.21.15.22:0/2422356834 wait complete. 2024-04-20T21:33:42.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.893+0000 7f4891400700 1 Processor -- start 2024-04-20T21:33:42.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.894+0000 7f4891400700 1 -- start start 2024-04-20T21:33:42.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.894+0000 7f4891400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488c076f80 0x7f488c1a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:42.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.894+0000 7f4891400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f488c0778a0 0x7f488c1a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:42.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.894+0000 7f4891400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488c113800 0x7f488c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:42.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.894+0000 7f4891400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f488c119330 con 0x7f488c076f80 2024-04-20T21:33:42.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.894+0000 7f4891400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f488c1191b0 con 0x7f488c0778a0 2024-04-20T21:33:42.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.894+0000 7f4891400700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f488c1194b0 con 0x7f488c113800 2024-04-20T21:33:42.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.894+0000 7f488b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f488c0778a0 0x7f488c1a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:42.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.895+0000 7f488b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f488c0778a0 0x7f488c1a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39778/0 (socket says 172.21.15.22:39778) 2024-04-20T21:33:42.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.895+0000 7f488b400700 1 -- 172.21.15.22:0/1485695310 learned_addr learned my addr 172.21.15.22:0/1485695310 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:42.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.895+0000 7f4890a00700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488c113800 0x7f488c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:42.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.895+0000 7f488be00700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488c076f80 0x7f488c1a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:42.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.895+0000 7f488b400700 1 -- 172.21.15.22:0/1485695310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488c113800 msgr2=0x7f488c19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:42.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.895+0000 7f488b400700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488c113800 0x7f488c19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:42.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.895+0000 7f488b400700 1 -- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488c076f80 msgr2=0x7f488c1a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:42.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.895+0000 7f488b400700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488c076f80 0x7f488c1a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:42.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.895+0000 7f488b400700 1 -- 172.21.15.22:0/1485695310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c19fd90 con 0x7f488c0778a0 2024-04-20T21:33:42.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.895+0000 7f488be00700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488c076f80 0x7f488c1a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:42.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.895+0000 7f488b400700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f488c0778a0 0x7f488c1a5910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f487800e9a0 tx=0x7f487800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:42.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.895+0000 7f4888c00700 1 -- 172.21.15.22:0/1485695310 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487800cbd0 con 0x7f488c0778a0 2024-04-20T21:33:42.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.896+0000 7f4888c00700 1 -- 172.21.15.22:0/1485695310 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487800cd70 con 0x7f488c0778a0 2024-04-20T21:33:42.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.896+0000 7f4891400700 1 -- 172.21.15.22:0/1485695310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488c1a0080 con 0x7f488c0778a0 2024-04-20T21:33:42.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.896+0000 7f4888c00700 1 -- 172.21.15.22:0/1485695310 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878018750 con 0x7f488c0778a0 2024-04-20T21:33:42.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.896+0000 7f4891400700 1 -- 172.21.15.22:0/1485695310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f488c1ac2a0 con 0x7f488c0778a0 2024-04-20T21:33:42.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.897+0000 7f4891400700 1 -- 172.21.15.22:0/1485695310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f488c069740 con 0x7f488c0778a0 2024-04-20T21:33:42.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.900+0000 7f4888c00700 1 -- 172.21.15.22:0/1485695310 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f4878010430 con 0x7f488c0778a0 2024-04-20T21:33:42.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.902+0000 7f4888c00700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f48700a3d10 0x7f48700a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:42.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.902+0000 7f488be00700 1 -- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f48700a3d10 msgr2=0x7f48700a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1911146506 2024-04-20T21:33:42.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.902+0000 7f4888c00700 1 -- 172.21.15.22:0/1485695310 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7f48780d0770 con 0x7f488c0778a0 2024-04-20T21:33:42.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.902+0000 7f488be00700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f48700a3d10 0x7f48700a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:33:42.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:42.903+0000 7f4888c00700 1 -- 172.21.15.22:0/1485695310 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f487809d0b0 con 0x7f488c0778a0 2024-04-20T21:33:43.016 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:43 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:43.016+0000 7f2439076200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:33:43.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.103+0000 7f488be00700 1 -- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f48700a3d10 msgr2=0x7f48700a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1911146506 2024-04-20T21:33:43.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.103+0000 7f488be00700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f48700a3d10 0x7f48700a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:33:43.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.182+0000 7f4891400700 1 -- 172.21.15.22:0/1485695310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f488c1a0ac0 con 0x7f488c0778a0 2024-04-20T21:33:43.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.182+0000 7f4888c00700 1 -- 172.21.15.22:0/1485695310 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f487809c9b0 con 0x7f488c0778a0 2024-04-20T21:33:43.182 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:33:43.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.185+0000 7f4891400700 1 -- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f48700a3d10 msgr2=0x7f48700a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:33:43.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.185+0000 7f4891400700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f48700a3d10 0x7f48700a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:43.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.185+0000 7f4891400700 1 -- 172.21.15.22:0/1485695310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f488c0778a0 msgr2=0x7f488c1a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:43.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.185+0000 7f4891400700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f488c0778a0 0x7f488c1a5910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f487800e9a0 tx=0x7f487800ecb0 comp rx=0 tx=0).stop 2024-04-20T21:33:43.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.185+0000 7f4891400700 1 -- 172.21.15.22:0/1485695310 shutdown_connections 2024-04-20T21:33:43.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.185+0000 7f4891400700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f48700a3d10 0x7f48700a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:43.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.185+0000 7f4891400700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488c113800 0x7f488c19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:43.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.185+0000 7f4891400700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f488c076f80 0x7f488c1a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:43.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.185+0000 7f4891400700 1 --2- 172.21.15.22:0/1485695310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f488c0778a0 0x7f488c1a5910 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:43.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.185+0000 7f4891400700 1 -- 172.21.15.22:0/1485695310 >> 172.21.15.22:0/1485695310 conn(0x7f488c100630 msgr2=0x7f488c102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:43.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.186+0000 7f4891400700 1 -- 172.21.15.22:0/1485695310 shutdown_connections 2024-04-20T21:33:43.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:43.186+0000 7f4891400700 1 -- 172.21.15.22:0/1485695310 wait complete. 2024-04-20T21:33:43.234 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:42 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:42.920+0000 7fe29b37d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:33:43.407 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:43 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:43.148+0000 7f2439076200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:33:43.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:43 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1485695310' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T21:33:43.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:43 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1485695310' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T21:33:43.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:43 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:43.339+0000 7fe29b37d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:33:43.525 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:43 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:43.434+0000 7fe29b37d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:33:43.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:43 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1485695310' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T21:33:43.554 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:43 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:43.120+0000 7f55e3a4d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:33:43.694 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:43 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:43.421+0000 7f2439076200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:33:43.870 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:43 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:43.562+0000 7f55e3a4d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:33:43.870 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:43 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:43.662+0000 7f55e3a4d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:33:43.912 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:33:43.912 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:33:43.912 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-04-20T21:33:43.912 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:33:43.912 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:43.912 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:33:43.912 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:33:43.913 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:33:43.913 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:33:43.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:43.913 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-04-20T21:33:43.964 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:33:43.964 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:33:43.964 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:33:43.964 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:33:43.964 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-04-20T21:33:43.964 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 41851sec preferred_lft 41851sec 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link noprefixroute 2024-04-20T21:33:43.965 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:33:43.966 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:33:43.966 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:33:43.966 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-04-20T21:33:43.966 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T21:33:43.966 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:33:43.966 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-20T21:33:43.966 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2024-04-20T21:33:43.967 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2024-04-20T21:33:43.967 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2024-04-20T21:33:43.967 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T21:33:43.967 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-20T21:33:44.024 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:43 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:43.638+0000 7fe29b37d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:33:44.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:33:44.088 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:43 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:43.694+0000 7f2439076200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:33:44.303 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:43 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:43.871+0000 7f55e3a4d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:33:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:44.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:44.457 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:33:44.490 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:44.156+0000 7fe29b37d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:33:44.490 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:44.248+0000 7fe29b37d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:33:44.521 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-04-20T21:33:44.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:44.525 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-04-20T21:33:44.580 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-04-20T21:33:44.581 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:33:44.581 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-20T21:33:44.581 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-20T21:33:44.581 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:33:44.581 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:33:44.581 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-20T21:33:44.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-20T21:33:44.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-20T21:33:44.582 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-20T21:33:44.582 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-20T21:33:44.656 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:44 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:44.541+0000 7f2439076200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:33:44.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:33:44.761 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:44 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:44.414+0000 7f55e3a4d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:33:44.761 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:44 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:44.510+0000 7f55e3a4d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:33:44.774 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:44.491+0000 7fe29b37d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:33:44.775 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:44.622+0000 7fe29b37d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:33:44.775 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:44.716+0000 7fe29b37d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:33:45.053 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:44 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:44.762+0000 7f55e3a4d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:33:45.053 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:44 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:44.900+0000 7f55e3a4d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:33:45.053 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:44.999+0000 7f55e3a4d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:33:45.127 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:44.830+0000 7fe29b37d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:33:45.267 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:45 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:45.131+0000 7f2439076200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:33:45.267 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:45 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:45.267+0000 7f2439076200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:33:45.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:45.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:45.294 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-04-20T21:33:45.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:45.344 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-04-20T21:33:45.388 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:33:45.388 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-20T21:33:45.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:33:45.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-20T21:33:45.389 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-20T21:33:45.430 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:45.119+0000 7f55e3a4d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:33:45.524 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:45 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:45.128+0000 7fe29b37d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:33:45.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:33:45.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:45.605 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-20T21:33:45.606 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:33:45.606 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-20T21:33:45.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-20T21:33:45.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T21:33:45.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-20T21:33:45.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-20T21:33:45.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-20T21:33:45.606 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-20T21:33:45.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:33:45.807 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:45.431+0000 7f55e3a4d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:33:45.809 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:45 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:45.551+0000 7f2439076200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:33:45.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:45.907 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:33:45.907 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-04-20T21:33:45.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-04-20T21:33:45.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T21:33:45.907 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-04-20T21:33:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:33:46.018 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:45 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:45.728+0000 7fe29b37d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:33:46.018 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:33:45 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z[24020]: 2024-04-20T21:33:45.945+0000 7fe29b37d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:33:46.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:46 smithi022 ceph-mon[16843]: Standby manager daemon z restarted 2024-04-20T21:33:46.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:46 smithi022 ceph-mon[16843]: Standby manager daemon z started 2024-04-20T21:33:46.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:46 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.122:0/78753322' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T21:33:46.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:46 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.122:0/78753322' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:33:46.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:46 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.122:0/78753322' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T21:33:46.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:46 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.122:0/78753322' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:33:46.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:46.101 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:33:46.101 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:46.138 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:33:46.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:33:46.139 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:46.232 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-20T21:33:46.232 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-20T21:33:46.232 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/0bd71eb4-0894-49b6-ab83-fa3698d7b690 2024-04-20T21:33:46.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:46.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/0bd71eb4-0894-49b6-ab83-fa3698d7b690 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-20T21:33:46.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:46 smithi122 ceph-mon[22147]: Standby manager daemon z restarted 2024-04-20T21:33:46.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:46 smithi122 ceph-mon[22147]: Standby manager daemon z started 2024-04-20T21:33:46.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:46 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.122:0/78753322' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T21:33:46.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:46 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.122:0/78753322' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:33:46.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:46 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.122:0/78753322' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T21:33:46.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:46 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.122:0/78753322' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:33:46.292 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:46 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:46.068+0000 7f55e3a4d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:33:46.292 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:46 smithi033 ceph-mon[21626]: Standby manager daemon z restarted 2024-04-20T21:33:46.292 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:46 smithi033 ceph-mon[21626]: Standby manager daemon z started 2024-04-20T21:33:46.292 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:46 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.122:0/78753322' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T21:33:46.292 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:46 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.122:0/78753322' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:33:46.292 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:46 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.122:0/78753322' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T21:33:46.292 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:46 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.122:0/78753322' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:33:46.363 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-20T21:33:46.364 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-20T21:33:46.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:46 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:46.283+0000 7f2439076200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:33:46.553 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:33:46 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y[23003]: 2024-04-20T21:33:46.293+0000 7f55e3a4d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:33:46.770 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:46 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:46.422+0000 7f2439076200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:33:47.048 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:47 smithi022 ceph-mon[16843]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-20T21:33:47.048 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:47 smithi022 ceph-mon[16843]: Standby manager daemon y restarted 2024-04-20T21:33:47.048 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:47 smithi022 ceph-mon[16843]: Standby manager daemon y started 2024-04-20T21:33:47.048 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:47 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.33:0/1762133496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T21:33:47.048 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:47 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.33:0/1762133496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:33:47.048 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:47 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.33:0/1762133496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T21:33:47.049 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:47 smithi022 ceph-mon[16843]: from='mgr.? 172.21.15.33:0/1762133496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:33:47.049 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:46 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:46.787+0000 7f2439076200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:33:47.049 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:46 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:46.976+0000 7f2439076200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:33:47.055 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-20T21:33:47.055 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-20T21:33:47.055 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.33:3300,172.21.15.122:3300" 2024-04-20T21:33:47.055 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with new device syntax: 0@92c49c5c-ff5b-11ee-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/0bd71eb4-0894-49b6-ab83-fa3698d7b690 2024-04-20T21:33:47.055 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.22:3300/172.21.15.33:3300/172.21.15.122:3300" will pass to kernel 2024-04-20T21:33:47.058 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-20T21:33:47.059 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-04-20T21:33:47.059 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-20T21:33:47.059 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-04-20T21:33:47.059 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-20T21:33:47.062 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-20T21:33:47.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:47.062 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:47.132 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T21:33:47.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T21:33:47.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:47.243 INFO:teuthology.orchestra.run.smithi022.stdout:client15045 2024-04-20T21:33:47.244 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:33:47.244 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:33:47.291 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T21:33:47.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:47 smithi033 ceph-mon[21626]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-20T21:33:47.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:47 smithi033 ceph-mon[21626]: Standby manager daemon y restarted 2024-04-20T21:33:47.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:47 smithi033 ceph-mon[21626]: Standby manager daemon y started 2024-04-20T21:33:47.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:47 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.33:0/1762133496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T21:33:47.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:47 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.33:0/1762133496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:33:47.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:47 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.33:0/1762133496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T21:33:47.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:47 smithi033 ceph-mon[21626]: from='mgr.? 172.21.15.33:0/1762133496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:33:47.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:47 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:47.116+0000 7f2439076200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:33:47.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:47 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:47.279+0000 7f2439076200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:33:47.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T21:33:47.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:47.505 INFO:teuthology.orchestra.run.smithi022.stdout:client15045 2024-04-20T21:33:47.505 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T21:33:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:47 smithi122 ceph-mon[22147]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-20T21:33:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:47 smithi122 ceph-mon[22147]: Standby manager daemon y restarted 2024-04-20T21:33:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:47 smithi122 ceph-mon[22147]: Standby manager daemon y started 2024-04-20T21:33:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:47 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.33:0/1762133496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T21:33:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:47 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.33:0/1762133496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:33:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:47 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.33:0/1762133496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T21:33:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:47 smithi122 ceph-mon[22147]: from='mgr.? 172.21.15.33:0/1762133496' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:33:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T21:33:47.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:47.630 INFO:teuthology.orchestra.run.smithi022.stdout:client15045 2024-04-20T21:33:47.631 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968.client15045/status') 2024-04-20T21:33:47.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:33:47.766 INFO:teuthology.orchestra.run.smithi022.stdout:instance: client.15045 (3)192.168.0.1:0/1247955792 2024-04-20T21:33:47.766 INFO:teuthology.orchestra.run.smithi022.stdout:blocklisted: false 2024-04-20T21:33:47.767 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records in 2024-04-20T21:33:47.767 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records out 2024-04-20T21:33:47.767 INFO:teuthology.orchestra.run.smithi022.stderr:70 bytes copied, 8.4229e-05 s, 831 kB/s 2024-04-20T21:33:47.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:47.778 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-20T21:33:47.790 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-20T21:33:47.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:48.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.001+0000 7ff543a00640 1 -- 172.21.15.22:0/1576351227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff53c1041c0 msgr2=0x7ff53c1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:48.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.001+0000 7ff543a00640 1 --2- 172.21.15.22:0/1576351227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff53c1041c0 0x7ff53c1045e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff530009860 tx=0x7ff53002bb70 comp rx=0 tx=0).stop 2024-04-20T21:33:48.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.001+0000 7ff543a00640 1 -- 172.21.15.22:0/1576351227 shutdown_connections 2024-04-20T21:33:48.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.001+0000 7ff543a00640 1 --2- 172.21.15.22:0/1576351227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff53c105d70 0x7ff53c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.001+0000 7ff543a00640 1 --2- 172.21.15.22:0/1576351227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff53c1053b0 0x7ff53c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.001+0000 7ff543a00640 1 --2- 172.21.15.22:0/1576351227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff53c1041c0 0x7ff53c1045e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.001+0000 7ff543a00640 1 -- 172.21.15.22:0/1576351227 >> 172.21.15.22:0/1576351227 conn(0x7ff53c0ff750 msgr2=0x7ff53c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:48.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.001+0000 7ff543a00640 1 -- 172.21.15.22:0/1576351227 shutdown_connections 2024-04-20T21:33:48.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.001+0000 7ff543a00640 1 -- 172.21.15.22:0/1576351227 wait complete. 2024-04-20T21:33:48.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.003+0000 7ff543a00640 1 Processor -- start 2024-04-20T21:33:48.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.003+0000 7ff543a00640 1 -- start start 2024-04-20T21:33:48.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.003+0000 7ff543a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff53c1041c0 0x7ff53c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:48.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff543a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff53c1053b0 0x7ff53c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:48.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff543a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff53c105d70 0x7ff53c1a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:48.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff543a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c10fe50 con 0x7ff53c1041c0 2024-04-20T21:33:48.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff543a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c10fcd0 con 0x7ff53c105d70 2024-04-20T21:33:48.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff543a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c10ffd0 con 0x7ff53c1053b0 2024-04-20T21:33:48.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff541000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff53c1041c0 0x7ff53c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:48.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff53be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff53c1053b0 0x7ff53c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:48.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff53be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff53c1053b0 0x7ff53c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:59538/0 (socket says 172.21.15.22:59538) 2024-04-20T21:33:48.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff53be00640 1 -- 172.21.15.22:0/2606653755 learned_addr learned my addr 172.21.15.22:0/2606653755 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:48.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff541a00640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff53c105d70 0x7ff53c1a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:48.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff53be00640 1 -- 172.21.15.22:0/2606653755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff53c105d70 msgr2=0x7ff53c1a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:48.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff53be00640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff53c105d70 0x7ff53c1a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff53be00640 1 -- 172.21.15.22:0/2606653755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff53c1041c0 msgr2=0x7ff53c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:48.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff53be00640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff53c1041c0 0x7ff53c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff53be00640 1 -- 172.21.15.22:0/2606653755 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53c1a2100 con 0x7ff53c1053b0 2024-04-20T21:33:48.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.004+0000 7ff541000640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff53c1041c0 0x7ff53c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:48.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.005+0000 7ff53be00640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff53c1053b0 0x7ff53c19aa20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff52800e9c0 tx=0x7ff52800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:48.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.005+0000 7ff539600640 1 -- 172.21.15.22:0/2606653755 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52800ccc0 con 0x7ff53c1053b0 2024-04-20T21:33:48.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.005+0000 7ff543a00640 1 -- 172.21.15.22:0/2606653755 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53c1a23f0 con 0x7ff53c1053b0 2024-04-20T21:33:48.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.005+0000 7ff539600640 1 -- 172.21.15.22:0/2606653755 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff528005500 con 0x7ff53c1053b0 2024-04-20T21:33:48.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.005+0000 7ff543a00640 1 -- 172.21.15.22:0/2606653755 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53c1a2990 con 0x7ff53c1053b0 2024-04-20T21:33:48.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.005+0000 7ff539600640 1 -- 172.21.15.22:0/2606653755 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528010430 con 0x7ff53c1053b0 2024-04-20T21:33:48.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.006+0000 7ff543a00640 1 -- 172.21.15.22:0/2606653755 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff53c10b530 con 0x7ff53c1053b0 2024-04-20T21:33:48.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.009+0000 7ff539600640 1 -- 172.21.15.22:0/2606653755 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 21) ==== 138663+0+0 (secure 0 0 0) 0x7ff5280046c0 con 0x7ff53c1053b0 2024-04-20T21:33:48.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.012+0000 7ff539600640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5240a3c80 0x7ff5240a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:48.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.012+0000 7ff541000640 1 -- 172.21.15.22:0/2606653755 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5240a3c80 msgr2=0x7ff5240a6140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1911146506 2024-04-20T21:33:48.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.012+0000 7ff539600640 1 -- 172.21.15.22:0/2606653755 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7ff5280d08d0 con 0x7ff53c1053b0 2024-04-20T21:33:48.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.012+0000 7ff541000640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5240a3c80 0x7ff5240a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:33:48.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.012+0000 7ff539600640 1 -- 172.21.15.22:0/2606653755 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff528053ff0 con 0x7ff53c1053b0 2024-04-20T21:33:48.033 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:47 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:47.669+0000 7f2439076200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:33:48.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.213+0000 7ff541000640 1 -- 172.21.15.22:0/2606653755 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5240a3c80 msgr2=0x7ff5240a6140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1911146506 2024-04-20T21:33:48.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.213+0000 7ff541000640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5240a3c80 0x7ff5240a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:33:48.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.219+0000 7ff543a00640 1 -- 172.21.15.22:0/2606653755 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff53c02d7d0 con 0x7ff53c1053b0 2024-04-20T21:33:48.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.221+0000 7ff539600640 1 -- 172.21.15.22:0/2606653755 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22768 (secure 0 0 0) 0x7ff52809d210 con 0x7ff53c1053b0 2024-04-20T21:33:48.221 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:48.221 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:33:32.135126+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:48.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.224+0000 7ff543a00640 1 -- 172.21.15.22:0/2606653755 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5240a3c80 msgr2=0x7ff5240a6140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:33:48.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.224+0000 7ff543a00640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5240a3c80 0x7ff5240a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.224+0000 7ff543a00640 1 -- 172.21.15.22:0/2606653755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff53c1053b0 msgr2=0x7ff53c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:48.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.224+0000 7ff543a00640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff53c1053b0 0x7ff53c19aa20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff52800e9c0 tx=0x7ff52800ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:48.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.224+0000 7ff543a00640 1 -- 172.21.15.22:0/2606653755 shutdown_connections 2024-04-20T21:33:48.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.224+0000 7ff543a00640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7ff5240a3c80 0x7ff5240a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.224+0000 7ff543a00640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff53c105d70 0x7ff53c1a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.224+0000 7ff543a00640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff53c1053b0 0x7ff53c19aa20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.224+0000 7ff543a00640 1 --2- 172.21.15.22:0/2606653755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff53c1041c0 0x7ff53c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.224+0000 7ff543a00640 1 -- 172.21.15.22:0/2606653755 >> 172.21.15.22:0/2606653755 conn(0x7ff53c0ff750 msgr2=0x7ff53c108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:48.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.224+0000 7ff543a00640 1 -- 172.21.15.22:0/2606653755 shutdown_connections 2024-04-20T21:33:48.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.224+0000 7ff543a00640 1 -- 172.21.15.22:0/2606653755 wait complete. 2024-04-20T21:33:48.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:26:15.688818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:31:25.506074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:31:26.126261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:33:48.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:33:48.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:48 smithi033 ceph-mon[21626]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-04-20T21:33:48.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:48 smithi022 ceph-mon[16843]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-04-20T21:33:48.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.459+0000 7f6025800640 1 -- 172.21.15.22:0/3746107030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6020101e70 msgr2=0x7f60201022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:48.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.459+0000 7f6025800640 1 --2- 172.21.15.22:0/3746107030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6020101e70 0x7f60201022f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f600c009900 tx=0x7f600c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:48.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.459+0000 7f6025800640 1 -- 172.21.15.22:0/3746107030 shutdown_connections 2024-04-20T21:33:48.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.459+0000 7f6025800640 1 --2- 172.21.15.22:0/3746107030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6020102830 0x7f60201090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.459+0000 7f6025800640 1 --2- 172.21.15.22:0/3746107030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6020101e70 0x7f60201022f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.459+0000 7f6025800640 1 --2- 172.21.15.22:0/3746107030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6020100c80 0x7f60201010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.459+0000 7f6025800640 1 -- 172.21.15.22:0/3746107030 >> 172.21.15.22:0/3746107030 conn(0x7f60200fc230 msgr2=0x7f60200fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:48.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.459+0000 7f6025800640 1 -- 172.21.15.22:0/3746107030 shutdown_connections 2024-04-20T21:33:48.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.459+0000 7f6025800640 1 -- 172.21.15.22:0/3746107030 wait complete. 2024-04-20T21:33:48.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.460+0000 7f6025800640 1 Processor -- start 2024-04-20T21:33:48.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.460+0000 7f6025800640 1 -- start start 2024-04-20T21:33:48.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.460+0000 7f6025800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6020100c80 0x7f602019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:48.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.460+0000 7f6025800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6020101e70 0x7f602019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:48.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.460+0000 7f6025800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6020102830 0x7f60201a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:48.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.460+0000 7f6025800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f602010da80 con 0x7f6020102830 2024-04-20T21:33:48.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.460+0000 7f6025800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f602010d900 con 0x7f6020101e70 2024-04-20T21:33:48.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.460+0000 7f6025800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f602010dc00 con 0x7f6020100c80 2024-04-20T21:33:48.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.461+0000 7f601fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6020100c80 0x7f602019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:48.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.461+0000 7f601fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6020100c80 0x7f602019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:59550/0 (socket says 172.21.15.22:59550) 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.461+0000 7f601fe00640 1 -- 172.21.15.22:0/2342886945 learned_addr learned my addr 172.21.15.22:0/2342886945 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.461+0000 7f6024e00640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6020102830 0x7f60201a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.461+0000 7f601f400640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6020101e70 0x7f602019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.461+0000 7f601fe00640 1 -- 172.21.15.22:0/2342886945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6020101e70 msgr2=0x7f602019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.461+0000 7f601fe00640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6020101e70 0x7f602019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.461+0000 7f601fe00640 1 -- 172.21.15.22:0/2342886945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6020102830 msgr2=0x7f60201a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.461+0000 7f601fe00640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6020102830 0x7f60201a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.461+0000 7f601fe00640 1 -- 172.21.15.22:0/2342886945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60201a1e80 con 0x7f6020100c80 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.462+0000 7f601fe00640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6020100c80 0x7f602019a260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f601000e9c0 tx=0x7f601000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.462+0000 7f601cc00640 1 -- 172.21.15.22:0/2342886945 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601000cd80 con 0x7f6020100c80 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.462+0000 7f601cc00640 1 -- 172.21.15.22:0/2342886945 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60100035f0 con 0x7f6020100c80 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.462+0000 7f6025800640 1 -- 172.21.15.22:0/2342886945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60201a2170 con 0x7f6020100c80 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.462+0000 7f601cc00640 1 -- 172.21.15.22:0/2342886945 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010010690 con 0x7f6020100c80 2024-04-20T21:33:48.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.462+0000 7f6025800640 1 -- 172.21.15.22:0/2342886945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60201a2690 con 0x7f6020100c80 2024-04-20T21:33:48.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.463+0000 7f6025800640 1 -- 172.21.15.22:0/2342886945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60201a2950 con 0x7f6020100c80 2024-04-20T21:33:48.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.466+0000 7f601cc00640 1 -- 172.21.15.22:0/2342886945 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 21) ==== 138663+0+0 (secure 0 0 0) 0x7f60100037a0 con 0x7f6020100c80 2024-04-20T21:33:48.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.468+0000 7f601cc00640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f60040a3c00 0x7f60040a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:48.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.468+0000 7f601cc00640 1 -- 172.21.15.22:0/2342886945 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8298+0+0 (secure 0 0 0) 0x7f60100d09a0 con 0x7f6020100c80 2024-04-20T21:33:48.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.468+0000 7f601f400640 1 -- 172.21.15.22:0/2342886945 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f60040a3c00 msgr2=0x7f60040a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1911146506 2024-04-20T21:33:48.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.468+0000 7f601f400640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f60040a3c00 0x7f60040a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:33:48.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.470+0000 7f601cc00640 1 -- 172.21.15.22:0/2342886945 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f601009d370 con 0x7f6020100c80 2024-04-20T21:33:48.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:48 smithi122 ceph-mon[22147]: mgrmap e21: x(active, since 10m), standbys: z, y 2024-04-20T21:33:48.656 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:48 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:48.509+0000 7f2439076200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:33:48.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.669+0000 7f601f400640 1 -- 172.21.15.22:0/2342886945 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f60040a3c00 msgr2=0x7f60040a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1911146506 2024-04-20T21:33:48.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.669+0000 7f601f400640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f60040a3c00 0x7f60040a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:33:48.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.671+0000 7f6025800640 1 -- 172.21.15.22:0/2342886945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f60201a2b20 con 0x7f6020100c80 2024-04-20T21:33:48.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.672+0000 7f601cc00640 1 -- 172.21.15.22:0/2342886945 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f60100a3020 con 0x7f6020100c80 2024-04-20T21:33:48.671 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-20T21:33:48.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.674+0000 7f6025800640 1 -- 172.21.15.22:0/2342886945 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f60040a3c00 msgr2=0x7f60040a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:33:48.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.674+0000 7f6025800640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f60040a3c00 0x7f60040a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.674+0000 7f6025800640 1 -- 172.21.15.22:0/2342886945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6020100c80 msgr2=0x7f602019a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:48.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.674+0000 7f6025800640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6020100c80 0x7f602019a260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f601000e9c0 tx=0x7f601000ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:48.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.674+0000 7f6025800640 1 -- 172.21.15.22:0/2342886945 shutdown_connections 2024-04-20T21:33:48.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.674+0000 7f6025800640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.22:6800/1911146506,v1:172.21.15.22:6801/1911146506] conn(0x7f60040a3c00 0x7f60040a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.674+0000 7f6025800640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6020102830 0x7f60201a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.674+0000 7f6025800640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6020101e70 0x7f602019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.674+0000 7f6025800640 1 --2- 172.21.15.22:0/2342886945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6020100c80 0x7f602019a260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.674+0000 7f6025800640 1 -- 172.21.15.22:0/2342886945 >> 172.21.15.22:0/2342886945 conn(0x7f60200fc230 msgr2=0x7f6020109a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:48.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.674+0000 7f6025800640 1 -- 172.21.15.22:0/2342886945 shutdown_connections 2024-04-20T21:33:48.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.674+0000 7f6025800640 1 -- 172.21.15.22:0/2342886945 wait complete. 2024-04-20T21:33:48.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:33:48.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.898+0000 7ff4efc00640 1 -- 172.21.15.22:0/3060912732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81041e0 msgr2=0x7ff4e8104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:48.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.898+0000 7ff4efc00640 1 --2- 172.21.15.22:0/3060912732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81041e0 0x7ff4e8104600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009900 tx=0x7ff4dc02bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:48.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.898+0000 7ff4efc00640 1 -- 172.21.15.22:0/3060912732 shutdown_connections 2024-04-20T21:33:48.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.898+0000 7ff4efc00640 1 --2- 172.21.15.22:0/3060912732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e8105d90 0x7ff4e810c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.898+0000 7ff4efc00640 1 --2- 172.21.15.22:0/3060912732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e81053d0 0x7ff4e8105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.898+0000 7ff4efc00640 1 --2- 172.21.15.22:0/3060912732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81041e0 0x7ff4e8104600 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.898+0000 7ff4efc00640 1 -- 172.21.15.22:0/3060912732 >> 172.21.15.22:0/3060912732 conn(0x7ff4e80ff750 msgr2=0x7ff4e8101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:48.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.898+0000 7ff4efc00640 1 -- 172.21.15.22:0/3060912732 shutdown_connections 2024-04-20T21:33:48.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.898+0000 7ff4efc00640 1 -- 172.21.15.22:0/3060912732 wait complete. 2024-04-20T21:33:48.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.899+0000 7ff4efc00640 1 Processor -- start 2024-04-20T21:33:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.899+0000 7ff4efc00640 1 -- start start 2024-04-20T21:33:48.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.899+0000 7ff4efc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81041e0 0x7ff4e819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.899+0000 7ff4efc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e81053d0 0x7ff4e819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.899+0000 7ff4efc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e8105d90 0x7ff4e81a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.899+0000 7ff4efc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e810fec0 con 0x7ff4e81041e0 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.899+0000 7ff4efc00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e810fd40 con 0x7ff4e8105d90 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.899+0000 7ff4efc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e8110040 con 0x7ff4e81053d0 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.900+0000 7ff4ed200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81041e0 0x7ff4e819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.900+0000 7ff4e7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e81053d0 0x7ff4e819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.900+0000 7ff4ed200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81041e0 0x7ff4e819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45846/0 (socket says 172.21.15.22:45846) 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.900+0000 7ff4ed200640 1 -- 172.21.15.22:0/3735890471 learned_addr learned my addr 172.21.15.22:0/3735890471 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.900+0000 7ff4edc00640 1 --2- 172.21.15.22:0/3735890471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e8105d90 0x7ff4e81a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.900+0000 7ff4e7e00640 1 -- 172.21.15.22:0/3735890471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e8105d90 msgr2=0x7ff4e81a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.900+0000 7ff4e7e00640 1 --2- 172.21.15.22:0/3735890471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e8105d90 0x7ff4e81a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.900+0000 7ff4e7e00640 1 -- 172.21.15.22:0/3735890471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81041e0 msgr2=0x7ff4e819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:48.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.900+0000 7ff4e7e00640 1 --2- 172.21.15.22:0/3735890471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81041e0 0x7ff4e819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:48.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.900+0000 7ff4e7e00640 1 -- 172.21.15.22:0/3735890471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e81a2130 con 0x7ff4e81053d0 2024-04-20T21:33:48.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.900+0000 7ff4ed200640 1 --2- 172.21.15.22:0/3735890471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81041e0 0x7ff4e819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:48.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.901+0000 7ff4e7e00640 1 --2- 172.21.15.22:0/3735890471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e81053d0 0x7ff4e819aa50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400e9f0 tx=0x7ff4d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:48.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.901+0000 7ff4e5600640 1 -- 172.21.15.22:0/3735890471 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d400ccc0 con 0x7ff4e81053d0 2024-04-20T21:33:48.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.901+0000 7ff4efc00640 1 -- 172.21.15.22:0/3735890471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e81a2420 con 0x7ff4e81053d0 2024-04-20T21:33:48.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.901+0000 7ff4e5600640 1 -- 172.21.15.22:0/3735890471 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4d4005020 con 0x7ff4e81053d0 2024-04-20T21:33:48.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.901+0000 7ff4e5600640 1 -- 172.21.15.22:0/3735890471 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40049e0 con 0x7ff4e81053d0 2024-04-20T21:33:48.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.901+0000 7ff4efc00640 1 -- 172.21.15.22:0/3735890471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e81a2970 con 0x7ff4e81053d0 2024-04-20T21:33:48.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.902+0000 7ff4efc00640 1 -- 172.21.15.22:0/3735890471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e810b500 con 0x7ff4e81053d0 2024-04-20T21:33:48.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.904+0000 7ff4e5600640 1 -- 172.21.15.22:0/3735890471 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7ff4d4005560 con 0x7ff4e81053d0 2024-04-20T21:33:48.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.906+0000 7ff4e5600640 1 -- 172.21.15.22:0/3735890471 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7ff4d40cff70 con 0x7ff4e81053d0 2024-04-20T21:33:48.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:48.910+0000 7ff4e5600640 1 -- 172.21.15.22:0/3735890471 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff4d40051d0 con 0x7ff4e81053d0 2024-04-20T21:33:49.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2606653755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:49.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2342886945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:49.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: Active manager daemon x restarted 2024-04-20T21:33:49.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: Activating manager daemon x 2024-04-20T21:33:49.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: osdmap e93: 12 total, 12 up, 12 in 2024-04-20T21:33:49.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: mgrmap e22: x(active, starting, since 0.0147004s), standbys: z, y 2024-04-20T21:33:49.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:33:49.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:33:49.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:33:49.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T21:33:49.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T21:33:49.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T21:33:49.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T21:33:49.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T21:33:49.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:33:49.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:33:49.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:33:49.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:33:49.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:33:49.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:33:49.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:33:49.044 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:33:49.045 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:33:49.045 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T21:33:49.045 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T21:33:49.045 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T21:33:49.045 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: Manager daemon x is now available 2024-04-20T21:33:49.045 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:33:49.045 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:33:49.045 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:33:49.045 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:48 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:48.812+0000 7f2439076200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:33:49.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.175+0000 7ff4efc00640 1 -- 172.21.15.22:0/3735890471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff4e8062b60 con 0x7ff4e81053d0 2024-04-20T21:33:49.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.175+0000 7ff4e5600640 1 -- 172.21.15.22:0/3735890471 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7ff4d409cbb0 con 0x7ff4e81053d0 2024-04-20T21:33:49.173 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-20T21:33:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.177+0000 7ff4efc00640 1 -- 172.21.15.22:0/3735890471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e81053d0 msgr2=0x7ff4e819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.177+0000 7ff4efc00640 1 --2- 172.21.15.22:0/3735890471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e81053d0 0x7ff4e819aa50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400e9f0 tx=0x7ff4d400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.178+0000 7ff4efc00640 1 -- 172.21.15.22:0/3735890471 shutdown_connections 2024-04-20T21:33:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.178+0000 7ff4efc00640 1 --2- 172.21.15.22:0/3735890471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e8105d90 0x7ff4e81a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.178+0000 7ff4efc00640 1 --2- 172.21.15.22:0/3735890471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e81053d0 0x7ff4e819aa50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.178+0000 7ff4efc00640 1 --2- 172.21.15.22:0/3735890471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81041e0 0x7ff4e819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.178+0000 7ff4efc00640 1 -- 172.21.15.22:0/3735890471 >> 172.21.15.22:0/3735890471 conn(0x7ff4e80ff750 msgr2=0x7ff4e81086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.178+0000 7ff4efc00640 1 -- 172.21.15.22:0/3735890471 shutdown_connections 2024-04-20T21:33:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.178+0000 7ff4efc00640 1 -- 172.21.15.22:0/3735890471 wait complete. 2024-04-20T21:33:49.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:33:49.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2606653755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:49.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2342886945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:49.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: Active manager daemon x restarted 2024-04-20T21:33:49.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: Activating manager daemon x 2024-04-20T21:33:49.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: osdmap e93: 12 total, 12 up, 12 in 2024-04-20T21:33:49.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: mgrmap e22: x(active, starting, since 0.0147004s), standbys: z, y 2024-04-20T21:33:49.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:33:49.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:33:49.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T21:33:49.305 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T21:33:49.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T21:33:49.307 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T21:33:49.307 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: Manager daemon x is now available 2024-04-20T21:33:49.307 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:33:49.307 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:33:49.307 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:49 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:33:49.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.385+0000 7f96e6c00640 1 -- 172.21.15.22:0/145116617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e01053b0 msgr2=0x7f96e0105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:49.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.385+0000 7f96e6c00640 1 --2- 172.21.15.22:0/145116617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e01053b0 0x7f96e0105830 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009900 tx=0x7f96cc02bc50 comp rx=0 tx=0).stop 2024-04-20T21:33:49.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.386+0000 7f96e6c00640 1 -- 172.21.15.22:0/145116617 shutdown_connections 2024-04-20T21:33:49.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.386+0000 7f96e6c00640 1 --2- 172.21.15.22:0/145116617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96e0105d70 0x7f96e010c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.386+0000 7f96e6c00640 1 --2- 172.21.15.22:0/145116617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e01053b0 0x7f96e0105830 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.386+0000 7f96e6c00640 1 --2- 172.21.15.22:0/145116617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96e01041c0 0x7f96e01045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.386+0000 7f96e6c00640 1 -- 172.21.15.22:0/145116617 >> 172.21.15.22:0/145116617 conn(0x7f96e00ff750 msgr2=0x7f96e0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:49.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.386+0000 7f96e6c00640 1 -- 172.21.15.22:0/145116617 shutdown_connections 2024-04-20T21:33:49.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.386+0000 7f96e6c00640 1 -- 172.21.15.22:0/145116617 wait complete. 2024-04-20T21:33:49.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.387+0000 7f96e6c00640 1 Processor -- start 2024-04-20T21:33:49.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.387+0000 7f96e6c00640 1 -- start start 2024-04-20T21:33:49.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.387+0000 7f96e6c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96e01041c0 0x7f96e019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:49.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.387+0000 7f96e6c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96e01053b0 0x7f96e019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:49.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.387+0000 7f96e6c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e0105d70 0x7f96e01a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:49.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.387+0000 7f96e6c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f96e010fea0 con 0x7f96e01041c0 2024-04-20T21:33:49.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.387+0000 7f96e6c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96e010fd20 con 0x7f96e01053b0 2024-04-20T21:33:49.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.387+0000 7f96e6c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96e0110020 con 0x7f96e0105d70 2024-04-20T21:33:49.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.387+0000 7f96dfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96e01041c0 0x7f96e019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:49.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.387+0000 7f96dfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96e01041c0 0x7f96e019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45956/0 (socket says 172.21.15.22:45956) 2024-04-20T21:33:49.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.387+0000 7f96dfe00640 1 -- 172.21.15.22:0/1359201663 learned_addr learned my addr 172.21.15.22:0/1359201663 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:49.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.387+0000 7f96e4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e0105d70 0x7f96e01a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:49.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.388+0000 7f96df400640 1 --2- 172.21.15.22:0/1359201663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96e01053b0 0x7f96e019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:49.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.388+0000 7f96e4c00640 1 -- 172.21.15.22:0/1359201663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96e01053b0 msgr2=0x7f96e019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:49.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.388+0000 7f96e4c00640 1 --2- 172.21.15.22:0/1359201663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96e01053b0 0x7f96e019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.388+0000 7f96e4c00640 1 -- 172.21.15.22:0/1359201663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96e01041c0 msgr2=0x7f96e019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:49.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.388+0000 7f96e4c00640 1 --2- 172.21.15.22:0/1359201663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96e01041c0 0x7f96e019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.388+0000 7f96e4c00640 1 -- 172.21.15.22:0/1359201663 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e01a20e0 con 0x7f96e0105d70 2024-04-20T21:33:49.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.388+0000 7f96e4c00640 1 --2- 172.21.15.22:0/1359201663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e0105d70 0x7f96e01a1a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009fd0 tx=0x7f96d800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:49.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.388+0000 7f96dcc00640 1 -- 172.21.15.22:0/1359201663 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d800ce20 con 0x7f96e0105d70 2024-04-20T21:33:49.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.388+0000 7f96e6c00640 1 -- 172.21.15.22:0/1359201663 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e01a23d0 con 0x7f96e0105d70 2024-04-20T21:33:49.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.389+0000 7f96dcc00640 1 -- 172.21.15.22:0/1359201663 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d8005020 con 0x7f96e0105d70 2024-04-20T21:33:49.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.389+0000 7f96dcc00640 1 -- 172.21.15.22:0/1359201663 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8010720 con 0x7f96e0105d70 2024-04-20T21:33:49.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.389+0000 7f96e6c00640 1 -- 172.21.15.22:0/1359201663 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e01a28f0 con 0x7f96e0105d70 2024-04-20T21:33:49.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.390+0000 7f96e6c00640 1 -- 172.21.15.22:0/1359201663 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e010b480 con 0x7f96e0105d70 2024-04-20T21:33:49.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.393+0000 7f96dcc00640 1 -- 172.21.15.22:0/1359201663 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f96d8005540 con 0x7f96e0105d70 2024-04-20T21:33:49.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.395+0000 7f96dcc00640 1 -- 172.21.15.22:0/1359201663 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f96d80d0710 con 0x7f96e0105d70 2024-04-20T21:33:49.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.399+0000 7f96dcc00640 1 -- 172.21.15.22:0/1359201663 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f96d809d350 con 0x7f96e0105d70 2024-04-20T21:33:49.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:49.170+0000 7f238e000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:49.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:49.170+0000 7f238e000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:49.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:49.170+0000 7f238e000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:49.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:49.170+0000 7f238e000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:49.406 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:49.170+0000 7f238e000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:49.407 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:49.171+0000 7f238b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:49.407 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:49.171+0000 7f238b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:49.407 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:49.171+0000 7f238b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:49.407 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:49.171+0000 7f238b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:49.407 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:33:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:33:49.171+0000 7f238b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:49.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2606653755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:49.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2342886945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:49.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: Active manager daemon x restarted 2024-04-20T21:33:49.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: Activating manager daemon x 2024-04-20T21:33:49.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: osdmap e93: 12 total, 12 up, 12 in 2024-04-20T21:33:49.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: mgrmap e22: x(active, starting, since 0.0147004s), standbys: z, y 2024-04-20T21:33:49.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:33:49.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:33:49.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:33:49.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T21:33:49.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T21:33:49.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T21:33:49.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:33:49.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: Manager daemon x is now available 2024-04-20T21:33:49.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:33:49.529 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:33:49.529 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:49 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:33:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.601+0000 7f96e6c00640 1 -- 172.21.15.22:0/1359201663 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f96e0062b60 con 0x7f96e0105d70 2024-04-20T21:33:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.602+0000 7f96dcc00640 1 -- 172.21.15.22:0/1359201663 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f96d809cc50 con 0x7f96e0105d70 2024-04-20T21:33:49.600 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-20T21:33:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.604+0000 7f96e6c00640 1 -- 172.21.15.22:0/1359201663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e0105d70 msgr2=0x7f96e01a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.604+0000 7f96e6c00640 1 --2- 172.21.15.22:0/1359201663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e0105d70 0x7f96e01a1a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009fd0 tx=0x7f96d800ef10 comp rx=0 tx=0).stop 2024-04-20T21:33:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.605+0000 7f96e6c00640 1 -- 172.21.15.22:0/1359201663 shutdown_connections 2024-04-20T21:33:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.605+0000 7f96e6c00640 1 --2- 172.21.15.22:0/1359201663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e0105d70 0x7f96e01a1a70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.605+0000 7f96e6c00640 1 --2- 172.21.15.22:0/1359201663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96e01053b0 0x7f96e019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.605+0000 7f96e6c00640 1 --2- 172.21.15.22:0/1359201663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96e01041c0 0x7f96e019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.605+0000 7f96e6c00640 1 -- 172.21.15.22:0/1359201663 >> 172.21.15.22:0/1359201663 conn(0x7f96e00ff750 msgr2=0x7f96e0108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.605+0000 7f96e6c00640 1 -- 172.21.15.22:0/1359201663 shutdown_connections 2024-04-20T21:33:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.605+0000 7f96e6c00640 1 -- 172.21.15.22:0/1359201663 wait complete. 2024-04-20T21:33:49.620 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:49.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.802+0000 7f6489200640 1 -- 172.21.15.22:0/3534444524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6484103980 msgr2=0x7f648410a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:49.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.802+0000 7f6489200640 1 --2- 172.21.15.22:0/3534444524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6484103980 0x7f648410a210 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f647c009ca0 tx=0x7f647c02c100 comp rx=0 tx=0).stop 2024-04-20T21:33:49.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.802+0000 7f6489200640 1 -- 172.21.15.22:0/3534444524 shutdown_connections 2024-04-20T21:33:49.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.802+0000 7f6489200640 1 --2- 172.21.15.22:0/3534444524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6484103980 0x7f648410a210 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.802+0000 7f6489200640 1 --2- 172.21.15.22:0/3534444524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6484102fc0 0x7f6484103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.802+0000 7f6489200640 1 --2- 172.21.15.22:0/3534444524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6484101dd0 0x7f64841021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.803+0000 7f6489200640 1 -- 172.21.15.22:0/3534444524 >> 172.21.15.22:0/3534444524 conn(0x7f64840fd340 msgr2=0x7f64840ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:49.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.803+0000 7f6489200640 1 -- 172.21.15.22:0/3534444524 shutdown_connections 2024-04-20T21:33:49.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.803+0000 7f6489200640 1 -- 172.21.15.22:0/3534444524 wait complete. 2024-04-20T21:33:49.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6489200640 1 Processor -- start 2024-04-20T21:33:49.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6489200640 1 -- start start 2024-04-20T21:33:49.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6489200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6484101dd0 0x7f648419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:49.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6489200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6484102fc0 0x7f648419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:49.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6489200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6484103980 0x7f64841a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:49.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6489200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f648410da30 con 0x7f6484101dd0 2024-04-20T21:33:49.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6489200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f648410d8b0 con 0x7f6484103980 2024-04-20T21:33:49.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6489200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f648410dbb0 con 0x7f6484102fc0 2024-04-20T21:33:49.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6482a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6484102fc0 0x7f648419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:49.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6483400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6484101dd0 0x7f648419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:49.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6483e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6484103980 0x7f64841a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:49.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6482a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6484102fc0 0x7f648419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:59680/0 (socket says 172.21.15.22:59680) 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.804+0000 7f6482a00640 1 -- 172.21.15.22:0/3825608189 learned_addr learned my addr 172.21.15.22:0/3825608189 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.805+0000 7f6483e00640 1 -- 172.21.15.22:0/3825608189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6484102fc0 msgr2=0x7f648419a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.805+0000 7f6483e00640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6484102fc0 0x7f648419a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.805+0000 7f6483e00640 1 -- 172.21.15.22:0/3825608189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6484101dd0 msgr2=0x7f648419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.805+0000 7f6483e00640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6484101dd0 0x7f648419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.805+0000 7f6483e00640 1 -- 172.21.15.22:0/3825608189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64841a1eb0 con 0x7f6484103980 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.805+0000 7f6482a00640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6484102fc0 0x7f648419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.805+0000 7f6483e00640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6484103980 0x7f64841a1840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f647c037040 tx=0x7f647c035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.805+0000 7f647be00640 1 -- 172.21.15.22:0/3825608189 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c039040 con 0x7f6484103980 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.806+0000 7f647be00640 1 -- 172.21.15.22:0/3825608189 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f647c044070 con 0x7f6484103980 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.806+0000 7f647be00640 1 -- 172.21.15.22:0/3825608189 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c004350 con 0x7f6484103980 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.806+0000 7f6489200640 1 -- 172.21.15.22:0/3825608189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64841a2140 con 0x7f6484103980 2024-04-20T21:33:49.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.806+0000 7f6489200640 1 -- 172.21.15.22:0/3825608189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64841a2600 con 0x7f6484103980 2024-04-20T21:33:49.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.807+0000 7f6489200640 1 -- 172.21.15.22:0/3825608189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64841a2920 con 0x7f6484103980 2024-04-20T21:33:49.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.810+0000 7f647be00640 1 -- 172.21.15.22:0/3825608189 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f647c0044f0 con 0x7f6484103980 2024-04-20T21:33:49.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.812+0000 7f647be00640 1 -- 172.21.15.22:0/3825608189 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f647c0f24c0 con 0x7f6484103980 2024-04-20T21:33:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.816+0000 7f647be00640 1 -- 172.21.15.22:0/3825608189 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f647c0bf160 con 0x7f6484103980 2024-04-20T21:33:49.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.847+0000 7f647be00640 1 -- 172.21.15.22:0/3825608189 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f647c0ba7b0 con 0x7f6484103980 2024-04-20T21:33:49.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.847+0000 7f647be00640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f646801db50 0x7f64680aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:49.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.852+0000 7f6483400640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f646801db50 0x7f64680aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:49.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:49.859+0000 7f6483400640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f646801db50 0x7f64680aca30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6484102e20 tx=0x7f6474005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:50.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.046+0000 7f6489200640 1 -- 172.21.15.22:0/3825608189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64841a2fb0 con 0x7f6484103980 2024-04-20T21:33:50.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.048+0000 7f647be00640 1 -- 172.21.15.22:0/3825608189 <== mon.2 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+23146 (secure 0 0 0) 0x7f647c075640 con 0x7f6484103980 2024-04-20T21:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:33:48.816036+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/1815958949":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/3359736029":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/3485677447":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/1911146506":"2024-04-21T21:33:48.815984+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1408863783":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/1911146506":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:50.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.050+0000 7f6489200640 1 -- 172.21.15.22:0/3825608189 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f646801db50 msgr2=0x7f64680aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:50.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.050+0000 7f6489200640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f646801db50 0x7f64680aca30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6484102e20 tx=0x7f6474005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:50.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.050+0000 7f6489200640 1 -- 172.21.15.22:0/3825608189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6484103980 msgr2=0x7f64841a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:50.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.050+0000 7f6489200640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6484103980 0x7f64841a1840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f647c037040 tx=0x7f647c035d10 comp rx=0 tx=0).stop 2024-04-20T21:33:50.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.050+0000 7f6489200640 1 -- 172.21.15.22:0/3825608189 shutdown_connections 2024-04-20T21:33:50.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.050+0000 7f6489200640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f646801db50 0x7f64680aca30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.050+0000 7f6489200640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6484103980 0x7f64841a1840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.050+0000 7f6489200640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6484102fc0 0x7f648419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.050+0000 7f6489200640 1 --2- 172.21.15.22:0/3825608189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6484101dd0 0x7f648419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.050+0000 7f6489200640 1 -- 172.21.15.22:0/3825608189 >> 172.21.15.22:0/3825608189 conn(0x7f64840fd340 msgr2=0x7f6484074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:50.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.050+0000 7f6489200640 1 -- 172.21.15.22:0/3825608189 shutdown_connections 2024-04-20T21:33:50.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.050+0000 7f6489200640 1 -- 172.21.15.22:0/3825608189 wait complete. 2024-04-20T21:33:50.065 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:26:15.688818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:31:25.506074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:31:26.126261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:33:50.065 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:33:50.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.247+0000 7f4525e00640 1 -- 172.21.15.22:0/3493294742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4520075f40 msgr2=0x7f4520074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:50.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.247+0000 7f4525e00640 1 --2- 172.21.15.22:0/3493294742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4520075f40 0x7f4520074390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4514009900 tx=0x7f451402bcc0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.247+0000 7f4525e00640 1 -- 172.21.15.22:0/3493294742 shutdown_connections 2024-04-20T21:33:50.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.247+0000 7f4525e00640 1 --2- 172.21.15.22:0/3493294742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4520109c30 0x7f452010c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.247+0000 7f4525e00640 1 --2- 172.21.15.22:0/3493294742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520074960 0x7f4520074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.247+0000 7f4525e00640 1 --2- 172.21.15.22:0/3493294742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4520075f40 0x7f4520074390 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.247+0000 7f4525e00640 1 -- 172.21.15.22:0/3493294742 >> 172.21.15.22:0/3493294742 conn(0x7f45200fd310 msgr2=0x7f45200ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:50.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.247+0000 7f4525e00640 1 -- 172.21.15.22:0/3493294742 shutdown_connections 2024-04-20T21:33:50.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.247+0000 7f4525e00640 1 -- 172.21.15.22:0/3493294742 wait complete. 2024-04-20T21:33:50.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.248+0000 7f4525e00640 1 Processor -- start 2024-04-20T21:33:50.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.248+0000 7f4525e00640 1 -- start start 2024-04-20T21:33:50.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.248+0000 7f4525e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4520074960 0x7f452019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:50.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.248+0000 7f4525e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4520075f40 0x7f452019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:50.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.248+0000 7f4525e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520109c30 0x7f45201a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:50.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.248+0000 7f4525e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f452010fc80 con 0x7f4520109c30 2024-04-20T21:33:50.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.248+0000 7f4525e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f452010fb00 con 0x7f4520075f40 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.248+0000 7f4525e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f452010fe00 con 0x7f4520074960 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.248+0000 7f451fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4520075f40 0x7f452019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.248+0000 7f4524a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4520074960 0x7f452019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.249+0000 7f451fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4520075f40 0x7f452019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:40038/0 (socket says 172.21.15.22:40038) 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.249+0000 7f4524a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4520074960 0x7f452019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:59706/0 (socket says 172.21.15.22:59706) 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.249+0000 7f451fe00640 1 -- 172.21.15.22:0/1636142490 learned_addr learned my addr 172.21.15.22:0/1636142490 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.249+0000 7f4525400640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520109c30 0x7f45201a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.249+0000 7f451fe00640 1 -- 172.21.15.22:0/1636142490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4520074960 msgr2=0x7f452019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.249+0000 7f451fe00640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4520074960 0x7f452019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.249+0000 7f451fe00640 1 -- 172.21.15.22:0/1636142490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520109c30 msgr2=0x7f45201a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.249+0000 7f451fe00640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520109c30 0x7f45201a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.249+0000 7f451fe00640 1 -- 172.21.15.22:0/1636142490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45201a1e50 con 0x7f4520075f40 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.249+0000 7f451fe00640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4520075f40 0x7f452019a770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f450c00e9f0 tx=0x7f450c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:50.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.249+0000 7f451d600640 1 -- 172.21.15.22:0/1636142490 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c00cd80 con 0x7f4520075f40 2024-04-20T21:33:50.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.250+0000 7f4525e00640 1 -- 172.21.15.22:0/1636142490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45201a2140 con 0x7f4520075f40 2024-04-20T21:33:50.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.250+0000 7f451d600640 1 -- 172.21.15.22:0/1636142490 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f450c010910 con 0x7f4520075f40 2024-04-20T21:33:50.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.250+0000 7f451d600640 1 -- 172.21.15.22:0/1636142490 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c0188f0 con 0x7f4520075f40 2024-04-20T21:33:50.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.250+0000 7f4525e00640 1 -- 172.21.15.22:0/1636142490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45201a26e0 con 0x7f4520075f40 2024-04-20T21:33:50.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.251+0000 7f4525e00640 1 -- 172.21.15.22:0/1636142490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45201089f0 con 0x7f4520075f40 2024-04-20T21:33:50.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.253+0000 7f451d600640 1 -- 172.21.15.22:0/1636142490 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f450c010ac0 con 0x7f4520075f40 2024-04-20T21:33:50.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.255+0000 7f451d600640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f45080a39b0 0x7f45080a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:50.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.255+0000 7f451d600640 1 -- 172.21.15.22:0/1636142490 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f450c0d0510 con 0x7f4520075f40 2024-04-20T21:33:50.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.256+0000 7f4524a00640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f45080a39b0 0x7f45080a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:50.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.256+0000 7f4524a00640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f45080a39b0 0x7f45080a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f451402c2d0 tx=0x7f451402c910 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:50.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.263+0000 7f451d600640 1 -- 172.21.15.22:0/1636142490 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f450c09ca10 con 0x7f4520075f40 2024-04-20T21:33:50.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:50 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3735890471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:50.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:50 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1359201663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:50.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:50 smithi033 ceph-mon[21626]: mgrmap e23: x(active, since 1.03246s), standbys: z, y 2024-04-20T21:33:50.335 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:50 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3735890471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:50.335 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:50 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1359201663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:50.335 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:50 smithi122 ceph-mon[22147]: mgrmap e23: x(active, since 1.03246s), standbys: z, y 2024-04-20T21:33:50.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:50 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3735890471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:50.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:50 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1359201663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:50.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:50 smithi022 ceph-mon[16843]: mgrmap e23: x(active, since 1.03246s), standbys: z, y 2024-04-20T21:33:50.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.466+0000 7f4525e00640 1 -- 172.21.15.22:0/1636142490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4520062b60 con 0x7f4520075f40 2024-04-20T21:33:50.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.466+0000 7f451d600640 1 -- 172.21.15.22:0/1636142490 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f450c0a0d10 con 0x7f4520075f40 2024-04-20T21:33:50.464 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-20T21:33:50.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.468+0000 7f4525e00640 1 -- 172.21.15.22:0/1636142490 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f45080a39b0 msgr2=0x7f45080a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:50.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.468+0000 7f4525e00640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f45080a39b0 0x7f45080a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f451402c2d0 tx=0x7f451402c910 comp rx=0 tx=0).stop 2024-04-20T21:33:50.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.469+0000 7f4525e00640 1 -- 172.21.15.22:0/1636142490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4520075f40 msgr2=0x7f452019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:50.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.469+0000 7f4525e00640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4520075f40 0x7f452019a770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f450c00e9f0 tx=0x7f450c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.469+0000 7f4525e00640 1 -- 172.21.15.22:0/1636142490 shutdown_connections 2024-04-20T21:33:50.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.469+0000 7f4525e00640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f45080a39b0 0x7f45080a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.469+0000 7f4525e00640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520109c30 0x7f45201a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.469+0000 7f4525e00640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4520075f40 0x7f452019a770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.469+0000 7f4525e00640 1 --2- 172.21.15.22:0/1636142490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4520074960 0x7f452019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.469+0000 7f4525e00640 1 -- 172.21.15.22:0/1636142490 >> 172.21.15.22:0/1636142490 conn(0x7f45200fd310 msgr2=0x7f4520105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:50.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.469+0000 7f4525e00640 1 -- 172.21.15.22:0/1636142490 shutdown_connections 2024-04-20T21:33:50.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.469+0000 7f4525e00640 1 -- 172.21.15.22:0/1636142490 wait complete. 2024-04-20T21:33:50.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:33:50.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.686+0000 7fa1cbe00640 1 -- 172.21.15.22:0/3423020920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc1028f0 msgr2=0x7fa1cc104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:50.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.686+0000 7fa1cbe00640 1 --2- 172.21.15.22:0/3423020920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc1028f0 0x7fa1cc104ce0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009900 tx=0x7fa1bc02bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:50.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.686+0000 7fa1cbe00640 1 -- 172.21.15.22:0/3423020920 shutdown_connections 2024-04-20T21:33:50.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.686+0000 7fa1cbe00640 1 --2- 172.21.15.22:0/3423020920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1cc107b50 0x7fa1cc109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.686+0000 7fa1cbe00640 1 --2- 172.21.15.22:0/3423020920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1cc105220 0x7fa1cc107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.686+0000 7fa1cbe00640 1 --2- 172.21.15.22:0/3423020920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc1028f0 0x7fa1cc104ce0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.686+0000 7fa1cbe00640 1 -- 172.21.15.22:0/3423020920 >> 172.21.15.22:0/3423020920 conn(0x7fa1cc0fc300 msgr2=0x7fa1cc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:50.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.686+0000 7fa1cbe00640 1 -- 172.21.15.22:0/3423020920 shutdown_connections 2024-04-20T21:33:50.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.686+0000 7fa1cbe00640 1 -- 172.21.15.22:0/3423020920 wait complete. 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.687+0000 7fa1cbe00640 1 Processor -- start 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.687+0000 7fa1cbe00640 1 -- start start 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.687+0000 7fa1cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1cc1028f0 0x7fa1cc19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.687+0000 7fa1cbe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1cc105220 0x7fa1cc19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.687+0000 7fa1cbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc107b50 0x7fa1cc1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.687+0000 7fa1cbe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc10db10 con 0x7fa1cc1028f0 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.687+0000 7fa1cbe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc10d990 con 0x7fa1cc105220 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.687+0000 7fa1cbe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc10dc90 con 0x7fa1cc107b50 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.688+0000 7fa1ca000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1cc105220 0x7fa1cc19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.688+0000 7fa1ca000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1cc105220 0x7fa1cc19a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:40052/0 (socket says 172.21.15.22:40052) 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.688+0000 7fa1ca000640 1 -- 172.21.15.22:0/1343741548 learned_addr learned my addr 172.21.15.22:0/1343741548 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.688+0000 7fa1caa00640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1cc1028f0 0x7fa1cc19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.688+0000 7fa1cb400640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc107b50 0x7fa1cc1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.688+0000 7fa1ca000640 1 -- 172.21.15.22:0/1343741548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc107b50 msgr2=0x7fa1cc1a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.688+0000 7fa1ca000640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc107b50 0x7fa1cc1a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.688+0000 7fa1ca000640 1 -- 172.21.15.22:0/1343741548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1cc1028f0 msgr2=0x7fa1cc19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.688+0000 7fa1ca000640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1cc1028f0 0x7fa1cc19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.688+0000 7fa1ca000640 1 -- 172.21.15.22:0/1343741548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1cc1a1fc0 con 0x7fa1cc105220 2024-04-20T21:33:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.688+0000 7fa1ca000640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1cc105220 0x7fa1cc19a8e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800e9a0 tx=0x7fa1b800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.689+0000 7fa1c3400640 1 -- 172.21.15.22:0/1343741548 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b800cd80 con 0x7fa1cc105220 2024-04-20T21:33:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.689+0000 7fa1cbe00640 1 -- 172.21.15.22:0/1343741548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1cc1a22b0 con 0x7fa1cc105220 2024-04-20T21:33:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.689+0000 7fa1c3400640 1 -- 172.21.15.22:0/1343741548 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b80035f0 con 0x7fa1cc105220 2024-04-20T21:33:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.689+0000 7fa1c3400640 1 -- 172.21.15.22:0/1343741548 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8010670 con 0x7fa1cc105220 2024-04-20T21:33:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.689+0000 7fa1cbe00640 1 -- 172.21.15.22:0/1343741548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1cc1a2850 con 0x7fa1cc105220 2024-04-20T21:33:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.690+0000 7fa1cbe00640 1 -- 172.21.15.22:0/1343741548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1cc1096a0 con 0x7fa1cc105220 2024-04-20T21:33:50.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.693+0000 7fa1c3400640 1 -- 172.21.15.22:0/1343741548 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7fa1b80037a0 con 0x7fa1cc105220 2024-04-20T21:33:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.695+0000 7fa1c3400640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa1b00a39b0 0x7fa1b00a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.695+0000 7fa1c3400640 1 -- 172.21.15.22:0/1343741548 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fa1b80d03a0 con 0x7fa1cc105220 2024-04-20T21:33:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.695+0000 7fa1caa00640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa1b00a39b0 0x7fa1b00a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:50.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.696+0000 7fa1caa00640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa1b00a39b0 0x7fa1b00a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc02c7c0 tx=0x7fa1bc005fd0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:50.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.700+0000 7fa1c3400640 1 -- 172.21.15.22:0/1343741548 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa1b809cef0 con 0x7fa1cc105220 2024-04-20T21:33:50.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.903+0000 7fa1cbe00640 1 -- 172.21.15.22:0/1343741548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa1cc062b60 con 0x7fa1cc105220 2024-04-20T21:33:50.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.903+0000 7fa1c3400640 1 -- 172.21.15.22:0/1343741548 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fa1b809c7f0 con 0x7fa1cc105220 2024-04-20T21:33:50.901 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-20T21:33:50.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.905+0000 7fa1cbe00640 1 -- 172.21.15.22:0/1343741548 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa1b00a39b0 msgr2=0x7fa1b00a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:50.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.905+0000 7fa1cbe00640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa1b00a39b0 0x7fa1b00a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc02c7c0 tx=0x7fa1bc005fd0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.905+0000 7fa1cbe00640 1 -- 172.21.15.22:0/1343741548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1cc105220 msgr2=0x7fa1cc19a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:50.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.905+0000 7fa1cbe00640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1cc105220 0x7fa1cc19a8e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800e9a0 tx=0x7fa1b800ee60 comp rx=0 tx=0).stop 2024-04-20T21:33:50.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.906+0000 7fa1cbe00640 1 -- 172.21.15.22:0/1343741548 shutdown_connections 2024-04-20T21:33:50.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.906+0000 7fa1cbe00640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa1b00a39b0 0x7fa1b00a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.906+0000 7fa1cbe00640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc107b50 0x7fa1cc1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.906+0000 7fa1cbe00640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1cc105220 0x7fa1cc19a8e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.906+0000 7fa1cbe00640 1 --2- 172.21.15.22:0/1343741548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1cc1028f0 0x7fa1cc19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:50.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.906+0000 7fa1cbe00640 1 -- 172.21.15.22:0/1343741548 >> 172.21.15.22:0/1343741548 conn(0x7fa1cc0fc300 msgr2=0x7fa1cc102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:50.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.906+0000 7fa1cbe00640 1 -- 172.21.15.22:0/1343741548 shutdown_connections 2024-04-20T21:33:50.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:50.906+0000 7fa1cbe00640 1 -- 172.21.15.22:0/1343741548 wait complete. 2024-04-20T21:33:50.921 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:33:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.118+0000 7fb0f3200640 1 -- 172.21.15.22:0/3853470158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec101b70 msgr2=0x7fb0ec101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.118+0000 7fb0f3200640 1 --2- 172.21.15.22:0/3853470158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec101b70 0x7fb0ec101ff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009ca0 tx=0x7fb0e402c190 comp rx=0 tx=0).stop 2024-04-20T21:33:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.119+0000 7fb0f3200640 1 -- 172.21.15.22:0/3853470158 shutdown_connections 2024-04-20T21:33:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.119+0000 7fb0f3200640 1 --2- 172.21.15.22:0/3853470158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec101b70 0x7fb0ec101ff0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.119+0000 7fb0f3200640 1 --2- 172.21.15.22:0/3853470158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0ec1011d0 0x7fb0ec101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.119+0000 7fb0f3200640 1 --2- 172.21.15.22:0/3853470158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0ec100870 0x7fb0ec100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.119+0000 7fb0f3200640 1 -- 172.21.15.22:0/3853470158 >> 172.21.15.22:0/3853470158 conn(0x7fb0ec0fc3f0 msgr2=0x7fb0ec0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.119+0000 7fb0f3200640 1 -- 172.21.15.22:0/3853470158 shutdown_connections 2024-04-20T21:33:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.119+0000 7fb0f3200640 1 -- 172.21.15.22:0/3853470158 wait complete. 2024-04-20T21:33:51.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.120+0000 7fb0f3200640 1 Processor -- start 2024-04-20T21:33:51.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.120+0000 7fb0f3200640 1 -- start start 2024-04-20T21:33:51.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.120+0000 7fb0f3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0ec100870 0x7fb0ec19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:51.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.120+0000 7fb0f3200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec1011d0 0x7fb0ec19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:51.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.120+0000 7fb0f3200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0ec101b70 0x7fb0ec1a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:51.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.120+0000 7fb0f3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec10fe10 con 0x7fb0ec100870 2024-04-20T21:33:51.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.120+0000 7fb0f3200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec10fc90 con 0x7fb0ec1011d0 2024-04-20T21:33:51.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.120+0000 7fb0f3200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec10ff90 con 0x7fb0ec101b70 2024-04-20T21:33:51.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.121+0000 7fb0e3e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec1011d0 0x7fb0ec19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:51.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.121+0000 7fb0f1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0ec101b70 0x7fb0ec1a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.121+0000 7fb0e3e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec1011d0 0x7fb0ec19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:40084/0 (socket says 172.21.15.22:40084) 2024-04-20T21:33:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.121+0000 7fb0e3e00640 1 -- 172.21.15.22:0/3851136016 learned_addr learned my addr 172.21.15.22:0/3851136016 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.121+0000 7fb0ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0ec100870 0x7fb0ec19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.121+0000 7fb0f1200640 1 -- 172.21.15.22:0/3851136016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec1011d0 msgr2=0x7fb0ec19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.121+0000 7fb0f1200640 1 --2- 172.21.15.22:0/3851136016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec1011d0 0x7fb0ec19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.121+0000 7fb0f1200640 1 -- 172.21.15.22:0/3851136016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0ec100870 msgr2=0x7fb0ec19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.121+0000 7fb0f1200640 1 --2- 172.21.15.22:0/3851136016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0ec100870 0x7fb0ec19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.121+0000 7fb0f1200640 1 -- 172.21.15.22:0/3851136016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ec1a1e30 con 0x7fb0ec101b70 2024-04-20T21:33:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.121+0000 7fb0f1200640 1 --2- 172.21.15.22:0/3851136016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0ec101b70 0x7fb0ec1a17c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009bf0 tx=0x7fb0e4035f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.122+0000 7fb0e9600640 1 -- 172.21.15.22:0/3851136016 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4039040 con 0x7fb0ec101b70 2024-04-20T21:33:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.122+0000 7fb0e9600640 1 -- 172.21.15.22:0/3851136016 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e4044070 con 0x7fb0ec101b70 2024-04-20T21:33:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.122+0000 7fb0e9600640 1 -- 172.21.15.22:0/3851136016 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e40046a0 con 0x7fb0ec101b70 2024-04-20T21:33:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.122+0000 7fb0f3200640 1 -- 172.21.15.22:0/3851136016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ec1a20c0 con 0x7fb0ec101b70 2024-04-20T21:33:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.122+0000 7fb0f3200640 1 -- 172.21.15.22:0/3851136016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ec1a2530 con 0x7fb0ec101b70 2024-04-20T21:33:51.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.124+0000 7fb0f3200640 1 -- 172.21.15.22:0/3851136016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0ec10b5c0 con 0x7fb0ec101b70 2024-04-20T21:33:51.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.125+0000 7fb0e9600640 1 -- 172.21.15.22:0/3851136016 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7fb0e40326e0 con 0x7fb0ec101b70 2024-04-20T21:33:51.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.126+0000 7fb0e9600640 1 --2- 172.21.15.22:0/3851136016 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb0d00a3a30 0x7fb0d00a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:51.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.127+0000 7fb0e9600640 1 -- 172.21.15.22:0/3851136016 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fb0e40f28b0 con 0x7fb0ec101b70 2024-04-20T21:33:51.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.127+0000 7fb0ebe00640 1 --2- 172.21.15.22:0/3851136016 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb0d00a3a30 0x7fb0d00a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:51.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.127+0000 7fb0ebe00640 1 --2- 172.21.15.22:0/3851136016 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb0d00a3a30 0x7fb0d00a5ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc005d00 tx=0x7fb0dc00b410 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:51.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.131+0000 7fb0e9600640 1 -- 172.21.15.22:0/3851136016 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb0e40bf400 con 0x7fb0ec101b70 2024-04-20T21:33:51.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.218+0000 7fb0e9600640 1 -- 172.21.15.22:0/3851136016 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7fb0e40bb370 con 0x7fb0ec101b70 2024-04-20T21:33:51.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:51 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3825608189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:51.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:51 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:51.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:51 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:51.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:51 smithi033 ceph-mon[21626]: [20/Apr/2024:21:33:50] ENGINE Bus STARTING 2024-04-20T21:33:51.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:51 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1636142490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:51.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:51 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1343741548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:51.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:51 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.344+0000 7fb0f3200640 1 -- 172.21.15.22:0/3851136016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb0ec062b60 con 0x7fb0ec101b70 2024-04-20T21:33:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.344+0000 7fb0e9600640 1 -- 172.21.15.22:0/3851136016 <== mon.1 v2:172.21.15.122:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fb0e407a410 con 0x7fb0ec101b70 2024-04-20T21:33:51.342 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-20T21:33:51.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.347+0000 7fb0f3200640 1 -- 172.21.15.22:0/3851136016 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb0d00a3a30 msgr2=0x7fb0d00a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:51.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.347+0000 7fb0f3200640 1 --2- 172.21.15.22:0/3851136016 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb0d00a3a30 0x7fb0d00a5ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc005d00 tx=0x7fb0dc00b410 comp rx=0 tx=0).stop 2024-04-20T21:33:51.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.347+0000 7fb0f3200640 1 -- 172.21.15.22:0/3851136016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0ec101b70 msgr2=0x7fb0ec1a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:51.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.347+0000 7fb0f3200640 1 --2- 172.21.15.22:0/3851136016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0ec101b70 0x7fb0ec1a17c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009bf0 tx=0x7fb0e4035f90 comp rx=0 tx=0).stop 2024-04-20T21:33:51.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.347+0000 7fb0f3200640 1 -- 172.21.15.22:0/3851136016 shutdown_connections 2024-04-20T21:33:51.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.347+0000 7fb0f3200640 1 --2- 172.21.15.22:0/3851136016 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb0d00a3a30 0x7fb0d00a5ef0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.347+0000 7fb0f3200640 1 --2- 172.21.15.22:0/3851136016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0ec101b70 0x7fb0ec1a17c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.347+0000 7fb0f3200640 1 --2- 172.21.15.22:0/3851136016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec1011d0 0x7fb0ec19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.347+0000 7fb0f3200640 1 --2- 172.21.15.22:0/3851136016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0ec100870 0x7fb0ec19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.347+0000 7fb0f3200640 1 -- 172.21.15.22:0/3851136016 >> 172.21.15.22:0/3851136016 conn(0x7fb0ec0fc3f0 msgr2=0x7fb0ec0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:51.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.347+0000 7fb0f3200640 1 -- 172.21.15.22:0/3851136016 shutdown_connections 2024-04-20T21:33:51.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.347+0000 7fb0f3200640 1 -- 172.21.15.22:0/3851136016 wait complete. 2024-04-20T21:33:51.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:51 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3825608189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:51.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:51 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:51.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:51 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:51.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:51 smithi022 ceph-mon[16843]: [20/Apr/2024:21:33:50] ENGINE Bus STARTING 2024-04-20T21:33:51.361 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:51 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1636142490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:51.361 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:51 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1343741548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:51.361 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:51 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:51.362 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-20T21:33:51.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:51.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:51 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3825608189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:51.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:51 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:51.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:51 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:51.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:51 smithi122 ceph-mon[22147]: [20/Apr/2024:21:33:50] ENGINE Bus STARTING 2024-04-20T21:33:51.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:51 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1636142490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:51.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:51 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1343741548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:51.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:51 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:51.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.576+0000 7ff13f600640 1 -- 172.21.15.22:0/1542188065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138100270 msgr2=0x7ff138100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:51.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.576+0000 7ff13f600640 1 --2- 172.21.15.22:0/1542188065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138100270 0x7ff138100690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009900 tx=0x7ff12c02bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:51.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.576+0000 7ff13f600640 1 -- 172.21.15.22:0/1542188065 shutdown_connections 2024-04-20T21:33:51.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.576+0000 7ff13f600640 1 --2- 172.21.15.22:0/1542188065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff138101990 0x7ff138106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.576+0000 7ff13f600640 1 --2- 172.21.15.22:0/1542188065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138100fd0 0x7ff138101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.576+0000 7ff13f600640 1 --2- 172.21.15.22:0/1542188065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138100270 0x7ff138100690 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.576+0000 7ff13f600640 1 -- 172.21.15.22:0/1542188065 >> 172.21.15.22:0/1542188065 conn(0x7ff1380fbba0 msgr2=0x7ff1380fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:51.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.576+0000 7ff13f600640 1 -- 172.21.15.22:0/1542188065 shutdown_connections 2024-04-20T21:33:51.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.576+0000 7ff13f600640 1 -- 172.21.15.22:0/1542188065 wait complete. 2024-04-20T21:33:51.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.577+0000 7ff13f600640 1 Processor -- start 2024-04-20T21:33:51.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.577+0000 7ff13f600640 1 -- start start 2024-04-20T21:33:51.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.577+0000 7ff13f600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138100270 0x7ff13819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:51.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.577+0000 7ff13f600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138100fd0 0x7ff13819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:51.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.577+0000 7ff13f600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff138101990 0x7ff1381a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:51.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.577+0000 7ff13f600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff138114280 con 0x7ff138101990 2024-04-20T21:33:51.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.577+0000 7ff13f600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff138114100 con 0x7ff138100270 2024-04-20T21:33:51.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.577+0000 7ff13f600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff138114400 con 0x7ff138100fd0 2024-04-20T21:33:51.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.578+0000 7ff13cc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138100270 0x7ff13819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.578+0000 7ff137e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138100fd0 0x7ff13819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.578+0000 7ff13cc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138100270 0x7ff13819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:40106/0 (socket says 172.21.15.22:40106) 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.578+0000 7ff13cc00640 1 -- 172.21.15.22:0/495420605 learned_addr learned my addr 172.21.15.22:0/495420605 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.578+0000 7ff13d600640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff138101990 0x7ff1381a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.578+0000 7ff13cc00640 1 -- 172.21.15.22:0/495420605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138100fd0 msgr2=0x7ff13819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.578+0000 7ff13cc00640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138100fd0 0x7ff13819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.578+0000 7ff13cc00640 1 -- 172.21.15.22:0/495420605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff138101990 msgr2=0x7ff1381a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.578+0000 7ff13cc00640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff138101990 0x7ff1381a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.578+0000 7ff13cc00640 1 -- 172.21.15.22:0/495420605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1381a64c0 con 0x7ff138100270 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.578+0000 7ff137e00640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138100fd0 0x7ff13819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.579+0000 7ff13cc00640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138100270 0x7ff13819e8f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff12c02c8f0 tx=0x7ff12c02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.579+0000 7ff135600640 1 -- 172.21.15.22:0/495420605 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c041070 con 0x7ff138100270 2024-04-20T21:33:51.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.579+0000 7ff135600640 1 -- 172.21.15.22:0/495420605 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff12c004450 con 0x7ff138100270 2024-04-20T21:33:51.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.579+0000 7ff135600640 1 -- 172.21.15.22:0/495420605 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c034440 con 0x7ff138100270 2024-04-20T21:33:51.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.579+0000 7ff13f600640 1 -- 172.21.15.22:0/495420605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1381a6750 con 0x7ff138100270 2024-04-20T21:33:51.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.579+0000 7ff13f600640 1 -- 172.21.15.22:0/495420605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1381a6bf0 con 0x7ff138100270 2024-04-20T21:33:51.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.580+0000 7ff13f600640 1 -- 172.21.15.22:0/495420605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1381a6e50 con 0x7ff138100270 2024-04-20T21:33:51.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.582+0000 7ff135600640 1 -- 172.21.15.22:0/495420605 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7ff12c044070 con 0x7ff138100270 2024-04-20T21:33:51.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.584+0000 7ff135600640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff1200a3d20 0x7ff1200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:51.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.584+0000 7ff135600640 1 -- 172.21.15.22:0/495420605 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7ff12c0f2450 con 0x7ff138100270 2024-04-20T21:33:51.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.584+0000 7ff137e00640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff1200a3d20 0x7ff1200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:51.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.585+0000 7ff137e00640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff1200a3d20 0x7ff1200a61e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff13819fdb0 tx=0x7ff124005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:51.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.586+0000 7ff135600640 1 -- 172.21.15.22:0/495420605 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff12c0be530 con 0x7ff138100270 2024-04-20T21:33:51.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.832+0000 7ff13f600640 1 -- 172.21.15.22:0/495420605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1381a7020 con 0x7ff138100270 2024-04-20T21:33:51.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.833+0000 7ff135600640 1 -- 172.21.15.22:0/495420605 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7ff12c0c2830 con 0x7ff138100270 2024-04-20T21:33:51.831 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:51.832 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:33:51.832 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:33:51.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.836+0000 7ff13f600640 1 -- 172.21.15.22:0/495420605 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff1200a3d20 msgr2=0x7ff1200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:51.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.836+0000 7ff13f600640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff1200a3d20 0x7ff1200a61e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff13819fdb0 tx=0x7ff124005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:51.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.836+0000 7ff13f600640 1 -- 172.21.15.22:0/495420605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138100270 msgr2=0x7ff13819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:51.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.836+0000 7ff13f600640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138100270 0x7ff13819e8f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff12c02c8f0 tx=0x7ff12c02c660 comp rx=0 tx=0).stop 2024-04-20T21:33:51.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.836+0000 7ff13f600640 1 -- 172.21.15.22:0/495420605 shutdown_connections 2024-04-20T21:33:51.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.836+0000 7ff13f600640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff1200a3d20 0x7ff1200a61e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.836+0000 7ff13f600640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff138101990 0x7ff1381a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.836+0000 7ff13f600640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138100fd0 0x7ff13819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.836+0000 7ff13f600640 1 --2- 172.21.15.22:0/495420605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138100270 0x7ff13819e8f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:51.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.836+0000 7ff13f600640 1 -- 172.21.15.22:0/495420605 >> 172.21.15.22:0/495420605 conn(0x7ff1380fbba0 msgr2=0x7ff1380fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:51.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.836+0000 7ff13f600640 1 -- 172.21.15.22:0/495420605 shutdown_connections 2024-04-20T21:33:51.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:51.836+0000 7ff13f600640 1 -- 172.21.15.22:0/495420605 wait complete. 2024-04-20T21:33:51.854 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-20T21:33:51.854 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-20T21:33:51.854 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:33:52.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.053+0000 7f794a800640 1 -- 172.21.15.22:0/3570582305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7944075fa0 msgr2=0x7f79440743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:52.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.053+0000 7f794a800640 1 --2- 172.21.15.22:0/3570582305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7944075fa0 0x7f79440743f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7938009900 tx=0x7f793802bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:52.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.054+0000 7f794a800640 1 -- 172.21.15.22:0/3570582305 shutdown_connections 2024-04-20T21:33:52.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.054+0000 7f794a800640 1 --2- 172.21.15.22:0/3570582305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7944109cc0 0x7f794410c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.054+0000 7f794a800640 1 --2- 172.21.15.22:0/3570582305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79440749c0 0x7f7944074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.054+0000 7f794a800640 1 --2- 172.21.15.22:0/3570582305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7944075fa0 0x7f79440743f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.054+0000 7f794a800640 1 -- 172.21.15.22:0/3570582305 >> 172.21.15.22:0/3570582305 conn(0x7f79440fd370 msgr2=0x7f79440ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:52.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.054+0000 7f794a800640 1 -- 172.21.15.22:0/3570582305 shutdown_connections 2024-04-20T21:33:52.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.054+0000 7f794a800640 1 -- 172.21.15.22:0/3570582305 wait complete. 2024-04-20T21:33:52.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.055+0000 7f794a800640 1 Processor -- start 2024-04-20T21:33:52.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.055+0000 7f794a800640 1 -- start start 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.056+0000 7f794a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79440749c0 0x7f794419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.056+0000 7f794a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7944075fa0 0x7f794419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.056+0000 7f794a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7944109cc0 0x7f79441a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.056+0000 7f794a800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f794410fc90 con 0x7f7944075fa0 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.056+0000 7f794a800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f794410fb10 con 0x7f79440749c0 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.056+0000 7f794a800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f794410fe10 con 0x7f7944109cc0 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.056+0000 7f7949400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79440749c0 0x7f794419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.056+0000 7f7949400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79440749c0 0x7f794419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:40122/0 (socket says 172.21.15.22:40122) 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.056+0000 7f7949e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7944109cc0 0x7f79441a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.056+0000 7f7949400640 1 -- 172.21.15.22:0/1280151188 learned_addr learned my addr 172.21.15.22:0/1280151188 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.056+0000 7f7948a00640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7944075fa0 0x7f794419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.057+0000 7f7949400640 1 -- 172.21.15.22:0/1280151188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7944109cc0 msgr2=0x7f79441a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.057+0000 7f7949400640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7944109cc0 0x7f79441a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.057+0000 7f7949400640 1 -- 172.21.15.22:0/1280151188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7944075fa0 msgr2=0x7f794419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:52.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.057+0000 7f7949400640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7944075fa0 0x7f794419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.057+0000 7f7949400640 1 -- 172.21.15.22:0/1280151188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79441a1f50 con 0x7f79440749c0 2024-04-20T21:33:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.057+0000 7f7949e00640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7944109cc0 0x7f79441a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.057+0000 7f7949400640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79440749c0 0x7f794419a330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7938000c00 tx=0x7f793802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.057+0000 7f7942000640 1 -- 172.21.15.22:0/1280151188 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938043020 con 0x7f79440749c0 2024-04-20T21:33:52.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.057+0000 7f7942000640 1 -- 172.21.15.22:0/1280151188 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79380054e0 con 0x7f79440749c0 2024-04-20T21:33:52.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.057+0000 7f794a800640 1 -- 172.21.15.22:0/1280151188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79441a21e0 con 0x7f79440749c0 2024-04-20T21:33:52.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.057+0000 7f7942000640 1 -- 172.21.15.22:0/1280151188 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938003bd0 con 0x7f79440749c0 2024-04-20T21:33:52.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.057+0000 7f794a800640 1 -- 172.21.15.22:0/1280151188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79441a2730 con 0x7f79440749c0 2024-04-20T21:33:52.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.059+0000 7f794a800640 1 -- 172.21.15.22:0/1280151188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79441089f0 con 0x7f79440749c0 2024-04-20T21:33:52.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.061+0000 7f7942000640 1 -- 172.21.15.22:0/1280151188 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7f7938046020 con 0x7f79440749c0 2024-04-20T21:33:52.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.062+0000 7f7942000640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f792c0a3d20 0x7f792c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:52.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.062+0000 7f7942000640 1 -- 172.21.15.22:0/1280151188 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f79380f27d0 con 0x7f79440749c0 2024-04-20T21:33:52.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.062+0000 7f7948a00640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f792c0a3d20 0x7f792c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:52.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.063+0000 7f7948a00640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f792c0a3d20 0x7f792c0a61e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f794419b840 tx=0x7f7930005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:52.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.066+0000 7f7942000640 1 -- 172.21.15.22:0/1280151188 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f793804d020 con 0x7f79440749c0 2024-04-20T21:33:52.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:52 smithi033 ceph-mon[21626]: [20/Apr/2024:21:33:50] ENGINE Serving on https://172.21.15.22:7150 2024-04-20T21:33:52.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:52 smithi033 ceph-mon[21626]: [20/Apr/2024:21:33:50] ENGINE Serving on http://172.21.15.22:8765 2024-04-20T21:33:52.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:52 smithi033 ceph-mon[21626]: [20/Apr/2024:21:33:50] ENGINE Bus STARTED 2024-04-20T21:33:52.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:52 smithi033 ceph-mon[21626]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:52.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:52 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:52.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:52 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:52.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:52 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:52.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:52 smithi033 ceph-mon[21626]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-04-20T21:33:52.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:52 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3851136016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:52.304 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:52 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/495420605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:52.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.308+0000 7f794a800640 1 -- 172.21.15.22:0/1280151188 --> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f79440623a0 con 0x7f792c0a3d20 2024-04-20T21:33:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.318+0000 7f7942000640 1 -- 172.21.15.22:0/1280151188 <== mgr.15024 v2:172.21.15.22:6800/975472489 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257707 (secure 0 0 0) 0x7f79440623a0 con 0x7f792c0a3d20 2024-04-20T21:33:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:52.321 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-20T21:33:52.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.322+0000 7f794a800640 1 -- 172.21.15.22:0/1280151188 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f792c0a3d20 msgr2=0x7f792c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:52.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.322+0000 7f794a800640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f792c0a3d20 0x7f792c0a61e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f794419b840 tx=0x7f7930005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:52.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.322+0000 7f794a800640 1 -- 172.21.15.22:0/1280151188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79440749c0 msgr2=0x7f794419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:52.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.322+0000 7f794a800640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79440749c0 0x7f794419a330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7938000c00 tx=0x7f793802c650 comp rx=0 tx=0).stop 2024-04-20T21:33:52.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.322+0000 7f794a800640 1 -- 172.21.15.22:0/1280151188 shutdown_connections 2024-04-20T21:33:52.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.322+0000 7f794a800640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f792c0a3d20 0x7f792c0a61e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.322+0000 7f794a800640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7944109cc0 0x7f79441a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.322+0000 7f794a800640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7944075fa0 0x7f794419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.322+0000 7f794a800640 1 --2- 172.21.15.22:0/1280151188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79440749c0 0x7f794419a330 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.322+0000 7f794a800640 1 -- 172.21.15.22:0/1280151188 >> 172.21.15.22:0/1280151188 conn(0x7f79440fd370 msgr2=0x7f7944105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.322+0000 7f794a800640 1 -- 172.21.15.22:0/1280151188 shutdown_connections 2024-04-20T21:33:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.323+0000 7f794a800640 1 -- 172.21.15.22:0/1280151188 wait complete. 2024-04-20T21:33:52.338 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:52 smithi022 ceph-mon[16843]: [20/Apr/2024:21:33:50] ENGINE Serving on https://172.21.15.22:7150 2024-04-20T21:33:52.338 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:52 smithi022 ceph-mon[16843]: [20/Apr/2024:21:33:50] ENGINE Serving on http://172.21.15.22:8765 2024-04-20T21:33:52.338 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:52 smithi022 ceph-mon[16843]: [20/Apr/2024:21:33:50] ENGINE Bus STARTED 2024-04-20T21:33:52.338 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:52 smithi022 ceph-mon[16843]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:52.338 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:52 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:52.338 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:52 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:52.338 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:52 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:52.338 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:52 smithi022 ceph-mon[16843]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-04-20T21:33:52.338 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:52 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3851136016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:52.338 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:52 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/495420605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:52.340 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-20T21:33:50.845424+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1183976,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":523,"num_read_kb":415,"num_write":383,"num_write_kb":4344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385388,"kb_used_data":15144,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124556436,"statfs":{"total":1151940427776,"available":1151545790464,"internally_reserved":0,"allocated":15507456,"data_stored":9002119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.977533"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966208+0000","last_change":"2024-04-20T21:31:27.527151+0000","last_active":"2024-04-20T21:33:48.966208+0000","last_peered":"2024-04-20T21:33:48.966208+0000","last_clean":"2024-04-20T21:33:48.966208+0000","last_became_active":"2024-04-20T21:31:27.526739+0000","last_became_peered":"2024-04-20T21:31:27.526739+0000","last_unstale":"2024-04-20T21:33:48.966208+0000","last_undegraded":"2024-04-20T21:33:48.966208+0000","last_fullsized":"2024-04-20T21:33:48.966208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:16:39.204523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120131+0000","last_change":"2024-04-20T21:31:27.527437+0000","last_active":"2024-04-20T21:33:49.120131+0000","last_peered":"2024-04-20T21:33:49.120131+0000","last_clean":"2024-04-20T21:33:49.120131+0000","last_became_active":"2024-04-20T21:31:27.527040+0000","last_became_peered":"2024-04-20T21:31:27.527040+0000","last_unstale":"2024-04-20T21:33:49.120131+0000","last_undegraded":"2024-04-20T21:33:49.120131+0000","last_fullsized":"2024-04-20T21:33:49.120131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:53:39.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119810+0000","last_change":"2024-04-20T21:31:27.531671+0000","last_active":"2024-04-20T21:33:49.119810+0000","last_peered":"2024-04-20T21:33:49.119810+0000","last_clean":"2024-04-20T21:33:49.119810+0000","last_became_active":"2024-04-20T21:31:27.531267+0000","last_became_peered":"2024-04-20T21:31:27.531267+0000","last_unstale":"2024-04-20T21:33:49.119810+0000","last_undegraded":"2024-04-20T21:33:49.119810+0000","last_fullsized":"2024-04-20T21:33:49.119810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:47.002481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119241+0000","last_change":"2024-04-20T21:31:27.512033+0000","last_active":"2024-04-20T21:33:49.119241+0000","last_peered":"2024-04-20T21:33:49.119241+0000","last_clean":"2024-04-20T21:33:49.119241+0000","last_became_active":"2024-04-20T21:31:27.510801+0000","last_became_peered":"2024-04-20T21:31:27.510801+0000","last_unstale":"2024-04-20T21:33:49.119241+0000","last_undegraded":"2024-04-20T21:33:49.119241+0000","last_fullsized":"2024-04-20T21:33:49.119241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:43:25.407173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148706+0000","last_change":"2024-04-20T21:31:27.545862+0000","last_active":"2024-04-20T21:33:32.148706+0000","last_peered":"2024-04-20T21:33:32.148706+0000","last_clean":"2024-04-20T21:33:32.148706+0000","last_became_active":"2024-04-20T21:31:27.545500+0000","last_became_peered":"2024-04-20T21:31:27.545500+0000","last_unstale":"2024-04-20T21:33:32.148706+0000","last_undegraded":"2024-04-20T21:33:32.148706+0000","last_fullsized":"2024-04-20T21:33:32.148706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:37.640777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828391+0000","last_change":"2024-04-20T21:31:27.526408+0000","last_active":"2024-04-20T21:33:48.828391+0000","last_peered":"2024-04-20T21:33:48.828391+0000","last_clean":"2024-04-20T21:33:48.828391+0000","last_became_active":"2024-04-20T21:31:27.526015+0000","last_became_peered":"2024-04-20T21:31:27.526015+0000","last_unstale":"2024-04-20T21:33:48.828391+0000","last_undegraded":"2024-04-20T21:33:48.828391+0000","last_fullsized":"2024-04-20T21:33:48.828391+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:56.904696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119156+0000","last_change":"2024-04-20T21:31:27.526272+0000","last_active":"2024-04-20T21:33:49.119156+0000","last_peered":"2024-04-20T21:33:49.119156+0000","last_clean":"2024-04-20T21:33:49.119156+0000","last_became_active":"2024-04-20T21:31:27.525852+0000","last_became_peered":"2024-04-20T21:31:27.525852+0000","last_unstale":"2024-04-20T21:33:49.119156+0000","last_undegraded":"2024-04-20T21:33:49.119156+0000","last_fullsized":"2024-04-20T21:33:49.119156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:33:57.811339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.962898+0000","last_change":"2024-04-20T21:32:31.998671+0000","last_active":"2024-04-20T21:33:48.962898+0000","last_peered":"2024-04-20T21:33:48.962898+0000","last_clean":"2024-04-20T21:33:48.962898+0000","last_became_active":"2024-04-20T21:32:31.996769+0000","last_became_peered":"2024-04-20T21:32:31.996769+0000","last_unstale":"2024-04-20T21:33:48.962898+0000","last_undegraded":"2024-04-20T21:33:48.962898+0000","last_fullsized":"2024-04-20T21:33:48.962898+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:03.135794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120675+0000","last_change":"2024-04-20T21:31:27.514810+0000","last_active":"2024-04-20T21:33:49.120675+0000","last_peered":"2024-04-20T21:33:49.120675+0000","last_clean":"2024-04-20T21:33:49.120675+0000","last_became_active":"2024-04-20T21:31:27.512548+0000","last_became_peered":"2024-04-20T21:31:27.512548+0000","last_unstale":"2024-04-20T21:33:49.120675+0000","last_undegraded":"2024-04-20T21:33:49.120675+0000","last_fullsized":"2024-04-20T21:33:49.120675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:11:30.375294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828858+0000","last_change":"2024-04-20T21:31:27.510303+0000","last_active":"2024-04-20T21:33:48.828858+0000","last_peered":"2024-04-20T21:33:48.828858+0000","last_clean":"2024-04-20T21:33:48.828858+0000","last_became_active":"2024-04-20T21:31:27.509907+0000","last_became_peered":"2024-04-20T21:31:27.509907+0000","last_unstale":"2024-04-20T21:33:48.828858+0000","last_undegraded":"2024-04-20T21:33:48.828858+0000","last_fullsized":"2024-04-20T21:33:48.828858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:31.160785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119674+0000","last_change":"2024-04-20T21:31:27.513749+0000","last_active":"2024-04-20T21:33:49.119674+0000","last_peered":"2024-04-20T21:33:49.119674+0000","last_clean":"2024-04-20T21:33:49.119674+0000","last_became_active":"2024-04-20T21:31:27.513404+0000","last_became_peered":"2024-04-20T21:31:27.513404+0000","last_unstale":"2024-04-20T21:33:49.119674+0000","last_undegraded":"2024-04-20T21:33:49.119674+0000","last_fullsized":"2024-04-20T21:33:49.119674+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:26.279697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118958+0000","last_change":"2024-04-20T21:31:27.507727+0000","last_active":"2024-04-20T21:33:49.118958+0000","last_peered":"2024-04-20T21:33:49.118958+0000","last_clean":"2024-04-20T21:33:49.118958+0000","last_became_active":"2024-04-20T21:31:27.507363+0000","last_became_peered":"2024-04-20T21:31:27.507363+0000","last_unstale":"2024-04-20T21:33:49.118958+0000","last_undegraded":"2024-04-20T21:33:49.118958+0000","last_fullsized":"2024-04-20T21:33:49.118958+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:01.139963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.834435+0000","last_change":"2024-04-20T21:31:27.512941+0000","last_active":"2024-04-20T21:33:48.834435+0000","last_peered":"2024-04-20T21:33:48.834435+0000","last_clean":"2024-04-20T21:33:48.834435+0000","last_became_active":"2024-04-20T21:31:27.512571+0000","last_became_peered":"2024-04-20T21:31:27.512571+0000","last_unstale":"2024-04-20T21:33:48.834435+0000","last_undegraded":"2024-04-20T21:33:48.834435+0000","last_fullsized":"2024-04-20T21:33:48.834435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:30.977624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119024+0000","last_change":"2024-04-20T21:31:27.521439+0000","last_active":"2024-04-20T21:33:49.119024+0000","last_peered":"2024-04-20T21:33:49.119024+0000","last_clean":"2024-04-20T21:33:49.119024+0000","last_became_active":"2024-04-20T21:31:27.520245+0000","last_became_peered":"2024-04-20T21:31:27.520245+0000","last_unstale":"2024-04-20T21:33:49.119024+0000","last_undegraded":"2024-04-20T21:33:49.119024+0000","last_fullsized":"2024-04-20T21:33:49.119024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:53:26.510845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118750+0000","last_change":"2024-04-20T21:32:31.619209+0000","last_active":"2024-04-20T21:33:49.118750+0000","last_peered":"2024-04-20T21:33:49.118750+0000","last_clean":"2024-04-20T21:33:49.118750+0000","last_became_active":"2024-04-20T21:32:31.618527+0000","last_became_peered":"2024-04-20T21:32:31.618527+0000","last_unstale":"2024-04-20T21:33:49.118750+0000","last_undegraded":"2024-04-20T21:33:49.118750+0000","last_fullsized":"2024-04-20T21:33:49.118750+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:54:21.400896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118476+0000","last_change":"2024-04-20T21:31:27.521533+0000","last_active":"2024-04-20T21:33:49.118476+0000","last_peered":"2024-04-20T21:33:49.118476+0000","last_clean":"2024-04-20T21:33:49.118476+0000","last_became_active":"2024-04-20T21:31:27.520920+0000","last_became_peered":"2024-04-20T21:31:27.520920+0000","last_unstale":"2024-04-20T21:33:49.118476+0000","last_undegraded":"2024-04-20T21:33:49.118476+0000","last_fullsized":"2024-04-20T21:33:49.118476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:28:13.605238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966062+0000","last_change":"2024-04-20T21:31:27.527484+0000","last_active":"2024-04-20T21:33:48.966062+0000","last_peered":"2024-04-20T21:33:48.966062+0000","last_clean":"2024-04-20T21:33:48.966062+0000","last_became_active":"2024-04-20T21:31:27.527048+0000","last_became_peered":"2024-04-20T21:31:27.527048+0000","last_unstale":"2024-04-20T21:33:48.966062+0000","last_undegraded":"2024-04-20T21:33:48.966062+0000","last_fullsized":"2024-04-20T21:33:48.966062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:35:56.467661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963479+0000","last_change":"2024-04-20T21:31:27.527870+0000","last_active":"2024-04-20T21:33:48.963479+0000","last_peered":"2024-04-20T21:33:48.963479+0000","last_clean":"2024-04-20T21:33:48.963479+0000","last_became_active":"2024-04-20T21:31:27.525918+0000","last_became_peered":"2024-04-20T21:31:27.525918+0000","last_unstale":"2024-04-20T21:33:48.963479+0000","last_undegraded":"2024-04-20T21:33:48.963479+0000","last_fullsized":"2024-04-20T21:33:48.963479+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:34.425370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118470+0000","last_change":"2024-04-20T21:31:27.544987+0000","last_active":"2024-04-20T21:33:49.118470+0000","last_peered":"2024-04-20T21:33:49.118470+0000","last_clean":"2024-04-20T21:33:49.118470+0000","last_became_active":"2024-04-20T21:31:27.544628+0000","last_became_peered":"2024-04-20T21:31:27.544628+0000","last_unstale":"2024-04-20T21:33:49.118470+0000","last_undegraded":"2024-04-20T21:33:49.118470+0000","last_fullsized":"2024-04-20T21:33:49.118470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:05:42.396775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963603+0000","last_change":"2024-04-20T21:31:26.702803+0000","last_active":"2024-04-20T21:33:48.963603+0000","last_peered":"2024-04-20T21:33:48.963603+0000","last_clean":"2024-04-20T21:33:48.963603+0000","last_became_active":"2024-04-20T21:31:26.695781+0000","last_became_peered":"2024-04-20T21:31:26.695781+0000","last_unstale":"2024-04-20T21:33:48.963603+0000","last_undegraded":"2024-04-20T21:33:48.963603+0000","last_fullsized":"2024-04-20T21:33:48.963603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:49:17.131832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.830615+0000","last_change":"2024-04-20T21:31:27.535264+0000","last_active":"2024-04-20T21:33:48.830615+0000","last_peered":"2024-04-20T21:33:48.830615+0000","last_clean":"2024-04-20T21:33:48.830615+0000","last_became_active":"2024-04-20T21:31:27.534888+0000","last_became_peered":"2024-04-20T21:31:27.534888+0000","last_unstale":"2024-04-20T21:33:48.830615+0000","last_undegraded":"2024-04-20T21:33:48.830615+0000","last_fullsized":"2024-04-20T21:33:48.830615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:04:20.951916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.830901+0000","last_change":"2024-04-20T21:31:26.700378+0000","last_active":"2024-04-20T21:33:48.830901+0000","last_peered":"2024-04-20T21:33:48.830901+0000","last_clean":"2024-04-20T21:33:48.830901+0000","last_became_active":"2024-04-20T21:31:26.700077+0000","last_became_peered":"2024-04-20T21:31:26.700077+0000","last_unstale":"2024-04-20T21:33:48.830901+0000","last_undegraded":"2024-04-20T21:33:48.830901+0000","last_fullsized":"2024-04-20T21:33:48.830901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:34:21.646402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120548+0000","last_change":"2024-04-20T21:31:27.531069+0000","last_active":"2024-04-20T21:33:49.120548+0000","last_peered":"2024-04-20T21:33:49.120548+0000","last_clean":"2024-04-20T21:33:49.120548+0000","last_became_active":"2024-04-20T21:31:27.530457+0000","last_became_peered":"2024-04-20T21:31:27.530457+0000","last_unstale":"2024-04-20T21:33:49.120548+0000","last_undegraded":"2024-04-20T21:33:49.120548+0000","last_fullsized":"2024-04-20T21:33:49.120548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:46:59.829623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.149054+0000","last_change":"2024-04-20T21:31:26.684150+0000","last_active":"2024-04-20T21:33:32.149054+0000","last_peered":"2024-04-20T21:33:32.149054+0000","last_clean":"2024-04-20T21:33:32.149054+0000","last_became_active":"2024-04-20T21:31:26.683823+0000","last_became_peered":"2024-04-20T21:31:26.683823+0000","last_unstale":"2024-04-20T21:33:32.149054+0000","last_undegraded":"2024-04-20T21:33:32.149054+0000","last_fullsized":"2024-04-20T21:33:32.149054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:14.413606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120262+0000","last_change":"2024-04-20T21:31:27.507499+0000","last_active":"2024-04-20T21:33:49.120262+0000","last_peered":"2024-04-20T21:33:49.120262+0000","last_clean":"2024-04-20T21:33:49.120262+0000","last_became_active":"2024-04-20T21:31:27.506922+0000","last_became_peered":"2024-04-20T21:31:27.506922+0000","last_unstale":"2024-04-20T21:33:49.120262+0000","last_undegraded":"2024-04-20T21:33:49.120262+0000","last_fullsized":"2024-04-20T21:33:49.120262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:40.865232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.965750+0000","last_change":"2024-04-20T21:32:31.613577+0000","last_active":"2024-04-20T21:33:48.965750+0000","last_peered":"2024-04-20T21:33:48.965750+0000","last_clean":"2024-04-20T21:33:48.965750+0000","last_became_active":"2024-04-20T21:32:31.608943+0000","last_became_peered":"2024-04-20T21:32:31.608943+0000","last_unstale":"2024-04-20T21:33:48.965750+0000","last_undegraded":"2024-04-20T21:33:48.965750+0000","last_fullsized":"2024-04-20T21:33:48.965750+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:06.561157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976247+0000","last_change":"2024-04-20T21:31:27.517839+0000","last_active":"2024-04-20T21:33:48.976247+0000","last_peered":"2024-04-20T21:33:48.976247+0000","last_clean":"2024-04-20T21:33:48.976247+0000","last_became_active":"2024-04-20T21:31:27.517044+0000","last_became_peered":"2024-04-20T21:31:27.517044+0000","last_unstale":"2024-04-20T21:33:48.976247+0000","last_undegraded":"2024-04-20T21:33:48.976247+0000","last_fullsized":"2024-04-20T21:33:48.976247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:50.042665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976109+0000","last_change":"2024-04-20T21:31:26.686151+0000","last_active":"2024-04-20T21:33:48.976109+0000","last_peered":"2024-04-20T21:33:48.976109+0000","last_clean":"2024-04-20T21:33:48.976109+0000","last_became_active":"2024-04-20T21:31:26.685827+0000","last_became_peered":"2024-04-20T21:31:26.685827+0000","last_unstale":"2024-04-20T21:33:48.976109+0000","last_undegraded":"2024-04-20T21:33:48.976109+0000","last_fullsized":"2024-04-20T21:33:48.976109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:41:17.263420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148585+0000","last_change":"2024-04-20T21:32:31.620563+0000","last_active":"2024-04-20T21:33:32.148585+0000","last_peered":"2024-04-20T21:33:32.148585+0000","last_clean":"2024-04-20T21:33:32.148585+0000","last_became_active":"2024-04-20T21:32:31.620145+0000","last_became_peered":"2024-04-20T21:32:31.620145+0000","last_unstale":"2024-04-20T21:33:32.148585+0000","last_undegraded":"2024-04-20T21:33:32.148585+0000","last_fullsized":"2024-04-20T21:33:32.148585+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:43.609490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828355+0000","last_change":"2024-04-20T21:31:26.684573+0000","last_active":"2024-04-20T21:33:48.828355+0000","last_peered":"2024-04-20T21:33:48.828355+0000","last_clean":"2024-04-20T21:33:48.828355+0000","last_became_active":"2024-04-20T21:31:26.684163+0000","last_became_peered":"2024-04-20T21:31:26.684163+0000","last_unstale":"2024-04-20T21:33:48.828355+0000","last_undegraded":"2024-04-20T21:33:48.828355+0000","last_fullsized":"2024-04-20T21:33:48.828355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:39.445068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976276+0000","last_change":"2024-04-20T21:31:27.543680+0000","last_active":"2024-04-20T21:33:48.976276+0000","last_peered":"2024-04-20T21:33:48.976276+0000","last_clean":"2024-04-20T21:33:48.976276+0000","last_became_active":"2024-04-20T21:31:27.543379+0000","last_became_peered":"2024-04-20T21:31:27.543379+0000","last_unstale":"2024-04-20T21:33:48.976276+0000","last_undegraded":"2024-04-20T21:33:48.976276+0000","last_fullsized":"2024-04-20T21:33:48.976276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:58:17.168813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.827019+0000","last_change":"2024-04-20T21:31:26.689665+0000","last_active":"2024-04-20T21:33:48.827019+0000","last_peered":"2024-04-20T21:33:48.827019+0000","last_clean":"2024-04-20T21:33:48.827019+0000","last_became_active":"2024-04-20T21:31:26.689175+0000","last_became_peered":"2024-04-20T21:31:26.689175+0000","last_unstale":"2024-04-20T21:33:48.827019+0000","last_undegraded":"2024-04-20T21:33:48.827019+0000","last_fullsized":"2024-04-20T21:33:48.827019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:21.342700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.829688+0000","last_change":"2024-04-20T21:31:27.544137+0000","last_active":"2024-04-20T21:33:48.829688+0000","last_peered":"2024-04-20T21:33:48.829688+0000","last_clean":"2024-04-20T21:33:48.829688+0000","last_became_active":"2024-04-20T21:31:27.542841+0000","last_became_peered":"2024-04-20T21:31:27.542841+0000","last_unstale":"2024-04-20T21:33:48.829688+0000","last_undegraded":"2024-04-20T21:33:48.829688+0000","last_fullsized":"2024-04-20T21:33:48.829688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:04.610144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.975963+0000","last_change":"2024-04-20T21:31:26.681937+0000","last_active":"2024-04-20T21:33:48.975963+0000","last_peered":"2024-04-20T21:33:48.975963+0000","last_clean":"2024-04-20T21:33:48.975963+0000","last_became_active":"2024-04-20T21:31:26.681621+0000","last_became_peered":"2024-04-20T21:31:26.681621+0000","last_unstale":"2024-04-20T21:33:48.975963+0000","last_undegraded":"2024-04-20T21:33:48.975963+0000","last_fullsized":"2024-04-20T21:33:48.975963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:06.641608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963226+0000","last_change":"2024-04-20T21:31:27.540225+0000","last_active":"2024-04-20T21:33:48.963226+0000","last_peered":"2024-04-20T21:33:48.963226+0000","last_clean":"2024-04-20T21:33:48.963226+0000","last_became_active":"2024-04-20T21:31:27.539860+0000","last_became_peered":"2024-04-20T21:31:27.539860+0000","last_unstale":"2024-04-20T21:33:48.963226+0000","last_undegraded":"2024-04-20T21:33:48.963226+0000","last_fullsized":"2024-04-20T21:33:48.963226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:50.146761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828972+0000","last_change":"2024-04-20T21:31:26.701816+0000","last_active":"2024-04-20T21:33:48.828972+0000","last_peered":"2024-04-20T21:33:48.828972+0000","last_clean":"2024-04-20T21:33:48.828972+0000","last_became_active":"2024-04-20T21:31:26.701530+0000","last_became_peered":"2024-04-20T21:31:26.701530+0000","last_unstale":"2024-04-20T21:33:48.828972+0000","last_undegraded":"2024-04-20T21:33:48.828972+0000","last_fullsized":"2024-04-20T21:33:48.828972+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:31.175330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120556+0000","last_change":"2024-04-20T21:31:27.527222+0000","last_active":"2024-04-20T21:33:49.120556+0000","last_peered":"2024-04-20T21:33:49.120556+0000","last_clean":"2024-04-20T21:33:49.120556+0000","last_became_active":"2024-04-20T21:31:27.526779+0000","last_became_peered":"2024-04-20T21:31:27.526779+0000","last_unstale":"2024-04-20T21:33:49.120556+0000","last_undegraded":"2024-04-20T21:33:49.120556+0000","last_fullsized":"2024-04-20T21:33:49.120556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:13:54.500516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119650+0000","last_change":"2024-04-20T21:31:26.685969+0000","last_active":"2024-04-20T21:33:49.119650+0000","last_peered":"2024-04-20T21:33:49.119650+0000","last_clean":"2024-04-20T21:33:49.119650+0000","last_became_active":"2024-04-20T21:31:26.685580+0000","last_became_peered":"2024-04-20T21:31:26.685580+0000","last_unstale":"2024-04-20T21:33:49.119650+0000","last_undegraded":"2024-04-20T21:33:49.119650+0000","last_fullsized":"2024-04-20T21:33:49.119650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:22:50.558526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119011+0000","last_change":"2024-04-20T21:31:27.525862+0000","last_active":"2024-04-20T21:33:49.119011+0000","last_peered":"2024-04-20T21:33:49.119011+0000","last_clean":"2024-04-20T21:33:49.119011+0000","last_became_active":"2024-04-20T21:31:27.525372+0000","last_became_peered":"2024-04-20T21:31:27.525372+0000","last_unstale":"2024-04-20T21:33:49.119011+0000","last_undegraded":"2024-04-20T21:33:49.119011+0000","last_fullsized":"2024-04-20T21:33:49.119011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:44:15.527137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976460+0000","last_change":"2024-04-20T21:31:26.691091+0000","last_active":"2024-04-20T21:33:48.976460+0000","last_peered":"2024-04-20T21:33:48.976460+0000","last_clean":"2024-04-20T21:33:48.976460+0000","last_became_active":"2024-04-20T21:31:26.690608+0000","last_became_peered":"2024-04-20T21:31:26.690608+0000","last_unstale":"2024-04-20T21:33:48.976460+0000","last_undegraded":"2024-04-20T21:33:48.976460+0000","last_fullsized":"2024-04-20T21:33:48.976460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:44:25.336193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120411+0000","last_change":"2024-04-20T21:31:27.507900+0000","last_active":"2024-04-20T21:33:49.120411+0000","last_peered":"2024-04-20T21:33:49.120411+0000","last_clean":"2024-04-20T21:33:49.120411+0000","last_became_active":"2024-04-20T21:31:27.507360+0000","last_became_peered":"2024-04-20T21:31:27.507360+0000","last_unstale":"2024-04-20T21:33:49.120411+0000","last_undegraded":"2024-04-20T21:33:49.120411+0000","last_fullsized":"2024-04-20T21:33:49.120411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:27.110877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'33","reported_seq":348,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.265110+0000","last_change":"2024-04-20T21:31:26.685736+0000","last_active":"2024-04-20T21:33:49.265110+0000","last_peered":"2024-04-20T21:33:49.265110+0000","last_clean":"2024-04-20T21:33:49.265110+0000","last_became_active":"2024-04-20T21:31:26.685309+0000","last_became_peered":"2024-04-20T21:31:26.685309+0000","last_unstale":"2024-04-20T21:33:49.265110+0000","last_undegraded":"2024-04-20T21:33:49.265110+0000","last_fullsized":"2024-04-20T21:33:49.265110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:06.501753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67760,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":289,"num_read_kb":211,"num_write":33,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148851+0000","last_change":"2024-04-20T21:31:27.507532+0000","last_active":"2024-04-20T21:33:32.148851+0000","last_peered":"2024-04-20T21:33:32.148851+0000","last_clean":"2024-04-20T21:33:32.148851+0000","last_became_active":"2024-04-20T21:31:27.507188+0000","last_became_peered":"2024-04-20T21:31:27.507188+0000","last_unstale":"2024-04-20T21:33:32.148851+0000","last_undegraded":"2024-04-20T21:33:32.148851+0000","last_fullsized":"2024-04-20T21:33:32.148851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:58.814116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118339+0000","last_change":"2024-04-20T21:31:26.701361+0000","last_active":"2024-04-20T21:33:49.118339+0000","last_peered":"2024-04-20T21:33:49.118339+0000","last_clean":"2024-04-20T21:33:49.118339+0000","last_became_active":"2024-04-20T21:31:26.701068+0000","last_became_peered":"2024-04-20T21:31:26.701068+0000","last_unstale":"2024-04-20T21:33:49.118339+0000","last_undegraded":"2024-04-20T21:33:49.118339+0000","last_fullsized":"2024-04-20T21:33:49.118339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:22:51.745294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.830874+0000","last_change":"2024-04-20T21:31:27.542189+0000","last_active":"2024-04-20T21:33:48.830874+0000","last_peered":"2024-04-20T21:33:48.830874+0000","last_clean":"2024-04-20T21:33:48.830874+0000","last_became_active":"2024-04-20T21:31:27.541800+0000","last_became_peered":"2024-04-20T21:31:27.541800+0000","last_unstale":"2024-04-20T21:33:48.830874+0000","last_undegraded":"2024-04-20T21:33:48.830874+0000","last_fullsized":"2024-04-20T21:33:48.830874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:31.898443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976103+0000","last_change":"2024-04-20T21:31:26.687652+0000","last_active":"2024-04-20T21:33:48.976103+0000","last_peered":"2024-04-20T21:33:48.976103+0000","last_clean":"2024-04-20T21:33:48.976103+0000","last_became_active":"2024-04-20T21:31:26.687333+0000","last_became_peered":"2024-04-20T21:31:26.687333+0000","last_unstale":"2024-04-20T21:33:48.976103+0000","last_undegraded":"2024-04-20T21:33:48.976103+0000","last_fullsized":"2024-04-20T21:33:48.976103+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:12.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120140+0000","last_change":"2024-04-20T21:31:27.522535+0000","last_active":"2024-04-20T21:33:49.120140+0000","last_peered":"2024-04-20T21:33:49.120140+0000","last_clean":"2024-04-20T21:33:49.120140+0000","last_became_active":"2024-04-20T21:31:27.521821+0000","last_became_peered":"2024-04-20T21:31:27.521821+0000","last_unstale":"2024-04-20T21:33:49.120140+0000","last_undegraded":"2024-04-20T21:33:49.120140+0000","last_fullsized":"2024-04-20T21:33:49.120140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:47:53.401129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118630+0000","last_change":"2024-04-20T21:31:26.681758+0000","last_active":"2024-04-20T21:33:49.118630+0000","last_peered":"2024-04-20T21:33:49.118630+0000","last_clean":"2024-04-20T21:33:49.118630+0000","last_became_active":"2024-04-20T21:31:26.681395+0000","last_became_peered":"2024-04-20T21:31:26.681395+0000","last_unstale":"2024-04-20T21:33:49.118630+0000","last_undegraded":"2024-04-20T21:33:49.118630+0000","last_fullsized":"2024-04-20T21:33:49.118630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:33:00.924879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119081+0000","last_change":"2024-04-20T21:31:27.512873+0000","last_active":"2024-04-20T21:33:49.119081+0000","last_peered":"2024-04-20T21:33:49.119081+0000","last_clean":"2024-04-20T21:33:49.119081+0000","last_became_active":"2024-04-20T21:31:27.512554+0000","last_became_peered":"2024-04-20T21:31:27.512554+0000","last_unstale":"2024-04-20T21:33:49.119081+0000","last_undegraded":"2024-04-20T21:33:49.119081+0000","last_fullsized":"2024-04-20T21:33:49.119081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:26:06.670726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148549+0000","last_change":"2024-04-20T21:31:26.691210+0000","last_active":"2024-04-20T21:33:32.148549+0000","last_peered":"2024-04-20T21:33:32.148549+0000","last_clean":"2024-04-20T21:33:32.148549+0000","last_became_active":"2024-04-20T21:31:26.690859+0000","last_became_peered":"2024-04-20T21:31:26.690859+0000","last_unstale":"2024-04-20T21:33:32.148549+0000","last_undegraded":"2024-04-20T21:33:32.148549+0000","last_fullsized":"2024-04-20T21:33:32.148549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:36.789429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976179+0000","last_change":"2024-04-20T21:32:31.616413+0000","last_active":"2024-04-20T21:33:48.976179+0000","last_peered":"2024-04-20T21:33:48.976179+0000","last_clean":"2024-04-20T21:33:48.976179+0000","last_became_active":"2024-04-20T21:32:31.616011+0000","last_became_peered":"2024-04-20T21:32:31.616011+0000","last_unstale":"2024-04-20T21:33:48.976179+0000","last_undegraded":"2024-04-20T21:33:48.976179+0000","last_fullsized":"2024-04-20T21:33:48.976179+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:11:05.630337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.149077+0000","last_change":"2024-04-20T21:31:26.682311+0000","last_active":"2024-04-20T21:33:32.149077+0000","last_peered":"2024-04-20T21:33:32.149077+0000","last_clean":"2024-04-20T21:33:32.149077+0000","last_became_active":"2024-04-20T21:31:26.681996+0000","last_became_peered":"2024-04-20T21:31:26.681996+0000","last_unstale":"2024-04-20T21:33:32.149077+0000","last_undegraded":"2024-04-20T21:33:32.149077+0000","last_fullsized":"2024-04-20T21:33:32.149077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:32:37.659946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966120+0000","last_change":"2024-04-20T21:31:27.510922+0000","last_active":"2024-04-20T21:33:48.966120+0000","last_peered":"2024-04-20T21:33:48.966120+0000","last_clean":"2024-04-20T21:33:48.966120+0000","last_became_active":"2024-04-20T21:31:27.510553+0000","last_became_peered":"2024-04-20T21:31:27.510553+0000","last_unstale":"2024-04-20T21:33:48.966120+0000","last_undegraded":"2024-04-20T21:33:48.966120+0000","last_fullsized":"2024-04-20T21:33:48.966120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:23.754768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963235+0000","last_change":"2024-04-20T21:31:26.697010+0000","last_active":"2024-04-20T21:33:48.963235+0000","last_peered":"2024-04-20T21:33:48.963235+0000","last_clean":"2024-04-20T21:33:48.963235+0000","last_became_active":"2024-04-20T21:31:26.696112+0000","last_became_peered":"2024-04-20T21:31:26.696112+0000","last_unstale":"2024-04-20T21:33:48.963235+0000","last_undegraded":"2024-04-20T21:33:48.963235+0000","last_fullsized":"2024-04-20T21:33:48.963235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:57:02.715884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828760+0000","last_change":"2024-04-20T21:31:27.545025+0000","last_active":"2024-04-20T21:33:48.828760+0000","last_peered":"2024-04-20T21:33:48.828760+0000","last_clean":"2024-04-20T21:33:48.828760+0000","last_became_active":"2024-04-20T21:31:27.544630+0000","last_became_peered":"2024-04-20T21:31:27.544630+0000","last_unstale":"2024-04-20T21:33:48.828760+0000","last_undegraded":"2024-04-20T21:33:48.828760+0000","last_fullsized":"2024-04-20T21:33:48.828760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:57.292481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119540+0000","last_change":"2024-04-20T21:31:26.682909+0000","last_active":"2024-04-20T21:33:49.119540+0000","last_peered":"2024-04-20T21:33:49.119540+0000","last_clean":"2024-04-20T21:33:49.119540+0000","last_became_active":"2024-04-20T21:31:26.682597+0000","last_became_peered":"2024-04-20T21:31:26.682597+0000","last_unstale":"2024-04-20T21:33:49.119540+0000","last_undegraded":"2024-04-20T21:33:49.119540+0000","last_fullsized":"2024-04-20T21:33:49.119540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:31.889513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828594+0000","last_change":"2024-04-20T21:31:27.530217+0000","last_active":"2024-04-20T21:33:48.828594+0000","last_peered":"2024-04-20T21:33:48.828594+0000","last_clean":"2024-04-20T21:33:48.828594+0000","last_became_active":"2024-04-20T21:31:27.529780+0000","last_became_peered":"2024-04-20T21:31:27.529780+0000","last_unstale":"2024-04-20T21:33:48.828594+0000","last_undegraded":"2024-04-20T21:33:48.828594+0000","last_fullsized":"2024-04-20T21:33:48.828594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:21.613460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976309+0000","last_change":"2024-04-20T21:32:31.610727+0000","last_active":"2024-04-20T21:33:48.976309+0000","last_peered":"2024-04-20T21:33:48.976309+0000","last_clean":"2024-04-20T21:33:48.976309+0000","last_became_active":"2024-04-20T21:32:31.610321+0000","last_became_peered":"2024-04-20T21:32:31.610321+0000","last_unstale":"2024-04-20T21:33:48.976309+0000","last_undegraded":"2024-04-20T21:33:48.976309+0000","last_fullsized":"2024-04-20T21:33:48.976309+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:01:35.697137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.826910+0000","last_change":"2024-04-20T21:31:26.692845+0000","last_active":"2024-04-20T21:33:48.826910+0000","last_peered":"2024-04-20T21:33:48.826910+0000","last_clean":"2024-04-20T21:33:48.826910+0000","last_became_active":"2024-04-20T21:31:26.691915+0000","last_became_peered":"2024-04-20T21:31:26.691915+0000","last_unstale":"2024-04-20T21:33:48.826910+0000","last_undegraded":"2024-04-20T21:33:48.826910+0000","last_fullsized":"2024-04-20T21:33:48.826910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:24:00.436955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.832281+0000","last_change":"2024-04-20T21:31:27.541623+0000","last_active":"2024-04-20T21:33:48.832281+0000","last_peered":"2024-04-20T21:33:48.832281+0000","last_clean":"2024-04-20T21:33:48.832281+0000","last_became_active":"2024-04-20T21:31:27.540953+0000","last_became_peered":"2024-04-20T21:31:27.540953+0000","last_unstale":"2024-04-20T21:33:48.832281+0000","last_undegraded":"2024-04-20T21:33:48.832281+0000","last_fullsized":"2024-04-20T21:33:48.832281+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:03:15.091886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118799+0000","last_change":"2024-04-20T21:31:26.687373+0000","last_active":"2024-04-20T21:33:49.118799+0000","last_peered":"2024-04-20T21:33:49.118799+0000","last_clean":"2024-04-20T21:33:49.118799+0000","last_became_active":"2024-04-20T21:31:26.687009+0000","last_became_peered":"2024-04-20T21:31:26.687009+0000","last_unstale":"2024-04-20T21:33:49.118799+0000","last_undegraded":"2024-04-20T21:33:49.118799+0000","last_fullsized":"2024-04-20T21:33:49.118799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:23:42.708129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119190+0000","last_change":"2024-04-20T21:30:06.995711+0000","last_active":"2024-04-20T21:33:49.119190+0000","last_peered":"2024-04-20T21:33:49.119190+0000","last_clean":"2024-04-20T21:33:49.119190+0000","last_became_active":"2024-04-20T21:30:06.995280+0000","last_became_peered":"2024-04-20T21:30:06.995280+0000","last_unstale":"2024-04-20T21:33:49.119190+0000","last_undegraded":"2024-04-20T21:33:49.119190+0000","last_fullsized":"2024-04-20T21:33:49.119190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_clean_scrub_stamp":"2024-04-20T21:26:16.477494+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:49:26.014145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.826753+0000","last_change":"2024-04-20T21:31:27.514064+0000","last_active":"2024-04-20T21:33:48.826753+0000","last_peered":"2024-04-20T21:33:48.826753+0000","last_clean":"2024-04-20T21:33:48.826753+0000","last_became_active":"2024-04-20T21:31:27.513416+0000","last_became_peered":"2024-04-20T21:31:27.513416+0000","last_unstale":"2024-04-20T21:33:48.826753+0000","last_undegraded":"2024-04-20T21:33:48.826753+0000","last_fullsized":"2024-04-20T21:33:48.826753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:12.658265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119384+0000","last_change":"2024-04-20T21:31:26.680948+0000","last_active":"2024-04-20T21:33:49.119384+0000","last_peered":"2024-04-20T21:33:49.119384+0000","last_clean":"2024-04-20T21:33:49.119384+0000","last_became_active":"2024-04-20T21:31:26.680624+0000","last_became_peered":"2024-04-20T21:31:26.680624+0000","last_unstale":"2024-04-20T21:33:49.119384+0000","last_undegraded":"2024-04-20T21:33:49.119384+0000","last_fullsized":"2024-04-20T21:33:49.119384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:02.750166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148990+0000","last_change":"2024-04-20T21:31:26.697632+0000","last_active":"2024-04-20T21:33:32.148990+0000","last_peered":"2024-04-20T21:33:32.148990+0000","last_clean":"2024-04-20T21:33:32.148990+0000","last_became_active":"2024-04-20T21:31:26.697393+0000","last_became_peered":"2024-04-20T21:31:26.697393+0000","last_unstale":"2024-04-20T21:33:32.148990+0000","last_undegraded":"2024-04-20T21:33:32.148990+0000","last_fullsized":"2024-04-20T21:33:32.148990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:08:39.856082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.832878+0000","last_change":"2024-04-20T21:31:27.526613+0000","last_active":"2024-04-20T21:33:48.832878+0000","last_peered":"2024-04-20T21:33:48.832878+0000","last_clean":"2024-04-20T21:33:48.832878+0000","last_became_active":"2024-04-20T21:31:27.526234+0000","last_became_peered":"2024-04-20T21:31:27.526234+0000","last_unstale":"2024-04-20T21:33:48.832878+0000","last_undegraded":"2024-04-20T21:33:48.832878+0000","last_fullsized":"2024-04-20T21:33:48.832878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:06:44.114924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'45","reported_seq":104,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.131406+0000","last_change":"2024-04-20T21:32:31.625214+0000","last_active":"2024-04-20T21:33:49.131406+0000","last_peered":"2024-04-20T21:33:49.131406+0000","last_clean":"2024-04-20T21:33:49.131406+0000","last_became_active":"2024-04-20T21:32:31.619956+0000","last_became_peered":"2024-04-20T21:32:31.619956+0000","last_unstale":"2024-04-20T21:33:49.131406+0000","last_undegraded":"2024-04-20T21:33:49.131406+0000","last_fullsized":"2024-04-20T21:33:49.131406+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:19:41.620846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":81,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.831220+0000","last_change":"2024-04-20T21:31:27.538361+0000","last_active":"2024-04-20T21:33:48.831220+0000","last_peered":"2024-04-20T21:33:48.831220+0000","last_clean":"2024-04-20T21:33:48.831220+0000","last_became_active":"2024-04-20T21:31:27.537494+0000","last_became_peered":"2024-04-20T21:31:27.537494+0000","last_unstale":"2024-04-20T21:33:48.831220+0000","last_undegraded":"2024-04-20T21:33:48.831220+0000","last_fullsized":"2024-04-20T21:33:48.831220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:46.212759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119090+0000","last_change":"2024-04-20T21:31:26.686960+0000","last_active":"2024-04-20T21:33:49.119090+0000","last_peered":"2024-04-20T21:33:49.119090+0000","last_clean":"2024-04-20T21:33:49.119090+0000","last_became_active":"2024-04-20T21:31:26.686565+0000","last_became_peered":"2024-04-20T21:31:26.686565+0000","last_unstale":"2024-04-20T21:33:49.119090+0000","last_undegraded":"2024-04-20T21:33:49.119090+0000","last_fullsized":"2024-04-20T21:33:49.119090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.608834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120313+0000","last_change":"2024-04-20T21:31:27.546978+0000","last_active":"2024-04-20T21:33:49.120313+0000","last_peered":"2024-04-20T21:33:49.120313+0000","last_clean":"2024-04-20T21:33:49.120313+0000","last_became_active":"2024-04-20T21:31:27.545941+0000","last_became_peered":"2024-04-20T21:31:27.545941+0000","last_unstale":"2024-04-20T21:33:49.120313+0000","last_undegraded":"2024-04-20T21:33:49.120313+0000","last_fullsized":"2024-04-20T21:33:49.120313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:08.363519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118443+0000","last_change":"2024-04-20T21:31:26.690797+0000","last_active":"2024-04-20T21:33:49.118443+0000","last_peered":"2024-04-20T21:33:49.118443+0000","last_clean":"2024-04-20T21:33:49.118443+0000","last_became_active":"2024-04-20T21:31:26.690486+0000","last_became_peered":"2024-04-20T21:31:26.690486+0000","last_unstale":"2024-04-20T21:33:49.118443+0000","last_undegraded":"2024-04-20T21:33:49.118443+0000","last_fullsized":"2024-04-20T21:33:49.118443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:17:33.943212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148549+0000","last_change":"2024-04-20T21:31:27.507175+0000","last_active":"2024-04-20T21:33:32.148549+0000","last_peered":"2024-04-20T21:33:32.148549+0000","last_clean":"2024-04-20T21:33:32.148549+0000","last_became_active":"2024-04-20T21:31:27.506808+0000","last_became_peered":"2024-04-20T21:31:27.506808+0000","last_unstale":"2024-04-20T21:33:32.148549+0000","last_undegraded":"2024-04-20T21:33:32.148549+0000","last_fullsized":"2024-04-20T21:33:32.148549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:38:15.212880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828658+0000","last_change":"2024-04-20T21:31:26.698302+0000","last_active":"2024-04-20T21:33:48.828658+0000","last_peered":"2024-04-20T21:33:48.828658+0000","last_clean":"2024-04-20T21:33:48.828658+0000","last_became_active":"2024-04-20T21:31:26.696209+0000","last_became_peered":"2024-04-20T21:31:26.696209+0000","last_unstale":"2024-04-20T21:33:48.828658+0000","last_undegraded":"2024-04-20T21:33:48.828658+0000","last_fullsized":"2024-04-20T21:33:48.828658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:43.495880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.965838+0000","last_change":"2024-04-20T21:31:27.521594+0000","last_active":"2024-04-20T21:33:48.965838+0000","last_peered":"2024-04-20T21:33:48.965838+0000","last_clean":"2024-04-20T21:33:48.965838+0000","last_became_active":"2024-04-20T21:31:27.521185+0000","last_became_peered":"2024-04-20T21:31:27.521185+0000","last_unstale":"2024-04-20T21:33:48.965838+0000","last_undegraded":"2024-04-20T21:33:48.965838+0000","last_fullsized":"2024-04-20T21:33:48.965838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:22:08.647624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118193+0000","last_change":"2024-04-20T21:31:26.691536+0000","last_active":"2024-04-20T21:33:49.118193+0000","last_peered":"2024-04-20T21:33:49.118193+0000","last_clean":"2024-04-20T21:33:49.118193+0000","last_became_active":"2024-04-20T21:31:26.691202+0000","last_became_peered":"2024-04-20T21:31:26.691202+0000","last_unstale":"2024-04-20T21:33:49.118193+0000","last_undegraded":"2024-04-20T21:33:49.118193+0000","last_fullsized":"2024-04-20T21:33:49.118193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:58:59.331644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963190+0000","last_change":"2024-04-20T21:32:31.998258+0000","last_active":"2024-04-20T21:33:48.963190+0000","last_peered":"2024-04-20T21:33:48.963190+0000","last_clean":"2024-04-20T21:33:48.963190+0000","last_became_active":"2024-04-20T21:32:31.997119+0000","last_became_peered":"2024-04-20T21:32:31.997119+0000","last_unstale":"2024-04-20T21:33:48.963190+0000","last_undegraded":"2024-04-20T21:33:48.963190+0000","last_fullsized":"2024-04-20T21:33:48.963190+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:41:08.565434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148696+0000","last_change":"2024-04-20T21:32:31.616245+0000","last_active":"2024-04-20T21:33:32.148696+0000","last_peered":"2024-04-20T21:33:32.148696+0000","last_clean":"2024-04-20T21:33:32.148696+0000","last_became_active":"2024-04-20T21:32:31.611535+0000","last_became_peered":"2024-04-20T21:32:31.611535+0000","last_unstale":"2024-04-20T21:33:32.148696+0000","last_undegraded":"2024-04-20T21:33:32.148696+0000","last_fullsized":"2024-04-20T21:33:32.148696+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:28:17.609890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119779+0000","last_change":"2024-04-20T21:31:27.518346+0000","last_active":"2024-04-20T21:33:49.119779+0000","last_peered":"2024-04-20T21:33:49.119779+0000","last_clean":"2024-04-20T21:33:49.119779+0000","last_became_active":"2024-04-20T21:31:27.517782+0000","last_became_peered":"2024-04-20T21:31:27.517782+0000","last_unstale":"2024-04-20T21:33:49.119779+0000","last_undegraded":"2024-04-20T21:33:49.119779+0000","last_fullsized":"2024-04-20T21:33:49.119779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:21.674663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119132+0000","last_change":"2024-04-20T21:32:31.615626+0000","last_active":"2024-04-20T21:33:49.119132+0000","last_peered":"2024-04-20T21:33:49.119132+0000","last_clean":"2024-04-20T21:33:49.119132+0000","last_became_active":"2024-04-20T21:32:31.615181+0000","last_became_peered":"2024-04-20T21:32:31.615181+0000","last_unstale":"2024-04-20T21:33:49.119132+0000","last_undegraded":"2024-04-20T21:33:49.119132+0000","last_fullsized":"2024-04-20T21:33:49.119132+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:58:18.344583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119853+0000","last_change":"2024-04-20T21:31:27.510920+0000","last_active":"2024-04-20T21:33:49.119853+0000","last_peered":"2024-04-20T21:33:49.119853+0000","last_clean":"2024-04-20T21:33:49.119853+0000","last_became_active":"2024-04-20T21:31:27.510564+0000","last_became_peered":"2024-04-20T21:31:27.510564+0000","last_unstale":"2024-04-20T21:33:49.119853+0000","last_undegraded":"2024-04-20T21:33:49.119853+0000","last_fullsized":"2024-04-20T21:33:49.119853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:05.196178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.128160+0000","last_change":"2024-04-20T21:31:26.693159+0000","last_active":"2024-04-20T21:33:49.128160+0000","last_peered":"2024-04-20T21:33:49.128160+0000","last_clean":"2024-04-20T21:33:49.128160+0000","last_became_active":"2024-04-20T21:31:26.692902+0000","last_became_peered":"2024-04-20T21:31:26.692902+0000","last_unstale":"2024-04-20T21:33:49.128160+0000","last_undegraded":"2024-04-20T21:33:49.128160+0000","last_fullsized":"2024-04-20T21:33:49.128160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:54.704343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119400+0000","last_change":"2024-04-20T21:31:27.541557+0000","last_active":"2024-04-20T21:33:49.119400+0000","last_peered":"2024-04-20T21:33:49.119400+0000","last_clean":"2024-04-20T21:33:49.119400+0000","last_became_active":"2024-04-20T21:31:27.541161+0000","last_became_peered":"2024-04-20T21:31:27.541161+0000","last_unstale":"2024-04-20T21:33:49.119400+0000","last_undegraded":"2024-04-20T21:33:49.119400+0000","last_fullsized":"2024-04-20T21:33:49.119400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:55:25.558918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.833193+0000","last_change":"2024-04-20T21:31:26.688019+0000","last_active":"2024-04-20T21:33:48.833193+0000","last_peered":"2024-04-20T21:33:48.833193+0000","last_clean":"2024-04-20T21:33:48.833193+0000","last_became_active":"2024-04-20T21:31:26.686370+0000","last_became_peered":"2024-04-20T21:31:26.686370+0000","last_unstale":"2024-04-20T21:33:48.833193+0000","last_undegraded":"2024-04-20T21:33:48.833193+0000","last_fullsized":"2024-04-20T21:33:48.833193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:42:44.059365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119798+0000","last_change":"2024-04-20T21:31:27.511592+0000","last_active":"2024-04-20T21:33:49.119798+0000","last_peered":"2024-04-20T21:33:49.119798+0000","last_clean":"2024-04-20T21:33:49.119798+0000","last_became_active":"2024-04-20T21:31:27.511311+0000","last_became_peered":"2024-04-20T21:31:27.511311+0000","last_unstale":"2024-04-20T21:33:49.119798+0000","last_undegraded":"2024-04-20T21:33:49.119798+0000","last_fullsized":"2024-04-20T21:33:49.119798+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:35:27.665578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.965845+0000","last_change":"2024-04-20T21:31:27.511827+0000","last_active":"2024-04-20T21:33:48.965845+0000","last_peered":"2024-04-20T21:33:48.965845+0000","last_clean":"2024-04-20T21:33:48.965845+0000","last_became_active":"2024-04-20T21:31:27.511344+0000","last_became_peered":"2024-04-20T21:31:27.511344+0000","last_unstale":"2024-04-20T21:33:48.965845+0000","last_undegraded":"2024-04-20T21:33:48.965845+0000","last_fullsized":"2024-04-20T21:33:48.965845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:11:37.973050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118126+0000","last_change":"2024-04-20T21:31:27.510747+0000","last_active":"2024-04-20T21:33:49.118126+0000","last_peered":"2024-04-20T21:33:49.118126+0000","last_clean":"2024-04-20T21:33:49.118126+0000","last_became_active":"2024-04-20T21:31:27.510253+0000","last_became_peered":"2024-04-20T21:31:27.510253+0000","last_unstale":"2024-04-20T21:33:49.118126+0000","last_undegraded":"2024-04-20T21:33:49.118126+0000","last_fullsized":"2024-04-20T21:33:49.118126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:37:13.855840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118089+0000","last_change":"2024-04-20T21:31:27.541185+0000","last_active":"2024-04-20T21:33:49.118089+0000","last_peered":"2024-04-20T21:33:49.118089+0000","last_clean":"2024-04-20T21:33:49.118089+0000","last_became_active":"2024-04-20T21:31:27.540881+0000","last_became_peered":"2024-04-20T21:31:27.540881+0000","last_unstale":"2024-04-20T21:33:49.118089+0000","last_undegraded":"2024-04-20T21:33:49.118089+0000","last_fullsized":"2024-04-20T21:33:49.118089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:46:41.897081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119309+0000","last_change":"2024-04-20T21:31:27.527204+0000","last_active":"2024-04-20T21:33:49.119309+0000","last_peered":"2024-04-20T21:33:49.119309+0000","last_clean":"2024-04-20T21:33:49.119309+0000","last_became_active":"2024-04-20T21:31:27.526805+0000","last_became_peered":"2024-04-20T21:31:27.526805+0000","last_unstale":"2024-04-20T21:33:49.119309+0000","last_undegraded":"2024-04-20T21:33:49.119309+0000","last_fullsized":"2024-04-20T21:33:49.119309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:43.032609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.832649+0000","last_change":"2024-04-20T21:31:27.525521+0000","last_active":"2024-04-20T21:33:48.832649+0000","last_peered":"2024-04-20T21:33:48.832649+0000","last_clean":"2024-04-20T21:33:48.832649+0000","last_became_active":"2024-04-20T21:31:27.521892+0000","last_became_peered":"2024-04-20T21:31:27.521892+0000","last_unstale":"2024-04-20T21:33:48.832649+0000","last_undegraded":"2024-04-20T21:33:48.832649+0000","last_fullsized":"2024-04-20T21:33:48.832649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:32:47.683115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966118+0000","last_change":"2024-04-20T21:33:32.432716+0000","last_active":"2024-04-20T21:33:48.966118+0000","last_peered":"2024-04-20T21:33:48.966118+0000","last_clean":"2024-04-20T21:33:48.966118+0000","last_became_active":"2024-04-20T21:33:32.432201+0000","last_became_peered":"2024-04-20T21:33:32.432201+0000","last_unstale":"2024-04-20T21:33:48.966118+0000","last_undegraded":"2024-04-20T21:33:48.966118+0000","last_fullsized":"2024-04-20T21:33:48.966118+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:14:13.089095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118560+0000","last_change":"2024-04-20T21:31:27.512542+0000","last_active":"2024-04-20T21:33:49.118560+0000","last_peered":"2024-04-20T21:33:49.118560+0000","last_clean":"2024-04-20T21:33:49.118560+0000","last_became_active":"2024-04-20T21:31:27.511907+0000","last_became_peered":"2024-04-20T21:31:27.511907+0000","last_unstale":"2024-04-20T21:33:49.118560+0000","last_undegraded":"2024-04-20T21:33:49.118560+0000","last_fullsized":"2024-04-20T21:33:49.118560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:10:09.265471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.831096+0000","last_change":"2024-04-20T21:31:27.538697+0000","last_active":"2024-04-20T21:33:48.831096+0000","last_peered":"2024-04-20T21:33:48.831096+0000","last_clean":"2024-04-20T21:33:48.831096+0000","last_became_active":"2024-04-20T21:31:27.537972+0000","last_became_peered":"2024-04-20T21:31:27.537972+0000","last_unstale":"2024-04-20T21:33:48.831096+0000","last_undegraded":"2024-04-20T21:33:48.831096+0000","last_fullsized":"2024-04-20T21:33:48.831096+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:51.974485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976126+0000","last_change":"2024-04-20T21:31:27.538092+0000","last_active":"2024-04-20T21:33:48.976126+0000","last_peered":"2024-04-20T21:33:48.976126+0000","last_clean":"2024-04-20T21:33:48.976126+0000","last_became_active":"2024-04-20T21:31:27.537821+0000","last_became_peered":"2024-04-20T21:31:27.537821+0000","last_unstale":"2024-04-20T21:33:48.976126+0000","last_undegraded":"2024-04-20T21:33:48.976126+0000","last_fullsized":"2024-04-20T21:33:48.976126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:52.773583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828402+0000","last_change":"2024-04-20T21:31:27.535498+0000","last_active":"2024-04-20T21:33:48.828402+0000","last_peered":"2024-04-20T21:33:48.828402+0000","last_clean":"2024-04-20T21:33:48.828402+0000","last_became_active":"2024-04-20T21:31:27.535162+0000","last_became_peered":"2024-04-20T21:31:27.535162+0000","last_unstale":"2024-04-20T21:33:48.828402+0000","last_undegraded":"2024-04-20T21:33:48.828402+0000","last_fullsized":"2024-04-20T21:33:48.828402+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:46.638825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120232+0000","last_change":"2024-04-20T21:31:27.537776+0000","last_active":"2024-04-20T21:33:49.120232+0000","last_peered":"2024-04-20T21:33:49.120232+0000","last_clean":"2024-04-20T21:33:49.120232+0000","last_became_active":"2024-04-20T21:31:27.537428+0000","last_became_peered":"2024-04-20T21:31:27.537428+0000","last_unstale":"2024-04-20T21:33:49.120232+0000","last_undegraded":"2024-04-20T21:33:49.120232+0000","last_fullsized":"2024-04-20T21:33:49.120232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:15:03.344277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.834707+0000","last_change":"2024-04-20T21:31:27.535943+0000","last_active":"2024-04-20T21:33:48.834707+0000","last_peered":"2024-04-20T21:33:48.834707+0000","last_clean":"2024-04-20T21:33:48.834707+0000","last_became_active":"2024-04-20T21:31:27.533694+0000","last_became_peered":"2024-04-20T21:31:27.533694+0000","last_unstale":"2024-04-20T21:33:48.834707+0000","last_undegraded":"2024-04-20T21:33:48.834707+0000","last_fullsized":"2024-04-20T21:33:48.834707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:42.603232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966013+0000","last_change":"2024-04-20T21:31:27.511345+0000","last_active":"2024-04-20T21:33:48.966013+0000","last_peered":"2024-04-20T21:33:48.966013+0000","last_clean":"2024-04-20T21:33:48.966013+0000","last_became_active":"2024-04-20T21:31:27.510984+0000","last_became_peered":"2024-04-20T21:31:27.510984+0000","last_unstale":"2024-04-20T21:33:48.966013+0000","last_undegraded":"2024-04-20T21:33:48.966013+0000","last_fullsized":"2024-04-20T21:33:48.966013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:19.965636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":593374,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":387,"num_read_kb":296,"num_write":200,"num_write_kb":1404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1880064,"data_stored":1780122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285609,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1421312,"data_stored":888395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547246,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28816,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716336,"statfs":{"total":95995035648,"available":95965528064,"internally_reserved":0,"allocated":1777664,"data_stored":1213519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710772,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":486747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972411,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":486769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266750,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1613824,"data_stored":1077232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430279,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":495296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":562939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":495269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":562973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149781,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1572864,"data_stored":1020333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888145,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":495201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691915,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37248,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707904,"statfs":{"total":95995035648,"available":95956893696,"internally_reserved":0,"allocated":1773568,"data_stored":1217446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":524920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":131782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":199554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":393216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:33:52.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:33:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:52 smithi122 ceph-mon[22147]: [20/Apr/2024:21:33:50] ENGINE Serving on https://172.21.15.22:7150 2024-04-20T21:33:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:52 smithi122 ceph-mon[22147]: [20/Apr/2024:21:33:50] ENGINE Serving on http://172.21.15.22:8765 2024-04-20T21:33:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:52 smithi122 ceph-mon[22147]: [20/Apr/2024:21:33:50] ENGINE Bus STARTED 2024-04-20T21:33:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:52 smithi122 ceph-mon[22147]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:52 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:52 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:52 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:52 smithi122 ceph-mon[22147]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-04-20T21:33:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:52 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3851136016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:52 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/495420605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.564+0000 7fc465400640 1 -- 172.21.15.22:0/1099793858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc460077b60 msgr2=0x7fc460077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.564+0000 7fc465400640 1 --2- 172.21.15.22:0/1099793858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc460077b60 0x7fc460077fc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009900 tx=0x7fc44c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.564+0000 7fc465400640 1 -- 172.21.15.22:0/1099793858 shutdown_connections 2024-04-20T21:33:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.564+0000 7fc465400640 1 --2- 172.21.15.22:0/1099793858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc460078500 0x7fc460078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.564+0000 7fc465400640 1 --2- 172.21.15.22:0/1099793858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc460077b60 0x7fc460077fc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.564+0000 7fc465400640 1 --2- 172.21.15.22:0/1099793858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc460076920 0x7fc460076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.564+0000 7fc465400640 1 -- 172.21.15.22:0/1099793858 >> 172.21.15.22:0/1099793858 conn(0x7fc4600ff750 msgr2=0x7fc460101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.564+0000 7fc465400640 1 -- 172.21.15.22:0/1099793858 shutdown_connections 2024-04-20T21:33:52.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.564+0000 7fc465400640 1 -- 172.21.15.22:0/1099793858 wait complete. 2024-04-20T21:33:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.565+0000 7fc465400640 1 Processor -- start 2024-04-20T21:33:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.565+0000 7fc465400640 1 -- start start 2024-04-20T21:33:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.565+0000 7fc465400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc460076920 0x7fc46019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.565+0000 7fc465400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc460077b60 0x7fc46019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:52.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.565+0000 7fc465400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc460078500 0x7fc4601a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.565+0000 7fc465400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601142b0 con 0x7fc460077b60 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.565+0000 7fc465400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc460114130 con 0x7fc460078500 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.565+0000 7fc465400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc460114430 con 0x7fc460076920 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.566+0000 7fc45e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc460076920 0x7fc46019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.566+0000 7fc45de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc460077b60 0x7fc46019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.566+0000 7fc45e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc460076920 0x7fc46019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:35014/0 (socket says 172.21.15.22:35014) 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.566+0000 7fc45e800640 1 -- 172.21.15.22:0/2127705757 learned_addr learned my addr 172.21.15.22:0/2127705757 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.566+0000 7fc45f200640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc460078500 0x7fc4601a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.566+0000 7fc45e800640 1 -- 172.21.15.22:0/2127705757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc460078500 msgr2=0x7fc4601a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.566+0000 7fc45e800640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc460078500 0x7fc4601a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.566+0000 7fc45e800640 1 -- 172.21.15.22:0/2127705757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc460077b60 msgr2=0x7fc46019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.566+0000 7fc45e800640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc460077b60 0x7fc46019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.566+0000 7fc45e800640 1 -- 172.21.15.22:0/2127705757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4601a64f0 con 0x7fc460076920 2024-04-20T21:33:52.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.566+0000 7fc45e800640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc460076920 0x7fc46019e8d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc45000e9f0 tx=0x7fc45000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:52.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.567+0000 7fc457400640 1 -- 172.21.15.22:0/2127705757 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45000cd80 con 0x7fc460076920 2024-04-20T21:33:52.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.567+0000 7fc457400640 1 -- 172.21.15.22:0/2127705757 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc450005500 con 0x7fc460076920 2024-04-20T21:33:52.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.567+0000 7fc465400640 1 -- 172.21.15.22:0/2127705757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4601a67e0 con 0x7fc460076920 2024-04-20T21:33:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.567+0000 7fc457400640 1 -- 172.21.15.22:0/2127705757 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450010690 con 0x7fc460076920 2024-04-20T21:33:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.567+0000 7fc465400640 1 -- 172.21.15.22:0/2127705757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4601a6d00 con 0x7fc460076920 2024-04-20T21:33:52.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.568+0000 7fc465400640 1 -- 172.21.15.22:0/2127705757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc460110cb0 con 0x7fc460076920 2024-04-20T21:33:52.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.570+0000 7fc457400640 1 -- 172.21.15.22:0/2127705757 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7fc450004f60 con 0x7fc460076920 2024-04-20T21:33:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.571+0000 7fc457400640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc4440a3da0 0x7fc4440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:52.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.571+0000 7fc457400640 1 -- 172.21.15.22:0/2127705757 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc4500d0bf0 con 0x7fc460076920 2024-04-20T21:33:52.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.572+0000 7fc45de00640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc4440a3da0 0x7fc4440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:52.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.572+0000 7fc45de00640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc4440a3da0 0x7fc4440a6260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc44c02c240 tx=0x7fc44c02df90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:52.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.575+0000 7fc457400640 1 -- 172.21.15.22:0/2127705757 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc45009d210 con 0x7fc460076920 2024-04-20T21:33:52.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.798+0000 7fc465400640 1 -- 172.21.15.22:0/2127705757 --> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4600623a0 con 0x7fc4440a3da0 2024-04-20T21:33:52.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.812+0000 7fc457400640 1 -- 172.21.15.22:0/2127705757 <== mgr.15024 v2:172.21.15.22:6800/975472489 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257707 (secure 0 0 0) 0x7fc4600623a0 con 0x7fc4440a3da0 2024-04-20T21:33:52.811 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:52.821 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-20T21:33:52.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.816+0000 7fc465400640 1 -- 172.21.15.22:0/2127705757 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc4440a3da0 msgr2=0x7fc4440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:52.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.816+0000 7fc465400640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc4440a3da0 0x7fc4440a6260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc44c02c240 tx=0x7fc44c02df90 comp rx=0 tx=0).stop 2024-04-20T21:33:52.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.817+0000 7fc465400640 1 -- 172.21.15.22:0/2127705757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc460076920 msgr2=0x7fc46019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:52.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.817+0000 7fc465400640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc460076920 0x7fc46019e8d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc45000e9f0 tx=0x7fc45000edf0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.817+0000 7fc465400640 1 -- 172.21.15.22:0/2127705757 shutdown_connections 2024-04-20T21:33:52.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.817+0000 7fc465400640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc4440a3da0 0x7fc4440a6260 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.817+0000 7fc465400640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc460078500 0x7fc4601a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.817+0000 7fc465400640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc460077b60 0x7fc46019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.817+0000 7fc465400640 1 --2- 172.21.15.22:0/2127705757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc460076920 0x7fc46019e8d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:52.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.817+0000 7fc465400640 1 -- 172.21.15.22:0/2127705757 >> 172.21.15.22:0/2127705757 conn(0x7fc4600ff750 msgr2=0x7fc460100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:52.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.817+0000 7fc465400640 1 -- 172.21.15.22:0/2127705757 shutdown_connections 2024-04-20T21:33:52.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:52.817+0000 7fc465400640 1 -- 172.21.15.22:0/2127705757 wait complete. 2024-04-20T21:33:52.837 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-20T21:33:50.845424+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1183976,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":523,"num_read_kb":415,"num_write":383,"num_write_kb":4344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385388,"kb_used_data":15144,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124556436,"statfs":{"total":1151940427776,"available":1151545790464,"internally_reserved":0,"allocated":15507456,"data_stored":9002119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.977533"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966208+0000","last_change":"2024-04-20T21:31:27.527151+0000","last_active":"2024-04-20T21:33:48.966208+0000","last_peered":"2024-04-20T21:33:48.966208+0000","last_clean":"2024-04-20T21:33:48.966208+0000","last_became_active":"2024-04-20T21:31:27.526739+0000","last_became_peered":"2024-04-20T21:31:27.526739+0000","last_unstale":"2024-04-20T21:33:48.966208+0000","last_undegraded":"2024-04-20T21:33:48.966208+0000","last_fullsized":"2024-04-20T21:33:48.966208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:16:39.204523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120131+0000","last_change":"2024-04-20T21:31:27.527437+0000","last_active":"2024-04-20T21:33:49.120131+0000","last_peered":"2024-04-20T21:33:49.120131+0000","last_clean":"2024-04-20T21:33:49.120131+0000","last_became_active":"2024-04-20T21:31:27.527040+0000","last_became_peered":"2024-04-20T21:31:27.527040+0000","last_unstale":"2024-04-20T21:33:49.120131+0000","last_undegraded":"2024-04-20T21:33:49.120131+0000","last_fullsized":"2024-04-20T21:33:49.120131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:53:39.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119810+0000","last_change":"2024-04-20T21:31:27.531671+0000","last_active":"2024-04-20T21:33:49.119810+0000","last_peered":"2024-04-20T21:33:49.119810+0000","last_clean":"2024-04-20T21:33:49.119810+0000","last_became_active":"2024-04-20T21:31:27.531267+0000","last_became_peered":"2024-04-20T21:31:27.531267+0000","last_unstale":"2024-04-20T21:33:49.119810+0000","last_undegraded":"2024-04-20T21:33:49.119810+0000","last_fullsized":"2024-04-20T21:33:49.119810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:47.002481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119241+0000","last_change":"2024-04-20T21:31:27.512033+0000","last_active":"2024-04-20T21:33:49.119241+0000","last_peered":"2024-04-20T21:33:49.119241+0000","last_clean":"2024-04-20T21:33:49.119241+0000","last_became_active":"2024-04-20T21:31:27.510801+0000","last_became_peered":"2024-04-20T21:31:27.510801+0000","last_unstale":"2024-04-20T21:33:49.119241+0000","last_undegraded":"2024-04-20T21:33:49.119241+0000","last_fullsized":"2024-04-20T21:33:49.119241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:43:25.407173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148706+0000","last_change":"2024-04-20T21:31:27.545862+0000","last_active":"2024-04-20T21:33:32.148706+0000","last_peered":"2024-04-20T21:33:32.148706+0000","last_clean":"2024-04-20T21:33:32.148706+0000","last_became_active":"2024-04-20T21:31:27.545500+0000","last_became_peered":"2024-04-20T21:31:27.545500+0000","last_unstale":"2024-04-20T21:33:32.148706+0000","last_undegraded":"2024-04-20T21:33:32.148706+0000","last_fullsized":"2024-04-20T21:33:32.148706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:37.640777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828391+0000","last_change":"2024-04-20T21:31:27.526408+0000","last_active":"2024-04-20T21:33:48.828391+0000","last_peered":"2024-04-20T21:33:48.828391+0000","last_clean":"2024-04-20T21:33:48.828391+0000","last_became_active":"2024-04-20T21:31:27.526015+0000","last_became_peered":"2024-04-20T21:31:27.526015+0000","last_unstale":"2024-04-20T21:33:48.828391+0000","last_undegraded":"2024-04-20T21:33:48.828391+0000","last_fullsized":"2024-04-20T21:33:48.828391+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:56.904696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119156+0000","last_change":"2024-04-20T21:31:27.526272+0000","last_active":"2024-04-20T21:33:49.119156+0000","last_peered":"2024-04-20T21:33:49.119156+0000","last_clean":"2024-04-20T21:33:49.119156+0000","last_became_active":"2024-04-20T21:31:27.525852+0000","last_became_peered":"2024-04-20T21:31:27.525852+0000","last_unstale":"2024-04-20T21:33:49.119156+0000","last_undegraded":"2024-04-20T21:33:49.119156+0000","last_fullsized":"2024-04-20T21:33:49.119156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:33:57.811339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.962898+0000","last_change":"2024-04-20T21:32:31.998671+0000","last_active":"2024-04-20T21:33:48.962898+0000","last_peered":"2024-04-20T21:33:48.962898+0000","last_clean":"2024-04-20T21:33:48.962898+0000","last_became_active":"2024-04-20T21:32:31.996769+0000","last_became_peered":"2024-04-20T21:32:31.996769+0000","last_unstale":"2024-04-20T21:33:48.962898+0000","last_undegraded":"2024-04-20T21:33:48.962898+0000","last_fullsized":"2024-04-20T21:33:48.962898+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:03.135794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120675+0000","last_change":"2024-04-20T21:31:27.514810+0000","last_active":"2024-04-20T21:33:49.120675+0000","last_peered":"2024-04-20T21:33:49.120675+0000","last_clean":"2024-04-20T21:33:49.120675+0000","last_became_active":"2024-04-20T21:31:27.512548+0000","last_became_peered":"2024-04-20T21:31:27.512548+0000","last_unstale":"2024-04-20T21:33:49.120675+0000","last_undegraded":"2024-04-20T21:33:49.120675+0000","last_fullsized":"2024-04-20T21:33:49.120675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:11:30.375294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828858+0000","last_change":"2024-04-20T21:31:27.510303+0000","last_active":"2024-04-20T21:33:48.828858+0000","last_peered":"2024-04-20T21:33:48.828858+0000","last_clean":"2024-04-20T21:33:48.828858+0000","last_became_active":"2024-04-20T21:31:27.509907+0000","last_became_peered":"2024-04-20T21:31:27.509907+0000","last_unstale":"2024-04-20T21:33:48.828858+0000","last_undegraded":"2024-04-20T21:33:48.828858+0000","last_fullsized":"2024-04-20T21:33:48.828858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:31.160785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119674+0000","last_change":"2024-04-20T21:31:27.513749+0000","last_active":"2024-04-20T21:33:49.119674+0000","last_peered":"2024-04-20T21:33:49.119674+0000","last_clean":"2024-04-20T21:33:49.119674+0000","last_became_active":"2024-04-20T21:31:27.513404+0000","last_became_peered":"2024-04-20T21:31:27.513404+0000","last_unstale":"2024-04-20T21:33:49.119674+0000","last_undegraded":"2024-04-20T21:33:49.119674+0000","last_fullsized":"2024-04-20T21:33:49.119674+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:26.279697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118958+0000","last_change":"2024-04-20T21:31:27.507727+0000","last_active":"2024-04-20T21:33:49.118958+0000","last_peered":"2024-04-20T21:33:49.118958+0000","last_clean":"2024-04-20T21:33:49.118958+0000","last_became_active":"2024-04-20T21:31:27.507363+0000","last_became_peered":"2024-04-20T21:31:27.507363+0000","last_unstale":"2024-04-20T21:33:49.118958+0000","last_undegraded":"2024-04-20T21:33:49.118958+0000","last_fullsized":"2024-04-20T21:33:49.118958+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:01.139963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.834435+0000","last_change":"2024-04-20T21:31:27.512941+0000","last_active":"2024-04-20T21:33:48.834435+0000","last_peered":"2024-04-20T21:33:48.834435+0000","last_clean":"2024-04-20T21:33:48.834435+0000","last_became_active":"2024-04-20T21:31:27.512571+0000","last_became_peered":"2024-04-20T21:31:27.512571+0000","last_unstale":"2024-04-20T21:33:48.834435+0000","last_undegraded":"2024-04-20T21:33:48.834435+0000","last_fullsized":"2024-04-20T21:33:48.834435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:30.977624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119024+0000","last_change":"2024-04-20T21:31:27.521439+0000","last_active":"2024-04-20T21:33:49.119024+0000","last_peered":"2024-04-20T21:33:49.119024+0000","last_clean":"2024-04-20T21:33:49.119024+0000","last_became_active":"2024-04-20T21:31:27.520245+0000","last_became_peered":"2024-04-20T21:31:27.520245+0000","last_unstale":"2024-04-20T21:33:49.119024+0000","last_undegraded":"2024-04-20T21:33:49.119024+0000","last_fullsized":"2024-04-20T21:33:49.119024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:53:26.510845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118750+0000","last_change":"2024-04-20T21:32:31.619209+0000","last_active":"2024-04-20T21:33:49.118750+0000","last_peered":"2024-04-20T21:33:49.118750+0000","last_clean":"2024-04-20T21:33:49.118750+0000","last_became_active":"2024-04-20T21:32:31.618527+0000","last_became_peered":"2024-04-20T21:32:31.618527+0000","last_unstale":"2024-04-20T21:33:49.118750+0000","last_undegraded":"2024-04-20T21:33:49.118750+0000","last_fullsized":"2024-04-20T21:33:49.118750+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:54:21.400896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118476+0000","last_change":"2024-04-20T21:31:27.521533+0000","last_active":"2024-04-20T21:33:49.118476+0000","last_peered":"2024-04-20T21:33:49.118476+0000","last_clean":"2024-04-20T21:33:49.118476+0000","last_became_active":"2024-04-20T21:31:27.520920+0000","last_became_peered":"2024-04-20T21:31:27.520920+0000","last_unstale":"2024-04-20T21:33:49.118476+0000","last_undegraded":"2024-04-20T21:33:49.118476+0000","last_fullsized":"2024-04-20T21:33:49.118476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:28:13.605238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966062+0000","last_change":"2024-04-20T21:31:27.527484+0000","last_active":"2024-04-20T21:33:48.966062+0000","last_peered":"2024-04-20T21:33:48.966062+0000","last_clean":"2024-04-20T21:33:48.966062+0000","last_became_active":"2024-04-20T21:31:27.527048+0000","last_became_peered":"2024-04-20T21:31:27.527048+0000","last_unstale":"2024-04-20T21:33:48.966062+0000","last_undegraded":"2024-04-20T21:33:48.966062+0000","last_fullsized":"2024-04-20T21:33:48.966062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:35:56.467661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963479+0000","last_change":"2024-04-20T21:31:27.527870+0000","last_active":"2024-04-20T21:33:48.963479+0000","last_peered":"2024-04-20T21:33:48.963479+0000","last_clean":"2024-04-20T21:33:48.963479+0000","last_became_active":"2024-04-20T21:31:27.525918+0000","last_became_peered":"2024-04-20T21:31:27.525918+0000","last_unstale":"2024-04-20T21:33:48.963479+0000","last_undegraded":"2024-04-20T21:33:48.963479+0000","last_fullsized":"2024-04-20T21:33:48.963479+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:34.425370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118470+0000","last_change":"2024-04-20T21:31:27.544987+0000","last_active":"2024-04-20T21:33:49.118470+0000","last_peered":"2024-04-20T21:33:49.118470+0000","last_clean":"2024-04-20T21:33:49.118470+0000","last_became_active":"2024-04-20T21:31:27.544628+0000","last_became_peered":"2024-04-20T21:31:27.544628+0000","last_unstale":"2024-04-20T21:33:49.118470+0000","last_undegraded":"2024-04-20T21:33:49.118470+0000","last_fullsized":"2024-04-20T21:33:49.118470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:05:42.396775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963603+0000","last_change":"2024-04-20T21:31:26.702803+0000","last_active":"2024-04-20T21:33:48.963603+0000","last_peered":"2024-04-20T21:33:48.963603+0000","last_clean":"2024-04-20T21:33:48.963603+0000","last_became_active":"2024-04-20T21:31:26.695781+0000","last_became_peered":"2024-04-20T21:31:26.695781+0000","last_unstale":"2024-04-20T21:33:48.963603+0000","last_undegraded":"2024-04-20T21:33:48.963603+0000","last_fullsized":"2024-04-20T21:33:48.963603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:49:17.131832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.830615+0000","last_change":"2024-04-20T21:31:27.535264+0000","last_active":"2024-04-20T21:33:48.830615+0000","last_peered":"2024-04-20T21:33:48.830615+0000","last_clean":"2024-04-20T21:33:48.830615+0000","last_became_active":"2024-04-20T21:31:27.534888+0000","last_became_peered":"2024-04-20T21:31:27.534888+0000","last_unstale":"2024-04-20T21:33:48.830615+0000","last_undegraded":"2024-04-20T21:33:48.830615+0000","last_fullsized":"2024-04-20T21:33:48.830615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:04:20.951916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.830901+0000","last_change":"2024-04-20T21:31:26.700378+0000","last_active":"2024-04-20T21:33:48.830901+0000","last_peered":"2024-04-20T21:33:48.830901+0000","last_clean":"2024-04-20T21:33:48.830901+0000","last_became_active":"2024-04-20T21:31:26.700077+0000","last_became_peered":"2024-04-20T21:31:26.700077+0000","last_unstale":"2024-04-20T21:33:48.830901+0000","last_undegraded":"2024-04-20T21:33:48.830901+0000","last_fullsized":"2024-04-20T21:33:48.830901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:34:21.646402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120548+0000","last_change":"2024-04-20T21:31:27.531069+0000","last_active":"2024-04-20T21:33:49.120548+0000","last_peered":"2024-04-20T21:33:49.120548+0000","last_clean":"2024-04-20T21:33:49.120548+0000","last_became_active":"2024-04-20T21:31:27.530457+0000","last_became_peered":"2024-04-20T21:31:27.530457+0000","last_unstale":"2024-04-20T21:33:49.120548+0000","last_undegraded":"2024-04-20T21:33:49.120548+0000","last_fullsized":"2024-04-20T21:33:49.120548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:46:59.829623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.149054+0000","last_change":"2024-04-20T21:31:26.684150+0000","last_active":"2024-04-20T21:33:32.149054+0000","last_peered":"2024-04-20T21:33:32.149054+0000","last_clean":"2024-04-20T21:33:32.149054+0000","last_became_active":"2024-04-20T21:31:26.683823+0000","last_became_peered":"2024-04-20T21:31:26.683823+0000","last_unstale":"2024-04-20T21:33:32.149054+0000","last_undegraded":"2024-04-20T21:33:32.149054+0000","last_fullsized":"2024-04-20T21:33:32.149054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:14.413606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120262+0000","last_change":"2024-04-20T21:31:27.507499+0000","last_active":"2024-04-20T21:33:49.120262+0000","last_peered":"2024-04-20T21:33:49.120262+0000","last_clean":"2024-04-20T21:33:49.120262+0000","last_became_active":"2024-04-20T21:31:27.506922+0000","last_became_peered":"2024-04-20T21:31:27.506922+0000","last_unstale":"2024-04-20T21:33:49.120262+0000","last_undegraded":"2024-04-20T21:33:49.120262+0000","last_fullsized":"2024-04-20T21:33:49.120262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:40.865232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.965750+0000","last_change":"2024-04-20T21:32:31.613577+0000","last_active":"2024-04-20T21:33:48.965750+0000","last_peered":"2024-04-20T21:33:48.965750+0000","last_clean":"2024-04-20T21:33:48.965750+0000","last_became_active":"2024-04-20T21:32:31.608943+0000","last_became_peered":"2024-04-20T21:32:31.608943+0000","last_unstale":"2024-04-20T21:33:48.965750+0000","last_undegraded":"2024-04-20T21:33:48.965750+0000","last_fullsized":"2024-04-20T21:33:48.965750+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:06.561157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976247+0000","last_change":"2024-04-20T21:31:27.517839+0000","last_active":"2024-04-20T21:33:48.976247+0000","last_peered":"2024-04-20T21:33:48.976247+0000","last_clean":"2024-04-20T21:33:48.976247+0000","last_became_active":"2024-04-20T21:31:27.517044+0000","last_became_peered":"2024-04-20T21:31:27.517044+0000","last_unstale":"2024-04-20T21:33:48.976247+0000","last_undegraded":"2024-04-20T21:33:48.976247+0000","last_fullsized":"2024-04-20T21:33:48.976247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:50.042665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976109+0000","last_change":"2024-04-20T21:31:26.686151+0000","last_active":"2024-04-20T21:33:48.976109+0000","last_peered":"2024-04-20T21:33:48.976109+0000","last_clean":"2024-04-20T21:33:48.976109+0000","last_became_active":"2024-04-20T21:31:26.685827+0000","last_became_peered":"2024-04-20T21:31:26.685827+0000","last_unstale":"2024-04-20T21:33:48.976109+0000","last_undegraded":"2024-04-20T21:33:48.976109+0000","last_fullsized":"2024-04-20T21:33:48.976109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:41:17.263420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148585+0000","last_change":"2024-04-20T21:32:31.620563+0000","last_active":"2024-04-20T21:33:32.148585+0000","last_peered":"2024-04-20T21:33:32.148585+0000","last_clean":"2024-04-20T21:33:32.148585+0000","last_became_active":"2024-04-20T21:32:31.620145+0000","last_became_peered":"2024-04-20T21:32:31.620145+0000","last_unstale":"2024-04-20T21:33:32.148585+0000","last_undegraded":"2024-04-20T21:33:32.148585+0000","last_fullsized":"2024-04-20T21:33:32.148585+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:43.609490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828355+0000","last_change":"2024-04-20T21:31:26.684573+0000","last_active":"2024-04-20T21:33:48.828355+0000","last_peered":"2024-04-20T21:33:48.828355+0000","last_clean":"2024-04-20T21:33:48.828355+0000","last_became_active":"2024-04-20T21:31:26.684163+0000","last_became_peered":"2024-04-20T21:31:26.684163+0000","last_unstale":"2024-04-20T21:33:48.828355+0000","last_undegraded":"2024-04-20T21:33:48.828355+0000","last_fullsized":"2024-04-20T21:33:48.828355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:39.445068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976276+0000","last_change":"2024-04-20T21:31:27.543680+0000","last_active":"2024-04-20T21:33:48.976276+0000","last_peered":"2024-04-20T21:33:48.976276+0000","last_clean":"2024-04-20T21:33:48.976276+0000","last_became_active":"2024-04-20T21:31:27.543379+0000","last_became_peered":"2024-04-20T21:31:27.543379+0000","last_unstale":"2024-04-20T21:33:48.976276+0000","last_undegraded":"2024-04-20T21:33:48.976276+0000","last_fullsized":"2024-04-20T21:33:48.976276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:58:17.168813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.827019+0000","last_change":"2024-04-20T21:31:26.689665+0000","last_active":"2024-04-20T21:33:48.827019+0000","last_peered":"2024-04-20T21:33:48.827019+0000","last_clean":"2024-04-20T21:33:48.827019+0000","last_became_active":"2024-04-20T21:31:26.689175+0000","last_became_peered":"2024-04-20T21:31:26.689175+0000","last_unstale":"2024-04-20T21:33:48.827019+0000","last_undegraded":"2024-04-20T21:33:48.827019+0000","last_fullsized":"2024-04-20T21:33:48.827019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:21.342700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.829688+0000","last_change":"2024-04-20T21:31:27.544137+0000","last_active":"2024-04-20T21:33:48.829688+0000","last_peered":"2024-04-20T21:33:48.829688+0000","last_clean":"2024-04-20T21:33:48.829688+0000","last_became_active":"2024-04-20T21:31:27.542841+0000","last_became_peered":"2024-04-20T21:31:27.542841+0000","last_unstale":"2024-04-20T21:33:48.829688+0000","last_undegraded":"2024-04-20T21:33:48.829688+0000","last_fullsized":"2024-04-20T21:33:48.829688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:04.610144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.975963+0000","last_change":"2024-04-20T21:31:26.681937+0000","last_active":"2024-04-20T21:33:48.975963+0000","last_peered":"2024-04-20T21:33:48.975963+0000","last_clean":"2024-04-20T21:33:48.975963+0000","last_became_active":"2024-04-20T21:31:26.681621+0000","last_became_peered":"2024-04-20T21:31:26.681621+0000","last_unstale":"2024-04-20T21:33:48.975963+0000","last_undegraded":"2024-04-20T21:33:48.975963+0000","last_fullsized":"2024-04-20T21:33:48.975963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:06.641608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963226+0000","last_change":"2024-04-20T21:31:27.540225+0000","last_active":"2024-04-20T21:33:48.963226+0000","last_peered":"2024-04-20T21:33:48.963226+0000","last_clean":"2024-04-20T21:33:48.963226+0000","last_became_active":"2024-04-20T21:31:27.539860+0000","last_became_peered":"2024-04-20T21:31:27.539860+0000","last_unstale":"2024-04-20T21:33:48.963226+0000","last_undegraded":"2024-04-20T21:33:48.963226+0000","last_fullsized":"2024-04-20T21:33:48.963226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:50.146761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828972+0000","last_change":"2024-04-20T21:31:26.701816+0000","last_active":"2024-04-20T21:33:48.828972+0000","last_peered":"2024-04-20T21:33:48.828972+0000","last_clean":"2024-04-20T21:33:48.828972+0000","last_became_active":"2024-04-20T21:31:26.701530+0000","last_became_peered":"2024-04-20T21:31:26.701530+0000","last_unstale":"2024-04-20T21:33:48.828972+0000","last_undegraded":"2024-04-20T21:33:48.828972+0000","last_fullsized":"2024-04-20T21:33:48.828972+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:31.175330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120556+0000","last_change":"2024-04-20T21:31:27.527222+0000","last_active":"2024-04-20T21:33:49.120556+0000","last_peered":"2024-04-20T21:33:49.120556+0000","last_clean":"2024-04-20T21:33:49.120556+0000","last_became_active":"2024-04-20T21:31:27.526779+0000","last_became_peered":"2024-04-20T21:31:27.526779+0000","last_unstale":"2024-04-20T21:33:49.120556+0000","last_undegraded":"2024-04-20T21:33:49.120556+0000","last_fullsized":"2024-04-20T21:33:49.120556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:13:54.500516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119650+0000","last_change":"2024-04-20T21:31:26.685969+0000","last_active":"2024-04-20T21:33:49.119650+0000","last_peered":"2024-04-20T21:33:49.119650+0000","last_clean":"2024-04-20T21:33:49.119650+0000","last_became_active":"2024-04-20T21:31:26.685580+0000","last_became_peered":"2024-04-20T21:31:26.685580+0000","last_unstale":"2024-04-20T21:33:49.119650+0000","last_undegraded":"2024-04-20T21:33:49.119650+0000","last_fullsized":"2024-04-20T21:33:49.119650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:22:50.558526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119011+0000","last_change":"2024-04-20T21:31:27.525862+0000","last_active":"2024-04-20T21:33:49.119011+0000","last_peered":"2024-04-20T21:33:49.119011+0000","last_clean":"2024-04-20T21:33:49.119011+0000","last_became_active":"2024-04-20T21:31:27.525372+0000","last_became_peered":"2024-04-20T21:31:27.525372+0000","last_unstale":"2024-04-20T21:33:49.119011+0000","last_undegraded":"2024-04-20T21:33:49.119011+0000","last_fullsized":"2024-04-20T21:33:49.119011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:44:15.527137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976460+0000","last_change":"2024-04-20T21:31:26.691091+0000","last_active":"2024-04-20T21:33:48.976460+0000","last_peered":"2024-04-20T21:33:48.976460+0000","last_clean":"2024-04-20T21:33:48.976460+0000","last_became_active":"2024-04-20T21:31:26.690608+0000","last_became_peered":"2024-04-20T21:31:26.690608+0000","last_unstale":"2024-04-20T21:33:48.976460+0000","last_undegraded":"2024-04-20T21:33:48.976460+0000","last_fullsized":"2024-04-20T21:33:48.976460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:44:25.336193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120411+0000","last_change":"2024-04-20T21:31:27.507900+0000","last_active":"2024-04-20T21:33:49.120411+0000","last_peered":"2024-04-20T21:33:49.120411+0000","last_clean":"2024-04-20T21:33:49.120411+0000","last_became_active":"2024-04-20T21:31:27.507360+0000","last_became_peered":"2024-04-20T21:31:27.507360+0000","last_unstale":"2024-04-20T21:33:49.120411+0000","last_undegraded":"2024-04-20T21:33:49.120411+0000","last_fullsized":"2024-04-20T21:33:49.120411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:27.110877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'33","reported_seq":348,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.265110+0000","last_change":"2024-04-20T21:31:26.685736+0000","last_active":"2024-04-20T21:33:49.265110+0000","last_peered":"2024-04-20T21:33:49.265110+0000","last_clean":"2024-04-20T21:33:49.265110+0000","last_became_active":"2024-04-20T21:31:26.685309+0000","last_became_peered":"2024-04-20T21:31:26.685309+0000","last_unstale":"2024-04-20T21:33:49.265110+0000","last_undegraded":"2024-04-20T21:33:49.265110+0000","last_fullsized":"2024-04-20T21:33:49.265110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:06.501753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67760,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":289,"num_read_kb":211,"num_write":33,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148851+0000","last_change":"2024-04-20T21:31:27.507532+0000","last_active":"2024-04-20T21:33:32.148851+0000","last_peered":"2024-04-20T21:33:32.148851+0000","last_clean":"2024-04-20T21:33:32.148851+0000","last_became_active":"2024-04-20T21:31:27.507188+0000","last_became_peered":"2024-04-20T21:31:27.507188+0000","last_unstale":"2024-04-20T21:33:32.148851+0000","last_undegraded":"2024-04-20T21:33:32.148851+0000","last_fullsized":"2024-04-20T21:33:32.148851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:58.814116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118339+0000","last_change":"2024-04-20T21:31:26.701361+0000","last_active":"2024-04-20T21:33:49.118339+0000","last_peered":"2024-04-20T21:33:49.118339+0000","last_clean":"2024-04-20T21:33:49.118339+0000","last_became_active":"2024-04-20T21:31:26.701068+0000","last_became_peered":"2024-04-20T21:31:26.701068+0000","last_unstale":"2024-04-20T21:33:49.118339+0000","last_undegraded":"2024-04-20T21:33:49.118339+0000","last_fullsized":"2024-04-20T21:33:49.118339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:22:51.745294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.830874+0000","last_change":"2024-04-20T21:31:27.542189+0000","last_active":"2024-04-20T21:33:48.830874+0000","last_peered":"2024-04-20T21:33:48.830874+0000","last_clean":"2024-04-20T21:33:48.830874+0000","last_became_active":"2024-04-20T21:31:27.541800+0000","last_became_peered":"2024-04-20T21:31:27.541800+0000","last_unstale":"2024-04-20T21:33:48.830874+0000","last_undegraded":"2024-04-20T21:33:48.830874+0000","last_fullsized":"2024-04-20T21:33:48.830874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:31.898443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976103+0000","last_change":"2024-04-20T21:31:26.687652+0000","last_active":"2024-04-20T21:33:48.976103+0000","last_peered":"2024-04-20T21:33:48.976103+0000","last_clean":"2024-04-20T21:33:48.976103+0000","last_became_active":"2024-04-20T21:31:26.687333+0000","last_became_peered":"2024-04-20T21:31:26.687333+0000","last_unstale":"2024-04-20T21:33:48.976103+0000","last_undegraded":"2024-04-20T21:33:48.976103+0000","last_fullsized":"2024-04-20T21:33:48.976103+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:12.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120140+0000","last_change":"2024-04-20T21:31:27.522535+0000","last_active":"2024-04-20T21:33:49.120140+0000","last_peered":"2024-04-20T21:33:49.120140+0000","last_clean":"2024-04-20T21:33:49.120140+0000","last_became_active":"2024-04-20T21:31:27.521821+0000","last_became_peered":"2024-04-20T21:31:27.521821+0000","last_unstale":"2024-04-20T21:33:49.120140+0000","last_undegraded":"2024-04-20T21:33:49.120140+0000","last_fullsized":"2024-04-20T21:33:49.120140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:47:53.401129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118630+0000","last_change":"2024-04-20T21:31:26.681758+0000","last_active":"2024-04-20T21:33:49.118630+0000","last_peered":"2024-04-20T21:33:49.118630+0000","last_clean":"2024-04-20T21:33:49.118630+0000","last_became_active":"2024-04-20T21:31:26.681395+0000","last_became_peered":"2024-04-20T21:31:26.681395+0000","last_unstale":"2024-04-20T21:33:49.118630+0000","last_undegraded":"2024-04-20T21:33:49.118630+0000","last_fullsized":"2024-04-20T21:33:49.118630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:33:00.924879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119081+0000","last_change":"2024-04-20T21:31:27.512873+0000","last_active":"2024-04-20T21:33:49.119081+0000","last_peered":"2024-04-20T21:33:49.119081+0000","last_clean":"2024-04-20T21:33:49.119081+0000","last_became_active":"2024-04-20T21:31:27.512554+0000","last_became_peered":"2024-04-20T21:31:27.512554+0000","last_unstale":"2024-04-20T21:33:49.119081+0000","last_undegraded":"2024-04-20T21:33:49.119081+0000","last_fullsized":"2024-04-20T21:33:49.119081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:26:06.670726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148549+0000","last_change":"2024-04-20T21:31:26.691210+0000","last_active":"2024-04-20T21:33:32.148549+0000","last_peered":"2024-04-20T21:33:32.148549+0000","last_clean":"2024-04-20T21:33:32.148549+0000","last_became_active":"2024-04-20T21:31:26.690859+0000","last_became_peered":"2024-04-20T21:31:26.690859+0000","last_unstale":"2024-04-20T21:33:32.148549+0000","last_undegraded":"2024-04-20T21:33:32.148549+0000","last_fullsized":"2024-04-20T21:33:32.148549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:36.789429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976179+0000","last_change":"2024-04-20T21:32:31.616413+0000","last_active":"2024-04-20T21:33:48.976179+0000","last_peered":"2024-04-20T21:33:48.976179+0000","last_clean":"2024-04-20T21:33:48.976179+0000","last_became_active":"2024-04-20T21:32:31.616011+0000","last_became_peered":"2024-04-20T21:32:31.616011+0000","last_unstale":"2024-04-20T21:33:48.976179+0000","last_undegraded":"2024-04-20T21:33:48.976179+0000","last_fullsized":"2024-04-20T21:33:48.976179+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:11:05.630337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.149077+0000","last_change":"2024-04-20T21:31:26.682311+0000","last_active":"2024-04-20T21:33:32.149077+0000","last_peered":"2024-04-20T21:33:32.149077+0000","last_clean":"2024-04-20T21:33:32.149077+0000","last_became_active":"2024-04-20T21:31:26.681996+0000","last_became_peered":"2024-04-20T21:31:26.681996+0000","last_unstale":"2024-04-20T21:33:32.149077+0000","last_undegraded":"2024-04-20T21:33:32.149077+0000","last_fullsized":"2024-04-20T21:33:32.149077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:32:37.659946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966120+0000","last_change":"2024-04-20T21:31:27.510922+0000","last_active":"2024-04-20T21:33:48.966120+0000","last_peered":"2024-04-20T21:33:48.966120+0000","last_clean":"2024-04-20T21:33:48.966120+0000","last_became_active":"2024-04-20T21:31:27.510553+0000","last_became_peered":"2024-04-20T21:31:27.510553+0000","last_unstale":"2024-04-20T21:33:48.966120+0000","last_undegraded":"2024-04-20T21:33:48.966120+0000","last_fullsized":"2024-04-20T21:33:48.966120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:23.754768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963235+0000","last_change":"2024-04-20T21:31:26.697010+0000","last_active":"2024-04-20T21:33:48.963235+0000","last_peered":"2024-04-20T21:33:48.963235+0000","last_clean":"2024-04-20T21:33:48.963235+0000","last_became_active":"2024-04-20T21:31:26.696112+0000","last_became_peered":"2024-04-20T21:31:26.696112+0000","last_unstale":"2024-04-20T21:33:48.963235+0000","last_undegraded":"2024-04-20T21:33:48.963235+0000","last_fullsized":"2024-04-20T21:33:48.963235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:57:02.715884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828760+0000","last_change":"2024-04-20T21:31:27.545025+0000","last_active":"2024-04-20T21:33:48.828760+0000","last_peered":"2024-04-20T21:33:48.828760+0000","last_clean":"2024-04-20T21:33:48.828760+0000","last_became_active":"2024-04-20T21:31:27.544630+0000","last_became_peered":"2024-04-20T21:31:27.544630+0000","last_unstale":"2024-04-20T21:33:48.828760+0000","last_undegraded":"2024-04-20T21:33:48.828760+0000","last_fullsized":"2024-04-20T21:33:48.828760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:57.292481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119540+0000","last_change":"2024-04-20T21:31:26.682909+0000","last_active":"2024-04-20T21:33:49.119540+0000","last_peered":"2024-04-20T21:33:49.119540+0000","last_clean":"2024-04-20T21:33:49.119540+0000","last_became_active":"2024-04-20T21:31:26.682597+0000","last_became_peered":"2024-04-20T21:31:26.682597+0000","last_unstale":"2024-04-20T21:33:49.119540+0000","last_undegraded":"2024-04-20T21:33:49.119540+0000","last_fullsized":"2024-04-20T21:33:49.119540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:31.889513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828594+0000","last_change":"2024-04-20T21:31:27.530217+0000","last_active":"2024-04-20T21:33:48.828594+0000","last_peered":"2024-04-20T21:33:48.828594+0000","last_clean":"2024-04-20T21:33:48.828594+0000","last_became_active":"2024-04-20T21:31:27.529780+0000","last_became_peered":"2024-04-20T21:31:27.529780+0000","last_unstale":"2024-04-20T21:33:48.828594+0000","last_undegraded":"2024-04-20T21:33:48.828594+0000","last_fullsized":"2024-04-20T21:33:48.828594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:21.613460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976309+0000","last_change":"2024-04-20T21:32:31.610727+0000","last_active":"2024-04-20T21:33:48.976309+0000","last_peered":"2024-04-20T21:33:48.976309+0000","last_clean":"2024-04-20T21:33:48.976309+0000","last_became_active":"2024-04-20T21:32:31.610321+0000","last_became_peered":"2024-04-20T21:32:31.610321+0000","last_unstale":"2024-04-20T21:33:48.976309+0000","last_undegraded":"2024-04-20T21:33:48.976309+0000","last_fullsized":"2024-04-20T21:33:48.976309+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:01:35.697137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.826910+0000","last_change":"2024-04-20T21:31:26.692845+0000","last_active":"2024-04-20T21:33:48.826910+0000","last_peered":"2024-04-20T21:33:48.826910+0000","last_clean":"2024-04-20T21:33:48.826910+0000","last_became_active":"2024-04-20T21:31:26.691915+0000","last_became_peered":"2024-04-20T21:31:26.691915+0000","last_unstale":"2024-04-20T21:33:48.826910+0000","last_undegraded":"2024-04-20T21:33:48.826910+0000","last_fullsized":"2024-04-20T21:33:48.826910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:24:00.436955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.832281+0000","last_change":"2024-04-20T21:31:27.541623+0000","last_active":"2024-04-20T21:33:48.832281+0000","last_peered":"2024-04-20T21:33:48.832281+0000","last_clean":"2024-04-20T21:33:48.832281+0000","last_became_active":"2024-04-20T21:31:27.540953+0000","last_became_peered":"2024-04-20T21:31:27.540953+0000","last_unstale":"2024-04-20T21:33:48.832281+0000","last_undegraded":"2024-04-20T21:33:48.832281+0000","last_fullsized":"2024-04-20T21:33:48.832281+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:03:15.091886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118799+0000","last_change":"2024-04-20T21:31:26.687373+0000","last_active":"2024-04-20T21:33:49.118799+0000","last_peered":"2024-04-20T21:33:49.118799+0000","last_clean":"2024-04-20T21:33:49.118799+0000","last_became_active":"2024-04-20T21:31:26.687009+0000","last_became_peered":"2024-04-20T21:31:26.687009+0000","last_unstale":"2024-04-20T21:33:49.118799+0000","last_undegraded":"2024-04-20T21:33:49.118799+0000","last_fullsized":"2024-04-20T21:33:49.118799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:23:42.708129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119190+0000","last_change":"2024-04-20T21:30:06.995711+0000","last_active":"2024-04-20T21:33:49.119190+0000","last_peered":"2024-04-20T21:33:49.119190+0000","last_clean":"2024-04-20T21:33:49.119190+0000","last_became_active":"2024-04-20T21:30:06.995280+0000","last_became_peered":"2024-04-20T21:30:06.995280+0000","last_unstale":"2024-04-20T21:33:49.119190+0000","last_undegraded":"2024-04-20T21:33:49.119190+0000","last_fullsized":"2024-04-20T21:33:49.119190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_clean_scrub_stamp":"2024-04-20T21:26:16.477494+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:49:26.014145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.826753+0000","last_change":"2024-04-20T21:31:27.514064+0000","last_active":"2024-04-20T21:33:48.826753+0000","last_peered":"2024-04-20T21:33:48.826753+0000","last_clean":"2024-04-20T21:33:48.826753+0000","last_became_active":"2024-04-20T21:31:27.513416+0000","last_became_peered":"2024-04-20T21:31:27.513416+0000","last_unstale":"2024-04-20T21:33:48.826753+0000","last_undegraded":"2024-04-20T21:33:48.826753+0000","last_fullsized":"2024-04-20T21:33:48.826753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:12.658265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119384+0000","last_change":"2024-04-20T21:31:26.680948+0000","last_active":"2024-04-20T21:33:49.119384+0000","last_peered":"2024-04-20T21:33:49.119384+0000","last_clean":"2024-04-20T21:33:49.119384+0000","last_became_active":"2024-04-20T21:31:26.680624+0000","last_became_peered":"2024-04-20T21:31:26.680624+0000","last_unstale":"2024-04-20T21:33:49.119384+0000","last_undegraded":"2024-04-20T21:33:49.119384+0000","last_fullsized":"2024-04-20T21:33:49.119384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:02.750166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148990+0000","last_change":"2024-04-20T21:31:26.697632+0000","last_active":"2024-04-20T21:33:32.148990+0000","last_peered":"2024-04-20T21:33:32.148990+0000","last_clean":"2024-04-20T21:33:32.148990+0000","last_became_active":"2024-04-20T21:31:26.697393+0000","last_became_peered":"2024-04-20T21:31:26.697393+0000","last_unstale":"2024-04-20T21:33:32.148990+0000","last_undegraded":"2024-04-20T21:33:32.148990+0000","last_fullsized":"2024-04-20T21:33:32.148990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:08:39.856082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.832878+0000","last_change":"2024-04-20T21:31:27.526613+0000","last_active":"2024-04-20T21:33:48.832878+0000","last_peered":"2024-04-20T21:33:48.832878+0000","last_clean":"2024-04-20T21:33:48.832878+0000","last_became_active":"2024-04-20T21:31:27.526234+0000","last_became_peered":"2024-04-20T21:31:27.526234+0000","last_unstale":"2024-04-20T21:33:48.832878+0000","last_undegraded":"2024-04-20T21:33:48.832878+0000","last_fullsized":"2024-04-20T21:33:48.832878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:06:44.114924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'45","reported_seq":104,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.131406+0000","last_change":"2024-04-20T21:32:31.625214+0000","last_active":"2024-04-20T21:33:49.131406+0000","last_peered":"2024-04-20T21:33:49.131406+0000","last_clean":"2024-04-20T21:33:49.131406+0000","last_became_active":"2024-04-20T21:32:31.619956+0000","last_became_peered":"2024-04-20T21:32:31.619956+0000","last_unstale":"2024-04-20T21:33:49.131406+0000","last_undegraded":"2024-04-20T21:33:49.131406+0000","last_fullsized":"2024-04-20T21:33:49.131406+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:19:41.620846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":81,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.831220+0000","last_change":"2024-04-20T21:31:27.538361+0000","last_active":"2024-04-20T21:33:48.831220+0000","last_peered":"2024-04-20T21:33:48.831220+0000","last_clean":"2024-04-20T21:33:48.831220+0000","last_became_active":"2024-04-20T21:31:27.537494+0000","last_became_peered":"2024-04-20T21:31:27.537494+0000","last_unstale":"2024-04-20T21:33:48.831220+0000","last_undegraded":"2024-04-20T21:33:48.831220+0000","last_fullsized":"2024-04-20T21:33:48.831220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:46.212759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119090+0000","last_change":"2024-04-20T21:31:26.686960+0000","last_active":"2024-04-20T21:33:49.119090+0000","last_peered":"2024-04-20T21:33:49.119090+0000","last_clean":"2024-04-20T21:33:49.119090+0000","last_became_active":"2024-04-20T21:31:26.686565+0000","last_became_peered":"2024-04-20T21:31:26.686565+0000","last_unstale":"2024-04-20T21:33:49.119090+0000","last_undegraded":"2024-04-20T21:33:49.119090+0000","last_fullsized":"2024-04-20T21:33:49.119090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.608834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120313+0000","last_change":"2024-04-20T21:31:27.546978+0000","last_active":"2024-04-20T21:33:49.120313+0000","last_peered":"2024-04-20T21:33:49.120313+0000","last_clean":"2024-04-20T21:33:49.120313+0000","last_became_active":"2024-04-20T21:31:27.545941+0000","last_became_peered":"2024-04-20T21:31:27.545941+0000","last_unstale":"2024-04-20T21:33:49.120313+0000","last_undegraded":"2024-04-20T21:33:49.120313+0000","last_fullsized":"2024-04-20T21:33:49.120313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:08.363519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118443+0000","last_change":"2024-04-20T21:31:26.690797+0000","last_active":"2024-04-20T21:33:49.118443+0000","last_peered":"2024-04-20T21:33:49.118443+0000","last_clean":"2024-04-20T21:33:49.118443+0000","last_became_active":"2024-04-20T21:31:26.690486+0000","last_became_peered":"2024-04-20T21:31:26.690486+0000","last_unstale":"2024-04-20T21:33:49.118443+0000","last_undegraded":"2024-04-20T21:33:49.118443+0000","last_fullsized":"2024-04-20T21:33:49.118443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:17:33.943212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148549+0000","last_change":"2024-04-20T21:31:27.507175+0000","last_active":"2024-04-20T21:33:32.148549+0000","last_peered":"2024-04-20T21:33:32.148549+0000","last_clean":"2024-04-20T21:33:32.148549+0000","last_became_active":"2024-04-20T21:31:27.506808+0000","last_became_peered":"2024-04-20T21:31:27.506808+0000","last_unstale":"2024-04-20T21:33:32.148549+0000","last_undegraded":"2024-04-20T21:33:32.148549+0000","last_fullsized":"2024-04-20T21:33:32.148549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:38:15.212880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828658+0000","last_change":"2024-04-20T21:31:26.698302+0000","last_active":"2024-04-20T21:33:48.828658+0000","last_peered":"2024-04-20T21:33:48.828658+0000","last_clean":"2024-04-20T21:33:48.828658+0000","last_became_active":"2024-04-20T21:31:26.696209+0000","last_became_peered":"2024-04-20T21:31:26.696209+0000","last_unstale":"2024-04-20T21:33:48.828658+0000","last_undegraded":"2024-04-20T21:33:48.828658+0000","last_fullsized":"2024-04-20T21:33:48.828658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:43.495880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.965838+0000","last_change":"2024-04-20T21:31:27.521594+0000","last_active":"2024-04-20T21:33:48.965838+0000","last_peered":"2024-04-20T21:33:48.965838+0000","last_clean":"2024-04-20T21:33:48.965838+0000","last_became_active":"2024-04-20T21:31:27.521185+0000","last_became_peered":"2024-04-20T21:31:27.521185+0000","last_unstale":"2024-04-20T21:33:48.965838+0000","last_undegraded":"2024-04-20T21:33:48.965838+0000","last_fullsized":"2024-04-20T21:33:48.965838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:22:08.647624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118193+0000","last_change":"2024-04-20T21:31:26.691536+0000","last_active":"2024-04-20T21:33:49.118193+0000","last_peered":"2024-04-20T21:33:49.118193+0000","last_clean":"2024-04-20T21:33:49.118193+0000","last_became_active":"2024-04-20T21:31:26.691202+0000","last_became_peered":"2024-04-20T21:31:26.691202+0000","last_unstale":"2024-04-20T21:33:49.118193+0000","last_undegraded":"2024-04-20T21:33:49.118193+0000","last_fullsized":"2024-04-20T21:33:49.118193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:58:59.331644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963190+0000","last_change":"2024-04-20T21:32:31.998258+0000","last_active":"2024-04-20T21:33:48.963190+0000","last_peered":"2024-04-20T21:33:48.963190+0000","last_clean":"2024-04-20T21:33:48.963190+0000","last_became_active":"2024-04-20T21:32:31.997119+0000","last_became_peered":"2024-04-20T21:32:31.997119+0000","last_unstale":"2024-04-20T21:33:48.963190+0000","last_undegraded":"2024-04-20T21:33:48.963190+0000","last_fullsized":"2024-04-20T21:33:48.963190+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:41:08.565434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148696+0000","last_change":"2024-04-20T21:32:31.616245+0000","last_active":"2024-04-20T21:33:32.148696+0000","last_peered":"2024-04-20T21:33:32.148696+0000","last_clean":"2024-04-20T21:33:32.148696+0000","last_became_active":"2024-04-20T21:32:31.611535+0000","last_became_peered":"2024-04-20T21:32:31.611535+0000","last_unstale":"2024-04-20T21:33:32.148696+0000","last_undegraded":"2024-04-20T21:33:32.148696+0000","last_fullsized":"2024-04-20T21:33:32.148696+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:28:17.609890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119779+0000","last_change":"2024-04-20T21:31:27.518346+0000","last_active":"2024-04-20T21:33:49.119779+0000","last_peered":"2024-04-20T21:33:49.119779+0000","last_clean":"2024-04-20T21:33:49.119779+0000","last_became_active":"2024-04-20T21:31:27.517782+0000","last_became_peered":"2024-04-20T21:31:27.517782+0000","last_unstale":"2024-04-20T21:33:49.119779+0000","last_undegraded":"2024-04-20T21:33:49.119779+0000","last_fullsized":"2024-04-20T21:33:49.119779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:21.674663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119132+0000","last_change":"2024-04-20T21:32:31.615626+0000","last_active":"2024-04-20T21:33:49.119132+0000","last_peered":"2024-04-20T21:33:49.119132+0000","last_clean":"2024-04-20T21:33:49.119132+0000","last_became_active":"2024-04-20T21:32:31.615181+0000","last_became_peered":"2024-04-20T21:32:31.615181+0000","last_unstale":"2024-04-20T21:33:49.119132+0000","last_undegraded":"2024-04-20T21:33:49.119132+0000","last_fullsized":"2024-04-20T21:33:49.119132+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:58:18.344583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119853+0000","last_change":"2024-04-20T21:31:27.510920+0000","last_active":"2024-04-20T21:33:49.119853+0000","last_peered":"2024-04-20T21:33:49.119853+0000","last_clean":"2024-04-20T21:33:49.119853+0000","last_became_active":"2024-04-20T21:31:27.510564+0000","last_became_peered":"2024-04-20T21:31:27.510564+0000","last_unstale":"2024-04-20T21:33:49.119853+0000","last_undegraded":"2024-04-20T21:33:49.119853+0000","last_fullsized":"2024-04-20T21:33:49.119853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:05.196178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.128160+0000","last_change":"2024-04-20T21:31:26.693159+0000","last_active":"2024-04-20T21:33:49.128160+0000","last_peered":"2024-04-20T21:33:49.128160+0000","last_clean":"2024-04-20T21:33:49.128160+0000","last_became_active":"2024-04-20T21:31:26.692902+0000","last_became_peered":"2024-04-20T21:31:26.692902+0000","last_unstale":"2024-04-20T21:33:49.128160+0000","last_undegraded":"2024-04-20T21:33:49.128160+0000","last_fullsized":"2024-04-20T21:33:49.128160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:54.704343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119400+0000","last_change":"2024-04-20T21:31:27.541557+0000","last_active":"2024-04-20T21:33:49.119400+0000","last_peered":"2024-04-20T21:33:49.119400+0000","last_clean":"2024-04-20T21:33:49.119400+0000","last_became_active":"2024-04-20T21:31:27.541161+0000","last_became_peered":"2024-04-20T21:31:27.541161+0000","last_unstale":"2024-04-20T21:33:49.119400+0000","last_undegraded":"2024-04-20T21:33:49.119400+0000","last_fullsized":"2024-04-20T21:33:49.119400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:55:25.558918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.833193+0000","last_change":"2024-04-20T21:31:26.688019+0000","last_active":"2024-04-20T21:33:48.833193+0000","last_peered":"2024-04-20T21:33:48.833193+0000","last_clean":"2024-04-20T21:33:48.833193+0000","last_became_active":"2024-04-20T21:31:26.686370+0000","last_became_peered":"2024-04-20T21:31:26.686370+0000","last_unstale":"2024-04-20T21:33:48.833193+0000","last_undegraded":"2024-04-20T21:33:48.833193+0000","last_fullsized":"2024-04-20T21:33:48.833193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:42:44.059365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119798+0000","last_change":"2024-04-20T21:31:27.511592+0000","last_active":"2024-04-20T21:33:49.119798+0000","last_peered":"2024-04-20T21:33:49.119798+0000","last_clean":"2024-04-20T21:33:49.119798+0000","last_became_active":"2024-04-20T21:31:27.511311+0000","last_became_peered":"2024-04-20T21:31:27.511311+0000","last_unstale":"2024-04-20T21:33:49.119798+0000","last_undegraded":"2024-04-20T21:33:49.119798+0000","last_fullsized":"2024-04-20T21:33:49.119798+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:35:27.665578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.965845+0000","last_change":"2024-04-20T21:31:27.511827+0000","last_active":"2024-04-20T21:33:48.965845+0000","last_peered":"2024-04-20T21:33:48.965845+0000","last_clean":"2024-04-20T21:33:48.965845+0000","last_became_active":"2024-04-20T21:31:27.511344+0000","last_became_peered":"2024-04-20T21:31:27.511344+0000","last_unstale":"2024-04-20T21:33:48.965845+0000","last_undegraded":"2024-04-20T21:33:48.965845+0000","last_fullsized":"2024-04-20T21:33:48.965845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:11:37.973050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118126+0000","last_change":"2024-04-20T21:31:27.510747+0000","last_active":"2024-04-20T21:33:49.118126+0000","last_peered":"2024-04-20T21:33:49.118126+0000","last_clean":"2024-04-20T21:33:49.118126+0000","last_became_active":"2024-04-20T21:31:27.510253+0000","last_became_peered":"2024-04-20T21:31:27.510253+0000","last_unstale":"2024-04-20T21:33:49.118126+0000","last_undegraded":"2024-04-20T21:33:49.118126+0000","last_fullsized":"2024-04-20T21:33:49.118126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:37:13.855840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118089+0000","last_change":"2024-04-20T21:31:27.541185+0000","last_active":"2024-04-20T21:33:49.118089+0000","last_peered":"2024-04-20T21:33:49.118089+0000","last_clean":"2024-04-20T21:33:49.118089+0000","last_became_active":"2024-04-20T21:31:27.540881+0000","last_became_peered":"2024-04-20T21:31:27.540881+0000","last_unstale":"2024-04-20T21:33:49.118089+0000","last_undegraded":"2024-04-20T21:33:49.118089+0000","last_fullsized":"2024-04-20T21:33:49.118089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:46:41.897081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119309+0000","last_change":"2024-04-20T21:31:27.527204+0000","last_active":"2024-04-20T21:33:49.119309+0000","last_peered":"2024-04-20T21:33:49.119309+0000","last_clean":"2024-04-20T21:33:49.119309+0000","last_became_active":"2024-04-20T21:31:27.526805+0000","last_became_peered":"2024-04-20T21:31:27.526805+0000","last_unstale":"2024-04-20T21:33:49.119309+0000","last_undegraded":"2024-04-20T21:33:49.119309+0000","last_fullsized":"2024-04-20T21:33:49.119309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:43.032609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.832649+0000","last_change":"2024-04-20T21:31:27.525521+0000","last_active":"2024-04-20T21:33:48.832649+0000","last_peered":"2024-04-20T21:33:48.832649+0000","last_clean":"2024-04-20T21:33:48.832649+0000","last_became_active":"2024-04-20T21:31:27.521892+0000","last_became_peered":"2024-04-20T21:31:27.521892+0000","last_unstale":"2024-04-20T21:33:48.832649+0000","last_undegraded":"2024-04-20T21:33:48.832649+0000","last_fullsized":"2024-04-20T21:33:48.832649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:32:47.683115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966118+0000","last_change":"2024-04-20T21:33:32.432716+0000","last_active":"2024-04-20T21:33:48.966118+0000","last_peered":"2024-04-20T21:33:48.966118+0000","last_clean":"2024-04-20T21:33:48.966118+0000","last_became_active":"2024-04-20T21:33:32.432201+0000","last_became_peered":"2024-04-20T21:33:32.432201+0000","last_unstale":"2024-04-20T21:33:48.966118+0000","last_undegraded":"2024-04-20T21:33:48.966118+0000","last_fullsized":"2024-04-20T21:33:48.966118+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:14:13.089095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118560+0000","last_change":"2024-04-20T21:31:27.512542+0000","last_active":"2024-04-20T21:33:49.118560+0000","last_peered":"2024-04-20T21:33:49.118560+0000","last_clean":"2024-04-20T21:33:49.118560+0000","last_became_active":"2024-04-20T21:31:27.511907+0000","last_became_peered":"2024-04-20T21:31:27.511907+0000","last_unstale":"2024-04-20T21:33:49.118560+0000","last_undegraded":"2024-04-20T21:33:49.118560+0000","last_fullsized":"2024-04-20T21:33:49.118560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:10:09.265471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.831096+0000","last_change":"2024-04-20T21:31:27.538697+0000","last_active":"2024-04-20T21:33:48.831096+0000","last_peered":"2024-04-20T21:33:48.831096+0000","last_clean":"2024-04-20T21:33:48.831096+0000","last_became_active":"2024-04-20T21:31:27.537972+0000","last_became_peered":"2024-04-20T21:31:27.537972+0000","last_unstale":"2024-04-20T21:33:48.831096+0000","last_undegraded":"2024-04-20T21:33:48.831096+0000","last_fullsized":"2024-04-20T21:33:48.831096+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:51.974485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976126+0000","last_change":"2024-04-20T21:31:27.538092+0000","last_active":"2024-04-20T21:33:48.976126+0000","last_peered":"2024-04-20T21:33:48.976126+0000","last_clean":"2024-04-20T21:33:48.976126+0000","last_became_active":"2024-04-20T21:31:27.537821+0000","last_became_peered":"2024-04-20T21:31:27.537821+0000","last_unstale":"2024-04-20T21:33:48.976126+0000","last_undegraded":"2024-04-20T21:33:48.976126+0000","last_fullsized":"2024-04-20T21:33:48.976126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:52.773583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828402+0000","last_change":"2024-04-20T21:31:27.535498+0000","last_active":"2024-04-20T21:33:48.828402+0000","last_peered":"2024-04-20T21:33:48.828402+0000","last_clean":"2024-04-20T21:33:48.828402+0000","last_became_active":"2024-04-20T21:31:27.535162+0000","last_became_peered":"2024-04-20T21:31:27.535162+0000","last_unstale":"2024-04-20T21:33:48.828402+0000","last_undegraded":"2024-04-20T21:33:48.828402+0000","last_fullsized":"2024-04-20T21:33:48.828402+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:46.638825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120232+0000","last_change":"2024-04-20T21:31:27.537776+0000","last_active":"2024-04-20T21:33:49.120232+0000","last_peered":"2024-04-20T21:33:49.120232+0000","last_clean":"2024-04-20T21:33:49.120232+0000","last_became_active":"2024-04-20T21:31:27.537428+0000","last_became_peered":"2024-04-20T21:31:27.537428+0000","last_unstale":"2024-04-20T21:33:49.120232+0000","last_undegraded":"2024-04-20T21:33:49.120232+0000","last_fullsized":"2024-04-20T21:33:49.120232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:15:03.344277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.834707+0000","last_change":"2024-04-20T21:31:27.535943+0000","last_active":"2024-04-20T21:33:48.834707+0000","last_peered":"2024-04-20T21:33:48.834707+0000","last_clean":"2024-04-20T21:33:48.834707+0000","last_became_active":"2024-04-20T21:31:27.533694+0000","last_became_peered":"2024-04-20T21:31:27.533694+0000","last_unstale":"2024-04-20T21:33:48.834707+0000","last_undegraded":"2024-04-20T21:33:48.834707+0000","last_fullsized":"2024-04-20T21:33:48.834707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:42.603232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966013+0000","last_change":"2024-04-20T21:31:27.511345+0000","last_active":"2024-04-20T21:33:48.966013+0000","last_peered":"2024-04-20T21:33:48.966013+0000","last_clean":"2024-04-20T21:33:48.966013+0000","last_became_active":"2024-04-20T21:31:27.510984+0000","last_became_peered":"2024-04-20T21:31:27.510984+0000","last_unstale":"2024-04-20T21:33:48.966013+0000","last_undegraded":"2024-04-20T21:33:48.966013+0000","last_fullsized":"2024-04-20T21:33:48.966013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:19.965636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":593374,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":387,"num_read_kb":296,"num_write":200,"num_write_kb":1404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1880064,"data_stored":1780122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285609,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1421312,"data_stored":888395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547246,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28816,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716336,"statfs":{"total":95995035648,"available":95965528064,"internally_reserved":0,"allocated":1777664,"data_stored":1213519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710772,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":486747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972411,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":486769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266750,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1613824,"data_stored":1077232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430279,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":495296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":562939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":495269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":562973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149781,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1572864,"data_stored":1020333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888145,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":495201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691915,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37248,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707904,"statfs":{"total":95995035648,"available":95956893696,"internally_reserved":0,"allocated":1773568,"data_stored":1217446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":524920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":131782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":199554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":393216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:33:52.841 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-20T21:33:52.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:33:53.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.075+0000 7f585a000640 1 -- 172.21.15.22:0/3503494845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5854109cb0 msgr2=0x7f585410c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:53.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.075+0000 7f585a000640 1 --2- 172.21.15.22:0/3503494845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5854109cb0 0x7f585410c140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f584c009be0 tx=0x7f584c02bf80 comp rx=0 tx=0).stop 2024-04-20T21:33:53.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.075+0000 7f585a000640 1 -- 172.21.15.22:0/3503494845 shutdown_connections 2024-04-20T21:33:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.075+0000 7f585a000640 1 --2- 172.21.15.22:0/3503494845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5854109cb0 0x7f585410c140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.075+0000 7f585a000640 1 --2- 172.21.15.22:0/3503494845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5854074940 0x7f5854074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.075+0000 7f585a000640 1 --2- 172.21.15.22:0/3503494845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5854075f20 0x7f5854074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.075+0000 7f585a000640 1 -- 172.21.15.22:0/3503494845 >> 172.21.15.22:0/3503494845 conn(0x7f58540fd370 msgr2=0x7f58540ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.075+0000 7f585a000640 1 -- 172.21.15.22:0/3503494845 shutdown_connections 2024-04-20T21:33:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.075+0000 7f585a000640 1 -- 172.21.15.22:0/3503494845 wait complete. 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.076+0000 7f585a000640 1 Processor -- start 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.076+0000 7f585a000640 1 -- start start 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f585a000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5854074940 0x7f585419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f585a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5854075f20 0x7f585419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f585a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5854109cb0 0x7f58541a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f585a000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f585410fcf0 con 0x7f5854075f20 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f585a000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f585410fb70 con 0x7f5854074940 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f585a000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f585410fe70 con 0x7f5854109cb0 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f5858c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5854074940 0x7f585419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f5853e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5854075f20 0x7f585419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f5858c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5854074940 0x7f585419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60090/0 (socket says 172.21.15.22:60090) 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f5859600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5854109cb0 0x7f58541a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f5858c00640 1 -- 172.21.15.22:0/1784093268 learned_addr learned my addr 172.21.15.22:0/1784093268 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f5858c00640 1 -- 172.21.15.22:0/1784093268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5854109cb0 msgr2=0x7f58541a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f5858c00640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5854109cb0 0x7f58541a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f5858c00640 1 -- 172.21.15.22:0/1784093268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5854075f20 msgr2=0x7f585419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f5858c00640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5854075f20 0x7f585419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.077+0000 7f5858c00640 1 -- 172.21.15.22:0/1784093268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58541a1ed0 con 0x7f5854074940 2024-04-20T21:33:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.078+0000 7f5859600640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5854109cb0 0x7f58541a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.078+0000 7f5853e00640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5854075f20 0x7f585419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:53.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.078+0000 7f5858c00640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5854074940 0x7f585419a2b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f584800e9c0 tx=0x7f584800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.078+0000 7f5851600640 1 -- 172.21.15.22:0/1784093268 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584800cde0 con 0x7f5854074940 2024-04-20T21:33:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.078+0000 7f5851600640 1 -- 172.21.15.22:0/1784093268 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58480035f0 con 0x7f5854074940 2024-04-20T21:33:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.078+0000 7f5851600640 1 -- 172.21.15.22:0/1784093268 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848010690 con 0x7f5854074940 2024-04-20T21:33:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.078+0000 7f585a000640 1 -- 172.21.15.22:0/1784093268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58541a21c0 con 0x7f5854074940 2024-04-20T21:33:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.078+0000 7f585a000640 1 -- 172.21.15.22:0/1784093268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58541a2760 con 0x7f5854074940 2024-04-20T21:33:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.080+0000 7f585a000640 1 -- 172.21.15.22:0/1784093268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5854108a70 con 0x7f5854074940 2024-04-20T21:33:53.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.083+0000 7f5851600640 1 -- 172.21.15.22:0/1784093268 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7f58480037a0 con 0x7f5854074940 2024-04-20T21:33:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.084+0000 7f5851600640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f583c0a3da0 0x7f583c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.084+0000 7f5851600640 1 -- 172.21.15.22:0/1784093268 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f58480d0ba0 con 0x7f5854074940 2024-04-20T21:33:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.084+0000 7f5853e00640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f583c0a3da0 0x7f583c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.085+0000 7f5853e00640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f583c0a3da0 0x7f583c0a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f585419b7c0 tx=0x7f5840005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.085+0000 7f5851600640 1 -- 172.21.15.22:0/1784093268 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f584809d250 con 0x7f5854074940 2024-04-20T21:33:53.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.282+0000 7f5851600640 1 -- 172.21.15.22:0/1784093268 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f58480991c0 con 0x7f5854074940 2024-04-20T21:33:53.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.310+0000 7f585a000640 1 -- 172.21.15.22:0/1784093268 --> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58540623a0 con 0x7f583c0a3da0 2024-04-20T21:33:53.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.320+0000 7f5851600640 1 -- 172.21.15.22:0/1784093268 <== mgr.15024 v2:172.21.15.22:6800/975472489 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257707 (secure 0 0 0) 0x7f58540623a0 con 0x7f583c0a3da0 2024-04-20T21:33:53.319 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:53.322 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-04-20T21:33:53.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.325+0000 7f585a000640 1 -- 172.21.15.22:0/1784093268 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f583c0a3da0 msgr2=0x7f583c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:53.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.325+0000 7f585a000640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f583c0a3da0 0x7f583c0a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f585419b7c0 tx=0x7f5840005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:53.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.326+0000 7f585a000640 1 -- 172.21.15.22:0/1784093268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5854074940 msgr2=0x7f585419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:53.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.326+0000 7f585a000640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5854074940 0x7f585419a2b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f584800e9c0 tx=0x7f584800ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:53.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.326+0000 7f585a000640 1 -- 172.21.15.22:0/1784093268 shutdown_connections 2024-04-20T21:33:53.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.326+0000 7f585a000640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f583c0a3da0 0x7f583c0a6260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.326+0000 7f585a000640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5854109cb0 0x7f58541a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.326+0000 7f585a000640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5854075f20 0x7f585419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.326+0000 7f585a000640 1 --2- 172.21.15.22:0/1784093268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5854074940 0x7f585419a2b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.326+0000 7f585a000640 1 -- 172.21.15.22:0/1784093268 >> 172.21.15.22:0/1784093268 conn(0x7f58540fd370 msgr2=0x7f5854105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:53.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.326+0000 7f585a000640 1 -- 172.21.15.22:0/1784093268 shutdown_connections 2024-04-20T21:33:53.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.326+0000 7f585a000640 1 -- 172.21.15.22:0/1784093268 wait complete. 2024-04-20T21:33:53.342 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-20T21:33:52.845978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1183976,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":523,"num_read_kb":415,"num_write":383,"num_write_kb":4344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385388,"kb_used_data":15144,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124556436,"statfs":{"total":1151940427776,"available":1151545790464,"internally_reserved":0,"allocated":15507456,"data_stored":9002119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.978086"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966208+0000","last_change":"2024-04-20T21:31:27.527151+0000","last_active":"2024-04-20T21:33:48.966208+0000","last_peered":"2024-04-20T21:33:48.966208+0000","last_clean":"2024-04-20T21:33:48.966208+0000","last_became_active":"2024-04-20T21:31:27.526739+0000","last_became_peered":"2024-04-20T21:31:27.526739+0000","last_unstale":"2024-04-20T21:33:48.966208+0000","last_undegraded":"2024-04-20T21:33:48.966208+0000","last_fullsized":"2024-04-20T21:33:48.966208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:16:39.204523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120131+0000","last_change":"2024-04-20T21:31:27.527437+0000","last_active":"2024-04-20T21:33:49.120131+0000","last_peered":"2024-04-20T21:33:49.120131+0000","last_clean":"2024-04-20T21:33:49.120131+0000","last_became_active":"2024-04-20T21:31:27.527040+0000","last_became_peered":"2024-04-20T21:31:27.527040+0000","last_unstale":"2024-04-20T21:33:49.120131+0000","last_undegraded":"2024-04-20T21:33:49.120131+0000","last_fullsized":"2024-04-20T21:33:49.120131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:53:39.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119810+0000","last_change":"2024-04-20T21:31:27.531671+0000","last_active":"2024-04-20T21:33:49.119810+0000","last_peered":"2024-04-20T21:33:49.119810+0000","last_clean":"2024-04-20T21:33:49.119810+0000","last_became_active":"2024-04-20T21:31:27.531267+0000","last_became_peered":"2024-04-20T21:31:27.531267+0000","last_unstale":"2024-04-20T21:33:49.119810+0000","last_undegraded":"2024-04-20T21:33:49.119810+0000","last_fullsized":"2024-04-20T21:33:49.119810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:47.002481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119241+0000","last_change":"2024-04-20T21:31:27.512033+0000","last_active":"2024-04-20T21:33:49.119241+0000","last_peered":"2024-04-20T21:33:49.119241+0000","last_clean":"2024-04-20T21:33:49.119241+0000","last_became_active":"2024-04-20T21:31:27.510801+0000","last_became_peered":"2024-04-20T21:31:27.510801+0000","last_unstale":"2024-04-20T21:33:49.119241+0000","last_undegraded":"2024-04-20T21:33:49.119241+0000","last_fullsized":"2024-04-20T21:33:49.119241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:43:25.407173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148706+0000","last_change":"2024-04-20T21:31:27.545862+0000","last_active":"2024-04-20T21:33:32.148706+0000","last_peered":"2024-04-20T21:33:32.148706+0000","last_clean":"2024-04-20T21:33:32.148706+0000","last_became_active":"2024-04-20T21:31:27.545500+0000","last_became_peered":"2024-04-20T21:31:27.545500+0000","last_unstale":"2024-04-20T21:33:32.148706+0000","last_undegraded":"2024-04-20T21:33:32.148706+0000","last_fullsized":"2024-04-20T21:33:32.148706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:37.640777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828391+0000","last_change":"2024-04-20T21:31:27.526408+0000","last_active":"2024-04-20T21:33:48.828391+0000","last_peered":"2024-04-20T21:33:48.828391+0000","last_clean":"2024-04-20T21:33:48.828391+0000","last_became_active":"2024-04-20T21:31:27.526015+0000","last_became_peered":"2024-04-20T21:31:27.526015+0000","last_unstale":"2024-04-20T21:33:48.828391+0000","last_undegraded":"2024-04-20T21:33:48.828391+0000","last_fullsized":"2024-04-20T21:33:48.828391+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:31:56.904696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119156+0000","last_change":"2024-04-20T21:31:27.526272+0000","last_active":"2024-04-20T21:33:49.119156+0000","last_peered":"2024-04-20T21:33:49.119156+0000","last_clean":"2024-04-20T21:33:49.119156+0000","last_became_active":"2024-04-20T21:31:27.525852+0000","last_became_peered":"2024-04-20T21:31:27.525852+0000","last_unstale":"2024-04-20T21:33:49.119156+0000","last_undegraded":"2024-04-20T21:33:49.119156+0000","last_fullsized":"2024-04-20T21:33:49.119156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:33:57.811339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.962898+0000","last_change":"2024-04-20T21:32:31.998671+0000","last_active":"2024-04-20T21:33:48.962898+0000","last_peered":"2024-04-20T21:33:48.962898+0000","last_clean":"2024-04-20T21:33:48.962898+0000","last_became_active":"2024-04-20T21:32:31.996769+0000","last_became_peered":"2024-04-20T21:32:31.996769+0000","last_unstale":"2024-04-20T21:33:48.962898+0000","last_undegraded":"2024-04-20T21:33:48.962898+0000","last_fullsized":"2024-04-20T21:33:48.962898+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:03.135794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120675+0000","last_change":"2024-04-20T21:31:27.514810+0000","last_active":"2024-04-20T21:33:49.120675+0000","last_peered":"2024-04-20T21:33:49.120675+0000","last_clean":"2024-04-20T21:33:49.120675+0000","last_became_active":"2024-04-20T21:31:27.512548+0000","last_became_peered":"2024-04-20T21:31:27.512548+0000","last_unstale":"2024-04-20T21:33:49.120675+0000","last_undegraded":"2024-04-20T21:33:49.120675+0000","last_fullsized":"2024-04-20T21:33:49.120675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:11:30.375294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828858+0000","last_change":"2024-04-20T21:31:27.510303+0000","last_active":"2024-04-20T21:33:48.828858+0000","last_peered":"2024-04-20T21:33:48.828858+0000","last_clean":"2024-04-20T21:33:48.828858+0000","last_became_active":"2024-04-20T21:31:27.509907+0000","last_became_peered":"2024-04-20T21:31:27.509907+0000","last_unstale":"2024-04-20T21:33:48.828858+0000","last_undegraded":"2024-04-20T21:33:48.828858+0000","last_fullsized":"2024-04-20T21:33:48.828858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:31.160785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119674+0000","last_change":"2024-04-20T21:31:27.513749+0000","last_active":"2024-04-20T21:33:49.119674+0000","last_peered":"2024-04-20T21:33:49.119674+0000","last_clean":"2024-04-20T21:33:49.119674+0000","last_became_active":"2024-04-20T21:31:27.513404+0000","last_became_peered":"2024-04-20T21:31:27.513404+0000","last_unstale":"2024-04-20T21:33:49.119674+0000","last_undegraded":"2024-04-20T21:33:49.119674+0000","last_fullsized":"2024-04-20T21:33:49.119674+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:26.279697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118958+0000","last_change":"2024-04-20T21:31:27.507727+0000","last_active":"2024-04-20T21:33:49.118958+0000","last_peered":"2024-04-20T21:33:49.118958+0000","last_clean":"2024-04-20T21:33:49.118958+0000","last_became_active":"2024-04-20T21:31:27.507363+0000","last_became_peered":"2024-04-20T21:31:27.507363+0000","last_unstale":"2024-04-20T21:33:49.118958+0000","last_undegraded":"2024-04-20T21:33:49.118958+0000","last_fullsized":"2024-04-20T21:33:49.118958+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:01.139963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.834435+0000","last_change":"2024-04-20T21:31:27.512941+0000","last_active":"2024-04-20T21:33:48.834435+0000","last_peered":"2024-04-20T21:33:48.834435+0000","last_clean":"2024-04-20T21:33:48.834435+0000","last_became_active":"2024-04-20T21:31:27.512571+0000","last_became_peered":"2024-04-20T21:31:27.512571+0000","last_unstale":"2024-04-20T21:33:48.834435+0000","last_undegraded":"2024-04-20T21:33:48.834435+0000","last_fullsized":"2024-04-20T21:33:48.834435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:30.977624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119024+0000","last_change":"2024-04-20T21:31:27.521439+0000","last_active":"2024-04-20T21:33:49.119024+0000","last_peered":"2024-04-20T21:33:49.119024+0000","last_clean":"2024-04-20T21:33:49.119024+0000","last_became_active":"2024-04-20T21:31:27.520245+0000","last_became_peered":"2024-04-20T21:31:27.520245+0000","last_unstale":"2024-04-20T21:33:49.119024+0000","last_undegraded":"2024-04-20T21:33:49.119024+0000","last_fullsized":"2024-04-20T21:33:49.119024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:53:26.510845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118750+0000","last_change":"2024-04-20T21:32:31.619209+0000","last_active":"2024-04-20T21:33:49.118750+0000","last_peered":"2024-04-20T21:33:49.118750+0000","last_clean":"2024-04-20T21:33:49.118750+0000","last_became_active":"2024-04-20T21:32:31.618527+0000","last_became_peered":"2024-04-20T21:32:31.618527+0000","last_unstale":"2024-04-20T21:33:49.118750+0000","last_undegraded":"2024-04-20T21:33:49.118750+0000","last_fullsized":"2024-04-20T21:33:49.118750+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:54:21.400896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118476+0000","last_change":"2024-04-20T21:31:27.521533+0000","last_active":"2024-04-20T21:33:49.118476+0000","last_peered":"2024-04-20T21:33:49.118476+0000","last_clean":"2024-04-20T21:33:49.118476+0000","last_became_active":"2024-04-20T21:31:27.520920+0000","last_became_peered":"2024-04-20T21:31:27.520920+0000","last_unstale":"2024-04-20T21:33:49.118476+0000","last_undegraded":"2024-04-20T21:33:49.118476+0000","last_fullsized":"2024-04-20T21:33:49.118476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:28:13.605238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966062+0000","last_change":"2024-04-20T21:31:27.527484+0000","last_active":"2024-04-20T21:33:48.966062+0000","last_peered":"2024-04-20T21:33:48.966062+0000","last_clean":"2024-04-20T21:33:48.966062+0000","last_became_active":"2024-04-20T21:31:27.527048+0000","last_became_peered":"2024-04-20T21:31:27.527048+0000","last_unstale":"2024-04-20T21:33:48.966062+0000","last_undegraded":"2024-04-20T21:33:48.966062+0000","last_fullsized":"2024-04-20T21:33:48.966062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:35:56.467661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963479+0000","last_change":"2024-04-20T21:31:27.527870+0000","last_active":"2024-04-20T21:33:48.963479+0000","last_peered":"2024-04-20T21:33:48.963479+0000","last_clean":"2024-04-20T21:33:48.963479+0000","last_became_active":"2024-04-20T21:31:27.525918+0000","last_became_peered":"2024-04-20T21:31:27.525918+0000","last_unstale":"2024-04-20T21:33:48.963479+0000","last_undegraded":"2024-04-20T21:33:48.963479+0000","last_fullsized":"2024-04-20T21:33:48.963479+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:34.425370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118470+0000","last_change":"2024-04-20T21:31:27.544987+0000","last_active":"2024-04-20T21:33:49.118470+0000","last_peered":"2024-04-20T21:33:49.118470+0000","last_clean":"2024-04-20T21:33:49.118470+0000","last_became_active":"2024-04-20T21:31:27.544628+0000","last_became_peered":"2024-04-20T21:31:27.544628+0000","last_unstale":"2024-04-20T21:33:49.118470+0000","last_undegraded":"2024-04-20T21:33:49.118470+0000","last_fullsized":"2024-04-20T21:33:49.118470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:05:42.396775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963603+0000","last_change":"2024-04-20T21:31:26.702803+0000","last_active":"2024-04-20T21:33:48.963603+0000","last_peered":"2024-04-20T21:33:48.963603+0000","last_clean":"2024-04-20T21:33:48.963603+0000","last_became_active":"2024-04-20T21:31:26.695781+0000","last_became_peered":"2024-04-20T21:31:26.695781+0000","last_unstale":"2024-04-20T21:33:48.963603+0000","last_undegraded":"2024-04-20T21:33:48.963603+0000","last_fullsized":"2024-04-20T21:33:48.963603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:49:17.131832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.830615+0000","last_change":"2024-04-20T21:31:27.535264+0000","last_active":"2024-04-20T21:33:48.830615+0000","last_peered":"2024-04-20T21:33:48.830615+0000","last_clean":"2024-04-20T21:33:48.830615+0000","last_became_active":"2024-04-20T21:31:27.534888+0000","last_became_peered":"2024-04-20T21:31:27.534888+0000","last_unstale":"2024-04-20T21:33:48.830615+0000","last_undegraded":"2024-04-20T21:33:48.830615+0000","last_fullsized":"2024-04-20T21:33:48.830615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:04:20.951916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.830901+0000","last_change":"2024-04-20T21:31:26.700378+0000","last_active":"2024-04-20T21:33:48.830901+0000","last_peered":"2024-04-20T21:33:48.830901+0000","last_clean":"2024-04-20T21:33:48.830901+0000","last_became_active":"2024-04-20T21:31:26.700077+0000","last_became_peered":"2024-04-20T21:31:26.700077+0000","last_unstale":"2024-04-20T21:33:48.830901+0000","last_undegraded":"2024-04-20T21:33:48.830901+0000","last_fullsized":"2024-04-20T21:33:48.830901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:34:21.646402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120548+0000","last_change":"2024-04-20T21:31:27.531069+0000","last_active":"2024-04-20T21:33:49.120548+0000","last_peered":"2024-04-20T21:33:49.120548+0000","last_clean":"2024-04-20T21:33:49.120548+0000","last_became_active":"2024-04-20T21:31:27.530457+0000","last_became_peered":"2024-04-20T21:31:27.530457+0000","last_unstale":"2024-04-20T21:33:49.120548+0000","last_undegraded":"2024-04-20T21:33:49.120548+0000","last_fullsized":"2024-04-20T21:33:49.120548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:46:59.829623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.149054+0000","last_change":"2024-04-20T21:31:26.684150+0000","last_active":"2024-04-20T21:33:32.149054+0000","last_peered":"2024-04-20T21:33:32.149054+0000","last_clean":"2024-04-20T21:33:32.149054+0000","last_became_active":"2024-04-20T21:31:26.683823+0000","last_became_peered":"2024-04-20T21:31:26.683823+0000","last_unstale":"2024-04-20T21:33:32.149054+0000","last_undegraded":"2024-04-20T21:33:32.149054+0000","last_fullsized":"2024-04-20T21:33:32.149054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:14.413606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120262+0000","last_change":"2024-04-20T21:31:27.507499+0000","last_active":"2024-04-20T21:33:49.120262+0000","last_peered":"2024-04-20T21:33:49.120262+0000","last_clean":"2024-04-20T21:33:49.120262+0000","last_became_active":"2024-04-20T21:31:27.506922+0000","last_became_peered":"2024-04-20T21:31:27.506922+0000","last_unstale":"2024-04-20T21:33:49.120262+0000","last_undegraded":"2024-04-20T21:33:49.120262+0000","last_fullsized":"2024-04-20T21:33:49.120262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:40.865232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.965750+0000","last_change":"2024-04-20T21:32:31.613577+0000","last_active":"2024-04-20T21:33:48.965750+0000","last_peered":"2024-04-20T21:33:48.965750+0000","last_clean":"2024-04-20T21:33:48.965750+0000","last_became_active":"2024-04-20T21:32:31.608943+0000","last_became_peered":"2024-04-20T21:32:31.608943+0000","last_unstale":"2024-04-20T21:33:48.965750+0000","last_undegraded":"2024-04-20T21:33:48.965750+0000","last_fullsized":"2024-04-20T21:33:48.965750+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:06.561157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976247+0000","last_change":"2024-04-20T21:31:27.517839+0000","last_active":"2024-04-20T21:33:48.976247+0000","last_peered":"2024-04-20T21:33:48.976247+0000","last_clean":"2024-04-20T21:33:48.976247+0000","last_became_active":"2024-04-20T21:31:27.517044+0000","last_became_peered":"2024-04-20T21:31:27.517044+0000","last_unstale":"2024-04-20T21:33:48.976247+0000","last_undegraded":"2024-04-20T21:33:48.976247+0000","last_fullsized":"2024-04-20T21:33:48.976247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:50.042665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976109+0000","last_change":"2024-04-20T21:31:26.686151+0000","last_active":"2024-04-20T21:33:48.976109+0000","last_peered":"2024-04-20T21:33:48.976109+0000","last_clean":"2024-04-20T21:33:48.976109+0000","last_became_active":"2024-04-20T21:31:26.685827+0000","last_became_peered":"2024-04-20T21:31:26.685827+0000","last_unstale":"2024-04-20T21:33:48.976109+0000","last_undegraded":"2024-04-20T21:33:48.976109+0000","last_fullsized":"2024-04-20T21:33:48.976109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:41:17.263420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148585+0000","last_change":"2024-04-20T21:32:31.620563+0000","last_active":"2024-04-20T21:33:32.148585+0000","last_peered":"2024-04-20T21:33:32.148585+0000","last_clean":"2024-04-20T21:33:32.148585+0000","last_became_active":"2024-04-20T21:32:31.620145+0000","last_became_peered":"2024-04-20T21:32:31.620145+0000","last_unstale":"2024-04-20T21:33:32.148585+0000","last_undegraded":"2024-04-20T21:33:32.148585+0000","last_fullsized":"2024-04-20T21:33:32.148585+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:43.609490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828355+0000","last_change":"2024-04-20T21:31:26.684573+0000","last_active":"2024-04-20T21:33:48.828355+0000","last_peered":"2024-04-20T21:33:48.828355+0000","last_clean":"2024-04-20T21:33:48.828355+0000","last_became_active":"2024-04-20T21:31:26.684163+0000","last_became_peered":"2024-04-20T21:31:26.684163+0000","last_unstale":"2024-04-20T21:33:48.828355+0000","last_undegraded":"2024-04-20T21:33:48.828355+0000","last_fullsized":"2024-04-20T21:33:48.828355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:39.445068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976276+0000","last_change":"2024-04-20T21:31:27.543680+0000","last_active":"2024-04-20T21:33:48.976276+0000","last_peered":"2024-04-20T21:33:48.976276+0000","last_clean":"2024-04-20T21:33:48.976276+0000","last_became_active":"2024-04-20T21:31:27.543379+0000","last_became_peered":"2024-04-20T21:31:27.543379+0000","last_unstale":"2024-04-20T21:33:48.976276+0000","last_undegraded":"2024-04-20T21:33:48.976276+0000","last_fullsized":"2024-04-20T21:33:48.976276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:58:17.168813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.827019+0000","last_change":"2024-04-20T21:31:26.689665+0000","last_active":"2024-04-20T21:33:48.827019+0000","last_peered":"2024-04-20T21:33:48.827019+0000","last_clean":"2024-04-20T21:33:48.827019+0000","last_became_active":"2024-04-20T21:31:26.689175+0000","last_became_peered":"2024-04-20T21:31:26.689175+0000","last_unstale":"2024-04-20T21:33:48.827019+0000","last_undegraded":"2024-04-20T21:33:48.827019+0000","last_fullsized":"2024-04-20T21:33:48.827019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:21.342700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.829688+0000","last_change":"2024-04-20T21:31:27.544137+0000","last_active":"2024-04-20T21:33:48.829688+0000","last_peered":"2024-04-20T21:33:48.829688+0000","last_clean":"2024-04-20T21:33:48.829688+0000","last_became_active":"2024-04-20T21:31:27.542841+0000","last_became_peered":"2024-04-20T21:31:27.542841+0000","last_unstale":"2024-04-20T21:33:48.829688+0000","last_undegraded":"2024-04-20T21:33:48.829688+0000","last_fullsized":"2024-04-20T21:33:48.829688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:04.610144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.975963+0000","last_change":"2024-04-20T21:31:26.681937+0000","last_active":"2024-04-20T21:33:48.975963+0000","last_peered":"2024-04-20T21:33:48.975963+0000","last_clean":"2024-04-20T21:33:48.975963+0000","last_became_active":"2024-04-20T21:31:26.681621+0000","last_became_peered":"2024-04-20T21:31:26.681621+0000","last_unstale":"2024-04-20T21:33:48.975963+0000","last_undegraded":"2024-04-20T21:33:48.975963+0000","last_fullsized":"2024-04-20T21:33:48.975963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:06.641608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963226+0000","last_change":"2024-04-20T21:31:27.540225+0000","last_active":"2024-04-20T21:33:48.963226+0000","last_peered":"2024-04-20T21:33:48.963226+0000","last_clean":"2024-04-20T21:33:48.963226+0000","last_became_active":"2024-04-20T21:31:27.539860+0000","last_became_peered":"2024-04-20T21:31:27.539860+0000","last_unstale":"2024-04-20T21:33:48.963226+0000","last_undegraded":"2024-04-20T21:33:48.963226+0000","last_fullsized":"2024-04-20T21:33:48.963226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:50.146761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828972+0000","last_change":"2024-04-20T21:31:26.701816+0000","last_active":"2024-04-20T21:33:48.828972+0000","last_peered":"2024-04-20T21:33:48.828972+0000","last_clean":"2024-04-20T21:33:48.828972+0000","last_became_active":"2024-04-20T21:31:26.701530+0000","last_became_peered":"2024-04-20T21:31:26.701530+0000","last_unstale":"2024-04-20T21:33:48.828972+0000","last_undegraded":"2024-04-20T21:33:48.828972+0000","last_fullsized":"2024-04-20T21:33:48.828972+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:31.175330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120556+0000","last_change":"2024-04-20T21:31:27.527222+0000","last_active":"2024-04-20T21:33:49.120556+0000","last_peered":"2024-04-20T21:33:49.120556+0000","last_clean":"2024-04-20T21:33:49.120556+0000","last_became_active":"2024-04-20T21:31:27.526779+0000","last_became_peered":"2024-04-20T21:31:27.526779+0000","last_unstale":"2024-04-20T21:33:49.120556+0000","last_undegraded":"2024-04-20T21:33:49.120556+0000","last_fullsized":"2024-04-20T21:33:49.120556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:13:54.500516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119650+0000","last_change":"2024-04-20T21:31:26.685969+0000","last_active":"2024-04-20T21:33:49.119650+0000","last_peered":"2024-04-20T21:33:49.119650+0000","last_clean":"2024-04-20T21:33:49.119650+0000","last_became_active":"2024-04-20T21:31:26.685580+0000","last_became_peered":"2024-04-20T21:31:26.685580+0000","last_unstale":"2024-04-20T21:33:49.119650+0000","last_undegraded":"2024-04-20T21:33:49.119650+0000","last_fullsized":"2024-04-20T21:33:49.119650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:22:50.558526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119011+0000","last_change":"2024-04-20T21:31:27.525862+0000","last_active":"2024-04-20T21:33:49.119011+0000","last_peered":"2024-04-20T21:33:49.119011+0000","last_clean":"2024-04-20T21:33:49.119011+0000","last_became_active":"2024-04-20T21:31:27.525372+0000","last_became_peered":"2024-04-20T21:31:27.525372+0000","last_unstale":"2024-04-20T21:33:49.119011+0000","last_undegraded":"2024-04-20T21:33:49.119011+0000","last_fullsized":"2024-04-20T21:33:49.119011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:44:15.527137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976460+0000","last_change":"2024-04-20T21:31:26.691091+0000","last_active":"2024-04-20T21:33:48.976460+0000","last_peered":"2024-04-20T21:33:48.976460+0000","last_clean":"2024-04-20T21:33:48.976460+0000","last_became_active":"2024-04-20T21:31:26.690608+0000","last_became_peered":"2024-04-20T21:31:26.690608+0000","last_unstale":"2024-04-20T21:33:48.976460+0000","last_undegraded":"2024-04-20T21:33:48.976460+0000","last_fullsized":"2024-04-20T21:33:48.976460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:44:25.336193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120411+0000","last_change":"2024-04-20T21:31:27.507900+0000","last_active":"2024-04-20T21:33:49.120411+0000","last_peered":"2024-04-20T21:33:49.120411+0000","last_clean":"2024-04-20T21:33:49.120411+0000","last_became_active":"2024-04-20T21:31:27.507360+0000","last_became_peered":"2024-04-20T21:31:27.507360+0000","last_unstale":"2024-04-20T21:33:49.120411+0000","last_undegraded":"2024-04-20T21:33:49.120411+0000","last_fullsized":"2024-04-20T21:33:49.120411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:27.110877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'33","reported_seq":348,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.265110+0000","last_change":"2024-04-20T21:31:26.685736+0000","last_active":"2024-04-20T21:33:49.265110+0000","last_peered":"2024-04-20T21:33:49.265110+0000","last_clean":"2024-04-20T21:33:49.265110+0000","last_became_active":"2024-04-20T21:31:26.685309+0000","last_became_peered":"2024-04-20T21:31:26.685309+0000","last_unstale":"2024-04-20T21:33:49.265110+0000","last_undegraded":"2024-04-20T21:33:49.265110+0000","last_fullsized":"2024-04-20T21:33:49.265110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:06.501753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67760,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":289,"num_read_kb":211,"num_write":33,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148851+0000","last_change":"2024-04-20T21:31:27.507532+0000","last_active":"2024-04-20T21:33:32.148851+0000","last_peered":"2024-04-20T21:33:32.148851+0000","last_clean":"2024-04-20T21:33:32.148851+0000","last_became_active":"2024-04-20T21:31:27.507188+0000","last_became_peered":"2024-04-20T21:31:27.507188+0000","last_unstale":"2024-04-20T21:33:32.148851+0000","last_undegraded":"2024-04-20T21:33:32.148851+0000","last_fullsized":"2024-04-20T21:33:32.148851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:58.814116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118339+0000","last_change":"2024-04-20T21:31:26.701361+0000","last_active":"2024-04-20T21:33:49.118339+0000","last_peered":"2024-04-20T21:33:49.118339+0000","last_clean":"2024-04-20T21:33:49.118339+0000","last_became_active":"2024-04-20T21:31:26.701068+0000","last_became_peered":"2024-04-20T21:31:26.701068+0000","last_unstale":"2024-04-20T21:33:49.118339+0000","last_undegraded":"2024-04-20T21:33:49.118339+0000","last_fullsized":"2024-04-20T21:33:49.118339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:22:51.745294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.830874+0000","last_change":"2024-04-20T21:31:27.542189+0000","last_active":"2024-04-20T21:33:48.830874+0000","last_peered":"2024-04-20T21:33:48.830874+0000","last_clean":"2024-04-20T21:33:48.830874+0000","last_became_active":"2024-04-20T21:31:27.541800+0000","last_became_peered":"2024-04-20T21:31:27.541800+0000","last_unstale":"2024-04-20T21:33:48.830874+0000","last_undegraded":"2024-04-20T21:33:48.830874+0000","last_fullsized":"2024-04-20T21:33:48.830874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:02:31.898443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976103+0000","last_change":"2024-04-20T21:31:26.687652+0000","last_active":"2024-04-20T21:33:48.976103+0000","last_peered":"2024-04-20T21:33:48.976103+0000","last_clean":"2024-04-20T21:33:48.976103+0000","last_became_active":"2024-04-20T21:31:26.687333+0000","last_became_peered":"2024-04-20T21:31:26.687333+0000","last_unstale":"2024-04-20T21:33:48.976103+0000","last_undegraded":"2024-04-20T21:33:48.976103+0000","last_fullsized":"2024-04-20T21:33:48.976103+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:12.819189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120140+0000","last_change":"2024-04-20T21:31:27.522535+0000","last_active":"2024-04-20T21:33:49.120140+0000","last_peered":"2024-04-20T21:33:49.120140+0000","last_clean":"2024-04-20T21:33:49.120140+0000","last_became_active":"2024-04-20T21:31:27.521821+0000","last_became_peered":"2024-04-20T21:31:27.521821+0000","last_unstale":"2024-04-20T21:33:49.120140+0000","last_undegraded":"2024-04-20T21:33:49.120140+0000","last_fullsized":"2024-04-20T21:33:49.120140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:47:53.401129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118630+0000","last_change":"2024-04-20T21:31:26.681758+0000","last_active":"2024-04-20T21:33:49.118630+0000","last_peered":"2024-04-20T21:33:49.118630+0000","last_clean":"2024-04-20T21:33:49.118630+0000","last_became_active":"2024-04-20T21:31:26.681395+0000","last_became_peered":"2024-04-20T21:31:26.681395+0000","last_unstale":"2024-04-20T21:33:49.118630+0000","last_undegraded":"2024-04-20T21:33:49.118630+0000","last_fullsized":"2024-04-20T21:33:49.118630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:33:00.924879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119081+0000","last_change":"2024-04-20T21:31:27.512873+0000","last_active":"2024-04-20T21:33:49.119081+0000","last_peered":"2024-04-20T21:33:49.119081+0000","last_clean":"2024-04-20T21:33:49.119081+0000","last_became_active":"2024-04-20T21:31:27.512554+0000","last_became_peered":"2024-04-20T21:31:27.512554+0000","last_unstale":"2024-04-20T21:33:49.119081+0000","last_undegraded":"2024-04-20T21:33:49.119081+0000","last_fullsized":"2024-04-20T21:33:49.119081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:26:06.670726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148549+0000","last_change":"2024-04-20T21:31:26.691210+0000","last_active":"2024-04-20T21:33:32.148549+0000","last_peered":"2024-04-20T21:33:32.148549+0000","last_clean":"2024-04-20T21:33:32.148549+0000","last_became_active":"2024-04-20T21:31:26.690859+0000","last_became_peered":"2024-04-20T21:31:26.690859+0000","last_unstale":"2024-04-20T21:33:32.148549+0000","last_undegraded":"2024-04-20T21:33:32.148549+0000","last_fullsized":"2024-04-20T21:33:32.148549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:36.789429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976179+0000","last_change":"2024-04-20T21:32:31.616413+0000","last_active":"2024-04-20T21:33:48.976179+0000","last_peered":"2024-04-20T21:33:48.976179+0000","last_clean":"2024-04-20T21:33:48.976179+0000","last_became_active":"2024-04-20T21:32:31.616011+0000","last_became_peered":"2024-04-20T21:32:31.616011+0000","last_unstale":"2024-04-20T21:33:48.976179+0000","last_undegraded":"2024-04-20T21:33:48.976179+0000","last_fullsized":"2024-04-20T21:33:48.976179+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:11:05.630337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.149077+0000","last_change":"2024-04-20T21:31:26.682311+0000","last_active":"2024-04-20T21:33:32.149077+0000","last_peered":"2024-04-20T21:33:32.149077+0000","last_clean":"2024-04-20T21:33:32.149077+0000","last_became_active":"2024-04-20T21:31:26.681996+0000","last_became_peered":"2024-04-20T21:31:26.681996+0000","last_unstale":"2024-04-20T21:33:32.149077+0000","last_undegraded":"2024-04-20T21:33:32.149077+0000","last_fullsized":"2024-04-20T21:33:32.149077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:32:37.659946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966120+0000","last_change":"2024-04-20T21:31:27.510922+0000","last_active":"2024-04-20T21:33:48.966120+0000","last_peered":"2024-04-20T21:33:48.966120+0000","last_clean":"2024-04-20T21:33:48.966120+0000","last_became_active":"2024-04-20T21:31:27.510553+0000","last_became_peered":"2024-04-20T21:31:27.510553+0000","last_unstale":"2024-04-20T21:33:48.966120+0000","last_undegraded":"2024-04-20T21:33:48.966120+0000","last_fullsized":"2024-04-20T21:33:48.966120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:23.754768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963235+0000","last_change":"2024-04-20T21:31:26.697010+0000","last_active":"2024-04-20T21:33:48.963235+0000","last_peered":"2024-04-20T21:33:48.963235+0000","last_clean":"2024-04-20T21:33:48.963235+0000","last_became_active":"2024-04-20T21:31:26.696112+0000","last_became_peered":"2024-04-20T21:31:26.696112+0000","last_unstale":"2024-04-20T21:33:48.963235+0000","last_undegraded":"2024-04-20T21:33:48.963235+0000","last_fullsized":"2024-04-20T21:33:48.963235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:57:02.715884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828760+0000","last_change":"2024-04-20T21:31:27.545025+0000","last_active":"2024-04-20T21:33:48.828760+0000","last_peered":"2024-04-20T21:33:48.828760+0000","last_clean":"2024-04-20T21:33:48.828760+0000","last_became_active":"2024-04-20T21:31:27.544630+0000","last_became_peered":"2024-04-20T21:31:27.544630+0000","last_unstale":"2024-04-20T21:33:48.828760+0000","last_undegraded":"2024-04-20T21:33:48.828760+0000","last_fullsized":"2024-04-20T21:33:48.828760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:57.292481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119540+0000","last_change":"2024-04-20T21:31:26.682909+0000","last_active":"2024-04-20T21:33:49.119540+0000","last_peered":"2024-04-20T21:33:49.119540+0000","last_clean":"2024-04-20T21:33:49.119540+0000","last_became_active":"2024-04-20T21:31:26.682597+0000","last_became_peered":"2024-04-20T21:31:26.682597+0000","last_unstale":"2024-04-20T21:33:49.119540+0000","last_undegraded":"2024-04-20T21:33:49.119540+0000","last_fullsized":"2024-04-20T21:33:49.119540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:18:31.889513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828594+0000","last_change":"2024-04-20T21:31:27.530217+0000","last_active":"2024-04-20T21:33:48.828594+0000","last_peered":"2024-04-20T21:33:48.828594+0000","last_clean":"2024-04-20T21:33:48.828594+0000","last_became_active":"2024-04-20T21:31:27.529780+0000","last_became_peered":"2024-04-20T21:31:27.529780+0000","last_unstale":"2024-04-20T21:33:48.828594+0000","last_undegraded":"2024-04-20T21:33:48.828594+0000","last_fullsized":"2024-04-20T21:33:48.828594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:21.613460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976309+0000","last_change":"2024-04-20T21:32:31.610727+0000","last_active":"2024-04-20T21:33:48.976309+0000","last_peered":"2024-04-20T21:33:48.976309+0000","last_clean":"2024-04-20T21:33:48.976309+0000","last_became_active":"2024-04-20T21:32:31.610321+0000","last_became_peered":"2024-04-20T21:32:31.610321+0000","last_unstale":"2024-04-20T21:33:48.976309+0000","last_undegraded":"2024-04-20T21:33:48.976309+0000","last_fullsized":"2024-04-20T21:33:48.976309+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:01:35.697137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.826910+0000","last_change":"2024-04-20T21:31:26.692845+0000","last_active":"2024-04-20T21:33:48.826910+0000","last_peered":"2024-04-20T21:33:48.826910+0000","last_clean":"2024-04-20T21:33:48.826910+0000","last_became_active":"2024-04-20T21:31:26.691915+0000","last_became_peered":"2024-04-20T21:31:26.691915+0000","last_unstale":"2024-04-20T21:33:48.826910+0000","last_undegraded":"2024-04-20T21:33:48.826910+0000","last_fullsized":"2024-04-20T21:33:48.826910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:24:00.436955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.832281+0000","last_change":"2024-04-20T21:31:27.541623+0000","last_active":"2024-04-20T21:33:48.832281+0000","last_peered":"2024-04-20T21:33:48.832281+0000","last_clean":"2024-04-20T21:33:48.832281+0000","last_became_active":"2024-04-20T21:31:27.540953+0000","last_became_peered":"2024-04-20T21:31:27.540953+0000","last_unstale":"2024-04-20T21:33:48.832281+0000","last_undegraded":"2024-04-20T21:33:48.832281+0000","last_fullsized":"2024-04-20T21:33:48.832281+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:03:15.091886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118799+0000","last_change":"2024-04-20T21:31:26.687373+0000","last_active":"2024-04-20T21:33:49.118799+0000","last_peered":"2024-04-20T21:33:49.118799+0000","last_clean":"2024-04-20T21:33:49.118799+0000","last_became_active":"2024-04-20T21:31:26.687009+0000","last_became_peered":"2024-04-20T21:31:26.687009+0000","last_unstale":"2024-04-20T21:33:49.118799+0000","last_undegraded":"2024-04-20T21:33:49.118799+0000","last_fullsized":"2024-04-20T21:33:49.118799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:23:42.708129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119190+0000","last_change":"2024-04-20T21:30:06.995711+0000","last_active":"2024-04-20T21:33:49.119190+0000","last_peered":"2024-04-20T21:33:49.119190+0000","last_clean":"2024-04-20T21:33:49.119190+0000","last_became_active":"2024-04-20T21:30:06.995280+0000","last_became_peered":"2024-04-20T21:30:06.995280+0000","last_unstale":"2024-04-20T21:33:49.119190+0000","last_undegraded":"2024-04-20T21:33:49.119190+0000","last_fullsized":"2024-04-20T21:33:49.119190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:26:16.477494+0000","last_clean_scrub_stamp":"2024-04-20T21:26:16.477494+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:49:26.014145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.826753+0000","last_change":"2024-04-20T21:31:27.514064+0000","last_active":"2024-04-20T21:33:48.826753+0000","last_peered":"2024-04-20T21:33:48.826753+0000","last_clean":"2024-04-20T21:33:48.826753+0000","last_became_active":"2024-04-20T21:31:27.513416+0000","last_became_peered":"2024-04-20T21:31:27.513416+0000","last_unstale":"2024-04-20T21:33:48.826753+0000","last_undegraded":"2024-04-20T21:33:48.826753+0000","last_fullsized":"2024-04-20T21:33:48.826753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:12.658265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119384+0000","last_change":"2024-04-20T21:31:26.680948+0000","last_active":"2024-04-20T21:33:49.119384+0000","last_peered":"2024-04-20T21:33:49.119384+0000","last_clean":"2024-04-20T21:33:49.119384+0000","last_became_active":"2024-04-20T21:31:26.680624+0000","last_became_peered":"2024-04-20T21:31:26.680624+0000","last_unstale":"2024-04-20T21:33:49.119384+0000","last_undegraded":"2024-04-20T21:33:49.119384+0000","last_fullsized":"2024-04-20T21:33:49.119384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:02.750166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148990+0000","last_change":"2024-04-20T21:31:26.697632+0000","last_active":"2024-04-20T21:33:32.148990+0000","last_peered":"2024-04-20T21:33:32.148990+0000","last_clean":"2024-04-20T21:33:32.148990+0000","last_became_active":"2024-04-20T21:31:26.697393+0000","last_became_peered":"2024-04-20T21:31:26.697393+0000","last_unstale":"2024-04-20T21:33:32.148990+0000","last_undegraded":"2024-04-20T21:33:32.148990+0000","last_fullsized":"2024-04-20T21:33:32.148990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:08:39.856082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.832878+0000","last_change":"2024-04-20T21:31:27.526613+0000","last_active":"2024-04-20T21:33:48.832878+0000","last_peered":"2024-04-20T21:33:48.832878+0000","last_clean":"2024-04-20T21:33:48.832878+0000","last_became_active":"2024-04-20T21:31:27.526234+0000","last_became_peered":"2024-04-20T21:31:27.526234+0000","last_unstale":"2024-04-20T21:33:48.832878+0000","last_undegraded":"2024-04-20T21:33:48.832878+0000","last_fullsized":"2024-04-20T21:33:48.832878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:06:44.114924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'45","reported_seq":104,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.131406+0000","last_change":"2024-04-20T21:32:31.625214+0000","last_active":"2024-04-20T21:33:49.131406+0000","last_peered":"2024-04-20T21:33:49.131406+0000","last_clean":"2024-04-20T21:33:49.131406+0000","last_became_active":"2024-04-20T21:32:31.619956+0000","last_became_peered":"2024-04-20T21:32:31.619956+0000","last_unstale":"2024-04-20T21:33:49.131406+0000","last_undegraded":"2024-04-20T21:33:49.131406+0000","last_fullsized":"2024-04-20T21:33:49.131406+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:19:41.620846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":81,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.831220+0000","last_change":"2024-04-20T21:31:27.538361+0000","last_active":"2024-04-20T21:33:48.831220+0000","last_peered":"2024-04-20T21:33:48.831220+0000","last_clean":"2024-04-20T21:33:48.831220+0000","last_became_active":"2024-04-20T21:31:27.537494+0000","last_became_peered":"2024-04-20T21:31:27.537494+0000","last_unstale":"2024-04-20T21:33:48.831220+0000","last_undegraded":"2024-04-20T21:33:48.831220+0000","last_fullsized":"2024-04-20T21:33:48.831220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:46.212759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119090+0000","last_change":"2024-04-20T21:31:26.686960+0000","last_active":"2024-04-20T21:33:49.119090+0000","last_peered":"2024-04-20T21:33:49.119090+0000","last_clean":"2024-04-20T21:33:49.119090+0000","last_became_active":"2024-04-20T21:31:26.686565+0000","last_became_peered":"2024-04-20T21:31:26.686565+0000","last_unstale":"2024-04-20T21:33:49.119090+0000","last_undegraded":"2024-04-20T21:33:49.119090+0000","last_fullsized":"2024-04-20T21:33:49.119090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:07.608834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120313+0000","last_change":"2024-04-20T21:31:27.546978+0000","last_active":"2024-04-20T21:33:49.120313+0000","last_peered":"2024-04-20T21:33:49.120313+0000","last_clean":"2024-04-20T21:33:49.120313+0000","last_became_active":"2024-04-20T21:31:27.545941+0000","last_became_peered":"2024-04-20T21:31:27.545941+0000","last_unstale":"2024-04-20T21:33:49.120313+0000","last_undegraded":"2024-04-20T21:33:49.120313+0000","last_fullsized":"2024-04-20T21:33:49.120313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:08.363519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118443+0000","last_change":"2024-04-20T21:31:26.690797+0000","last_active":"2024-04-20T21:33:49.118443+0000","last_peered":"2024-04-20T21:33:49.118443+0000","last_clean":"2024-04-20T21:33:49.118443+0000","last_became_active":"2024-04-20T21:31:26.690486+0000","last_became_peered":"2024-04-20T21:31:26.690486+0000","last_unstale":"2024-04-20T21:33:49.118443+0000","last_undegraded":"2024-04-20T21:33:49.118443+0000","last_fullsized":"2024-04-20T21:33:49.118443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:17:33.943212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148549+0000","last_change":"2024-04-20T21:31:27.507175+0000","last_active":"2024-04-20T21:33:32.148549+0000","last_peered":"2024-04-20T21:33:32.148549+0000","last_clean":"2024-04-20T21:33:32.148549+0000","last_became_active":"2024-04-20T21:31:27.506808+0000","last_became_peered":"2024-04-20T21:31:27.506808+0000","last_unstale":"2024-04-20T21:33:32.148549+0000","last_undegraded":"2024-04-20T21:33:32.148549+0000","last_fullsized":"2024-04-20T21:33:32.148549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:38:15.212880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828658+0000","last_change":"2024-04-20T21:31:26.698302+0000","last_active":"2024-04-20T21:33:48.828658+0000","last_peered":"2024-04-20T21:33:48.828658+0000","last_clean":"2024-04-20T21:33:48.828658+0000","last_became_active":"2024-04-20T21:31:26.696209+0000","last_became_peered":"2024-04-20T21:31:26.696209+0000","last_unstale":"2024-04-20T21:33:48.828658+0000","last_undegraded":"2024-04-20T21:33:48.828658+0000","last_fullsized":"2024-04-20T21:33:48.828658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:43.495880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.965838+0000","last_change":"2024-04-20T21:31:27.521594+0000","last_active":"2024-04-20T21:33:48.965838+0000","last_peered":"2024-04-20T21:33:48.965838+0000","last_clean":"2024-04-20T21:33:48.965838+0000","last_became_active":"2024-04-20T21:31:27.521185+0000","last_became_peered":"2024-04-20T21:31:27.521185+0000","last_unstale":"2024-04-20T21:33:48.965838+0000","last_undegraded":"2024-04-20T21:33:48.965838+0000","last_fullsized":"2024-04-20T21:33:48.965838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:22:08.647624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118193+0000","last_change":"2024-04-20T21:31:26.691536+0000","last_active":"2024-04-20T21:33:49.118193+0000","last_peered":"2024-04-20T21:33:49.118193+0000","last_clean":"2024-04-20T21:33:49.118193+0000","last_became_active":"2024-04-20T21:31:26.691202+0000","last_became_peered":"2024-04-20T21:31:26.691202+0000","last_unstale":"2024-04-20T21:33:49.118193+0000","last_undegraded":"2024-04-20T21:33:49.118193+0000","last_fullsized":"2024-04-20T21:33:49.118193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:58:59.331644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.963190+0000","last_change":"2024-04-20T21:32:31.998258+0000","last_active":"2024-04-20T21:33:48.963190+0000","last_peered":"2024-04-20T21:33:48.963190+0000","last_clean":"2024-04-20T21:33:48.963190+0000","last_became_active":"2024-04-20T21:32:31.997119+0000","last_became_peered":"2024-04-20T21:32:31.997119+0000","last_unstale":"2024-04-20T21:33:48.963190+0000","last_undegraded":"2024-04-20T21:33:48.963190+0000","last_fullsized":"2024-04-20T21:33:48.963190+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:41:08.565434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-20T21:33:32.148696+0000","last_change":"2024-04-20T21:32:31.616245+0000","last_active":"2024-04-20T21:33:32.148696+0000","last_peered":"2024-04-20T21:33:32.148696+0000","last_clean":"2024-04-20T21:33:32.148696+0000","last_became_active":"2024-04-20T21:32:31.611535+0000","last_became_peered":"2024-04-20T21:32:31.611535+0000","last_unstale":"2024-04-20T21:33:32.148696+0000","last_undegraded":"2024-04-20T21:33:32.148696+0000","last_fullsized":"2024-04-20T21:33:32.148696+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:28:17.609890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119779+0000","last_change":"2024-04-20T21:31:27.518346+0000","last_active":"2024-04-20T21:33:49.119779+0000","last_peered":"2024-04-20T21:33:49.119779+0000","last_clean":"2024-04-20T21:33:49.119779+0000","last_became_active":"2024-04-20T21:31:27.517782+0000","last_became_peered":"2024-04-20T21:31:27.517782+0000","last_unstale":"2024-04-20T21:33:49.119779+0000","last_undegraded":"2024-04-20T21:33:49.119779+0000","last_fullsized":"2024-04-20T21:33:49.119779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:21.674663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119132+0000","last_change":"2024-04-20T21:32:31.615626+0000","last_active":"2024-04-20T21:33:49.119132+0000","last_peered":"2024-04-20T21:33:49.119132+0000","last_clean":"2024-04-20T21:33:49.119132+0000","last_became_active":"2024-04-20T21:32:31.615181+0000","last_became_peered":"2024-04-20T21:32:31.615181+0000","last_unstale":"2024-04-20T21:33:49.119132+0000","last_undegraded":"2024-04-20T21:33:49.119132+0000","last_fullsized":"2024-04-20T21:33:49.119132+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:58:18.344583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119853+0000","last_change":"2024-04-20T21:31:27.510920+0000","last_active":"2024-04-20T21:33:49.119853+0000","last_peered":"2024-04-20T21:33:49.119853+0000","last_clean":"2024-04-20T21:33:49.119853+0000","last_became_active":"2024-04-20T21:31:27.510564+0000","last_became_peered":"2024-04-20T21:31:27.510564+0000","last_unstale":"2024-04-20T21:33:49.119853+0000","last_undegraded":"2024-04-20T21:33:49.119853+0000","last_fullsized":"2024-04-20T21:33:49.119853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:05.196178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.128160+0000","last_change":"2024-04-20T21:31:26.693159+0000","last_active":"2024-04-20T21:33:49.128160+0000","last_peered":"2024-04-20T21:33:49.128160+0000","last_clean":"2024-04-20T21:33:49.128160+0000","last_became_active":"2024-04-20T21:31:26.692902+0000","last_became_peered":"2024-04-20T21:31:26.692902+0000","last_unstale":"2024-04-20T21:33:49.128160+0000","last_undegraded":"2024-04-20T21:33:49.128160+0000","last_fullsized":"2024-04-20T21:33:49.128160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:54.704343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119400+0000","last_change":"2024-04-20T21:31:27.541557+0000","last_active":"2024-04-20T21:33:49.119400+0000","last_peered":"2024-04-20T21:33:49.119400+0000","last_clean":"2024-04-20T21:33:49.119400+0000","last_became_active":"2024-04-20T21:31:27.541161+0000","last_became_peered":"2024-04-20T21:31:27.541161+0000","last_unstale":"2024-04-20T21:33:49.119400+0000","last_undegraded":"2024-04-20T21:33:49.119400+0000","last_fullsized":"2024-04-20T21:33:49.119400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:55:25.558918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.833193+0000","last_change":"2024-04-20T21:31:26.688019+0000","last_active":"2024-04-20T21:33:48.833193+0000","last_peered":"2024-04-20T21:33:48.833193+0000","last_clean":"2024-04-20T21:33:48.833193+0000","last_became_active":"2024-04-20T21:31:26.686370+0000","last_became_peered":"2024-04-20T21:31:26.686370+0000","last_unstale":"2024-04-20T21:33:48.833193+0000","last_undegraded":"2024-04-20T21:33:48.833193+0000","last_fullsized":"2024-04-20T21:33:48.833193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:25.651053+0000","last_clean_scrub_stamp":"2024-04-20T21:31:25.651053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:42:44.059365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119798+0000","last_change":"2024-04-20T21:31:27.511592+0000","last_active":"2024-04-20T21:33:49.119798+0000","last_peered":"2024-04-20T21:33:49.119798+0000","last_clean":"2024-04-20T21:33:49.119798+0000","last_became_active":"2024-04-20T21:31:27.511311+0000","last_became_peered":"2024-04-20T21:31:27.511311+0000","last_unstale":"2024-04-20T21:33:49.119798+0000","last_undegraded":"2024-04-20T21:33:49.119798+0000","last_fullsized":"2024-04-20T21:33:49.119798+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:35:27.665578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.965845+0000","last_change":"2024-04-20T21:31:27.511827+0000","last_active":"2024-04-20T21:33:48.965845+0000","last_peered":"2024-04-20T21:33:48.965845+0000","last_clean":"2024-04-20T21:33:48.965845+0000","last_became_active":"2024-04-20T21:31:27.511344+0000","last_became_peered":"2024-04-20T21:31:27.511344+0000","last_unstale":"2024-04-20T21:33:48.965845+0000","last_undegraded":"2024-04-20T21:33:48.965845+0000","last_fullsized":"2024-04-20T21:33:48.965845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:11:37.973050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118126+0000","last_change":"2024-04-20T21:31:27.510747+0000","last_active":"2024-04-20T21:33:49.118126+0000","last_peered":"2024-04-20T21:33:49.118126+0000","last_clean":"2024-04-20T21:33:49.118126+0000","last_became_active":"2024-04-20T21:31:27.510253+0000","last_became_peered":"2024-04-20T21:31:27.510253+0000","last_unstale":"2024-04-20T21:33:49.118126+0000","last_undegraded":"2024-04-20T21:33:49.118126+0000","last_fullsized":"2024-04-20T21:33:49.118126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:37:13.855840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118089+0000","last_change":"2024-04-20T21:31:27.541185+0000","last_active":"2024-04-20T21:33:49.118089+0000","last_peered":"2024-04-20T21:33:49.118089+0000","last_clean":"2024-04-20T21:33:49.118089+0000","last_became_active":"2024-04-20T21:31:27.540881+0000","last_became_peered":"2024-04-20T21:31:27.540881+0000","last_unstale":"2024-04-20T21:33:49.118089+0000","last_undegraded":"2024-04-20T21:33:49.118089+0000","last_fullsized":"2024-04-20T21:33:49.118089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:46:41.897081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.119309+0000","last_change":"2024-04-20T21:31:27.527204+0000","last_active":"2024-04-20T21:33:49.119309+0000","last_peered":"2024-04-20T21:33:49.119309+0000","last_clean":"2024-04-20T21:33:49.119309+0000","last_became_active":"2024-04-20T21:31:27.526805+0000","last_became_peered":"2024-04-20T21:31:27.526805+0000","last_unstale":"2024-04-20T21:33:49.119309+0000","last_undegraded":"2024-04-20T21:33:49.119309+0000","last_fullsized":"2024-04-20T21:33:49.119309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:43.032609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.832649+0000","last_change":"2024-04-20T21:31:27.525521+0000","last_active":"2024-04-20T21:33:48.832649+0000","last_peered":"2024-04-20T21:33:48.832649+0000","last_clean":"2024-04-20T21:33:48.832649+0000","last_became_active":"2024-04-20T21:31:27.521892+0000","last_became_peered":"2024-04-20T21:31:27.521892+0000","last_unstale":"2024-04-20T21:33:48.832649+0000","last_undegraded":"2024-04-20T21:33:48.832649+0000","last_fullsized":"2024-04-20T21:33:48.832649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:32:47.683115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966118+0000","last_change":"2024-04-20T21:33:32.432716+0000","last_active":"2024-04-20T21:33:48.966118+0000","last_peered":"2024-04-20T21:33:48.966118+0000","last_clean":"2024-04-20T21:33:48.966118+0000","last_became_active":"2024-04-20T21:33:32.432201+0000","last_became_peered":"2024-04-20T21:33:32.432201+0000","last_unstale":"2024-04-20T21:33:48.966118+0000","last_undegraded":"2024-04-20T21:33:48.966118+0000","last_fullsized":"2024-04-20T21:33:48.966118+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:14:13.089095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.118560+0000","last_change":"2024-04-20T21:31:27.512542+0000","last_active":"2024-04-20T21:33:49.118560+0000","last_peered":"2024-04-20T21:33:49.118560+0000","last_clean":"2024-04-20T21:33:49.118560+0000","last_became_active":"2024-04-20T21:31:27.511907+0000","last_became_peered":"2024-04-20T21:31:27.511907+0000","last_unstale":"2024-04-20T21:33:49.118560+0000","last_undegraded":"2024-04-20T21:33:49.118560+0000","last_fullsized":"2024-04-20T21:33:49.118560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:10:09.265471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.831096+0000","last_change":"2024-04-20T21:31:27.538697+0000","last_active":"2024-04-20T21:33:48.831096+0000","last_peered":"2024-04-20T21:33:48.831096+0000","last_clean":"2024-04-20T21:33:48.831096+0000","last_became_active":"2024-04-20T21:31:27.537972+0000","last_became_peered":"2024-04-20T21:31:27.537972+0000","last_unstale":"2024-04-20T21:33:48.831096+0000","last_undegraded":"2024-04-20T21:33:48.831096+0000","last_fullsized":"2024-04-20T21:33:48.831096+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:51.974485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.976126+0000","last_change":"2024-04-20T21:31:27.538092+0000","last_active":"2024-04-20T21:33:48.976126+0000","last_peered":"2024-04-20T21:33:48.976126+0000","last_clean":"2024-04-20T21:33:48.976126+0000","last_became_active":"2024-04-20T21:31:27.537821+0000","last_became_peered":"2024-04-20T21:31:27.537821+0000","last_unstale":"2024-04-20T21:33:48.976126+0000","last_undegraded":"2024-04-20T21:33:48.976126+0000","last_fullsized":"2024-04-20T21:33:48.976126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:52.773583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.828402+0000","last_change":"2024-04-20T21:31:27.535498+0000","last_active":"2024-04-20T21:33:48.828402+0000","last_peered":"2024-04-20T21:33:48.828402+0000","last_clean":"2024-04-20T21:33:48.828402+0000","last_became_active":"2024-04-20T21:31:27.535162+0000","last_became_peered":"2024-04-20T21:31:27.535162+0000","last_unstale":"2024-04-20T21:33:48.828402+0000","last_undegraded":"2024-04-20T21:33:48.828402+0000","last_fullsized":"2024-04-20T21:33:48.828402+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:46.638825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:49.120232+0000","last_change":"2024-04-20T21:31:27.537776+0000","last_active":"2024-04-20T21:33:49.120232+0000","last_peered":"2024-04-20T21:33:49.120232+0000","last_clean":"2024-04-20T21:33:49.120232+0000","last_became_active":"2024-04-20T21:31:27.537428+0000","last_became_peered":"2024-04-20T21:31:27.537428+0000","last_unstale":"2024-04-20T21:33:49.120232+0000","last_undegraded":"2024-04-20T21:33:49.120232+0000","last_fullsized":"2024-04-20T21:33:49.120232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:15:03.344277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.834707+0000","last_change":"2024-04-20T21:31:27.535943+0000","last_active":"2024-04-20T21:33:48.834707+0000","last_peered":"2024-04-20T21:33:48.834707+0000","last_clean":"2024-04-20T21:33:48.834707+0000","last_became_active":"2024-04-20T21:31:27.533694+0000","last_became_peered":"2024-04-20T21:31:27.533694+0000","last_unstale":"2024-04-20T21:33:48.834707+0000","last_undegraded":"2024-04-20T21:33:48.834707+0000","last_fullsized":"2024-04-20T21:33:48.834707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:42.603232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T21:33:48.966013+0000","last_change":"2024-04-20T21:31:27.511345+0000","last_active":"2024-04-20T21:33:48.966013+0000","last_peered":"2024-04-20T21:33:48.966013+0000","last_clean":"2024-04-20T21:33:48.966013+0000","last_became_active":"2024-04-20T21:31:27.510984+0000","last_became_peered":"2024-04-20T21:31:27.510984+0000","last_unstale":"2024-04-20T21:33:48.966013+0000","last_undegraded":"2024-04-20T21:33:48.966013+0000","last_fullsized":"2024-04-20T21:33:48.966013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:31:26.657205+0000","last_clean_scrub_stamp":"2024-04-20T21:31:26.657205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:19.965636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":593374,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":387,"num_read_kb":296,"num_write":200,"num_write_kb":1404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1880064,"data_stored":1780122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285609,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1421312,"data_stored":888395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547246,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28816,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716336,"statfs":{"total":95995035648,"available":95965528064,"internally_reserved":0,"allocated":1777664,"data_stored":1213519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710772,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":486747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972411,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":486769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266750,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1613824,"data_stored":1077232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430279,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":495296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":562939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":495269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":562973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149781,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1572864,"data_stored":1020333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888145,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":495201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691915,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37248,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707904,"statfs":{"total":95995035648,"available":95956893696,"internally_reserved":0,"allocated":1773568,"data_stored":1217446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":524920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":131782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":199554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":393216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:33:53.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:53.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: Unable to set osd_memory_target on smithi122 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: Unable to set osd_memory_target on smithi033 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:53 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.499 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.499 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.499 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.499 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.499 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.499 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: Unable to set osd_memory_target on smithi122 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: Unable to set osd_memory_target on smithi033 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.500 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:53 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-20T21:33:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: Unable to set osd_memory_target on smithi122 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-20T21:33:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:53.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-20T21:33:53.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: Unable to set osd_memory_target on smithi033 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-20T21:33:53.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:53.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:53 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:53.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.579+0000 7f0cf3200640 1 -- 172.21.15.22:0/3110406620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf4100ce0 msgr2=0x7f0cf4101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:53.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.579+0000 7f0cf3200640 1 --2- 172.21.15.22:0/3110406620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf4100ce0 0x7f0cf4101100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009900 tx=0x7f0ce402bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:53.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.579+0000 7f0cf3200640 1 -- 172.21.15.22:0/3110406620 shutdown_connections 2024-04-20T21:33:53.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.579+0000 7f0cf3200640 1 --2- 172.21.15.22:0/3110406620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cf4102890 0x7f0cf4109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.579+0000 7f0cf3200640 1 --2- 172.21.15.22:0/3110406620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cf4101ed0 0x7f0cf4102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.579+0000 7f0cf3200640 1 --2- 172.21.15.22:0/3110406620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf4100ce0 0x7f0cf4101100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.579+0000 7f0cf3200640 1 -- 172.21.15.22:0/3110406620 >> 172.21.15.22:0/3110406620 conn(0x7f0cf40fc230 msgr2=0x7f0cf40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:53.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.579+0000 7f0cf3200640 1 -- 172.21.15.22:0/3110406620 shutdown_connections 2024-04-20T21:33:53.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.580+0000 7f0cf3200640 1 -- 172.21.15.22:0/3110406620 wait complete. 2024-04-20T21:33:53.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.581+0000 7f0cf3200640 1 Processor -- start 2024-04-20T21:33:53.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.581+0000 7f0cf3200640 1 -- start start 2024-04-20T21:33:53.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.581+0000 7f0cf3200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf4100ce0 0x7f0cf4198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:53.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.581+0000 7f0cf3200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cf4101ed0 0x7f0cf4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.581+0000 7f0cf3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cf4102890 0x7f0cf419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.581+0000 7f0cf3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410c980 con 0x7f0cf4102890 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.581+0000 7f0cf3200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410c800 con 0x7f0cf4100ce0 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.581+0000 7f0cf3200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410cb00 con 0x7f0cf4101ed0 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.581+0000 7f0cf1e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf4100ce0 0x7f0cf4198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.581+0000 7f0cf1e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf4100ce0 0x7f0cf4198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60120/0 (socket says 172.21.15.22:60120) 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.581+0000 7f0cf1e00640 1 -- 172.21.15.22:0/2615252195 learned_addr learned my addr 172.21.15.22:0/2615252195 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.581+0000 7f0cf2800640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cf4102890 0x7f0cf419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.582+0000 7f0cf1400640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cf4101ed0 0x7f0cf4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.582+0000 7f0cf1e00640 1 -- 172.21.15.22:0/2615252195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cf4101ed0 msgr2=0x7f0cf4198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.582+0000 7f0cf1e00640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cf4101ed0 0x7f0cf4198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.582+0000 7f0cf1e00640 1 -- 172.21.15.22:0/2615252195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cf4102890 msgr2=0x7f0cf419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:53.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.582+0000 7f0cf1e00640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cf4102890 0x7f0cf419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.582+0000 7f0cf1e00640 1 -- 172.21.15.22:0/2615252195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf419fd40 con 0x7f0cf4100ce0 2024-04-20T21:33:53.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.582+0000 7f0cf2800640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cf4102890 0x7f0cf419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:53.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.582+0000 7f0cf1e00640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf4100ce0 0x7f0cf4198120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0ce402efd0 tx=0x7f0ce4005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:53.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.582+0000 7f0ceaa00640 1 -- 172.21.15.22:0/2615252195 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4041070 con 0x7f0cf4100ce0 2024-04-20T21:33:53.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.582+0000 7f0ceaa00640 1 -- 172.21.15.22:0/2615252195 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce40044d0 con 0x7f0cf4100ce0 2024-04-20T21:33:53.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.583+0000 7f0cf3200640 1 -- 172.21.15.22:0/2615252195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf419ffd0 con 0x7f0cf4100ce0 2024-04-20T21:33:53.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.583+0000 7f0ceaa00640 1 -- 172.21.15.22:0/2615252195 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4034440 con 0x7f0cf4100ce0 2024-04-20T21:33:53.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.583+0000 7f0cf3200640 1 -- 172.21.15.22:0/2615252195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf41a03d0 con 0x7f0cf4100ce0 2024-04-20T21:33:53.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.584+0000 7f0cf3200640 1 -- 172.21.15.22:0/2615252195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf41a0600 con 0x7f0cf4100ce0 2024-04-20T21:33:53.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.586+0000 7f0ceaa00640 1 -- 172.21.15.22:0/2615252195 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f0ce4044070 con 0x7f0cf4100ce0 2024-04-20T21:33:53.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.587+0000 7f0ceaa00640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0cd80a3db0 0x7f0cd80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:53.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.587+0000 7f0ceaa00640 1 -- 172.21.15.22:0/2615252195 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f0ce40f24e0 con 0x7f0cf4100ce0 2024-04-20T21:33:53.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.587+0000 7f0cf1400640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0cd80a3db0 0x7f0cd80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:53.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.588+0000 7f0cf1400640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0cd80a3db0 0x7f0cd80a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4199630 tx=0x7f0ce0005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:53.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.591+0000 7f0ceaa00640 1 -- 172.21.15.22:0/2615252195 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0ce40beaa0 con 0x7f0cf4100ce0 2024-04-20T21:33:53.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.811+0000 7f0cf3200640 1 -- 172.21.15.22:0/2615252195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cf41a0960 con 0x7f0cf4100ce0 2024-04-20T21:33:53.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.813+0000 7f0ceaa00640 1 -- 172.21.15.22:0/2615252195 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+23146 (secure 0 0 0) 0x7f0ce40be3a0 con 0x7f0cf4100ce0 2024-04-20T21:33:53.812 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:53.812 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:33:48.816036+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/1815958949":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/3359736029":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/3485677447":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/1911146506":"2024-04-21T21:33:48.815984+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1408863783":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/1911146506":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.816+0000 7f0cf3200640 1 -- 172.21.15.22:0/2615252195 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0cd80a3db0 msgr2=0x7f0cd80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.816+0000 7f0cf3200640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0cd80a3db0 0x7f0cd80a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4199630 tx=0x7f0ce0005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.816+0000 7f0cf3200640 1 -- 172.21.15.22:0/2615252195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf4100ce0 msgr2=0x7f0cf4198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.816+0000 7f0cf3200640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf4100ce0 0x7f0cf4198120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0ce402efd0 tx=0x7f0ce4005d50 comp rx=0 tx=0).stop 2024-04-20T21:33:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.816+0000 7f0cf3200640 1 -- 172.21.15.22:0/2615252195 shutdown_connections 2024-04-20T21:33:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.816+0000 7f0cf3200640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0cd80a3db0 0x7f0cd80a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.816+0000 7f0cf3200640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cf4102890 0x7f0cf419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.816+0000 7f0cf3200640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cf4101ed0 0x7f0cf4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.816+0000 7f0cf3200640 1 --2- 172.21.15.22:0/2615252195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf4100ce0 0x7f0cf4198120 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.816+0000 7f0cf3200640 1 -- 172.21.15.22:0/2615252195 >> 172.21.15.22:0/2615252195 conn(0x7f0cf40fc230 msgr2=0x7f0cf40690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.817+0000 7f0cf3200640 1 -- 172.21.15.22:0/2615252195 shutdown_connections 2024-04-20T21:33:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:53.817+0000 7f0cf3200640 1 -- 172.21.15.22:0/2615252195 wait complete. 2024-04-20T21:33:53.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:26:15.688818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:31:25.506074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:31:26.126261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:33:53.835 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:33:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.086+0000 7fa328c00640 1 -- 172.21.15.22:0/2483437713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa324105190 msgr2=0x7fa324105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.086+0000 7fa328c00640 1 --2- 172.21.15.22:0/2483437713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa324105190 0x7fa324105610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa310009900 tx=0x7fa31002bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.086+0000 7fa328c00640 1 -- 172.21.15.22:0/2483437713 shutdown_connections 2024-04-20T21:33:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.086+0000 7fa328c00640 1 --2- 172.21.15.22:0/2483437713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa324105b50 0x7fa32410c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.086+0000 7fa328c00640 1 --2- 172.21.15.22:0/2483437713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa324105190 0x7fa324105610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.086+0000 7fa328c00640 1 --2- 172.21.15.22:0/2483437713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa324103fa0 0x7fa3241043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.086+0000 7fa328c00640 1 -- 172.21.15.22:0/2483437713 >> 172.21.15.22:0/2483437713 conn(0x7fa3240ff530 msgr2=0x7fa324101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.086+0000 7fa328c00640 1 -- 172.21.15.22:0/2483437713 shutdown_connections 2024-04-20T21:33:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.086+0000 7fa328c00640 1 -- 172.21.15.22:0/2483437713 wait complete. 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.087+0000 7fa328c00640 1 Processor -- start 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.088+0000 7fa328c00640 1 -- start start 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.088+0000 7fa328c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa324103fa0 0x7fa32419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.088+0000 7fa328c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa324105190 0x7fa32419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.088+0000 7fa328c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa324105b50 0x7fa3241a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.088+0000 7fa328c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410fc30 con 0x7fa324105190 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.088+0000 7fa328c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410fab0 con 0x7fa324105b50 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.088+0000 7fa328c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410fdb0 con 0x7fa324103fa0 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.088+0000 7fa322a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa324105190 0x7fa32419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.088+0000 7fa323e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa324105b50 0x7fa3241a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.088+0000 7fa323400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa324103fa0 0x7fa32419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.088+0000 7fa322a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa324105190 0x7fa32419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59242/0 (socket says 172.21.15.22:59242) 2024-04-20T21:33:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.088+0000 7fa322a00640 1 -- 172.21.15.22:0/2037063530 learned_addr learned my addr 172.21.15.22:0/2037063530 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.089+0000 7fa323e00640 1 -- 172.21.15.22:0/2037063530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa324103fa0 msgr2=0x7fa32419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.089+0000 7fa323e00640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa324103fa0 0x7fa32419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.089+0000 7fa323e00640 1 -- 172.21.15.22:0/2037063530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa324105190 msgr2=0x7fa32419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.089+0000 7fa323e00640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa324105190 0x7fa32419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.089+0000 7fa323e00640 1 -- 172.21.15.22:0/2037063530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3241a1e80 con 0x7fa324105b50 2024-04-20T21:33:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.089+0000 7fa322a00640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa324105190 0x7fa32419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.089+0000 7fa323e00640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa324105b50 0x7fa3241a1810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00e940 tx=0x7fa31c00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.089+0000 7fa31be00640 1 -- 172.21.15.22:0/2037063530 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c004d20 con 0x7fa324105b50 2024-04-20T21:33:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.089+0000 7fa31be00640 1 -- 172.21.15.22:0/2037063530 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa31c005020 con 0x7fa324105b50 2024-04-20T21:33:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.089+0000 7fa328c00640 1 -- 172.21.15.22:0/2037063530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3241a2170 con 0x7fa324105b50 2024-04-20T21:33:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.090+0000 7fa31be00640 1 -- 172.21.15.22:0/2037063530 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c0116b0 con 0x7fa324105b50 2024-04-20T21:33:54.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.090+0000 7fa328c00640 1 -- 172.21.15.22:0/2037063530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3241a2690 con 0x7fa324105b50 2024-04-20T21:33:54.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.091+0000 7fa328c00640 1 -- 172.21.15.22:0/2037063530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa32410b2c0 con 0x7fa324105b50 2024-04-20T21:33:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.093+0000 7fa31be00640 1 -- 172.21.15.22:0/2037063530 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7fa31c0054d0 con 0x7fa324105b50 2024-04-20T21:33:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.095+0000 7fa31be00640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa3080a3e30 0x7fa3080a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.095+0000 7fa31be00640 1 -- 172.21.15.22:0/2037063530 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fa31c0d0a10 con 0x7fa324105b50 2024-04-20T21:33:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.095+0000 7fa323400640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa3080a3e30 0x7fa3080a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.096+0000 7fa323400640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa3080a3e30 0x7fa3080a62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa3140095f0 tx=0x7fa314005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.099+0000 7fa31be00640 1 -- 172.21.15.22:0/2037063530 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa31c016030 con 0x7fa324105b50 2024-04-20T21:33:54.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.338+0000 7fa328c00640 1 -- 172.21.15.22:0/2037063530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa324062b60 con 0x7fa324105b50 2024-04-20T21:33:54.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.339+0000 7fa31be00640 1 -- 172.21.15.22:0/2037063530 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fa31c09c870 con 0x7fa324105b50 2024-04-20T21:33:54.337 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-20T21:33:54.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.341+0000 7fa328c00640 1 -- 172.21.15.22:0/2037063530 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa3080a3e30 msgr2=0x7fa3080a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:54.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.341+0000 7fa328c00640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa3080a3e30 0x7fa3080a62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa3140095f0 tx=0x7fa314005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:54.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.341+0000 7fa328c00640 1 -- 172.21.15.22:0/2037063530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa324105b50 msgr2=0x7fa3241a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:54.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.341+0000 7fa328c00640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa324105b50 0x7fa3241a1810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00e940 tx=0x7fa31c00ee00 comp rx=0 tx=0).stop 2024-04-20T21:33:54.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.341+0000 7fa328c00640 1 -- 172.21.15.22:0/2037063530 shutdown_connections 2024-04-20T21:33:54.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.341+0000 7fa328c00640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa3080a3e30 0x7fa3080a62f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.341+0000 7fa328c00640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa324105b50 0x7fa3241a1810 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.341+0000 7fa328c00640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa324105190 0x7fa32419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.341+0000 7fa328c00640 1 --2- 172.21.15.22:0/2037063530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa324103fa0 0x7fa32419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.341+0000 7fa328c00640 1 -- 172.21.15.22:0/2037063530 >> 172.21.15.22:0/2037063530 conn(0x7fa3240ff530 msgr2=0x7fa324108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:54.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.341+0000 7fa328c00640 1 -- 172.21.15.22:0/2037063530 shutdown_connections 2024-04-20T21:33:54.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.341+0000 7fa328c00640 1 -- 172.21.15.22:0/2037063530 wait complete. 2024-04-20T21:33:54.359 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:33:54.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:54 smithi122 ceph-mon[22147]: from='client.25288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:54.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:54 smithi122 ceph-mon[22147]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:54.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:54 smithi122 ceph-mon[22147]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-04-20T21:33:54.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:54 smithi122 ceph-mon[22147]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:54.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:54 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2615252195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:54.526 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:54 smithi022 ceph-mon[16843]: from='client.25288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:54.526 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:54 smithi022 ceph-mon[16843]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:54.526 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:54 smithi022 ceph-mon[16843]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-04-20T21:33:54.526 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:54 smithi022 ceph-mon[16843]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:54.526 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:54 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2615252195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:54.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.551+0000 7faf16800640 1 -- 172.21.15.22:0/1639788949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf10101fa0 msgr2=0x7faf101023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:54.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.551+0000 7faf16800640 1 --2- 172.21.15.22:0/1639788949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf10101fa0 0x7faf101023c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faf00009ca0 tx=0x7faf0002c100 comp rx=0 tx=0).stop 2024-04-20T21:33:54.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.552+0000 7faf16800640 1 -- 172.21.15.22:0/1639788949 shutdown_connections 2024-04-20T21:33:54.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.552+0000 7faf16800640 1 --2- 172.21.15.22:0/1639788949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf10103b50 0x7faf1010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.552+0000 7faf16800640 1 --2- 172.21.15.22:0/1639788949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf10103190 0x7faf10103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.552+0000 7faf16800640 1 --2- 172.21.15.22:0/1639788949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf10101fa0 0x7faf101023c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.552+0000 7faf16800640 1 -- 172.21.15.22:0/1639788949 >> 172.21.15.22:0/1639788949 conn(0x7faf100fd510 msgr2=0x7faf100ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:54.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.552+0000 7faf16800640 1 -- 172.21.15.22:0/1639788949 shutdown_connections 2024-04-20T21:33:54.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.552+0000 7faf16800640 1 -- 172.21.15.22:0/1639788949 wait complete. 2024-04-20T21:33:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.553+0000 7faf16800640 1 Processor -- start 2024-04-20T21:33:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.553+0000 7faf16800640 1 -- start start 2024-04-20T21:33:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.553+0000 7faf16800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf10101fa0 0x7faf1019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.553+0000 7faf16800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf10103190 0x7faf1019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.553+0000 7faf16800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf10103b50 0x7faf101a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf16800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010dc30 con 0x7faf10103b50 2024-04-20T21:33:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf16800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010dab0 con 0x7faf10103190 2024-04-20T21:33:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf16800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faf1010ddb0 con 0x7faf10101fa0 2024-04-20T21:33:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf0fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf10103b50 0x7faf101a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf0ea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf10103190 0x7faf1019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf0f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf10101fa0 0x7faf1019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf0ea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf10103190 0x7faf1019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60152/0 (socket says 172.21.15.22:60152) 2024-04-20T21:33:54.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf0ea00640 1 -- 172.21.15.22:0/3880331330 learned_addr learned my addr 172.21.15.22:0/3880331330 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:54.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:54 smithi033 ceph-mon[21626]: from='client.25288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:54.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:54 smithi033 ceph-mon[21626]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:54.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:54 smithi033 ceph-mon[21626]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-04-20T21:33:54.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:54 smithi033 ceph-mon[21626]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:54.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:54 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2615252195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf0f400640 1 -- 172.21.15.22:0/3880331330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf10103190 msgr2=0x7faf1019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf0f400640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf10103190 0x7faf1019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf0f400640 1 -- 172.21.15.22:0/3880331330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf10103b50 msgr2=0x7faf101a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf0f400640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf10103b50 0x7faf101a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf0f400640 1 -- 172.21.15.22:0/3880331330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf101a2150 con 0x7faf10101fa0 2024-04-20T21:33:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.554+0000 7faf0ea00640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf10103190 0x7faf1019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.555+0000 7faf0f400640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf10101fa0 0x7faf1019a530 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf0002c640 tx=0x7faf00035d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.555+0000 7faf07e00640 1 -- 172.21.15.22:0/3880331330 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00039040 con 0x7faf10101fa0 2024-04-20T21:33:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.555+0000 7faf16800640 1 -- 172.21.15.22:0/3880331330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf101a23e0 con 0x7faf10101fa0 2024-04-20T21:33:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.555+0000 7faf07e00640 1 -- 172.21.15.22:0/3880331330 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf00003970 con 0x7faf10101fa0 2024-04-20T21:33:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.555+0000 7faf07e00640 1 -- 172.21.15.22:0/3880331330 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf000042a0 con 0x7faf10101fa0 2024-04-20T21:33:54.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.555+0000 7faf16800640 1 -- 172.21.15.22:0/3880331330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf101a2920 con 0x7faf10101fa0 2024-04-20T21:33:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.556+0000 7faf16800640 1 -- 172.21.15.22:0/3880331330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf101a2b80 con 0x7faf10101fa0 2024-04-20T21:33:54.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.558+0000 7faf07e00640 1 -- 172.21.15.22:0/3880331330 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7faf00046030 con 0x7faf10101fa0 2024-04-20T21:33:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.560+0000 7faf07e00640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7faef40a3db0 0x7faef40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.560+0000 7faf07e00640 1 -- 172.21.15.22:0/3880331330 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7faf000f24d0 con 0x7faf10101fa0 2024-04-20T21:33:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.560+0000 7faf0ea00640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7faef40a3db0 0x7faef40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:54.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.561+0000 7faf0ea00640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7faef40a3db0 0x7faef40a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faef80095f0 tx=0x7faef8005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:54.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.565+0000 7faf07e00640 1 -- 172.21.15.22:0/3880331330 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7faf0004d060 con 0x7faf10101fa0 2024-04-20T21:33:54.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.773+0000 7faf16800640 1 -- 172.21.15.22:0/3880331330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faf101a2e30 con 0x7faf10101fa0 2024-04-20T21:33:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.774+0000 7faf07e00640 1 -- 172.21.15.22:0/3880331330 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7faf000beb20 con 0x7faf10101fa0 2024-04-20T21:33:54.772 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-20T21:33:54.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.777+0000 7faf16800640 1 -- 172.21.15.22:0/3880331330 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7faef40a3db0 msgr2=0x7faef40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:54.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.777+0000 7faf16800640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7faef40a3db0 0x7faef40a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faef80095f0 tx=0x7faef8005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:54.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.777+0000 7faf16800640 1 -- 172.21.15.22:0/3880331330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf10101fa0 msgr2=0x7faf1019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:54.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.777+0000 7faf16800640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf10101fa0 0x7faf1019a530 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf0002c640 tx=0x7faf00035d90 comp rx=0 tx=0).stop 2024-04-20T21:33:54.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.778+0000 7faf16800640 1 -- 172.21.15.22:0/3880331330 shutdown_connections 2024-04-20T21:33:54.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.778+0000 7faf16800640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7faef40a3db0 0x7faef40a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.778+0000 7faf16800640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf10103b50 0x7faf101a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.778+0000 7faf16800640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf10103190 0x7faf1019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.778+0000 7faf16800640 1 --2- 172.21.15.22:0/3880331330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf10101fa0 0x7faf1019a530 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.778+0000 7faf16800640 1 -- 172.21.15.22:0/3880331330 >> 172.21.15.22:0/3880331330 conn(0x7faf100fd510 msgr2=0x7faf10074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.778+0000 7faf16800640 1 -- 172.21.15.22:0/3880331330 shutdown_connections 2024-04-20T21:33:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:54.778+0000 7faf16800640 1 -- 172.21.15.22:0/3880331330 wait complete. 2024-04-20T21:33:54.804 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:33:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.018+0000 7f2b01000640 1 -- 172.21.15.22:0/2957490215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2afc103180 msgr2=0x7f2afc103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.018+0000 7f2b01000640 1 --2- 172.21.15.22:0/2957490215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2afc103180 0x7f2afc103600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009860 tx=0x7f2ae402bb70 comp rx=0 tx=0).stop 2024-04-20T21:33:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.019+0000 7f2b01000640 1 -- 172.21.15.22:0/2957490215 shutdown_connections 2024-04-20T21:33:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.019+0000 7f2b01000640 1 --2- 172.21.15.22:0/2957490215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2afc103b40 0x7f2afc10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.019+0000 7f2b01000640 1 --2- 172.21.15.22:0/2957490215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2afc103180 0x7f2afc103600 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.019+0000 7f2b01000640 1 --2- 172.21.15.22:0/2957490215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc101f90 0x7f2afc1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.019+0000 7f2b01000640 1 -- 172.21.15.22:0/2957490215 >> 172.21.15.22:0/2957490215 conn(0x7f2afc0fd520 msgr2=0x7f2afc0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:55.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.019+0000 7f2b01000640 1 -- 172.21.15.22:0/2957490215 shutdown_connections 2024-04-20T21:33:55.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.019+0000 7f2b01000640 1 -- 172.21.15.22:0/2957490215 wait complete. 2024-04-20T21:33:55.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.020+0000 7f2b01000640 1 Processor -- start 2024-04-20T21:33:55.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.020+0000 7f2b01000640 1 -- start start 2024-04-20T21:33:55.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.020+0000 7f2b01000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2afc101f90 0x7f2afc075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:55.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.020+0000 7f2b01000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc103180 0x7f2afc074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:55.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.020+0000 7f2b01000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2afc103b40 0x7f2afc074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:55.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.020+0000 7f2b01000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc10dc20 con 0x7f2afc103b40 2024-04-20T21:33:55.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.020+0000 7f2b01000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc10daa0 con 0x7f2afc103180 2024-04-20T21:33:55.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.020+0000 7f2b01000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc10dda0 con 0x7f2afc101f90 2024-04-20T21:33:55.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.021+0000 7f2afa800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2afc103b40 0x7f2afc074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.021+0000 7f2af3e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc103180 0x7f2afc074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.021+0000 7f2af3e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc103180 0x7f2afc074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60176/0 (socket says 172.21.15.22:60176) 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.021+0000 7f2af3e00640 1 -- 172.21.15.22:0/3755713781 learned_addr learned my addr 172.21.15.22:0/3755713781 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.021+0000 7f2af9e00640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2afc101f90 0x7f2afc075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.021+0000 7f2af3e00640 1 -- 172.21.15.22:0/3755713781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2afc101f90 msgr2=0x7f2afc075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.021+0000 7f2af3e00640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2afc101f90 0x7f2afc075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.021+0000 7f2af3e00640 1 -- 172.21.15.22:0/3755713781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2afc103b40 msgr2=0x7f2afc074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.021+0000 7f2af3e00640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2afc103b40 0x7f2afc074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.021+0000 7f2af3e00640 1 -- 172.21.15.22:0/3755713781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc075120 con 0x7f2afc103180 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.021+0000 7f2af3e00640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc103180 0x7f2afc074350 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40349e0 tx=0x7f2ae4032450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.022+0000 7f2af2a00640 1 -- 172.21.15.22:0/3755713781 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4041030 con 0x7f2afc103180 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.022+0000 7f2b01000640 1 -- 172.21.15.22:0/3755713781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc1a2370 con 0x7f2afc103180 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.022+0000 7f2af2a00640 1 -- 172.21.15.22:0/3755713781 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae402c510 con 0x7f2afc103180 2024-04-20T21:33:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.022+0000 7f2af2a00640 1 -- 172.21.15.22:0/3755713781 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4039040 con 0x7f2afc103180 2024-04-20T21:33:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.022+0000 7f2b01000640 1 -- 172.21.15.22:0/3755713781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc1a2910 con 0x7f2afc103180 2024-04-20T21:33:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.023+0000 7f2b01000640 1 -- 172.21.15.22:0/3755713781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc109300 con 0x7f2afc103180 2024-04-20T21:33:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.025+0000 7f2af2a00640 1 -- 172.21.15.22:0/3755713781 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f2ae4044070 con 0x7f2afc103180 2024-04-20T21:33:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.026+0000 7f2af2a00640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2ae00a3db0 0x7f2ae00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.026+0000 7f2af2a00640 1 -- 172.21.15.22:0/3755713781 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f2ae40f25f0 con 0x7f2afc103180 2024-04-20T21:33:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.027+0000 7f2af9e00640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2ae00a3db0 0x7f2ae00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:55.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.027+0000 7f2af9e00640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2ae00a3db0 0x7f2ae00a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0095f0 tx=0x7f2aec005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.029+0000 7f2af2a00640 1 -- 172.21.15.22:0/3755713781 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2ae40bebb0 con 0x7f2afc103180 2024-04-20T21:33:55.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.238+0000 7f2b01000640 1 -- 172.21.15.22:0/3755713781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2afc02d7d0 con 0x7f2afc103180 2024-04-20T21:33:55.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.239+0000 7f2af2a00640 1 -- 172.21.15.22:0/3755713781 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f2ae40be4b0 con 0x7f2afc103180 2024-04-20T21:33:55.237 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-20T21:33:55.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.241+0000 7f2b01000640 1 -- 172.21.15.22:0/3755713781 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2ae00a3db0 msgr2=0x7f2ae00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.241+0000 7f2b01000640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2ae00a3db0 0x7f2ae00a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0095f0 tx=0x7f2aec005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:55.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.241+0000 7f2b01000640 1 -- 172.21.15.22:0/3755713781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc103180 msgr2=0x7f2afc074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.241+0000 7f2b01000640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc103180 0x7f2afc074350 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40349e0 tx=0x7f2ae4032450 comp rx=0 tx=0).stop 2024-04-20T21:33:55.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.241+0000 7f2b01000640 1 -- 172.21.15.22:0/3755713781 shutdown_connections 2024-04-20T21:33:55.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.241+0000 7f2b01000640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2ae00a3db0 0x7f2ae00a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.241+0000 7f2b01000640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2afc103b40 0x7f2afc074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.241+0000 7f2b01000640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc103180 0x7f2afc074350 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.241+0000 7f2b01000640 1 --2- 172.21.15.22:0/3755713781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2afc101f90 0x7f2afc075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.241+0000 7f2b01000640 1 -- 172.21.15.22:0/3755713781 >> 172.21.15.22:0/3755713781 conn(0x7f2afc0fd520 msgr2=0x7f2afc1064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:55.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.241+0000 7f2b01000640 1 -- 172.21.15.22:0/3755713781 shutdown_connections 2024-04-20T21:33:55.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.241+0000 7f2b01000640 1 -- 172.21.15.22:0/3755713781 wait complete. 2024-04-20T21:33:55.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.462+0000 7f3ad7400640 1 -- 172.21.15.22:0/676143590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ad0074940 msgr2=0x7f3ad0074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.462+0000 7f3ad7400640 1 --2- 172.21.15.22:0/676143590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ad0074940 0x7f3ad0074da0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009900 tx=0x7f3abc02bcc0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.462+0000 7f3ad7400640 1 -- 172.21.15.22:0/676143590 shutdown_connections 2024-04-20T21:33:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.462+0000 7f3ad7400640 1 --2- 172.21.15.22:0/676143590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad0109e60 0x7f3ad010c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.462+0000 7f3ad7400640 1 --2- 172.21.15.22:0/676143590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ad0074940 0x7f3ad0074da0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.462+0000 7f3ad7400640 1 --2- 172.21.15.22:0/676143590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad0075f20 0x7f3ad0074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.462+0000 7f3ad7400640 1 -- 172.21.15.22:0/676143590 >> 172.21.15.22:0/676143590 conn(0x7f3ad00fd540 msgr2=0x7f3ad00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.462+0000 7f3ad7400640 1 -- 172.21.15.22:0/676143590 shutdown_connections 2024-04-20T21:33:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.462+0000 7f3ad7400640 1 -- 172.21.15.22:0/676143590 wait complete. 2024-04-20T21:33:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.463+0000 7f3ad7400640 1 Processor -- start 2024-04-20T21:33:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.463+0000 7f3ad7400640 1 -- start start 2024-04-20T21:33:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ad0074940 0x7f3ad019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad0075f20 0x7f3ad019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad0109e60 0x7f3ad01a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad7400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad010fea0 con 0x7f3ad0075f20 2024-04-20T21:33:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad7400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad010fd20 con 0x7f3ad0109e60 2024-04-20T21:33:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad7400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0110020 con 0x7f3ad0074940 2024-04-20T21:33:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad4a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ad0074940 0x7f3ad019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad4a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ad0074940 0x7f3ad019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:35132/0 (socket says 172.21.15.22:35132) 2024-04-20T21:33:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad4a00640 1 -- 172.21.15.22:0/3995174087 learned_addr learned my addr 172.21.15.22:0/3995174087 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3acfe00640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad0075f20 0x7f3ad019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad5400640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad0109e60 0x7f3ad01a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad4a00640 1 -- 172.21.15.22:0/3995174087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad0109e60 msgr2=0x7f3ad01a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad4a00640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad0109e60 0x7f3ad01a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad4a00640 1 -- 172.21.15.22:0/3995174087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad0075f20 msgr2=0x7f3ad019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.464+0000 7f3ad4a00640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad0075f20 0x7f3ad019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.465+0000 7f3ad4a00640 1 -- 172.21.15.22:0/3995174087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad01a20d0 con 0x7f3ad0074940 2024-04-20T21:33:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.465+0000 7f3ad4a00640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ad0074940 0x7f3ad019a4b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400ea60 tx=0x7f3ac400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.465+0000 7f3acd600640 1 -- 172.21.15.22:0/3995174087 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400cde0 con 0x7f3ad0074940 2024-04-20T21:33:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.465+0000 7f3acd600640 1 -- 172.21.15.22:0/3995174087 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac40035f0 con 0x7f3ad0074940 2024-04-20T21:33:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.465+0000 7f3ad7400640 1 -- 172.21.15.22:0/3995174087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad01a23c0 con 0x7f3ad0074940 2024-04-20T21:33:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.465+0000 7f3acd600640 1 -- 172.21.15.22:0/3995174087 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4010670 con 0x7f3ad0074940 2024-04-20T21:33:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.465+0000 7f3ad7400640 1 -- 172.21.15.22:0/3995174087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad01a2930 con 0x7f3ad0074940 2024-04-20T21:33:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.467+0000 7f3ad7400640 1 -- 172.21.15.22:0/3995174087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad0108c20 con 0x7f3ad0074940 2024-04-20T21:33:55.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.470+0000 7f3acd600640 1 -- 172.21.15.22:0/3995174087 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f3ac40037a0 con 0x7f3ad0074940 2024-04-20T21:33:55.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.471+0000 7f3acd600640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3ab80a3e30 0x7f3ab80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:55.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.472+0000 7f3acd600640 1 -- 172.21.15.22:0/3995174087 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f3ac40d0dd0 con 0x7f3ad0074940 2024-04-20T21:33:55.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.472+0000 7f3acfe00640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3ab80a3e30 0x7f3ab80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:55.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.473+0000 7f3acfe00640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3ab80a3e30 0x7f3ab80a62f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3abc02c2d0 tx=0x7f3abc005fd0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.476+0000 7f3acd600640 1 -- 172.21.15.22:0/3995174087 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3ac409d280 con 0x7f3ad0074940 2024-04-20T21:33:55.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:55 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2037063530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:55.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:55 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3880331330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:55.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:55 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:55.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:55 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:55.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:55 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:55.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:55 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:55.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:55 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:55.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:55 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:55.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:55 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:33:55.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:55 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:33:55.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:55 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3755713781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:55.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:55 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2037063530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:55.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:55 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3880331330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:55.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:55 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:55.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:55 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:55.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:55 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:55.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:55 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:55.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:55 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:55.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:55 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:55.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:55 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:33:55.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:55 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:33:55.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:55 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3755713781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:55.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.766+0000 7f3ad7400640 1 -- 172.21.15.22:0/3995174087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ad002d5d0 con 0x7f3ad0074940 2024-04-20T21:33:55.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.767+0000 7f3acd600640 1 -- 172.21.15.22:0/3995174087 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f3ac409cb80 con 0x7f3ad0074940 2024-04-20T21:33:55.767 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:55.767 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:33:55.767 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:33:55.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.770+0000 7f3ad7400640 1 -- 172.21.15.22:0/3995174087 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3ab80a3e30 msgr2=0x7f3ab80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.770+0000 7f3ad7400640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3ab80a3e30 0x7f3ab80a62f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3abc02c2d0 tx=0x7f3abc005fd0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.770+0000 7f3ad7400640 1 -- 172.21.15.22:0/3995174087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ad0074940 msgr2=0x7f3ad019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.770+0000 7f3ad7400640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ad0074940 0x7f3ad019a4b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400ea60 tx=0x7f3ac400ef20 comp rx=0 tx=0).stop 2024-04-20T21:33:55.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.770+0000 7f3ad7400640 1 -- 172.21.15.22:0/3995174087 shutdown_connections 2024-04-20T21:33:55.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.770+0000 7f3ad7400640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3ab80a3e30 0x7f3ab80a62f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.770+0000 7f3ad7400640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad0109e60 0x7f3ad01a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.770+0000 7f3ad7400640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ad0075f20 0x7f3ad019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.770+0000 7f3ad7400640 1 --2- 172.21.15.22:0/3995174087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ad0074940 0x7f3ad019a4b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.770+0000 7f3ad7400640 1 -- 172.21.15.22:0/3995174087 >> 172.21.15.22:0/3995174087 conn(0x7f3ad00fd540 msgr2=0x7f3ad0105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:55.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.770+0000 7f3ad7400640 1 -- 172.21.15.22:0/3995174087 shutdown_connections 2024-04-20T21:33:55.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.771+0000 7f3ad7400640 1 -- 172.21.15.22:0/3995174087 wait complete. 2024-04-20T21:33:55.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:55 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2037063530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:55.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:55 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3880331330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:55.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:55 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:55.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:55 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:55.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:55 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:55.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:55 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:55.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:55 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:55.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:55 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:55.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:55 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:33:55.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:55 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:33:55.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:55 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3755713781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:55.787 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.959+0000 7f85dee00640 1 -- 172.21.15.22:0/1777919411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85d8076920 msgr2=0x7f85d8076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.959+0000 7f85dee00640 1 --2- 172.21.15.22:0/1777919411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85d8076920 0x7f85d8076d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009900 tx=0x7f85c802bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.959+0000 7f85dee00640 1 -- 172.21.15.22:0/1777919411 shutdown_connections 2024-04-20T21:33:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.959+0000 7f85dee00640 1 --2- 172.21.15.22:0/1777919411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85d8078500 0x7f85d8078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.959+0000 7f85dee00640 1 --2- 172.21.15.22:0/1777919411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85d8077b60 0x7f85d8077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.959+0000 7f85dee00640 1 --2- 172.21.15.22:0/1777919411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85d8076920 0x7f85d8076d40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.959+0000 7f85dee00640 1 -- 172.21.15.22:0/1777919411 >> 172.21.15.22:0/1777919411 conn(0x7f85d80ff750 msgr2=0x7f85d8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.959+0000 7f85dee00640 1 -- 172.21.15.22:0/1777919411 shutdown_connections 2024-04-20T21:33:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.959+0000 7f85dee00640 1 -- 172.21.15.22:0/1777919411 wait complete. 2024-04-20T21:33:55.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.960+0000 7f85dee00640 1 Processor -- start 2024-04-20T21:33:55.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.961+0000 7f85dee00640 1 -- start start 2024-04-20T21:33:55.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.961+0000 7f85dee00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85d8076920 0x7f85d819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:55.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.961+0000 7f85dee00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85d8077b60 0x7f85d819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:55.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.961+0000 7f85dee00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85d8078500 0x7f85d81a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:55.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.961+0000 7f85dee00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8114290 con 0x7f85d8077b60 2024-04-20T21:33:55.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.961+0000 7f85dee00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8114110 con 0x7f85d8076920 2024-04-20T21:33:55.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.961+0000 7f85dee00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8114410 con 0x7f85d8078500 2024-04-20T21:33:55.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.961+0000 7f85d7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85d8076920 0x7f85d819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:55.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.961+0000 7f85d7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85d8076920 0x7f85d819e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60204/0 (socket says 172.21.15.22:60204) 2024-04-20T21:33:55.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.961+0000 7f85d7e00640 1 -- 172.21.15.22:0/3880469010 learned_addr learned my addr 172.21.15.22:0/3880469010 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:55.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.961+0000 7f85dce00640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85d8078500 0x7f85d81a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:55.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.961+0000 7f85d7400640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85d8077b60 0x7f85d819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:55.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.962+0000 7f85d7e00640 1 -- 172.21.15.22:0/3880469010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85d8078500 msgr2=0x7f85d81a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.962+0000 7f85d7e00640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85d8078500 0x7f85d81a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.962+0000 7f85d7e00640 1 -- 172.21.15.22:0/3880469010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85d8077b60 msgr2=0x7f85d819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:55.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.962+0000 7f85d7e00640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85d8077b60 0x7f85d819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:55.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.962+0000 7f85d7e00640 1 -- 172.21.15.22:0/3880469010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d81a64b0 con 0x7f85d8076920 2024-04-20T21:33:55.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.962+0000 7f85d7e00640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85d8076920 0x7f85d819e890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f85c8000c00 tx=0x7f85c802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:55.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.962+0000 7f85d4c00640 1 -- 172.21.15.22:0/3880469010 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8043020 con 0x7f85d8076920 2024-04-20T21:33:55.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.962+0000 7f85dee00640 1 -- 172.21.15.22:0/3880469010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d81a6740 con 0x7f85d8076920 2024-04-20T21:33:55.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.962+0000 7f85d4c00640 1 -- 172.21.15.22:0/3880469010 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c80054e0 con 0x7f85d8076920 2024-04-20T21:33:55.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.962+0000 7f85d4c00640 1 -- 172.21.15.22:0/3880469010 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8003c40 con 0x7f85d8076920 2024-04-20T21:33:55.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.963+0000 7f85dee00640 1 -- 172.21.15.22:0/3880469010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d81a6c90 con 0x7f85d8076920 2024-04-20T21:33:55.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.964+0000 7f85dee00640 1 -- 172.21.15.22:0/3880469010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8110c80 con 0x7f85d8076920 2024-04-20T21:33:55.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.966+0000 7f85d4c00640 1 -- 172.21.15.22:0/3880469010 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f85c8046020 con 0x7f85d8076920 2024-04-20T21:33:55.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.967+0000 7f85d4c00640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f85bc0a3db0 0x7f85bc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:55.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.967+0000 7f85d4c00640 1 -- 172.21.15.22:0/3880469010 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f85c80f28f0 con 0x7f85d8076920 2024-04-20T21:33:55.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.968+0000 7f85d7400640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f85bc0a3db0 0x7f85bc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:55.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.968+0000 7f85d7400640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f85bc0a3db0 0x7f85bc0a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f85d819fda0 tx=0x7f85c4005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:55.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:55.971+0000 7f85d4c00640 1 -- 172.21.15.22:0/3880469010 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f85c804d020 con 0x7f85d8076920 2024-04-20T21:33:56.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.182+0000 7f85dee00640 1 -- 172.21.15.22:0/3880469010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85d8062b60 con 0x7f85d8076920 2024-04-20T21:33:56.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.183+0000 7f85d4c00640 1 -- 172.21.15.22:0/3880469010 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+23146 (secure 0 0 0) 0x7f85c80beeb0 con 0x7f85d8076920 2024-04-20T21:33:56.183 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:33:56.183 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:33:48.816036+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/1815958949":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/3359736029":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/3485677447":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/1911146506":"2024-04-21T21:33:48.815984+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1408863783":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/1911146506":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:56.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.188+0000 7f85dee00640 1 -- 172.21.15.22:0/3880469010 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f85bc0a3db0 msgr2=0x7f85bc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.188+0000 7f85dee00640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f85bc0a3db0 0x7f85bc0a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f85d819fda0 tx=0x7f85c4005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:56.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.188+0000 7f85dee00640 1 -- 172.21.15.22:0/3880469010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85d8076920 msgr2=0x7f85d819e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.188+0000 7f85dee00640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85d8076920 0x7f85d819e890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f85c8000c00 tx=0x7f85c802c650 comp rx=0 tx=0).stop 2024-04-20T21:33:56.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.188+0000 7f85dee00640 1 -- 172.21.15.22:0/3880469010 shutdown_connections 2024-04-20T21:33:56.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.188+0000 7f85dee00640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f85bc0a3db0 0x7f85bc0a6270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.188+0000 7f85dee00640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85d8078500 0x7f85d81a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.188+0000 7f85dee00640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85d8077b60 0x7f85d819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.188+0000 7f85dee00640 1 --2- 172.21.15.22:0/3880469010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85d8076920 0x7f85d819e890 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.188+0000 7f85dee00640 1 -- 172.21.15.22:0/3880469010 >> 172.21.15.22:0/3880469010 conn(0x7f85d80ff750 msgr2=0x7f85d8100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.189+0000 7f85dee00640 1 -- 172.21.15.22:0/3880469010 shutdown_connections 2024-04-20T21:33:56.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.189+0000 7f85dee00640 1 -- 172.21.15.22:0/3880469010 wait complete. 2024-04-20T21:33:56.208 DEBUG:tasks.fwd_scrub:Yielding 2024-04-20T21:33:56.208 INFO:teuthology.run_tasks:Running task exec... 2024-04-20T21:33:56.216 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T21:33:56.217 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-04-20T21:33:56.217 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-20T21:33:56.217 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-20T21:33:56.217 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:33:56.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:56.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:33:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.405+0000 7f57eb200640 1 -- 172.21.15.22:0/3128776480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57ec102fc0 msgr2=0x7f57ec105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.405+0000 7f57eb200640 1 --2- 172.21.15.22:0/3128776480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57ec102fc0 0x7f57ec105430 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009900 tx=0x7f57d802bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.405+0000 7f57eb200640 1 -- 172.21.15.22:0/3128776480 shutdown_connections 2024-04-20T21:33:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.405+0000 7f57eb200640 1 --2- 172.21.15.22:0/3128776480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ec105970 0x7f57ec105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.405+0000 7f57eb200640 1 --2- 172.21.15.22:0/3128776480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57ec102fc0 0x7f57ec105430 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.405+0000 7f57eb200640 1 --2- 172.21.15.22:0/3128776480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57ec100650 0x7f57ec102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.405+0000 7f57eb200640 1 -- 172.21.15.22:0/3128776480 >> 172.21.15.22:0/3128776480 conn(0x7f57ec0fc250 msgr2=0x7f57ec0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.406+0000 7f57eb200640 1 -- 172.21.15.22:0/3128776480 shutdown_connections 2024-04-20T21:33:56.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.406+0000 7f57eb200640 1 -- 172.21.15.22:0/3128776480 wait complete. 2024-04-20T21:33:56.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.407+0000 7f57eb200640 1 Processor -- start 2024-04-20T21:33:56.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.407+0000 7f57eb200640 1 -- start start 2024-04-20T21:33:56.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.408+0000 7f57eb200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57ec100650 0x7f57ec19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.408+0000 7f57eb200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57ec102fc0 0x7f57ec19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.408+0000 7f57eb200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ec105970 0x7f57ec1a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.408+0000 7f57eb200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec1140a0 con 0x7f57ec105970 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.408+0000 7f57eb200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec113f20 con 0x7f57ec102fc0 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.408+0000 7f57eb200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec114220 con 0x7f57ec100650 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.408+0000 7f57e9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57ec100650 0x7f57ec19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.408+0000 7f57e9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57ec100650 0x7f57ec19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:35166/0 (socket says 172.21.15.22:35166) 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.408+0000 7f57e9e00640 1 -- 172.21.15.22:0/2056010900 learned_addr learned my addr 172.21.15.22:0/2056010900 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.408+0000 7f57e9400640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57ec102fc0 0x7f57ec19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.408+0000 7f57ea800640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ec105970 0x7f57ec1a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.408+0000 7f57e9e00640 1 -- 172.21.15.22:0/2056010900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57ec102fc0 msgr2=0x7f57ec19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.409+0000 7f57e9e00640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57ec102fc0 0x7f57ec19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.409+0000 7f57e9e00640 1 -- 172.21.15.22:0/2056010900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ec105970 msgr2=0x7f57ec1a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.409+0000 7f57e9e00640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ec105970 0x7f57ec1a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.409+0000 7f57e9e00640 1 -- 172.21.15.22:0/2056010900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ec1a62c0 con 0x7f57ec100650 2024-04-20T21:33:56.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.409+0000 7f57ea800640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ec105970 0x7f57ec1a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.409+0000 7f57e9400640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57ec102fc0 0x7f57ec19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.409+0000 7f57e9e00640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57ec100650 0x7f57ec19e6a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00e9f0 tx=0x7f57dc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.409+0000 7f57e2a00640 1 -- 172.21.15.22:0/2056010900 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00cd80 con 0x7f57ec100650 2024-04-20T21:33:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.409+0000 7f57e2a00640 1 -- 172.21.15.22:0/2056010900 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57dc010910 con 0x7f57ec100650 2024-04-20T21:33:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.409+0000 7f57eb200640 1 -- 172.21.15.22:0/2056010900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ec1a65b0 con 0x7f57ec100650 2024-04-20T21:33:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.409+0000 7f57e2a00640 1 -- 172.21.15.22:0/2056010900 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc018990 con 0x7f57ec100650 2024-04-20T21:33:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.409+0000 7f57eb200640 1 -- 172.21.15.22:0/2056010900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ec1a6b20 con 0x7f57ec100650 2024-04-20T21:33:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.411+0000 7f57eb200640 1 -- 172.21.15.22:0/2056010900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57ec050540 con 0x7f57ec100650 2024-04-20T21:33:56.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.412+0000 7f57e2a00640 1 -- 172.21.15.22:0/2056010900 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f57dc010ac0 con 0x7f57ec100650 2024-04-20T21:33:56.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.414+0000 7f57e2a00640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57d00a3e20 0x7f57d00a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.414+0000 7f57e2a00640 1 -- 172.21.15.22:0/2056010900 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f57dc0d0d50 con 0x7f57ec100650 2024-04-20T21:33:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.414+0000 7f57e9400640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57d00a3e20 0x7f57d00a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.415+0000 7f57e9400640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57d00a3e20 0x7f57d00a62e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f57d802c240 tx=0x7f57d802df90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.418+0000 7f57e2a00640 1 -- 172.21.15.22:0/2056010900 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f57dc09d200 con 0x7f57ec100650 2024-04-20T21:33:56.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.419+0000 7f08cc600640 1 -- 172.21.15.22:0/696023047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c4077b60 msgr2=0x7f08c4077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.419+0000 7f08cc600640 1 --2- 172.21.15.22:0/696023047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c4077b60 0x7f08c4077fc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009900 tx=0x7f08b002bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:56.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.419+0000 7f08cc600640 1 -- 172.21.15.22:0/696023047 shutdown_connections 2024-04-20T21:33:56.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.419+0000 7f08cc600640 1 --2- 172.21.15.22:0/696023047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c4078500 0x7f08c4078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.419+0000 7f08cc600640 1 --2- 172.21.15.22:0/696023047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c4077b60 0x7f08c4077fc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.419+0000 7f08cc600640 1 --2- 172.21.15.22:0/696023047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c4076920 0x7f08c4076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.419+0000 7f08cc600640 1 -- 172.21.15.22:0/696023047 >> 172.21.15.22:0/696023047 conn(0x7f08c40ff710 msgr2=0x7f08c4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.419+0000 7f08cc600640 1 -- 172.21.15.22:0/696023047 shutdown_connections 2024-04-20T21:33:56.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.419+0000 7f08cc600640 1 -- 172.21.15.22:0/696023047 wait complete. 2024-04-20T21:33:56.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.420+0000 7f08cc600640 1 Processor -- start 2024-04-20T21:33:56.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.420+0000 7f08cc600640 1 -- start start 2024-04-20T21:33:56.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08cc600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c4076920 0x7f08c419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08cc600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c4077b60 0x7f08c419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08cc600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c4078500 0x7f08c41a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08cc600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4114290 con 0x7f08c4078500 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08cc600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4114110 con 0x7f08c4077b60 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08cc600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4114410 con 0x7f08c4076920 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08c9200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c4077b60 0x7f08c419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08c9c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c4076920 0x7f08c419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08c9200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c4077b60 0x7f08c419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60250/0 (socket says 172.21.15.22:60250) 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08c9200640 1 -- 172.21.15.22:0/322062040 learned_addr learned my addr 172.21.15.22:0/322062040 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08ca600640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c4078500 0x7f08c41a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08c9200640 1 -- 172.21.15.22:0/322062040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c4076920 msgr2=0x7f08c419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08c9200640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c4076920 0x7f08c419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08c9200640 1 -- 172.21.15.22:0/322062040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c4078500 msgr2=0x7f08c41a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08c9200640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c4078500 0x7f08c41a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08c9200640 1 -- 172.21.15.22:0/322062040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c41a64e0 con 0x7f08c4077b60 2024-04-20T21:33:56.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.421+0000 7f08ca600640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c4078500 0x7f08c41a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:56.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.422+0000 7f08c9200640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c4077b60 0x7f08c419ee00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f08b0005ab0 tx=0x7f08b002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.422+0000 7f08c2000640 1 -- 172.21.15.22:0/322062040 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0043020 con 0x7f08c4077b60 2024-04-20T21:33:56.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.422+0000 7f08c2000640 1 -- 172.21.15.22:0/322062040 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b00054e0 con 0x7f08c4077b60 2024-04-20T21:33:56.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.422+0000 7f08c2000640 1 -- 172.21.15.22:0/322062040 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0003ce0 con 0x7f08c4077b60 2024-04-20T21:33:56.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.422+0000 7f08cc600640 1 -- 172.21.15.22:0/322062040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c41a6770 con 0x7f08c4077b60 2024-04-20T21:33:56.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.422+0000 7f08cc600640 1 -- 172.21.15.22:0/322062040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c41a6d10 con 0x7f08c4077b60 2024-04-20T21:33:56.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.426+0000 7f08cf439740 1 Processor -- start 2024-04-20T21:33:56.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.426+0000 7f08cf439740 1 -- start start 2024-04-20T21:33:56.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.426+0000 7f08c2000640 1 -- 172.21.15.22:0/322062040 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f08b0046020 con 0x7f08c4077b60 2024-04-20T21:33:56.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08cf439740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4857f30 0x55def4858350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08cf439740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def4858920 0x55def485d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08cf439740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def485da90 0x55def485ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08cf439740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55def47bdd20 con 0x55def485da90 2024-04-20T21:33:56.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08cf439740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55def47643e0 con 0x55def4857f30 2024-04-20T21:33:56.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08cf439740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55def47b9ce0 con 0x55def4858920 2024-04-20T21:33:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08c9c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def4858920 0x55def485d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08ca600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4857f30 0x55def4858350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08c9c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def4858920 0x55def485d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:35210/0 (socket says 172.21.15.22:35210) 2024-04-20T21:33:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08c9c00640 1 -- 172.21.15.22:0/2986458481 learned_addr learned my addr 172.21.15.22:0/2986458481 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08c9200640 1 --2- 172.21.15.22:0/2986458481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def485da90 0x55def485ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08c9c00640 1 -- 172.21.15.22:0/2986458481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4857f30 msgr2=0x55def4858350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08c9c00640 1 --2- 172.21.15.22:0/2986458481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4857f30 0x55def4858350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08c9c00640 1 -- 172.21.15.22:0/2986458481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def485da90 msgr2=0x55def485ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08c9c00640 1 --2- 172.21.15.22:0/2986458481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def485da90 0x55def485ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.427+0000 7f08c9c00640 1 -- 172.21.15.22:0/2986458481 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55def4783980 con 0x55def4858920 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.428+0000 7f08c9c00640 1 --2- 172.21.15.22:0/2986458481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def4858920 0x55def485d420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009cf0 tx=0x7f08b8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8b8b7a30b60dc7f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.428+0000 7f08b6a00640 1 -- 172.21.15.22:0/2986458481 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8004230 con 0x55def4858920 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.428+0000 7f08b6a00640 1 -- 172.21.15.22:0/2986458481 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b80043d0 con 0x55def4858920 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.428+0000 7f08b6a00640 1 -- 172.21.15.22:0/2986458481 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8019960 con 0x55def4858920 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.428+0000 7f08c2000640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f08ac0a3e30 0x7f08ac0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.429+0000 7f08c2000640 1 -- 172.21.15.22:0/322062040 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f08b00f2a10 con 0x7f08c4077b60 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.429+0000 7f08ca600640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f08ac0a3e30 0x7f08ac0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.429+0000 7f08cf439740 1 -- 172.21.15.22:0/2986458481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def4858920 msgr2=0x55def485d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.429+0000 7f08cf439740 1 --2- 172.21.15.22:0/2986458481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def4858920 0x55def485d420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009cf0 tx=0x7f08b8011810 comp rx=0 tx=0).stop 2024-04-20T21:33:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.430+0000 7f08ca600640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f08ac0a3e30 0x7f08ac0a62f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x55def4859180 tx=0x7f08bc009800 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.430+0000 7f08cf439740 1 -- 172.21.15.22:0/2986458481 shutdown_connections 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.430+0000 7f08cf439740 1 --2- 172.21.15.22:0/2986458481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def485da90 0x55def485ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.430+0000 7f08cf439740 1 --2- 172.21.15.22:0/2986458481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def4858920 0x55def485d420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.430+0000 7f08cf439740 1 --2- 172.21.15.22:0/2986458481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4857f30 0x55def4858350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.430+0000 7f08cf439740 1 -- 172.21.15.22:0/2986458481 >> 172.21.15.22:0/2986458481 conn(0x55def48534a0 msgr2=0x55def48538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.431+0000 7f08cf439740 1 -- 172.21.15.22:0/2986458481 shutdown_connections 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.431+0000 7f08cf439740 1 -- 172.21.15.22:0/2986458481 wait complete. 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.431+0000 7f08cf439740 1 -- start start 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.431+0000 7f08cf439740 1 Processor -- start 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.432+0000 7f08cf439740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4857f30 0x55def48624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.432+0000 7f08cf439740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 0x55def48629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.432+0000 7f08cf439740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4869a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.432+0000 7f08cf439740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55def4860a40 con 0x55def4857f30 2024-04-20T21:33:56.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.432+0000 7f08cf439740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55def47777a0 con 0x55def4858920 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.432+0000 7f08cf439740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55def4860bc0 con 0x55def485da90 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.432+0000 7f08cf439740 10 client.0 _ll_register_callbacks cb 0x55def4779b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.432+0000 7f08c9200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 0x55def48629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.432+0000 7f08c9c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4857f30 0x55def48624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.432+0000 7f08c9200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 0x55def48629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60268/0 (socket says 172.21.15.22:60268) 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.432+0000 7f08c9200640 1 -- 172.21.15.22:0/2581364036 learned_addr learned my addr 172.21.15.22:0/2581364036 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.432+0000 7f08ca600640 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4869a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.433+0000 7f08c9200640 1 -- 172.21.15.22:0/2581364036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 msgr2=0x55def4869a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.433+0000 7f08c9200640 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4869a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.433+0000 7f08c9200640 1 -- 172.21.15.22:0/2581364036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4857f30 msgr2=0x55def48624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.433+0000 7f08c9200640 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4857f30 0x55def48624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.433+0000 7f08c9200640 1 -- 172.21.15.22:0/2581364036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55def45987e0 con 0x55def4858920 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.433+0000 7f08c9c00640 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4857f30 0x55def48624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.433+0000 7f08c9200640 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 0x55def48629e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f08b0000c00 tx=0x7f08b00be6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81559b683170bdb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.433+0000 7f08ab400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:33:56.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.433+0000 7f08ab400640 1 -- 172.21.15.22:0/2581364036 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b00044a0 con 0x55def4858920 2024-04-20T21:33:56.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.433+0000 7f08ab400640 1 -- 172.21.15.22:0/2581364036 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f08b00bcbd0 con 0x55def4858920 2024-04-20T21:33:56.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.433+0000 7f08cf439740 1 -- 172.21.15.22:0/2581364036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b80043d0 con 0x55def4858920 2024-04-20T21:33:56.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.434+0000 7f08ab400640 1 -- 172.21.15.22:0/2581364036 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f08b0005690 con 0x55def4858920 2024-04-20T21:33:56.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.434+0000 7f08cf439740 10 client.25310 fetch_fsmap learned FSMap version 20 2024-04-20T21:33:56.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.434+0000 7f08cf439740 1 -- 172.21.15.22:0/2581364036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55def45f8f70 con 0x55def4858920 2024-04-20T21:33:56.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.434+0000 7f08ab400640 1 -- 172.21.15.22:0/2581364036 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b80043d0 con 0x55def4858920 2024-04-20T21:33:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.435+0000 7f08ab400640 1 -- 172.21.15.22:0/2581364036 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f08b00c70b0 con 0x55def4858920 2024-04-20T21:33:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.435+0000 7f08cf439740 10 client.25310 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:33:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.435+0000 7f08cf439740 10 client.25310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:33:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.435+0000 7f08cf439740 20 client.25310 populate_metadata read hostname 'smithi022' 2024-04-20T21:33:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.435+0000 7f08cf439740 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55def486e280 0x55def4870740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.435+0000 7f08cf439740 4 client.25310 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:33:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.435+0000 7f08cf439740 1 -- 172.21.15.22:0/2581364036 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55def47cb740 con 0x55def486e280 2024-04-20T21:33:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.435+0000 7f08c9c00640 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55def486e280 0x55def4870740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.436+0000 7f08c9c00640 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55def486e280 0x55def4870740 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0bb1e433696c105 server_cookie=ea45c11069f902b6 in_seq=0 out_seq=0 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.436+0000 7f08ab400640 10 client.25310 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.437+0000 7f08ab400640 1 -- 172.21.15.22:0/2581364036 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55def47cb740 con 0x55def486e280 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.437+0000 7f08ab400640 10 client.25310 handle_command_reply: tid=0 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.437+0000 7f08cf439740 1 client.25310 shutdown 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.437+0000 7f08cf439740 1 -- 172.21.15.22:0/2581364036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 msgr2=0x55def48629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.437+0000 7f08cf439740 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 0x55def48629e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f08b0000c00 tx=0x7f08b00be6a0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.437+0000 7f08cf439740 1 -- 172.21.15.22:0/2581364036 shutdown_connections 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.437+0000 7f08cf439740 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4869a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.437+0000 7f08cf439740 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55def486e280 0x55def4870740 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.437+0000 7f08cf439740 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 0x55def48629e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.437+0000 7f08cf439740 1 --2- 172.21.15.22:0/2581364036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4857f30 0x55def48624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.437+0000 7f08cf439740 1 -- 172.21.15.22:0/2581364036 >> 172.21.15.22:0/2581364036 conn(0x55def48534a0 msgr2=0x55def4852650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.437+0000 7f08ab400640 0 client.25310 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.438+0000 7f08cf439740 1 -- 172.21.15.22:0/2581364036 shutdown_connections 2024-04-20T21:33:56.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.438+0000 7f08cf439740 1 -- 172.21.15.22:0/2581364036 wait complete. 2024-04-20T21:33:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.438+0000 7f08cf439740 20 client.25310 trim_cache size 0 max 16384 2024-04-20T21:33:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08cf439740 1 Processor -- start 2024-04-20T21:33:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08cf439740 1 -- start start 2024-04-20T21:33:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08cf439740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4857f30 0x55def4851dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08cf439740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 0x55def48699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08cf439740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4852300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08cf439740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41adef0 con 0x55def4857f30 2024-04-20T21:33:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08cf439740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41b0770 con 0x55def4858920 2024-04-20T21:33:56.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08cf439740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41ae560 con 0x55def485da90 2024-04-20T21:33:56.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08c9200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 0x55def48699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08c9c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4857f30 0x55def4851dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08ca600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4852300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08c9200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 0x55def48699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60272/0 (socket says 172.21.15.22:60272) 2024-04-20T21:33:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.458+0000 7f08c9200640 1 -- 172.21.15.22:0/1745459270 learned_addr learned my addr 172.21.15.22:0/1745459270 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.459+0000 7f08c9200640 1 -- 172.21.15.22:0/1745459270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 msgr2=0x55def4852300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.459+0000 7f08c9200640 1 --2- 172.21.15.22:0/1745459270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4852300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.459+0000 7f08c9200640 1 -- 172.21.15.22:0/1745459270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4857f30 msgr2=0x55def4851dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.459+0000 7f08c9200640 1 --2- 172.21.15.22:0/1745459270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4857f30 0x55def4851dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.459+0000 7f08c9200640 1 -- 172.21.15.22:0/1745459270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55def4490170 con 0x55def4858920 2024-04-20T21:33:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.459+0000 7f08c9c00640 1 --2- 172.21.15.22:0/1745459270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4857f30 0x55def4851dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:56.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.459+0000 7f08c9200640 1 --2- 172.21.15.22:0/1745459270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 0x55def48699e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f08b00be860 tx=0x7f08b0003f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4439699119d8f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.459+0000 7f08b6000640 1 -- 172.21.15.22:0/1745459270 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b00c7590 con 0x55def4858920 2024-04-20T21:33:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.459+0000 7f08b6000640 1 -- 172.21.15.22:0/1745459270 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b00044a0 con 0x55def4858920 2024-04-20T21:33:56.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.459+0000 7f08b6000640 1 -- 172.21.15.22:0/1745459270 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b00a90e0 con 0x55def4858920 2024-04-20T21:33:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.460+0000 7f08cf439740 1 -- 172.21.15.22:0/1745459270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 msgr2=0x55def48699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.460+0000 7f08cf439740 1 --2- 172.21.15.22:0/1745459270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 0x55def48699e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f08b00be860 tx=0x7f08b0003f20 comp rx=0 tx=0).stop 2024-04-20T21:33:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.460+0000 7f08cf439740 1 -- 172.21.15.22:0/1745459270 shutdown_connections 2024-04-20T21:33:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.460+0000 7f08cf439740 1 --2- 172.21.15.22:0/1745459270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4852300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.460+0000 7f08cf439740 1 --2- 172.21.15.22:0/1745459270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4858920 0x55def48699e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.460+0000 7f08cf439740 1 --2- 172.21.15.22:0/1745459270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4857f30 0x55def4851dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.460+0000 7f08cf439740 1 -- 172.21.15.22:0/1745459270 >> 172.21.15.22:0/1745459270 conn(0x55def48534a0 msgr2=0x55def4859e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.460+0000 7f08cf439740 1 -- 172.21.15.22:0/1745459270 shutdown_connections 2024-04-20T21:33:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.460+0000 7f08cf439740 1 -- 172.21.15.22:0/1745459270 wait complete. 2024-04-20T21:33:56.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.461+0000 7f08cf439740 1 -- start start 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.461+0000 7f08cf439740 1 Processor -- start 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.461+0000 7f08cf439740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4857f30 0x55def4867550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.461+0000 7f08cf439740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4858920 0x55def4867a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.461+0000 7f08cf439740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4868a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.461+0000 7f08cf439740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41aec30 con 0x55def4858920 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.461+0000 7f08cf439740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41b00a0 con 0x55def4857f30 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.461+0000 7f08cf439740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08c41af300 con 0x55def485da90 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.462+0000 7f08cf439740 10 client.0 _ll_register_callbacks cb 0x7f08b00effc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.462+0000 7f08c9c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4857f30 0x55def4867550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.462+0000 7f08c9c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4857f30 0x55def4867550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60282/0 (socket says 172.21.15.22:60282) 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.462+0000 7f08c9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4858920 0x55def4867a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.462+0000 7f08c9c00640 1 -- 172.21.15.22:0/3286947653 learned_addr learned my addr 172.21.15.22:0/3286947653 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.462+0000 7f08ca600640 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4868a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.462+0000 7f08ca600640 1 -- 172.21.15.22:0/3286947653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4857f30 msgr2=0x55def4867550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.462+0000 7f08ca600640 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4857f30 0x55def4867550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.462+0000 7f08ca600640 1 -- 172.21.15.22:0/3286947653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4858920 msgr2=0x55def4867a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.462+0000 7f08ca600640 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4858920 0x55def4867a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.462+0000 7f08ca600640 1 -- 172.21.15.22:0/3286947653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55def4868ff0 con 0x55def485da90 2024-04-20T21:33:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.462+0000 7f08c9c00640 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4857f30 0x55def4867550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.463+0000 7f08ca600640 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4868a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009470 tx=0x7f08bc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f0d36239c872872 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.463+0000 7f08ab400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:33:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.463+0000 7f08ab400640 1 -- 172.21.15.22:0/3286947653 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc02f060 con 0x55def485da90 2024-04-20T21:33:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.463+0000 7f08cf439740 1 -- 172.21.15.22:0/3286947653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55def47c55b0 con 0x55def485da90 2024-04-20T21:33:56.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.463+0000 7f08ab400640 1 -- 172.21.15.22:0/3286947653 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f08bc020d40 con 0x55def485da90 2024-04-20T21:33:56.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.463+0000 7f08ab400640 1 -- 172.21.15.22:0/3286947653 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f08bc033430 con 0x55def485da90 2024-04-20T21:33:56.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.463+0000 7f08cf439740 10 client.25354 fetch_fsmap learned FSMap version 20 2024-04-20T21:33:56.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.463+0000 7f08cf439740 1 -- 172.21.15.22:0/3286947653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55def48d4db0 con 0x55def485da90 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.464+0000 7f08ab400640 1 -- 172.21.15.22:0/3286947653 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55def47c55b0 con 0x55def485da90 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.464+0000 7f08ab400640 1 -- 172.21.15.22:0/3286947653 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f08bc029070 con 0x55def485da90 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.464+0000 7f08cf439740 10 client.25354 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.464+0000 7f08cf439740 10 client.25354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.464+0000 7f08cf439740 20 client.25354 populate_metadata read hostname 'smithi022' 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.464+0000 7f08cf439740 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55def48d5790 0x55def48d7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.464+0000 7f08cf439740 4 client.25354 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.464+0000 7f08cf439740 1 -- 172.21.15.22:0/3286947653 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55def4741b10 con 0x55def48d5790 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.464+0000 7f08c9c00640 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55def48d5790 0x55def48d7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.465+0000 7f08c9c00640 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55def48d5790 0x55def48d7bb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee15d1fffd022d4 server_cookie=2a99f08b969799d2 in_seq=0 out_seq=0 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.465+0000 7f08ab400640 10 client.25354 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08ab400640 1 -- 172.21.15.22:0/3286947653 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55def4741b10 con 0x55def48d5790 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08ab400640 10 client.25354 handle_command_reply: tid=0 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08cf439740 1 client.25354 shutdown 2024-04-20T21:33:56.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08cf439740 1 -- 172.21.15.22:0/3286947653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 msgr2=0x55def4868a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08cf439740 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4868a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009470 tx=0x7f08bc004a40 comp rx=0 tx=0).stop 2024-04-20T21:33:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08cf439740 1 -- 172.21.15.22:0/3286947653 shutdown_connections 2024-04-20T21:33:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08cf439740 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55def485da90 0x55def4868a00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08cf439740 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55def4858920 0x55def4867a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08cf439740 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55def48d5790 0x55def48d7bb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08cf439740 1 --2- 172.21.15.22:0/3286947653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55def4857f30 0x55def4867550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08ab400640 0 client.25354 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08cf439740 1 -- 172.21.15.22:0/3286947653 >> 172.21.15.22:0/3286947653 conn(0x55def48534a0 msgr2=0x55def4851dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08cf439740 1 -- 172.21.15.22:0/3286947653 shutdown_connections 2024-04-20T21:33:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.466+0000 7f08cf439740 1 -- 172.21.15.22:0/3286947653 wait complete. 2024-04-20T21:33:56.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.467+0000 7f08cf439740 20 client.25354 trim_cache size 0 max 16384 2024-04-20T21:33:56.469 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:33:56.469 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:33:56.469 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "c1837e10-a234-483a-af7c-1bdc808e2840", 2024-04-20T21:33:56.469 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:33:56.469 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:33:56.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.467+0000 7f08cc600640 1 -- 172.21.15.22:0/322062040 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f08ac0a3e30 msgr2=0x7f08ac0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.467+0000 7f08cc600640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f08ac0a3e30 0x7f08ac0a62f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x55def4859180 tx=0x7f08bc009800 comp rx=0 tx=0).stop 2024-04-20T21:33:56.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.467+0000 7f08cc600640 1 -- 172.21.15.22:0/322062040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c4077b60 msgr2=0x7f08c419ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.467+0000 7f08cc600640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c4077b60 0x7f08c419ee00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f08b0005ab0 tx=0x7f08b002c6c0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.467+0000 7f08cc600640 1 -- 172.21.15.22:0/322062040 shutdown_connections 2024-04-20T21:33:56.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.467+0000 7f08cc600640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f08ac0a3e30 0x7f08ac0a62f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.467+0000 7f08cc600640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c4078500 0x7f08c41a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.467+0000 7f08cc600640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c4077b60 0x7f08c419ee00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.467+0000 7f08cc600640 1 --2- 172.21.15.22:0/322062040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c4076920 0x7f08c419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.467+0000 7f08cc600640 1 -- 172.21.15.22:0/322062040 >> 172.21.15.22:0/322062040 conn(0x7f08c40ff710 msgr2=0x7f08c4100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.468+0000 7f08cc600640 1 -- 172.21.15.22:0/322062040 shutdown_connections 2024-04-20T21:33:56.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.468+0000 7f08cc600640 1 -- 172.21.15.22:0/322062040 wait complete. 2024-04-20T21:33:56.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:56 smithi022 ceph-mon[16843]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:56.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:56 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi022 to 177.5M 2024-04-20T21:33:56.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:56 smithi022 ceph-mon[16843]: Unable to set osd_memory_target on smithi022 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T21:33:56.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:56 smithi022 ceph-mon[16843]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-20T21:33:56.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:56 smithi022 ceph-mon[16843]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-20T21:33:56.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:56 smithi022 ceph-mon[16843]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-20T21:33:56.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:56 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3995174087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:56.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:56 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3880469010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:56.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:33:56.486 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:33:56.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:56 smithi033 ceph-mon[21626]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:56.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:56 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi022 to 177.5M 2024-04-20T21:33:56.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:56 smithi033 ceph-mon[21626]: Unable to set osd_memory_target on smithi022 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T21:33:56.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:56 smithi033 ceph-mon[21626]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-20T21:33:56.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:56 smithi033 ceph-mon[21626]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-20T21:33:56.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:56 smithi033 ceph-mon[21626]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-20T21:33:56.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:56 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3995174087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:56.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:56 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3880469010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.665+0000 7fb373200640 1 -- 172.21.15.22:0/2628579843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb374105220 msgr2=0x7fb374107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.665+0000 7fb373200640 1 --2- 172.21.15.22:0/2628579843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb374105220 0x7fb374107610 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb360009900 tx=0x7fb36002bc50 comp rx=0 tx=0).stop 2024-04-20T21:33:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.665+0000 7fb373200640 1 -- 172.21.15.22:0/2628579843 shutdown_connections 2024-04-20T21:33:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.665+0000 7fb373200640 1 --2- 172.21.15.22:0/2628579843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb374107b50 0x7fb374109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.665+0000 7fb373200640 1 --2- 172.21.15.22:0/2628579843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb374105220 0x7fb374107610 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.665+0000 7fb373200640 1 --2- 172.21.15.22:0/2628579843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3741028f0 0x7fb374104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.665+0000 7fb373200640 1 -- 172.21.15.22:0/2628579843 >> 172.21.15.22:0/2628579843 conn(0x7fb3740fc300 msgr2=0x7fb3740fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.665+0000 7fb373200640 1 -- 172.21.15.22:0/2628579843 shutdown_connections 2024-04-20T21:33:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.665+0000 7fb373200640 1 -- 172.21.15.22:0/2628579843 wait complete. 2024-04-20T21:33:56.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.666+0000 7fb373200640 1 Processor -- start 2024-04-20T21:33:56.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.666+0000 7fb373200640 1 -- start start 2024-04-20T21:33:56.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.667+0000 7fb373200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3741028f0 0x7fb374102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.667+0000 7fb373200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb374105220 0x7fb3741008a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.667+0000 7fb373200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb374107b50 0x7fb374100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.667+0000 7fb373200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb37410db30 con 0x7fb374107b50 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.667+0000 7fb373200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb37410d9b0 con 0x7fb3741028f0 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.667+0000 7fb373200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb37410dcb0 con 0x7fb374105220 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.667+0000 7fb371400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb374105220 0x7fb3741008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.667+0000 7fb371e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3741028f0 0x7fb374102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.667+0000 7fb371400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb374105220 0x7fb3741008a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:35270/0 (socket says 172.21.15.22:35270) 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.667+0000 7fb371400640 1 -- 172.21.15.22:0/3894852822 learned_addr learned my addr 172.21.15.22:0/3894852822 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.667+0000 7fb372800640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb374107b50 0x7fb374100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.668+0000 7fb371e00640 1 -- 172.21.15.22:0/3894852822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb374105220 msgr2=0x7fb3741008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.668+0000 7fb371e00640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb374105220 0x7fb3741008a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.668+0000 7fb371e00640 1 -- 172.21.15.22:0/3894852822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb374107b50 msgr2=0x7fb374100de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.668+0000 7fb371e00640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb374107b50 0x7fb374100de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.668+0000 7fb371e00640 1 -- 172.21.15.22:0/3894852822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb374101320 con 0x7fb3741028f0 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.668+0000 7fb371400640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb374105220 0x7fb3741008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:56.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.668+0000 7fb371e00640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3741028f0 0x7fb374102260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb36400e9f0 tx=0x7fb36400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.668+0000 7fb36aa00640 1 -- 172.21.15.22:0/3894852822 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36400cd80 con 0x7fb3741028f0 2024-04-20T21:33:56.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.669+0000 7fb36aa00640 1 -- 172.21.15.22:0/3894852822 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3640035f0 con 0x7fb3741028f0 2024-04-20T21:33:56.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.669+0000 7fb36aa00640 1 -- 172.21.15.22:0/3894852822 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb364010690 con 0x7fb3741028f0 2024-04-20T21:33:56.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.669+0000 7fb373200640 1 -- 172.21.15.22:0/3894852822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb374101610 con 0x7fb3741028f0 2024-04-20T21:33:56.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.669+0000 7fb373200640 1 -- 172.21.15.22:0/3894852822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3741a6c80 con 0x7fb3741028f0 2024-04-20T21:33:56.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.671+0000 7f57eb200640 1 -- 172.21.15.22:0/2056010900 --> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f57ec062460 con 0x7f57d00a3e20 2024-04-20T21:33:56.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.673+0000 7fb37bdde740 1 Processor -- start 2024-04-20T21:33:56.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.673+0000 7fb37bdde740 1 -- start start 2024-04-20T21:33:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.673+0000 7fb37bdde740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 0x565532abc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.673+0000 7fb37bdde740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abcd90 0x565532ac1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.673+0000 7fb37bdde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532ac4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.673+0000 7fb37bdde740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565532a23150 con 0x565532abcd90 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.673+0000 7fb37bdde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5655329c9370 con 0x565532ac1f00 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.673+0000 7fb37bdde740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565532a1f110 con 0x565532abc3a0 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.673+0000 7fb36aa00640 1 -- 172.21.15.22:0/3894852822 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7fb364010830 con 0x7fb3741028f0 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.674+0000 7fb372800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 0x565532abc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.674+0000 7fb371400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532ac4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.674+0000 7fb371400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532ac4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60308/0 (socket says 172.21.15.22:60308) 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.674+0000 7fb371400640 1 -- 172.21.15.22:0/1285328861 learned_addr learned my addr 172.21.15.22:0/1285328861 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.674+0000 7fb372800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 0x565532abc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:35280/0 (socket says 172.21.15.22:35280) 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.674+0000 7fb371e00640 1 --2- 172.21.15.22:0/1285328861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abcd90 0x565532ac1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.674+0000 7fb372800640 1 -- 172.21.15.22:0/1285328861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 msgr2=0x565532ac4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.674+0000 7fb372800640 1 --2- 172.21.15.22:0/1285328861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532ac4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.674+0000 7fb372800640 1 -- 172.21.15.22:0/1285328861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abcd90 msgr2=0x565532ac1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.674+0000 7fb372800640 1 --2- 172.21.15.22:0/1285328861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abcd90 0x565532ac1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.674+0000 7fb372800640 1 -- 172.21.15.22:0/1285328861 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655327ff7d0 con 0x565532abc3a0 2024-04-20T21:33:56.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.675+0000 7fb372800640 1 --2- 172.21.15.22:0/1285328861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 0x565532abc7c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009f00 tx=0x7fb36c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b55b04fb7fdeea6a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.675+0000 7fb35f400640 1 -- 172.21.15.22:0/1285328861 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c003c30 con 0x565532abc3a0 2024-04-20T21:33:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.675+0000 7fb35f400640 1 -- 172.21.15.22:0/1285328861 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36c003dd0 con 0x565532abc3a0 2024-04-20T21:33:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.675+0000 7fb35f400640 1 -- 172.21.15.22:0/1285328861 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00fad0 con 0x565532abc3a0 2024-04-20T21:33:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.675+0000 7fb36aa00640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb3580a3e30 0x7fb3580a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.676+0000 7fb36aa00640 1 -- 172.21.15.22:0/3894852822 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fb3640d0bd0 con 0x7fb3741028f0 2024-04-20T21:33:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.676+0000 7fb371e00640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb3580a3e30 0x7fb3580a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.676+0000 7fb37bdde740 1 -- 172.21.15.22:0/1285328861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 msgr2=0x565532abc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.676+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1285328861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 0x565532abc7c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009f00 tx=0x7fb36c011810 comp rx=0 tx=0).stop 2024-04-20T21:33:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.677+0000 7fb371e00640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb3580a3e30 0x7fb3580a62f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3640ce2a0 tx=0x7fb364003aa0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.677+0000 7fb37bdde740 1 -- 172.21.15.22:0/1285328861 shutdown_connections 2024-04-20T21:33:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.677+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1285328861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532ac4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.677+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1285328861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abcd90 0x565532ac1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.677+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1285328861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 0x565532abc7c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.677+0000 7fb37bdde740 1 -- 172.21.15.22:0/1285328861 >> 172.21.15.22:0/1285328861 conn(0x565532ab7910 msgr2=0x565532ab7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.677+0000 7fb37bdde740 1 -- 172.21.15.22:0/1285328861 shutdown_connections 2024-04-20T21:33:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.677+0000 7fb37bdde740 1 -- 172.21.15.22:0/1285328861 wait complete. 2024-04-20T21:33:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.677+0000 7fb37bdde740 1 -- start start 2024-04-20T21:33:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.678+0000 7fb37bdde740 1 Processor -- start 2024-04-20T21:33:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.678+0000 7fb37bdde740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 0x565532ac68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.678+0000 7fb37bdde740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abcd90 0x565532ac6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.678+0000 7fb37bdde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532acde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.678+0000 7fb37bdde740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565532ac4eb0 con 0x565532abcd90 2024-04-20T21:33:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.678+0000 7fb37bdde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5655329dc740 con 0x565532ac1f00 2024-04-20T21:33:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.678+0000 7fb37bdde740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565532ac5030 con 0x565532abc3a0 2024-04-20T21:33:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.678+0000 7fb37bdde740 10 client.0 _ll_register_callbacks cb 0x5655329d7b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.678+0000 7fb372800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 0x565532ac68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.679+0000 7fb372800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 0x565532ac68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:35288/0 (socket says 172.21.15.22:35288) 2024-04-20T21:33:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.679+0000 7fb372800640 1 -- 172.21.15.22:0/1880474369 learned_addr learned my addr 172.21.15.22:0/1880474369 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.679+0000 7fb371400640 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abcd90 0x565532ac6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.679+0000 7fb372800640 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532acde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.679+0000 7fb371400640 1 -- 172.21.15.22:0/1880474369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 msgr2=0x565532ac68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.679+0000 7fb371400640 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 0x565532ac68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.679+0000 7fb371400640 1 -- 172.21.15.22:0/1880474369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 msgr2=0x565532acde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.679+0000 7fb371400640 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532acde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.679+0000 7fb371400640 1 -- 172.21.15.22:0/1880474369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655327353f0 con 0x565532abcd90 2024-04-20T21:33:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.680+0000 7fb371400640 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abcd90 0x565532ac6e30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb360005ab0 tx=0x7fb3600048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b71affa7bec687d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.680+0000 7fb353e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:33:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.680+0000 7fb353e00640 1 -- 172.21.15.22:0/1880474369 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360005d50 con 0x565532abcd90 2024-04-20T21:33:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.680+0000 7fb37bdde740 1 -- 172.21.15.22:0/1880474369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb36c003dd0 con 0x565532abcd90 2024-04-20T21:33:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.680+0000 7fb353e00640 1 -- 172.21.15.22:0/1880474369 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fb3600050f0 con 0x565532abcd90 2024-04-20T21:33:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.680+0000 7fb353e00640 1 -- 172.21.15.22:0/1880474369 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb36002cd50 con 0x565532abcd90 2024-04-20T21:33:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.680+0000 7fb37bdde740 10 client.15225 fetch_fsmap learned FSMap version 20 2024-04-20T21:33:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.680+0000 7fb37bdde740 1 -- 172.21.15.22:0/1880474369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5655326f50f0 con 0x565532abcd90 2024-04-20T21:33:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.681+0000 7fb353e00640 1 -- 172.21.15.22:0/1880474369 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36c003dd0 con 0x565532abcd90 2024-04-20T21:33:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.681+0000 7fb353e00640 1 -- 172.21.15.22:0/1880474369 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fb360046070 con 0x565532abcd90 2024-04-20T21:33:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.681+0000 7fb37bdde740 10 client.15225 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:33:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.681+0000 7fb37bdde740 10 client.15225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:33:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.681+0000 7fb37bdde740 20 client.15225 populate_metadata read hostname 'smithi022' 2024-04-20T21:33:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.681+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x565532ad26a0 0x565532ad4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.681+0000 7fb37bdde740 4 client.15225 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:33:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.681+0000 7fb37bdde740 1 -- 172.21.15.22:0/1880474369 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565532a30b70 con 0x565532ad26a0 2024-04-20T21:33:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.682+0000 7fb372800640 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x565532ad26a0 0x565532ad4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.682+0000 7fb372800640 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x565532ad26a0 0x565532ad4ad0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb756c7546754e89 server_cookie=8ebb631aa5e3931c in_seq=0 out_seq=0 2024-04-20T21:33:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.682+0000 7fb353e00640 10 client.15225 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:56.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.683+0000 7fb353e00640 1 -- 172.21.15.22:0/1880474369 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565532a30b70 con 0x565532ad26a0 2024-04-20T21:33:56.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.683+0000 7fb353e00640 10 client.15225 handle_command_reply: tid=0 2024-04-20T21:33:56.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.683+0000 7fb37bdde740 1 client.15225 shutdown 2024-04-20T21:33:56.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.683+0000 7fb37bdde740 1 -- 172.21.15.22:0/1880474369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abcd90 msgr2=0x565532ac6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.684+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abcd90 0x565532ac6e30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb360005ab0 tx=0x7fb3600048f0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.684+0000 7fb37bdde740 1 -- 172.21.15.22:0/1880474369 shutdown_connections 2024-04-20T21:33:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.684+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x565532ad26a0 0x565532ad4ad0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.684+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532acde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.684+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abcd90 0x565532ac6e30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.684+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1880474369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abc3a0 0x565532ac68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.684+0000 7fb353e00640 0 client.15225 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.684+0000 7fb37bdde740 1 -- 172.21.15.22:0/1880474369 >> 172.21.15.22:0/1880474369 conn(0x565532ab7910 msgr2=0x565532ab6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.684+0000 7fb37bdde740 1 -- 172.21.15.22:0/1880474369 shutdown_connections 2024-04-20T21:33:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.685+0000 7fb37bdde740 1 -- 172.21.15.22:0/1880474369 wait complete. 2024-04-20T21:33:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.685+0000 7fb37bdde740 20 client.15225 trim_cache size 0 max 16384 2024-04-20T21:33:56.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.712+0000 7fb37bdde740 1 Processor -- start 2024-04-20T21:33:56.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.713+0000 7fb37bdde740 1 -- start start 2024-04-20T21:33:56.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.713+0000 7fb37bdde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532abc3a0 0x565532ab6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.713+0000 7fb37bdde740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abcd90 0x565532acde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.713+0000 7fb37bdde740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532ac1f00 0x565532ab67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.713+0000 7fb37bdde740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741adc70 con 0x565532ac1f00 2024-04-20T21:33:56.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.713+0000 7fb37bdde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741b04f0 con 0x565532abc3a0 2024-04-20T21:33:56.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.713+0000 7fb37bdde740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741ae2e0 con 0x565532abcd90 2024-04-20T21:33:56.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.713+0000 7fb372800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532abc3a0 0x565532ab6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.713+0000 7fb372800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532abc3a0 0x565532ab6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60318/0 (socket says 172.21.15.22:60318) 2024-04-20T21:33:56.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.713+0000 7fb372800640 1 -- 172.21.15.22:0/1957847063 learned_addr learned my addr 172.21.15.22:0/1957847063 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:56.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.713+0000 7fb371400640 1 --2- 172.21.15.22:0/1957847063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abcd90 0x565532acde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.713+0000 7fb372800640 1 --2- 172.21.15.22:0/1957847063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532ac1f00 0x565532ab67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.714+0000 7fb371400640 1 -- 172.21.15.22:0/1957847063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532abc3a0 msgr2=0x565532ab6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.714+0000 7fb371400640 1 --2- 172.21.15.22:0/1957847063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532abc3a0 0x565532ab6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.714+0000 7fb371400640 1 -- 172.21.15.22:0/1957847063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532ac1f00 msgr2=0x565532ab67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.714+0000 7fb371400640 1 --2- 172.21.15.22:0/1957847063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532ac1f00 0x565532ab67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.714+0000 7fb371400640 1 -- 172.21.15.22:0/1957847063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565532ac15a0 con 0x565532abcd90 2024-04-20T21:33:56.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.714+0000 7fb371400640 1 --2- 172.21.15.22:0/1957847063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abcd90 0x565532acde30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb36002f020 tx=0x7fb3600045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c209b277106d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.714+0000 7fb35ea00640 1 -- 172.21.15.22:0/1957847063 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360055030 con 0x565532abcd90 2024-04-20T21:33:56.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.715+0000 7fb35ea00640 1 -- 172.21.15.22:0/1957847063 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36002cd50 con 0x565532abcd90 2024-04-20T21:33:56.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.715+0000 7fb35ea00640 1 -- 172.21.15.22:0/1957847063 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360000e60 con 0x565532abcd90 2024-04-20T21:33:56.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.715+0000 7fb37bdde740 1 -- 172.21.15.22:0/1957847063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abcd90 msgr2=0x565532acde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.715+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1957847063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abcd90 0x565532acde30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb36002f020 tx=0x7fb3600045d0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.716+0000 7fb37bdde740 1 -- 172.21.15.22:0/1957847063 shutdown_connections 2024-04-20T21:33:56.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.716+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1957847063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532ac1f00 0x565532ab67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.716+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1957847063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532abcd90 0x565532acde30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.716+0000 7fb37bdde740 1 --2- 172.21.15.22:0/1957847063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532abc3a0 0x565532ab6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.716+0000 7fb37bdde740 1 -- 172.21.15.22:0/1957847063 >> 172.21.15.22:0/1957847063 conn(0x565532ab7910 msgr2=0x565532ac68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.716+0000 7fb37bdde740 1 -- 172.21.15.22:0/1957847063 shutdown_connections 2024-04-20T21:33:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.716+0000 7fb37bdde740 1 -- 172.21.15.22:0/1957847063 wait complete. 2024-04-20T21:33:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.716+0000 7fb37bdde740 1 -- start start 2024-04-20T21:33:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.716+0000 7fb37bdde740 1 Processor -- start 2024-04-20T21:33:56.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.717+0000 7fb37bdde740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abc3a0 0x565532acd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.717+0000 7fb37bdde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532acdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.717+0000 7fb37bdde740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532acbe00 0x565532acc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.717+0000 7fb37bdde740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741ae9b0 con 0x565532abc3a0 2024-04-20T21:33:56.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.717+0000 7fb37bdde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741afe20 con 0x565532ac1f00 2024-04-20T21:33:56.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.717+0000 7fb37bdde740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741af080 con 0x565532acbe00 2024-04-20T21:33:56.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.717+0000 7fb37bdde740 10 client.0 _ll_register_callbacks cb 0x7fb360009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:56.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.717+0000 7fb372800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532acbe00 0x565532acc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.717+0000 7fb372800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532acbe00 0x565532acc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:35298/0 (socket says 172.21.15.22:35298) 2024-04-20T21:33:56.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.717+0000 7fb372800640 1 -- 172.21.15.22:0/3344656953 learned_addr learned my addr 172.21.15.22:0/3344656953 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.717+0000 7fb372800640 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abc3a0 0x565532acd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.718+0000 7fb371400640 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532acdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.718+0000 7fb372800640 1 -- 172.21.15.22:0/3344656953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532acbe00 msgr2=0x565532acc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.718+0000 7fb372800640 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532acbe00 0x565532acc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.718+0000 7fb372800640 1 -- 172.21.15.22:0/3344656953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 msgr2=0x565532acdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.718+0000 7fb372800640 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532acdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.718+0000 7fb372800640 1 -- 172.21.15.22:0/3344656953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565532acc9c0 con 0x565532abc3a0 2024-04-20T21:33:56.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.718+0000 7fb372800640 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abc3a0 0x565532acd7f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb36c000f80 tx=0x7fb36c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94379bd4ccd36e26 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.718+0000 7fb353e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:33:56.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.719+0000 7fb353e00640 1 -- 172.21.15.22:0/3344656953 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c01f040 con 0x565532abc3a0 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.719+0000 7fb353e00640 1 -- 172.21.15.22:0/3344656953 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fb36c00f540 con 0x565532abc3a0 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.719+0000 7fb37bdde740 1 -- 172.21.15.22:0/3344656953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb36002cd50 con 0x565532abc3a0 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.719+0000 7fb353e00640 1 -- 172.21.15.22:0/3344656953 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb36c00f740 con 0x565532abc3a0 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.719+0000 7fb37bdde740 10 client.15231 fetch_fsmap learned FSMap version 20 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.719+0000 7fb37bdde740 1 -- 172.21.15.22:0/3344656953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x565532b4aae0 con 0x565532abc3a0 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.720+0000 7fb353e00640 1 -- 172.21.15.22:0/3344656953 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36002cd50 con 0x565532abc3a0 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.720+0000 7fb353e00640 1 -- 172.21.15.22:0/3344656953 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fb36c018070 con 0x565532abc3a0 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.720+0000 7fb37bdde740 10 client.15231 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.720+0000 7fb37bdde740 10 client.15231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.720+0000 7fb37bdde740 20 client.15231 populate_metadata read hostname 'smithi022' 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.720+0000 7fb37bdde740 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x565532b4b300 0x565532b4d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.720+0000 7fb37bdde740 4 client.15231 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.720+0000 7fb37bdde740 1 -- 172.21.15.22:0/3344656953 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565532ac66e0 con 0x565532b4b300 2024-04-20T21:33:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.720+0000 7fb371400640 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x565532b4b300 0x565532b4d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.721+0000 7fb371400640 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x565532b4b300 0x565532b4d720 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e5cd30806e8247 server_cookie=a9ba87658336cf21 in_seq=0 out_seq=0 2024-04-20T21:33:56.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.721+0000 7fb353e00640 10 client.15231 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:56.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb353e00640 1 -- 172.21.15.22:0/3344656953 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x565532ac66e0 con 0x565532b4b300 2024-04-20T21:33:56.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb353e00640 10 client.15231 handle_command_reply: tid=0 2024-04-20T21:33:56.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb37bdde740 1 client.15231 shutdown 2024-04-20T21:33:56.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb37bdde740 1 -- 172.21.15.22:0/3344656953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abc3a0 msgr2=0x565532acd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb37bdde740 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abc3a0 0x565532acd7f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb36c000f80 tx=0x7fb36c009500 comp rx=0 tx=0).stop 2024-04-20T21:33:56.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb37bdde740 1 -- 172.21.15.22:0/3344656953 shutdown_connections 2024-04-20T21:33:56.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb37bdde740 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565532acbe00 0x565532acc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb37bdde740 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x565532b4b300 0x565532b4d720 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb37bdde740 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565532ac1f00 0x565532acdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb37bdde740 1 --2- 172.21.15.22:0/3344656953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565532abc3a0 0x565532acd7f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb37bdde740 1 -- 172.21.15.22:0/3344656953 >> 172.21.15.22:0/3344656953 conn(0x565532ab7910 msgr2=0x565532abab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb353e00640 0 client.15231 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:33:56.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb37bdde740 1 -- 172.21.15.22:0/3344656953 shutdown_connections 2024-04-20T21:33:56.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.722+0000 7fb37bdde740 1 -- 172.21.15.22:0/3344656953 wait complete. 2024-04-20T21:33:56.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.723+0000 7fb37bdde740 20 client.15231 trim_cache size 0 max 16384 2024-04-20T21:33:56.723 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:33:56.723 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:33:56.723 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:33:56.723 INFO:teuthology.orchestra.run.smithi022.stdout: "c1837e10-a234-483a-af7c-1bdc808e2840": { 2024-04-20T21:33:56.724 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:33:56.724 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "c1837e10-a234-483a-af7c-1bdc808e2840", 2024-04-20T21:33:56.724 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:33:56.724 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:33:56.724 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:33:56.724 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:33:56.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.723+0000 7fb373200640 1 -- 172.21.15.22:0/3894852822 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb3580a3e30 msgr2=0x7fb3580a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.723+0000 7fb373200640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb3580a3e30 0x7fb3580a62f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3640ce2a0 tx=0x7fb364003aa0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.723+0000 7fb373200640 1 -- 172.21.15.22:0/3894852822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3741028f0 msgr2=0x7fb374102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.723+0000 7fb373200640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3741028f0 0x7fb374102260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb36400e9f0 tx=0x7fb36400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.724+0000 7fb373200640 1 -- 172.21.15.22:0/3894852822 shutdown_connections 2024-04-20T21:33:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.724+0000 7fb373200640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb3580a3e30 0x7fb3580a62f0 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3640ce2a0 tx=0x7fb364003aa0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.724+0000 7fb373200640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb374107b50 0x7fb374100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.724+0000 7fb373200640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb374105220 0x7fb3741008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.724+0000 7fb373200640 1 --2- 172.21.15.22:0/3894852822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3741028f0 0x7fb374102260 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb36400e9f0 tx=0x7fb36400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.724+0000 7fb373200640 1 -- 172.21.15.22:0/3894852822 >> 172.21.15.22:0/3894852822 conn(0x7fb3740fc300 msgr2=0x7fb374103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.724+0000 7fb373200640 1 -- 172.21.15.22:0/3894852822 shutdown_connections 2024-04-20T21:33:56.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.724+0000 7fb373200640 1 -- 172.21.15.22:0/3894852822 wait complete. 2024-04-20T21:33:56.745 INFO:tasks.cephfs.filesystem:scrub status for tag:c1837e10-a234-483a-af7c-1bdc808e2840 - {'path': '/', 'tag': 'c1837e10-a234-483a-af7c-1bdc808e2840', 'options': 'recursive,force'} 2024-04-20T21:33:56.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.752+0000 7f57e2a00640 1 -- 172.21.15.22:0/2056010900 <== mgr.15024 v2:172.21.15.22:6800/975472489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f57ec062460 con 0x7f57d00a3e20 2024-04-20T21:33:56.751 INFO:teuthology.orchestra.run.smithi022.stdout:Schedule set for path / 2024-04-20T21:33:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.754+0000 7f57eb200640 1 -- 172.21.15.22:0/2056010900 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57d00a3e20 msgr2=0x7f57d00a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.754+0000 7f57eb200640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57d00a3e20 0x7f57d00a62e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f57d802c240 tx=0x7f57d802df90 comp rx=0 tx=0).stop 2024-04-20T21:33:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.754+0000 7f57eb200640 1 -- 172.21.15.22:0/2056010900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57ec100650 msgr2=0x7f57ec19e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.754+0000 7f57eb200640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57ec100650 0x7f57ec19e6a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00e9f0 tx=0x7f57dc00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.754+0000 7f57eb200640 1 -- 172.21.15.22:0/2056010900 shutdown_connections 2024-04-20T21:33:56.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.754+0000 7f57eb200640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57d00a3e20 0x7f57d00a62e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.754+0000 7f57eb200640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57ec105970 0x7f57ec1a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.754+0000 7f57eb200640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57ec102fc0 0x7f57ec19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.754+0000 7f57eb200640 1 --2- 172.21.15.22:0/2056010900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57ec100650 0x7f57ec19e6a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.754+0000 7f57eb200640 1 -- 172.21.15.22:0/2056010900 >> 172.21.15.22:0/2056010900 conn(0x7f57ec0fc250 msgr2=0x7f57ec100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.754+0000 7f57eb200640 1 -- 172.21.15.22:0/2056010900 shutdown_connections 2024-04-20T21:33:56.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.754+0000 7f57eb200640 1 -- 172.21.15.22:0/2056010900 wait complete. 2024-04-20T21:33:56.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-20T21:33:56.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:56 smithi122 ceph-mon[22147]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:56 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi022 to 177.5M 2024-04-20T21:33:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:56 smithi122 ceph-mon[22147]: Unable to set osd_memory_target on smithi022 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T21:33:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:56 smithi122 ceph-mon[22147]: Updating smithi022:/etc/ceph/ceph.conf 2024-04-20T21:33:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:56 smithi122 ceph-mon[22147]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-20T21:33:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:56 smithi122 ceph-mon[22147]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-20T21:33:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:56 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3995174087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:56.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:56 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3880469010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:56.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.991+0000 7f2f25e00640 1 -- 172.21.15.22:0/1792571416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f20103b30 msgr2=0x7f2f2010c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.991+0000 7f2f25e00640 1 --2- 172.21.15.22:0/1792571416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f20103b30 0x7f2f2010c010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009900 tx=0x7f2f0c02bcd0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.991+0000 7f2f25e00640 1 -- 172.21.15.22:0/1792571416 shutdown_connections 2024-04-20T21:33:56.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.991+0000 7f2f25e00640 1 --2- 172.21.15.22:0/1792571416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f2010c550 0x7f2f2010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.991+0000 7f2f25e00640 1 --2- 172.21.15.22:0/1792571416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f20103b30 0x7f2f2010c010 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.991+0000 7f2f25e00640 1 --2- 172.21.15.22:0/1792571416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f20103210 0x7f2f201035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.991+0000 7f2f25e00640 1 -- 172.21.15.22:0/1792571416 >> 172.21.15.22:0/1792571416 conn(0x7f2f20077fe0 msgr2=0x7f2f200783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:56.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.991+0000 7f2f25e00640 1 -- 172.21.15.22:0/1792571416 shutdown_connections 2024-04-20T21:33:56.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.991+0000 7f2f25e00640 1 -- 172.21.15.22:0/1792571416 wait complete. 2024-04-20T21:33:56.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.992+0000 7f2f25e00640 1 Processor -- start 2024-04-20T21:33:56.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.992+0000 7f2f25e00640 1 -- start start 2024-04-20T21:33:56.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.992+0000 7f2f25e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f20103210 0x7f2f201a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.992+0000 7f2f25e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f20103b30 0x7f2f201a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.992+0000 7f2f25e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f2010c550 0x7f2f2019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.992+0000 7f2f25e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20112330 con 0x7f2f20103210 2024-04-20T21:33:56.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.992+0000 7f2f25e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201121b0 con 0x7f2f20103b30 2024-04-20T21:33:56.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.992+0000 7f2f25e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201124b0 con 0x7f2f2010c550 2024-04-20T21:33:56.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.993+0000 7f2f1ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f20103210 0x7f2f201a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.993+0000 7f2f1f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f2010c550 0x7f2f2019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.993+0000 7f2f1e000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f20103b30 0x7f2f201a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.993+0000 7f2f1ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f20103210 0x7f2f201a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59468/0 (socket says 172.21.15.22:59468) 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.993+0000 7f2f1ea00640 1 -- 172.21.15.22:0/3085868928 learned_addr learned my addr 172.21.15.22:0/3085868928 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.993+0000 7f2f1e000640 1 -- 172.21.15.22:0/3085868928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f2010c550 msgr2=0x7f2f2019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.993+0000 7f2f1e000640 1 --2- 172.21.15.22:0/3085868928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f2010c550 0x7f2f2019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.993+0000 7f2f1e000640 1 -- 172.21.15.22:0/3085868928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f20103210 msgr2=0x7f2f201a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.993+0000 7f2f1e000640 1 --2- 172.21.15.22:0/3085868928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f20103210 0x7f2f201a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.993+0000 7f2f1e000640 1 -- 172.21.15.22:0/3085868928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2019b010 con 0x7f2f20103b30 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.993+0000 7f2f1e000640 1 --2- 172.21.15.22:0/3085868928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f20103b30 0x7f2f201a0c10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c02c210 tx=0x7f2f0c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.994+0000 7f2f24a00640 1 -- 172.21.15.22:0/3085868928 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c042020 con 0x7f2f20103b30 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.994+0000 7f2f25e00640 1 -- 172.21.15.22:0/3085868928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2019b2a0 con 0x7f2f20103b30 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.994+0000 7f2f25e00640 1 -- 172.21.15.22:0/3085868928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f201a7520 con 0x7f2f20103b30 2024-04-20T21:33:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.994+0000 7f2f24a00640 1 -- 172.21.15.22:0/3085868928 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f0c02ce00 con 0x7f2f20103b30 2024-04-20T21:33:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.994+0000 7f2f24a00640 1 -- 172.21.15.22:0/3085868928 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c003a30 con 0x7f2f20103b30 2024-04-20T21:33:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.996+0000 7f2f25e00640 1 -- 172.21.15.22:0/3085868928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f00005290 con 0x7f2f20103b30 2024-04-20T21:33:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.997+0000 7f2f24a00640 1 -- 172.21.15.22:0/3085868928 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f2f0c045020 con 0x7f2f20103b30 2024-04-20T21:33:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.999+0000 7f2f24a00640 1 --2- 172.21.15.22:0/3085868928 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2f040a3e60 0x7f2f040a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.999+0000 7f2f24a00640 1 -- 172.21.15.22:0/3085868928 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f2f0c0f1c20 con 0x7f2f20103b30 2024-04-20T21:33:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:56.999+0000 7f2f1ea00640 1 --2- 172.21.15.22:0/3085868928 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2f040a3e60 0x7f2f040a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.000+0000 7f2f1ea00640 1 --2- 172.21.15.22:0/3085868928 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2f040a3e60 0x7f2f040a6320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009c90 tx=0x7f2f10005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:57.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.003+0000 7f2f24a00640 1 -- 172.21.15.22:0/3085868928 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2f0c04c030 con 0x7f2f20103b30 2024-04-20T21:33:57.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.233+0000 7f2f25e00640 1 -- 172.21.15.22:0/3085868928 --> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f2f00000c60 con 0x7f2f040a3e60 2024-04-20T21:33:57.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.320+0000 7f2f24a00640 1 -- 172.21.15.22:0/3085868928 <== mgr.15024 v2:172.21.15.22:6800/975472489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f2f00000c60 con 0x7f2f040a3e60 2024-04-20T21:33:57.319 INFO:teuthology.orchestra.run.smithi022.stdout:Retention added to path / 2024-04-20T21:33:57.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.322+0000 7f2f25e00640 1 -- 172.21.15.22:0/3085868928 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2f040a3e60 msgr2=0x7f2f040a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:57.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.322+0000 7f2f25e00640 1 --2- 172.21.15.22:0/3085868928 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2f040a3e60 0x7f2f040a6320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009c90 tx=0x7f2f10005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:57.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.322+0000 7f2f25e00640 1 -- 172.21.15.22:0/3085868928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f20103b30 msgr2=0x7f2f201a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:57.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.322+0000 7f2f25e00640 1 --2- 172.21.15.22:0/3085868928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f20103b30 0x7f2f201a0c10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c02c210 tx=0x7f2f0c005e80 comp rx=0 tx=0).stop 2024-04-20T21:33:57.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.322+0000 7f2f25e00640 1 -- 172.21.15.22:0/3085868928 shutdown_connections 2024-04-20T21:33:57.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.322+0000 7f2f25e00640 1 --2- 172.21.15.22:0/3085868928 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2f040a3e60 0x7f2f040a6320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.322+0000 7f2f25e00640 1 --2- 172.21.15.22:0/3085868928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f2010c550 0x7f2f2019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.322+0000 7f2f25e00640 1 --2- 172.21.15.22:0/3085868928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f20103b30 0x7f2f201a0c10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.322+0000 7f2f25e00640 1 --2- 172.21.15.22:0/3085868928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f20103210 0x7f2f201a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.322+0000 7f2f25e00640 1 -- 172.21.15.22:0/3085868928 >> 172.21.15.22:0/3085868928 conn(0x7f2f20077fe0 msgr2=0x7f2f20108420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:57.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.322+0000 7f2f25e00640 1 -- 172.21.15.22:0/3085868928 shutdown_connections 2024-04-20T21:33:57.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.322+0000 7f2f25e00640 1 -- 172.21.15.22:0/3085868928 wait complete. 2024-04-20T21:33:57.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-20T21:33:57.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.507+0000 7f5dcb600640 1 -- 172.21.15.22:0/2772496654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dc406a790 msgr2=0x7f5dc406abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:57.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.507+0000 7f5dcb600640 1 --2- 172.21.15.22:0/2772496654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dc406a790 0x7f5dc406abf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009900 tx=0x7f5db002bcd0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.507+0000 7f5dcb600640 1 -- 172.21.15.22:0/2772496654 shutdown_connections 2024-04-20T21:33:57.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.507+0000 7f5dcb600640 1 --2- 172.21.15.22:0/2772496654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc406b130 0x7f5dc410ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.507+0000 7f5dcb600640 1 --2- 172.21.15.22:0/2772496654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dc406a790 0x7f5dc406abf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.507+0000 7f5dcb600640 1 --2- 172.21.15.22:0/2772496654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dc41080f0 0x7f5dc41084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.507+0000 7f5dcb600640 1 -- 172.21.15.22:0/2772496654 >> 172.21.15.22:0/2772496654 conn(0x7f5dc4076c60 msgr2=0x7f5dc4077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:57.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.507+0000 7f5dcb600640 1 -- 172.21.15.22:0/2772496654 shutdown_connections 2024-04-20T21:33:57.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.507+0000 7f5dcb600640 1 -- 172.21.15.22:0/2772496654 wait complete. 2024-04-20T21:33:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.508+0000 7f5dcb600640 1 Processor -- start 2024-04-20T21:33:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.508+0000 7f5dcb600640 1 -- start start 2024-04-20T21:33:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.508+0000 7f5dcb600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc406a790 0x7f5dc41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dcb600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dc406b130 0x7f5dc41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dcb600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dc41080f0 0x7f5dc419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dcb600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc4112330 con 0x7f5dc41080f0 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dcb600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc41121b0 con 0x7f5dc406a790 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dcb600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc41124b0 con 0x7f5dc406b130 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dc8c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc406a790 0x7f5dc41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dc8c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc406a790 0x7f5dc41a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60364/0 (socket says 172.21.15.22:60364) 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dc8c00640 1 -- 172.21.15.22:0/3141892469 learned_addr learned my addr 172.21.15.22:0/3141892469 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dc9600640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dc41080f0 0x7f5dc419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dc3e00640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dc406b130 0x7f5dc41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dc8c00640 1 -- 172.21.15.22:0/3141892469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dc406b130 msgr2=0x7f5dc41a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dc8c00640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dc406b130 0x7f5dc41a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dc8c00640 1 -- 172.21.15.22:0/3141892469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dc41080f0 msgr2=0x7f5dc419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dc8c00640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dc41080f0 0x7f5dc419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.509+0000 7f5dc8c00640 1 -- 172.21.15.22:0/3141892469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc419b010 con 0x7f5dc406a790 2024-04-20T21:33:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.510+0000 7f5dc8c00640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc406a790 0x7f5dc41a0690 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009d50 tx=0x7f5db800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:57.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.510+0000 7f5dc1600640 1 -- 172.21.15.22:0/3141892469 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8003b70 con 0x7f5dc406a790 2024-04-20T21:33:57.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.510+0000 7f5dcb600640 1 -- 172.21.15.22:0/3141892469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc419b300 con 0x7f5dc406a790 2024-04-20T21:33:57.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.510+0000 7f5dc1600640 1 -- 172.21.15.22:0/3141892469 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db8003d10 con 0x7f5dc406a790 2024-04-20T21:33:57.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.510+0000 7f5dc1600640 1 -- 172.21.15.22:0/3141892469 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8010a20 con 0x7f5dc406a790 2024-04-20T21:33:57.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.510+0000 7f5dcb600640 1 -- 172.21.15.22:0/3141892469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc41a7610 con 0x7f5dc406a790 2024-04-20T21:33:57.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.511+0000 7f5dcb600640 1 -- 172.21.15.22:0/3141892469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc4069fe0 con 0x7f5dc406a790 2024-04-20T21:33:57.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.513+0000 7f5dc1600640 1 -- 172.21.15.22:0/3141892469 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f5db8003ec0 con 0x7f5dc406a790 2024-04-20T21:33:57.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.515+0000 7f5dc1600640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5dac0a3e60 0x7f5dac0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:57.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.515+0000 7f5dc1600640 1 -- 172.21.15.22:0/3141892469 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5db80d10b0 con 0x7f5dc406a790 2024-04-20T21:33:57.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.515+0000 7f5dc3e00640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5dac0a3e60 0x7f5dac0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.516+0000 7f5dc3e00640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5dac0a3e60 0x7f5dac0a6320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5db0005ab0 tx=0x7f5db0005a20 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.519+0000 7f5dc1600640 1 -- 172.21.15.22:0/3141892469 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5db809d560 con 0x7f5dc406a790 2024-04-20T21:33:57.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: Updating smithi033:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:57.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: Updating smithi122:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:57.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:57.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:57.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: Updating smithi022:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:57.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: Updating smithi033:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:57.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: Updating smithi122:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:57.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:57.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:57.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:57.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:57.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:33:57.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:33:57.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:33:57.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:57 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:33:57.659 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: Updating smithi033:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:57.659 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: Updating smithi122:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: Updating smithi022:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: Updating smithi033:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: Updating smithi122:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:33:57.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:57 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:33:57.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.753+0000 7f5dcb600640 1 -- 172.21.15.22:0/3141892469 --> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f5dc410d810 con 0x7f5dac0a3e60 2024-04-20T21:33:57.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.763+0000 7f5dc1600640 1 -- 172.21.15.22:0/3141892469 <== mgr.15024 v2:172.21.15.22:6800/975472489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f5dc410d810 con 0x7f5dac0a3e60 2024-04-20T21:33:57.763 INFO:teuthology.orchestra.run.smithi022.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-20T00:00:00", "created": "2024-04-20T21:33:56", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-20T21:33:57.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.766+0000 7f5dcb600640 1 -- 172.21.15.22:0/3141892469 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5dac0a3e60 msgr2=0x7f5dac0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.766+0000 7f5dcb600640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5dac0a3e60 0x7f5dac0a6320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5db0005ab0 tx=0x7f5db0005a20 comp rx=0 tx=0).stop 2024-04-20T21:33:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.766+0000 7f5dcb600640 1 -- 172.21.15.22:0/3141892469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc406a790 msgr2=0x7f5dc41a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.766+0000 7f5dcb600640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc406a790 0x7f5dc41a0690 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009d50 tx=0x7f5db800c6a0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.766+0000 7f5dcb600640 1 -- 172.21.15.22:0/3141892469 shutdown_connections 2024-04-20T21:33:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.766+0000 7f5dcb600640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5dac0a3e60 0x7f5dac0a6320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.766+0000 7f5dcb600640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dc41080f0 0x7f5dc419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.766+0000 7f5dcb600640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dc406b130 0x7f5dc41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.766+0000 7f5dcb600640 1 --2- 172.21.15.22:0/3141892469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc406a790 0x7f5dc41a0690 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.766+0000 7f5dcb600640 1 -- 172.21.15.22:0/3141892469 >> 172.21.15.22:0/3141892469 conn(0x7f5dc4076c60 msgr2=0x7f5dc410d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.767+0000 7f5dcb600640 1 -- 172.21.15.22:0/3141892469 shutdown_connections 2024-04-20T21:33:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.767+0000 7f5dcb600640 1 -- 172.21.15.22:0/3141892469 wait complete. 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: Updating smithi033:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: Updating smithi122:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: Updating smithi022:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: Updating smithi033:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: Updating smithi122:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:33:57.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:33:57.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:33:57.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:57 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:33:57.785 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-20T21:33:57.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.992+0000 7f5db1c00640 1 -- 172.21.15.22:0/1105659370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dac1092e0 msgr2=0x7f5dac110eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:57.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.992+0000 7f5db1c00640 1 --2- 172.21.15.22:0/1105659370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dac1092e0 0x7f5dac110eb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009be0 tx=0x7f5da402bef0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.992+0000 7f5db1c00640 1 -- 172.21.15.22:0/1105659370 shutdown_connections 2024-04-20T21:33:57.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.992+0000 7f5db1c00640 1 --2- 172.21.15.22:0/1105659370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dac1092e0 0x7f5dac110eb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.992+0000 7f5db1c00640 1 --2- 172.21.15.22:0/1105659370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac104bc0 0x7f5dac108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.992+0000 7f5db1c00640 1 --2- 172.21.15.22:0/1105659370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dac104210 0x7f5dac1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.992+0000 7f5db1c00640 1 -- 172.21.15.22:0/1105659370 >> 172.21.15.22:0/1105659370 conn(0x7f5dac0ffa70 msgr2=0x7f5dac101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:57.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.992+0000 7f5db1c00640 1 -- 172.21.15.22:0/1105659370 shutdown_connections 2024-04-20T21:33:57.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.992+0000 7f5db1c00640 1 -- 172.21.15.22:0/1105659370 wait complete. 2024-04-20T21:33:57.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.993+0000 7f5db1c00640 1 Processor -- start 2024-04-20T21:33:57.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.993+0000 7f5db1c00640 1 -- start start 2024-04-20T21:33:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.994+0000 7f5db1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dac104210 0x7f5dac1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.994+0000 7f5db1c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dac1092e0 0x7f5dac1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.994+0000 7f5db1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac19a690 0x7f5dac19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.994+0000 7f5db1c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac1143f0 con 0x7f5dac19a690 2024-04-20T21:33:57.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.994+0000 7f5db1c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac114270 con 0x7f5dac1092e0 2024-04-20T21:33:57.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.994+0000 7f5db1c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac114570 con 0x7f5dac104210 2024-04-20T21:33:57.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.994+0000 7f5dab400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dac1092e0 0x7f5dac1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:57.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.994+0000 7f5dab400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dac1092e0 0x7f5dac1a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60394/0 (socket says 172.21.15.22:60394) 2024-04-20T21:33:57.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.994+0000 7f5dab400640 1 -- 172.21.15.22:0/17705952 learned_addr learned my addr 172.21.15.22:0/17705952 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:33:57.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.994+0000 7f5db1200640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac19a690 0x7f5dac19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:57.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.994+0000 7f5dabe00640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dac104210 0x7f5dac1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:57.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.995+0000 7f5dab400640 1 -- 172.21.15.22:0/17705952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dac104210 msgr2=0x7f5dac1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:57.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.995+0000 7f5dab400640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dac104210 0x7f5dac1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.995+0000 7f5dab400640 1 -- 172.21.15.22:0/17705952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac19a690 msgr2=0x7f5dac19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:57.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.995+0000 7f5dab400640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac19a690 0x7f5dac19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:57.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.995+0000 7f5dab400640 1 -- 172.21.15.22:0/17705952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac19b290 con 0x7f5dac1092e0 2024-04-20T21:33:57.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.995+0000 7f5dab400640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dac1092e0 0x7f5dac1a0b10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800e9a0 tx=0x7f5d9800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:57.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.995+0000 7f5da8c00640 1 -- 172.21.15.22:0/17705952 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9800cd80 con 0x7f5dac1092e0 2024-04-20T21:33:57.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.995+0000 7f5da8c00640 1 -- 172.21.15.22:0/17705952 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d980035f0 con 0x7f5dac1092e0 2024-04-20T21:33:57.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.995+0000 7f5db1c00640 1 -- 172.21.15.22:0/17705952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dac1a7370 con 0x7f5dac1092e0 2024-04-20T21:33:57.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.995+0000 7f5da8c00640 1 -- 172.21.15.22:0/17705952 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98010670 con 0x7f5dac1092e0 2024-04-20T21:33:57.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.996+0000 7f5db1c00640 1 -- 172.21.15.22:0/17705952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dac1a77d0 con 0x7f5dac1092e0 2024-04-20T21:33:57.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.997+0000 7f5db1c00640 1 -- 172.21.15.22:0/17705952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dac1056e0 con 0x7f5dac1092e0 2024-04-20T21:33:57.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:57.999+0000 7f5da8c00640 1 -- 172.21.15.22:0/17705952 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f5d980037a0 con 0x7f5dac1092e0 2024-04-20T21:33:58.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.001+0000 7f5da8c00640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5d900a3e20 0x7f5d900a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:58.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.001+0000 7f5da8c00640 1 -- 172.21.15.22:0/17705952 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5d980d0c50 con 0x7f5dac1092e0 2024-04-20T21:33:58.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.001+0000 7f5dabe00640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5d900a3e20 0x7f5d900a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:58.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.002+0000 7f5dabe00640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5d900a3e20 0x7f5d900a62e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009bd0 tx=0x7f5d9c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:58.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.007+0000 7f5da8c00640 1 -- 172.21.15.22:0/17705952 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5d9809d190 con 0x7f5dac1092e0 2024-04-20T21:33:58.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.239+0000 7f5db1c00640 1 -- 172.21.15.22:0/17705952 --> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5dac104d00 con 0x7f5d900a3e20 2024-04-20T21:33:58.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.251+0000 7f5da8c00640 1 -- 172.21.15.22:0/17705952 <== mgr.15024 v2:172.21.15.22:6800/975472489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5dac104d00 con 0x7f5d900a3e20 2024-04-20T21:33:58.250 INFO:teuthology.orchestra.run.smithi022.stdout:/ 1m 6m3h 2024-04-20T21:33:58.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.254+0000 7f5db1c00640 1 -- 172.21.15.22:0/17705952 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5d900a3e20 msgr2=0x7f5d900a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:58.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.254+0000 7f5db1c00640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5d900a3e20 0x7f5d900a62e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009bd0 tx=0x7f5d9c005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:58.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.254+0000 7f5db1c00640 1 -- 172.21.15.22:0/17705952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dac1092e0 msgr2=0x7f5dac1a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:58.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.254+0000 7f5db1c00640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dac1092e0 0x7f5dac1a0b10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800e9a0 tx=0x7f5d9800ee60 comp rx=0 tx=0).stop 2024-04-20T21:33:58.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.254+0000 7f5db1c00640 1 -- 172.21.15.22:0/17705952 shutdown_connections 2024-04-20T21:33:58.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.254+0000 7f5db1c00640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5d900a3e20 0x7f5d900a62e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:58.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.254+0000 7f5db1c00640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dac19a690 0x7f5dac19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:58.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.254+0000 7f5db1c00640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dac1092e0 0x7f5dac1a0b10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:58.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.254+0000 7f5db1c00640 1 --2- 172.21.15.22:0/17705952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dac104210 0x7f5dac1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:58.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.254+0000 7f5db1c00640 1 -- 172.21.15.22:0/17705952 >> 172.21.15.22:0/17705952 conn(0x7f5dac0ffa70 msgr2=0x7f5dac1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:58.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.254+0000 7f5db1c00640 1 -- 172.21.15.22:0/17705952 shutdown_connections 2024-04-20T21:33:58.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:33:58.254+0000 7f5db1c00640 1 -- 172.21.15.22:0/17705952 wait complete. 2024-04-20T21:33:58.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-20T21:33:58.366 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-20T21:33:58.377 INFO:teuthology.run_tasks:Running task workunit... 2024-04-20T21:33:58.390 INFO:tasks.workunit:Pulling workunits from ref a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:33:58.390 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-20T21:33:58.390 INFO:tasks.workunit:timeout=3h 2024-04-20T21:33:58.390 INFO:tasks.workunit:cleanup=True 2024-04-20T21:33:58.390 DEBUG:teuthology.orchestra.run.smithi022:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:58.446 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-20T21:33:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-20T21:33:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:33:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-20T21:33:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-04-20 21:31:34.041382087 +0000 2024-04-20T21:33:58.446 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-04-20 21:31:34.041382087 +0000 2024-04-20T21:33:58.447 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-04-20 21:33:47.127110851 +0000 2024-04-20T21:33:58.447 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-04-20 21:31:34.041382087 +0000 2024-04-20T21:33:58.447 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:58.447 DEBUG:teuthology.orchestra.run.smithi022:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-20T21:33:58.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:58 smithi022 ceph-mon[16843]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:58.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:58 smithi022 ceph-mon[16843]: from='client.25336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:58.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:58 smithi022 ceph-mon[16843]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-20T21:33:58.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:58 smithi022 ceph-mon[16843]: Updating smithi022:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:58.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:58 smithi022 ceph-mon[16843]: from='client.25340 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:58.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:58 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:58.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:58 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:58.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:58 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:58.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:58 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:33:58.529 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:33:58.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:58 smithi033 ceph-mon[21626]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:58.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:58 smithi033 ceph-mon[21626]: from='client.25336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:58.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:58 smithi033 ceph-mon[21626]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-20T21:33:58.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:58 smithi033 ceph-mon[21626]: Updating smithi022:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:58.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:58 smithi033 ceph-mon[21626]: from='client.25340 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:58.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:58 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:58.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:58 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:58.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:58 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:58.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:58 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:33:58.575 INFO:tasks.workunit.client.0.smithi022.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-20T21:33:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:58 smithi122 ceph-mon[22147]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:58 smithi122 ceph-mon[22147]: from='client.25336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:58 smithi122 ceph-mon[22147]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-20T21:33:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:58 smithi122 ceph-mon[22147]: Updating smithi022:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:58 smithi122 ceph-mon[22147]: from='client.25340 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:58 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:58 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:58 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:58.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:58 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:33:59.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:59 smithi033 ceph-mon[21626]: from='client.25346 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:59.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:59 smithi033 ceph-mon[21626]: from='client.25352 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:59.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:59 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:59.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:59 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:59.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:59 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:33:59.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:59 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:33:59.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:33:59 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:59.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:59 smithi022 ceph-mon[16843]: from='client.25346 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:59.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:59 smithi022 ceph-mon[16843]: from='client.25352 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:59.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:59 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:59.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:59 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:59.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:59 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:33:59.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:59 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:33:59.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:33:59 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:59.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:59 smithi122 ceph-mon[22147]: from='client.25346 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:59.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:59 smithi122 ceph-mon[22147]: from='client.25352 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:59.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:59 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:59.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:59 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:33:59.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:59 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:33:59.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:59 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:33:59.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:33:59 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:34:00.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:00 smithi033 ceph-mon[21626]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-20T21:34:00.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:00 smithi022 ceph-mon[16843]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-20T21:34:00.657 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:34:00 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:34:00.348+0000 7f235fc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:34:00.657 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:34:00 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:34:00.348+0000 7f235fc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:34:00.657 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:34:00 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:34:00.348+0000 7f235fc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:34:00.657 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:34:00 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:34:00.348+0000 7f235fc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:34:00.657 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:34:00 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:34:00.348+0000 7f235fc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:34:00.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:00 smithi122 ceph-mon[22147]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-20T21:34:01.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:01 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:34:01.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:01 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:34:01.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:01 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:34:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:02 smithi033 ceph-mon[21626]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 47 op/s 2024-04-20T21:34:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:02 smithi033 ceph-mon[21626]: mgrmap e26: x(active, since 12s), standbys: z, y 2024-04-20T21:34:02.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:02 smithi022 ceph-mon[16843]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 47 op/s 2024-04-20T21:34:02.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:02 smithi022 ceph-mon[16843]: mgrmap e26: x(active, since 12s), standbys: z, y 2024-04-20T21:34:02.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:02 smithi122 ceph-mon[22147]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 47 op/s 2024-04-20T21:34:02.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:02 smithi122 ceph-mon[22147]: mgrmap e26: x(active, since 12s), standbys: z, y 2024-04-20T21:34:04.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:04 smithi022 ceph-mon[16843]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-20T21:34:04.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:04 smithi122 ceph-mon[22147]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-20T21:34:04.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:04 smithi033 ceph-mon[21626]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-20T21:34:06.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:06 smithi022 ceph-mon[16843]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-20T21:34:06.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:06 smithi122 ceph-mon[22147]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-20T21:34:06.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:06 smithi033 ceph-mon[21626]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-20T21:34:08.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:08 smithi022 ceph-mon[16843]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-04-20T21:34:08.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:08 smithi122 ceph-mon[22147]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-04-20T21:34:08.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:08 smithi033 ceph-mon[21626]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-04-20T21:34:10.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:10 smithi022 ceph-mon[16843]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 99 KiB/s wr, 48 op/s 2024-04-20T21:34:10.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:10 smithi122 ceph-mon[22147]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 99 KiB/s wr, 48 op/s 2024-04-20T21:34:10.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:10 smithi033 ceph-mon[21626]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 99 KiB/s wr, 48 op/s 2024-04-20T21:34:12.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:12 smithi022 ceph-mon[16843]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 100 KiB/s wr, 49 op/s 2024-04-20T21:34:12.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:12 smithi122 ceph-mon[22147]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 100 KiB/s wr, 49 op/s 2024-04-20T21:34:12.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:12 smithi033 ceph-mon[21626]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 100 KiB/s wr, 49 op/s 2024-04-20T21:34:14.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:14 smithi022 ceph-mon[16843]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-20T21:34:14.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:14 smithi122 ceph-mon[22147]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-20T21:34:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:14 smithi033 ceph-mon[21626]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-20T21:34:16.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:16 smithi022 ceph-mon[16843]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-20T21:34:16.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:16 smithi122 ceph-mon[22147]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-20T21:34:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:16 smithi033 ceph-mon[21626]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-20T21:34:17.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:17 smithi022 ceph-mon[16843]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-20T21:34:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:17 smithi122 ceph-mon[22147]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-20T21:34:17.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:17 smithi033 ceph-mon[21626]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-20T21:34:20.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:20 smithi022 ceph-mon[16843]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-20T21:34:20.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:20 smithi122 ceph-mon[22147]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-20T21:34:20.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:20 smithi033 ceph-mon[21626]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-20T21:34:22.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:22 smithi022 ceph-mon[16843]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-20T21:34:22.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:22 smithi122 ceph-mon[22147]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-20T21:34:22.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:22 smithi033 ceph-mon[21626]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-20T21:34:24.355 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:24 smithi033 ceph-mon[21626]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:24.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:24 smithi022 ceph-mon[16843]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:24.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:24 smithi122 ceph-mon[22147]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:26.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:26 smithi022 ceph-mon[16843]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:26.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:26 smithi122 ceph-mon[22147]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:26.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:26 smithi033 ceph-mon[21626]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:26.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:34:26.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:34:26.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.980+0000 7fbabe000640 1 -- 172.21.15.22:0/243130163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab81061b0 msgr2=0x7fbab8106640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:26.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.980+0000 7fbabe000640 1 --2- 172.21.15.22:0/243130163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab81061b0 0x7fbab8106640 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009ca0 tx=0x7fbab002c1e0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.980+0000 7fbabe000640 1 -- 172.21.15.22:0/243130163 shutdown_connections 2024-04-20T21:34:26.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.980+0000 7fbabe000640 1 --2- 172.21.15.22:0/243130163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab81061b0 0x7fbab8106640 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.980+0000 7fbabe000640 1 --2- 172.21.15.22:0/243130163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab81016d0 0x7fbab8105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.980+0000 7fbabe000640 1 --2- 172.21.15.22:0/243130163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbab8100db0 0x7fbab8101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.980+0000 7fbabe000640 1 -- 172.21.15.22:0/243130163 >> 172.21.15.22:0/243130163 conn(0x7fbab80fc7e0 msgr2=0x7fbab80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:26.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.981+0000 7fbabe000640 1 -- 172.21.15.22:0/243130163 shutdown_connections 2024-04-20T21:34:26.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.981+0000 7fbabe000640 1 -- 172.21.15.22:0/243130163 wait complete. 2024-04-20T21:34:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbabe000640 1 Processor -- start 2024-04-20T21:34:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbabe000640 1 -- start start 2024-04-20T21:34:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbabe000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbab8100db0 0x7fbab81a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbabe000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab81016d0 0x7fbab81a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbabe000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab81061b0 0x7fbab819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbabe000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81187d0 con 0x7fbab8100db0 2024-04-20T21:34:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbabe000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8118650 con 0x7fbab81061b0 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbabe000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8118950 con 0x7fbab81016d0 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbabcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbab8100db0 0x7fbab81a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbabcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbab8100db0 0x7fbab81a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37858/0 (socket says 172.21.15.22:37858) 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbabcc00640 1 -- 172.21.15.22:0/2847728167 learned_addr learned my addr 172.21.15.22:0/2847728167 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbab7e00640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab81016d0 0x7fbab81a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.982+0000 7fbabd600640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab81061b0 0x7fbab819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.983+0000 7fbabcc00640 1 -- 172.21.15.22:0/2847728167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab81016d0 msgr2=0x7fbab81a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.983+0000 7fbabcc00640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab81016d0 0x7fbab81a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.983+0000 7fbabcc00640 1 -- 172.21.15.22:0/2847728167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab81061b0 msgr2=0x7fbab819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.983+0000 7fbabcc00640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab81061b0 0x7fbab819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.983+0000 7fbabcc00640 1 -- 172.21.15.22:0/2847728167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab819f270 con 0x7fbab8100db0 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.983+0000 7fbab7e00640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab81016d0 0x7fbab81a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.983+0000 7fbabcc00640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbab8100db0 0x7fbab81a4940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00e9f0 tx=0x7fbaac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.984+0000 7fbab5600640 1 -- 172.21.15.22:0/2847728167 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac00cd80 con 0x7fbab8100db0 2024-04-20T21:34:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.984+0000 7fbabe000640 1 -- 172.21.15.22:0/2847728167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab819f560 con 0x7fbab8100db0 2024-04-20T21:34:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.984+0000 7fbab5600640 1 -- 172.21.15.22:0/2847728167 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaac010910 con 0x7fbab8100db0 2024-04-20T21:34:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.984+0000 7fbab5600640 1 -- 172.21.15.22:0/2847728167 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0188f0 con 0x7fbab8100db0 2024-04-20T21:34:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.984+0000 7fbabe000640 1 -- 172.21.15.22:0/2847728167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab81ab800 con 0x7fbab8100db0 2024-04-20T21:34:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.987+0000 7fbac2213740 1 Processor -- start 2024-04-20T21:34:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.987+0000 7fbac2213740 1 -- start start 2024-04-20T21:34:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.987+0000 7fbab5600640 1 -- 172.21.15.22:0/2847728167 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7fbaac010ac0 con 0x7fbab8100db0 2024-04-20T21:34:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.987+0000 7fbac2213740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.987+0000 7fbac2213740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630ebca0 0x5647630f4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.987+0000 7fbac2213740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630f4880 0x5647630f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbac2213740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564763051150 con 0x5647630ebca0 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbac2213740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564762ff7370 con 0x5647630f4880 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbac2213740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56476304d110 con 0x5647630eb380 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbabcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630ebca0 0x5647630f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbabd600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbabcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630ebca0 0x5647630f4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37862/0 (socket says 172.21.15.22:37862) 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbab7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630f4880 0x5647630f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbabcc00640 1 -- 172.21.15.22:0/1137600325 learned_addr learned my addr 172.21.15.22:0/1137600325 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbabd600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630eb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36678/0 (socket says 172.21.15.22:36678) 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbab7e00640 1 -- 172.21.15.22:0/1137600325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 msgr2=0x5647630eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbab7e00640 1 --2- 172.21.15.22:0/1137600325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbab7e00640 1 -- 172.21.15.22:0/1137600325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630ebca0 msgr2=0x5647630f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbab7e00640 1 --2- 172.21.15.22:0/1137600325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630ebca0 0x5647630f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.988+0000 7fbab7e00640 1 -- 172.21.15.22:0/1137600325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564762e2d7d0 con 0x5647630f4880 2024-04-20T21:34:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.989+0000 7fbab7e00640 1 --2- 172.21.15.22:0/1137600325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630f4880 0x5647630f6c70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbab819fc30 tx=0x7fbaa40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b9bff3e8ecb91d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.989+0000 7fbaa9c00640 1 -- 172.21.15.22:0/1137600325 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4004310 con 0x5647630f4880 2024-04-20T21:34:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.989+0000 7fbaa9c00640 1 -- 172.21.15.22:0/1137600325 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaa40044b0 con 0x5647630f4880 2024-04-20T21:34:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.989+0000 7fbaa9c00640 1 -- 172.21.15.22:0/1137600325 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa40035f0 con 0x5647630f4880 2024-04-20T21:34:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.989+0000 7fbab5600640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbaa00a3e40 0x7fbaa00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.990+0000 7fbab5600640 1 -- 172.21.15.22:0/2847728167 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fbaac0d0e60 con 0x7fbab8100db0 2024-04-20T21:34:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.990+0000 7fbabd600640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbaa00a3e40 0x7fbaa00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.990+0000 7fbac2213740 1 -- 172.21.15.22:0/1137600325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630f4880 msgr2=0x5647630f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.990+0000 7fbac2213740 1 --2- 172.21.15.22:0/1137600325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630f4880 0x5647630f6c70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbab819fc30 tx=0x7fbaa40118d0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.990+0000 7fbac2213740 1 -- 172.21.15.22:0/1137600325 shutdown_connections 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.990+0000 7fbac2213740 1 --2- 172.21.15.22:0/1137600325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630f4880 0x5647630f6c70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.990+0000 7fbac2213740 1 --2- 172.21.15.22:0/1137600325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630ebca0 0x5647630f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.990+0000 7fbac2213740 1 --2- 172.21.15.22:0/1137600325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.990+0000 7fbac2213740 1 -- 172.21.15.22:0/1137600325 >> 172.21.15.22:0/1137600325 conn(0x5647630e5ab0 msgr2=0x5647630e5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.990+0000 7fbac2213740 1 -- 172.21.15.22:0/1137600325 shutdown_connections 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.991+0000 7fbac2213740 1 -- 172.21.15.22:0/1137600325 wait complete. 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.991+0000 7fbabd600640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbaa00a3e40 0x7fbaa00a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005e30 tx=0x7fbab0005d80 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.991+0000 7fbac2213740 1 -- start start 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.991+0000 7fbac2213740 1 Processor -- start 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbac2213740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbac2213740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 0x5647630f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbac2213740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630f4880 0x5647630f2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbac2213740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5647630f7670 con 0x5647630f4880 2024-04-20T21:34:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbac2213740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56476300a740 con 0x5647630ebca0 2024-04-20T21:34:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbac2213740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5647630f77f0 con 0x5647630eb380 2024-04-20T21:34:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbac2213740 10 client.0 _ll_register_callbacks cb 0x564763005b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbabd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630f4880 0x5647630f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbab7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 0x5647630f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbab7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 0x5647630f1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33654/0 (socket says 172.21.15.22:33654) 2024-04-20T21:34:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbabd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630f4880 0x5647630f2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37868/0 (socket says 172.21.15.22:37868) 2024-04-20T21:34:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbab7e00640 1 -- 172.21.15.22:0/1266132460 learned_addr learned my addr 172.21.15.22:0/1266132460 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.992+0000 7fbabcc00640 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:26.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.993+0000 7fbab7e00640 1 -- 172.21.15.22:0/1266132460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 msgr2=0x5647630f17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:26.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.993+0000 7fbab7e00640 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630f17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.993+0000 7fbab7e00640 1 -- 172.21.15.22:0/1266132460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630f4880 msgr2=0x5647630f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:26.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.993+0000 7fbab7e00640 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630f4880 0x5647630f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.993+0000 7fbab7e00640 1 -- 172.21.15.22:0/1266132460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564762d633f0 con 0x5647630ebca0 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.993+0000 7fbabcc00640 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630f17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.993+0000 7fbabd600640 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630f4880 0x5647630f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.993+0000 7fbab7e00640 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 0x5647630f1d20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4004a10 tx=0x7fbaa40043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=718ec8d0e40abc24 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.993+0000 7fba9e000640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.993+0000 7fba9e000640 1 -- 172.21.15.22:0/1266132460 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4029070 con 0x5647630ebca0 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.993+0000 7fbac2213740 1 -- 172.21.15.22:0/1266132460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaa40044b0 con 0x5647630ebca0 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.993+0000 7fba9e000640 1 -- 172.21.15.22:0/1266132460 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fbaa4003a80 con 0x5647630ebca0 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.994+0000 7fba9e000640 1 -- 172.21.15.22:0/1266132460 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbaa4035050 con 0x5647630ebca0 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.994+0000 7fbac2213740 10 client.25367 fetch_fsmap learned FSMap version 20 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.994+0000 7fbac2213740 1 -- 172.21.15.22:0/1266132460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564762d230f0 con 0x5647630ebca0 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.994+0000 7fba9e000640 1 -- 172.21.15.22:0/1266132460 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaa40044b0 con 0x5647630ebca0 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.994+0000 7fba9e000640 1 -- 172.21.15.22:0/1266132460 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fbaa400f480 con 0x5647630ebca0 2024-04-20T21:34:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.994+0000 7fbac2213740 10 client.25367 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.994+0000 7fbac2213740 10 client.25367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.994+0000 7fbac2213740 20 client.25367 populate_metadata read hostname 'smithi022' 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.995+0000 7fbac2213740 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5647631049f0 0x564763106de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.995+0000 7fbac2213740 4 client.25367 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.995+0000 7fbac2213740 1 -- 172.21.15.22:0/1266132460 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56476305eb70 con 0x5647631049f0 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.995+0000 7fbabcc00640 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5647631049f0 0x564763106de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.995+0000 7fbabcc00640 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5647631049f0 0x564763106de0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c107434e038db509 server_cookie=819f0dd95c125c3a in_seq=0 out_seq=0 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.995+0000 7fba9e000640 10 client.25367 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.996+0000 7fba9e000640 1 -- 172.21.15.22:0/1266132460 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56476305eb70 con 0x5647631049f0 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.996+0000 7fba9e000640 10 client.25367 handle_command_reply: tid=0 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.996+0000 7fbac2213740 1 client.25367 shutdown 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.997+0000 7fbac2213740 1 -- 172.21.15.22:0/1266132460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 msgr2=0x5647630f1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.997+0000 7fbac2213740 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 0x5647630f1d20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4004a10 tx=0x7fbaa40043c0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.997+0000 7fbac2213740 1 -- 172.21.15.22:0/1266132460 shutdown_connections 2024-04-20T21:34:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.997+0000 7fbac2213740 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630f4880 0x5647630f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.997+0000 7fbac2213740 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5647631049f0 0x564763106de0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.997+0000 7fbac2213740 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 0x5647630f1d20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.997+0000 7fbac2213740 1 --2- 172.21.15.22:0/1266132460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630f17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.997+0000 7fbac2213740 1 -- 172.21.15.22:0/1266132460 >> 172.21.15.22:0/1266132460 conn(0x5647630e5ab0 msgr2=0x5647630e7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.997+0000 7fba9e000640 0 client.25367 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.997+0000 7fbac2213740 1 -- 172.21.15.22:0/1266132460 shutdown_connections 2024-04-20T21:34:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.997+0000 7fbac2213740 1 -- 172.21.15.22:0/1266132460 wait complete. 2024-04-20T21:34:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:26.997+0000 7fbac2213740 20 client.25367 trim_cache size 0 max 16384 2024-04-20T21:34:27.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.016+0000 7fbac2213740 1 Processor -- start 2024-04-20T21:34:27.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.017+0000 7fbac2213740 1 -- start start 2024-04-20T21:34:27.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.017+0000 7fbac2213740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.017+0000 7fbac2213740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 0x564763100560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.017+0000 7fbac2213740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630f4880 0x5647630f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.017+0000 7fbac2213740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81b2260 con 0x5647630f4880 2024-04-20T21:34:27.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.017+0000 7fbac2213740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81b4ae0 con 0x5647630ebca0 2024-04-20T21:34:27.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.017+0000 7fbac2213740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81b28d0 con 0x5647630eb380 2024-04-20T21:34:27.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.017+0000 7fbabcc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.017+0000 7fbabd600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630f4880 0x5647630f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.017+0000 7fbabcc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630f3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36704/0 (socket says 172.21.15.22:36704) 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.017+0000 7fbabcc00640 1 -- 172.21.15.22:0/392352109 learned_addr learned my addr 172.21.15.22:0/392352109 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.017+0000 7fbab7e00640 1 --2- 172.21.15.22:0/392352109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 0x564763100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.018+0000 7fbabcc00640 1 -- 172.21.15.22:0/392352109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 msgr2=0x564763100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.018+0000 7fbabcc00640 1 --2- 172.21.15.22:0/392352109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 0x564763100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.018+0000 7fbabcc00640 1 -- 172.21.15.22:0/392352109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630f4880 msgr2=0x5647630f1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.018+0000 7fbabcc00640 1 --2- 172.21.15.22:0/392352109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630f4880 0x5647630f1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.018+0000 7fbabcc00640 1 -- 172.21.15.22:0/392352109 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564763107410 con 0x5647630eb380 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.018+0000 7fbab7e00640 1 --2- 172.21.15.22:0/392352109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 0x564763100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.018+0000 7fbabcc00640 1 --2- 172.21.15.22:0/392352109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630f3f10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00c6a0 tx=0x7fbaac0d3370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3733d7257c5268 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.018+0000 7fbaa9200640 1 -- 172.21.15.22:0/392352109 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac09cc40 con 0x5647630eb380 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.018+0000 7fbaa9200640 1 -- 172.21.15.22:0/392352109 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaac087040 con 0x5647630eb380 2024-04-20T21:34:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.018+0000 7fbaa9200640 1 -- 172.21.15.22:0/392352109 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0ce950 con 0x5647630eb380 2024-04-20T21:34:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.019+0000 7fbac2213740 1 -- 172.21.15.22:0/392352109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 msgr2=0x5647630f3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.019+0000 7fbac2213740 1 --2- 172.21.15.22:0/392352109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630f3f10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00c6a0 tx=0x7fbaac0d3370 comp rx=0 tx=0).stop 2024-04-20T21:34:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.019+0000 7fbac2213740 1 -- 172.21.15.22:0/392352109 shutdown_connections 2024-04-20T21:34:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.019+0000 7fbac2213740 1 --2- 172.21.15.22:0/392352109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630f4880 0x5647630f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.019+0000 7fbac2213740 1 --2- 172.21.15.22:0/392352109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630ebca0 0x564763100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.019+0000 7fbac2213740 1 --2- 172.21.15.22:0/392352109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630eb380 0x5647630f3f10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.019+0000 7fbac2213740 1 -- 172.21.15.22:0/392352109 >> 172.21.15.22:0/392352109 conn(0x5647630e5ab0 msgr2=0x5647630f17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.019+0000 7fbac2213740 1 -- 172.21.15.22:0/392352109 shutdown_connections 2024-04-20T21:34:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.019+0000 7fbac2213740 1 -- 172.21.15.22:0/392352109 wait complete. 2024-04-20T21:34:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.020+0000 7fbac2213740 1 -- start start 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.020+0000 7fbac2213740 1 Processor -- start 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.020+0000 7fbac2213740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630eb380 0x5647630fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.020+0000 7fbac2213740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630ebca0 0x5647630fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbac2213740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630f4880 0x5647630f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbac2213740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81b2fa0 con 0x5647630eb380 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbac2213740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81b4410 con 0x5647630f4880 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbac2213740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81b3670 con 0x5647630ebca0 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbac2213740 10 client.0 _ll_register_callbacks cb 0x7fbaac0d4840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbabcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630eb380 0x5647630fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbabcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630eb380 0x5647630fc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37886/0 (socket says 172.21.15.22:37886) 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbabcc00640 1 -- 172.21.15.22:0/480252841 learned_addr learned my addr 172.21.15.22:0/480252841 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbabd600640 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630f4880 0x5647630f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbab7e00640 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630ebca0 0x5647630fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbabcc00640 1 -- 172.21.15.22:0/480252841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630ebca0 msgr2=0x5647630fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbabcc00640 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630ebca0 0x5647630fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbabcc00640 1 -- 172.21.15.22:0/480252841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630f4880 msgr2=0x5647630f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbabcc00640 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630f4880 0x5647630f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.021+0000 7fbabcc00640 1 -- 172.21.15.22:0/480252841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647630e58e0 con 0x5647630eb380 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.022+0000 7fbabcc00640 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630eb380 0x5647630fc6d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbaac09cbd0 tx=0x7fbaac0d48f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a7e745c5e46fa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.022+0000 7fba9e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.022+0000 7fba9e000640 1 -- 172.21.15.22:0/480252841 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0b6020 con 0x5647630eb380 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.022+0000 7fba9e000640 1 -- 172.21.15.22:0/480252841 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fbaac0d4040 con 0x5647630eb380 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.022+0000 7fbac2213740 1 -- 172.21.15.22:0/480252841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaac087040 con 0x5647630eb380 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.022+0000 7fba9e000640 1 -- 172.21.15.22:0/480252841 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbaac0a0f40 con 0x5647630eb380 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.022+0000 7fbac2213740 10 client.15273 fetch_fsmap learned FSMap version 20 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.022+0000 7fbac2213740 1 -- 172.21.15.22:0/480252841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56476316a6e0 con 0x5647630eb380 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.022+0000 7fba9e000640 1 -- 172.21.15.22:0/480252841 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaac087040 con 0x5647630eb380 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.023+0000 7fba9e000640 1 -- 172.21.15.22:0/480252841 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fbaac0cece0 con 0x5647630eb380 2024-04-20T21:34:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.023+0000 7fbac2213740 10 client.15273 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:34:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.023+0000 7fbac2213740 10 client.15273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.023+0000 7fbac2213740 20 client.15273 populate_metadata read hostname 'smithi022' 2024-04-20T21:34:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.023+0000 7fbac2213740 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56476316ad60 0x56476316d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.023+0000 7fbac2213740 4 client.15273 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:34:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.023+0000 7fbac2213740 1 -- 172.21.15.22:0/480252841 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5647630589e0 con 0x56476316ad60 2024-04-20T21:34:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.023+0000 7fbab7e00640 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56476316ad60 0x56476316d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fbab7e00640 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56476316ad60 0x56476316d150 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e6350f0db22828d server_cookie=e4b968f4fe5444c in_seq=0 out_seq=0 2024-04-20T21:34:27.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fba9e000640 10 client.15273 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:27.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fba9e000640 1 -- 172.21.15.22:0/480252841 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5647630589e0 con 0x56476316ad60 2024-04-20T21:34:27.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fba9e000640 10 client.15273 handle_command_reply: tid=0 2024-04-20T21:34:27.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fbac2213740 1 client.15273 shutdown 2024-04-20T21:34:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fbac2213740 1 -- 172.21.15.22:0/480252841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630eb380 msgr2=0x5647630fc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fbac2213740 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630eb380 0x5647630fc6d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbaac09cbd0 tx=0x7fbaac0d48f0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fbac2213740 1 -- 172.21.15.22:0/480252841 shutdown_connections 2024-04-20T21:34:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fbac2213740 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56476316ad60 0x56476316d150 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fbac2213740 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647630f4880 0x5647630f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fbac2213740 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647630ebca0 0x5647630fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fbac2213740 1 --2- 172.21.15.22:0/480252841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5647630eb380 0x5647630fc6d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fba9e000640 0 client.15273 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fbac2213740 1 -- 172.21.15.22:0/480252841 >> 172.21.15.22:0/480252841 conn(0x5647630e5ab0 msgr2=0x5647630e76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fbac2213740 1 -- 172.21.15.22:0/480252841 shutdown_connections 2024-04-20T21:34:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.024+0000 7fbac2213740 1 -- 172.21.15.22:0/480252841 wait complete. 2024-04-20T21:34:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.025+0000 7fbac2213740 20 client.15273 trim_cache size 0 max 16384 2024-04-20T21:34:27.028 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:34:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:34:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:34:27.028 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:34:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.025+0000 7fbabe000640 1 -- 172.21.15.22:0/2847728167 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbaa00a3e40 msgr2=0x7fbaa00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.025+0000 7fbabe000640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbaa00a3e40 0x7fbaa00a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005e30 tx=0x7fbab0005d80 comp rx=0 tx=0).stop 2024-04-20T21:34:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.025+0000 7fbabe000640 1 -- 172.21.15.22:0/2847728167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbab8100db0 msgr2=0x7fbab81a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.025+0000 7fbabe000640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbab8100db0 0x7fbab81a4940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00e9f0 tx=0x7fbaac00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.025+0000 7fbabe000640 1 -- 172.21.15.22:0/2847728167 shutdown_connections 2024-04-20T21:34:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.025+0000 7fbabe000640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbaa00a3e40 0x7fbaa00a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.025+0000 7fbabe000640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab81061b0 0x7fbab819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.025+0000 7fbabe000640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab81016d0 0x7fbab81a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.025+0000 7fbabe000640 1 --2- 172.21.15.22:0/2847728167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbab8100db0 0x7fbab81a4940 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.025+0000 7fbabe000640 1 -- 172.21.15.22:0/2847728167 >> 172.21.15.22:0/2847728167 conn(0x7fbab80fc7e0 msgr2=0x7fbab80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.026+0000 7fbabe000640 1 -- 172.21.15.22:0/2847728167 shutdown_connections 2024-04-20T21:34:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.026+0000 7fbabe000640 1 -- 172.21.15.22:0/2847728167 wait complete. 2024-04-20T21:34:27.043 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:34:27.043 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:27.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.253+0000 7fc2ba000640 1 -- 172.21.15.22:0/200183281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2b41041f0 msgr2=0x7fc2b41045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.253+0000 7fc2ba000640 1 --2- 172.21.15.22:0/200183281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2b41041f0 0x7fc2b41045d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009860 tx=0x7fc29c02bb70 comp rx=0 tx=0).stop 2024-04-20T21:34:27.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.253+0000 7fc2ba000640 1 -- 172.21.15.22:0/200183281 shutdown_connections 2024-04-20T21:34:27.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.253+0000 7fc2ba000640 1 --2- 172.21.15.22:0/200183281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b41092c0 0x7fc2b4110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.253+0000 7fc2ba000640 1 --2- 172.21.15.22:0/200183281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b4104ba0 0x7fc2b4108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.253+0000 7fc2ba000640 1 --2- 172.21.15.22:0/200183281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2b41041f0 0x7fc2b41045d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.253+0000 7fc2ba000640 1 -- 172.21.15.22:0/200183281 >> 172.21.15.22:0/200183281 conn(0x7fc2b40ffa30 msgr2=0x7fc2b4101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:27.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.253+0000 7fc2ba000640 1 -- 172.21.15.22:0/200183281 shutdown_connections 2024-04-20T21:34:27.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.254+0000 7fc2ba000640 1 -- 172.21.15.22:0/200183281 wait complete. 2024-04-20T21:34:27.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.254+0000 7fc2ba000640 1 Processor -- start 2024-04-20T21:34:27.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2ba000640 1 -- start start 2024-04-20T21:34:27.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2ba000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2b41041f0 0x7fc2b41a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2ba000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b41092c0 0x7fc2b41a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2ba000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b419a6e0 0x7fc2b419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2ba000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4114380 con 0x7fc2b419a6e0 2024-04-20T21:34:27.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2ba000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4114200 con 0x7fc2b41092c0 2024-04-20T21:34:27.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2ba000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4114500 con 0x7fc2b41041f0 2024-04-20T21:34:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2b9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b419a6e0 0x7fc2b419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2b8c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2b41041f0 0x7fc2b41a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2abe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b41092c0 0x7fc2b41a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2b9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b419a6e0 0x7fc2b419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37910/0 (socket says 172.21.15.22:37910) 2024-04-20T21:34:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2b8c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2b41041f0 0x7fc2b41a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36732/0 (socket says 172.21.15.22:36732) 2024-04-20T21:34:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.255+0000 7fc2b9600640 1 -- 172.21.15.22:0/57666885 learned_addr learned my addr 172.21.15.22:0/57666885 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.256+0000 7fc2b8c00640 1 -- 172.21.15.22:0/57666885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b41092c0 msgr2=0x7fc2b41a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.256+0000 7fc2b8c00640 1 --2- 172.21.15.22:0/57666885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b41092c0 0x7fc2b41a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.256+0000 7fc2b8c00640 1 -- 172.21.15.22:0/57666885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b419a6e0 msgr2=0x7fc2b419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.256+0000 7fc2b8c00640 1 --2- 172.21.15.22:0/57666885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b419a6e0 0x7fc2b419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.256+0000 7fc2b8c00640 1 -- 172.21.15.22:0/57666885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b419b2e0 con 0x7fc2b41041f0 2024-04-20T21:34:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.256+0000 7fc2b8c00640 1 --2- 172.21.15.22:0/57666885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2b41041f0 0x7fc2b41a0620 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc29c02c0b0 tx=0x7fc29c0045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.256+0000 7fc2b2000640 1 -- 172.21.15.22:0/57666885 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c041020 con 0x7fc2b41041f0 2024-04-20T21:34:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.256+0000 7fc2b2000640 1 -- 172.21.15.22:0/57666885 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29c02c9f0 con 0x7fc2b41041f0 2024-04-20T21:34:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.256+0000 7fc2b2000640 1 -- 172.21.15.22:0/57666885 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c039040 con 0x7fc2b41041f0 2024-04-20T21:34:27.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.256+0000 7fc2ba000640 1 -- 172.21.15.22:0/57666885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b419b570 con 0x7fc2b41041f0 2024-04-20T21:34:27.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.257+0000 7fc2ba000640 1 -- 172.21.15.22:0/57666885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b41a76b0 con 0x7fc2b41041f0 2024-04-20T21:34:27.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.258+0000 7fc2ba000640 1 -- 172.21.15.22:0/57666885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b41056c0 con 0x7fc2b41041f0 2024-04-20T21:34:27.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.260+0000 7fc2b2000640 1 -- 172.21.15.22:0/57666885 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7fc29c044020 con 0x7fc2b41041f0 2024-04-20T21:34:27.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.261+0000 7fc2b2000640 1 --2- 172.21.15.22:0/57666885 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc2940a3e40 0x7fc2940a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.261+0000 7fc2b2000640 1 -- 172.21.15.22:0/57666885 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc29c0f2160 con 0x7fc2b41041f0 2024-04-20T21:34:27.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.261+0000 7fc2abe00640 1 --2- 172.21.15.22:0/57666885 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc2940a3e40 0x7fc2940a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.262+0000 7fc2abe00640 1 --2- 172.21.15.22:0/57666885 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc2940a3e40 0x7fc2940a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc2b419bf30 tx=0x7fc2a4009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.266+0000 7fc2b2000640 1 -- 172.21.15.22:0/57666885 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc29c04b020 con 0x7fc2b41041f0 2024-04-20T21:34:27.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.512+0000 7fc2ba000640 1 -- 172.21.15.22:0/57666885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2b410c550 con 0x7fc2b41041f0 2024-04-20T21:34:27.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.513+0000 7fc2b2000640 1 -- 172.21.15.22:0/57666885 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7fc29c0be640 con 0x7fc2b41041f0 2024-04-20T21:34:27.513 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:34:27.513 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:34:27.513 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:34:27.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.516+0000 7fc2ba000640 1 -- 172.21.15.22:0/57666885 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc2940a3e40 msgr2=0x7fc2940a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.516+0000 7fc2ba000640 1 --2- 172.21.15.22:0/57666885 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc2940a3e40 0x7fc2940a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc2b419bf30 tx=0x7fc2a4009260 comp rx=0 tx=0).stop 2024-04-20T21:34:27.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.516+0000 7fc2ba000640 1 -- 172.21.15.22:0/57666885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2b41041f0 msgr2=0x7fc2b41a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.516+0000 7fc2ba000640 1 --2- 172.21.15.22:0/57666885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2b41041f0 0x7fc2b41a0620 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc29c02c0b0 tx=0x7fc29c0045b0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.516+0000 7fc2ba000640 1 -- 172.21.15.22:0/57666885 shutdown_connections 2024-04-20T21:34:27.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.516+0000 7fc2ba000640 1 --2- 172.21.15.22:0/57666885 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc2940a3e40 0x7fc2940a6300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.516+0000 7fc2ba000640 1 --2- 172.21.15.22:0/57666885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2b419a6e0 0x7fc2b419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.516+0000 7fc2ba000640 1 --2- 172.21.15.22:0/57666885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b41092c0 0x7fc2b41a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.516+0000 7fc2ba000640 1 --2- 172.21.15.22:0/57666885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2b41041f0 0x7fc2b41a0620 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.516+0000 7fc2ba000640 1 -- 172.21.15.22:0/57666885 >> 172.21.15.22:0/57666885 conn(0x7fc2b40ffa30 msgr2=0x7fc2b4101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:27.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.516+0000 7fc2ba000640 1 -- 172.21.15.22:0/57666885 shutdown_connections 2024-04-20T21:34:27.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.516+0000 7fc2ba000640 1 -- 172.21.15.22:0/57666885 wait complete. 2024-04-20T21:34:27.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:34:27.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.752+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2417920905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb78076db0 msgr2=0x7fbb78112760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.752+0000 7fbb7ec00640 1 --2- 172.21.15.22:0/2417920905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb78076db0 0x7fbb78112760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009930 tx=0x7fbb6402bc20 comp rx=0 tx=0).stop 2024-04-20T21:34:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.752+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2417920905 shutdown_connections 2024-04-20T21:34:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.752+0000 7fbb7ec00640 1 --2- 172.21.15.22:0/2417920905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb78112ca0 0x7fbb78115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.752+0000 7fbb7ec00640 1 --2- 172.21.15.22:0/2417920905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb78076db0 0x7fbb78112760 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.752+0000 7fbb7ec00640 1 --2- 172.21.15.22:0/2417920905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb78076490 0x7fbb78076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.752+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2417920905 >> 172.21.15.22:0/2417920905 conn(0x7fbb780ffa70 msgr2=0x7fbb78101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.752+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2417920905 shutdown_connections 2024-04-20T21:34:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.752+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2417920905 wait complete. 2024-04-20T21:34:27.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.753+0000 7fbb7ec00640 1 Processor -- start 2024-04-20T21:34:27.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.754+0000 7fbb7ec00640 1 -- start start 2024-04-20T21:34:27.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.754+0000 7fbb7ec00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb78076490 0x7fbb7819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.754+0000 7fbb7ec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb78076db0 0x7fbb7819f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.754+0000 7fbb7ec00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb78112ca0 0x7fbb781a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.754+0000 7fbb7ec00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb78118840 con 0x7fbb78076490 2024-04-20T21:34:27.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.754+0000 7fbb7ec00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb781186c0 con 0x7fbb78112ca0 2024-04-20T21:34:27.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.754+0000 7fbb7ec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb781189c0 con 0x7fbb78076db0 2024-04-20T21:34:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.754+0000 7fbb7d800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb78076490 0x7fbb7819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.755+0000 7fbb7d800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb78076490 0x7fbb7819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37930/0 (socket says 172.21.15.22:37930) 2024-04-20T21:34:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.755+0000 7fbb7d800640 1 -- 172.21.15.22:0/2524139562 learned_addr learned my addr 172.21.15.22:0/2524139562 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.755+0000 7fbb7e200640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb78112ca0 0x7fbb781a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.755+0000 7fbb7ce00640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb78076db0 0x7fbb7819f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.755+0000 7fbb7e200640 1 -- 172.21.15.22:0/2524139562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb78076db0 msgr2=0x7fbb7819f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.755+0000 7fbb7e200640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb78076db0 0x7fbb7819f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.755+0000 7fbb7e200640 1 -- 172.21.15.22:0/2524139562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb78076490 msgr2=0x7fbb7819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.755+0000 7fbb7e200640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb78076490 0x7fbb7819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.755+0000 7fbb7e200640 1 -- 172.21.15.22:0/2524139562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb781a3d30 con 0x7fbb78112ca0 2024-04-20T21:34:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.755+0000 7fbb7e200640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb78112ca0 0x7fbb781a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbb70005fd0 tx=0x7fbb7000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:27.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.756+0000 7fbb76000640 1 -- 172.21.15.22:0/2524139562 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70003b70 con 0x7fbb78112ca0 2024-04-20T21:34:27.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.756+0000 7fbb76000640 1 -- 172.21.15.22:0/2524139562 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb70003d10 con 0x7fbb78112ca0 2024-04-20T21:34:27.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.756+0000 7fbb76000640 1 -- 172.21.15.22:0/2524139562 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70010830 con 0x7fbb78112ca0 2024-04-20T21:34:27.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.756+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2524139562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb781a4020 con 0x7fbb78112ca0 2024-04-20T21:34:27.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.756+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2524139562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb781ab8b0 con 0x7fbb78112ca0 2024-04-20T21:34:27.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.759+0000 7fbb82e0b740 1 Processor -- start 2024-04-20T21:34:27.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.759+0000 7fbb76000640 1 -- 172.21.15.22:0/2524139562 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7fbb70004020 con 0x7fbb78112ca0 2024-04-20T21:34:27.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.759+0000 7fbb82e0b740 1 -- start start 2024-04-20T21:34:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.760+0000 7fbb82e0b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8f170 0x561847b8f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.760+0000 7fbb82e0b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8fa90 0x561847b98130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.760+0000 7fbb82e0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b98670 0x561847b9aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.760+0000 7fbb82e0b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561847af4050 con 0x561847b8fa90 2024-04-20T21:34:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.760+0000 7fbb82e0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561847a9a490 con 0x561847b98670 2024-04-20T21:34:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.760+0000 7fbb82e0b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561847af0010 con 0x561847b8f170 2024-04-20T21:34:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.760+0000 7fbb7d800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8fa90 0x561847b98130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.760+0000 7fbb7ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8f170 0x561847b8f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.760+0000 7fbb7ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8f170 0x561847b8f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36764/0 (socket says 172.21.15.22:36764) 2024-04-20T21:34:27.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.760+0000 7fbb7ce00640 1 -- 172.21.15.22:0/3581427407 learned_addr learned my addr 172.21.15.22:0/3581427407 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:27.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.760+0000 7fbb7ce00640 1 --2- 172.21.15.22:0/3581427407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b98670 0x561847b9aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.761+0000 7fbb7d800640 1 -- 172.21.15.22:0/3581427407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8f170 msgr2=0x561847b8f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.761+0000 7fbb7d800640 1 --2- 172.21.15.22:0/3581427407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8f170 0x561847b8f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.761+0000 7fbb7d800640 1 -- 172.21.15.22:0/3581427407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b98670 msgr2=0x561847b9aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.761+0000 7fbb7d800640 1 --2- 172.21.15.22:0/3581427407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b98670 0x561847b9aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.761+0000 7fbb7d800640 1 -- 172.21.15.22:0/3581427407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561847aa2070 con 0x561847b8fa90 2024-04-20T21:34:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.761+0000 7fbb7d800640 1 --2- 172.21.15.22:0/3581427407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8fa90 0x561847b98130 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009c80 tx=0x7fbb6c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6ce06afa504ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.761+0000 7fbb76000640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbb600a3ec0 0x7fbb600a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.761+0000 7fbb76000640 1 -- 172.21.15.22:0/2524139562 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fbb700d0ff0 con 0x7fbb78112ca0 2024-04-20T21:34:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.762+0000 7fbb6aa00640 1 -- 172.21.15.22:0/3581427407 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c004250 con 0x561847b8fa90 2024-04-20T21:34:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.762+0000 7fbb7ce00640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbb600a3ec0 0x7fbb600a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.762+0000 7fbb6aa00640 1 -- 172.21.15.22:0/3581427407 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb6c009e30 con 0x561847b8fa90 2024-04-20T21:34:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.762+0000 7fbb6aa00640 1 -- 172.21.15.22:0/3581427407 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c004e20 con 0x561847b8fa90 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.762+0000 7fbb82e0b740 1 -- 172.21.15.22:0/3581427407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8fa90 msgr2=0x561847b98130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.762+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/3581427407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8fa90 0x561847b98130 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009c80 tx=0x7fbb6c011810 comp rx=0 tx=0).stop 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.762+0000 7fbb7ce00640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbb600a3ec0 0x7fbb600a6380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb64005ab0 tx=0x7fbb64005a20 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.762+0000 7fbb82e0b740 1 -- 172.21.15.22:0/3581427407 shutdown_connections 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.762+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/3581427407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b98670 0x561847b9aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.762+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/3581427407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8fa90 0x561847b98130 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.762+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/3581427407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8f170 0x561847b8f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.762+0000 7fbb82e0b740 1 -- 172.21.15.22:0/3581427407 >> 172.21.15.22:0/3581427407 conn(0x561847b898a0 msgr2=0x561847b89c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.763+0000 7fbb82e0b740 1 -- 172.21.15.22:0/3581427407 shutdown_connections 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.763+0000 7fbb82e0b740 1 -- 172.21.15.22:0/3581427407 wait complete. 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.763+0000 7fbb82e0b740 1 -- start start 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.763+0000 7fbb82e0b740 1 Processor -- start 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.764+0000 7fbb82e0b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8f170 0x561847b95670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.764+0000 7fbb82e0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8fa90 0x561847b95bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.764+0000 7fbb82e0b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b98670 0x561847b960f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.764+0000 7fbb82e0b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561847b9b460 con 0x561847b8f170 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.764+0000 7fbb82e0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561847aad860 con 0x561847b8fa90 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.764+0000 7fbb82e0b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561847b9b5e0 con 0x561847b98670 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.764+0000 7fbb82e0b740 10 client.0 _ll_register_callbacks cb 0x561847a7bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.764+0000 7fbb7d800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8f170 0x561847b95670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.764+0000 7fbb7d800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8f170 0x561847b95670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37956/0 (socket says 172.21.15.22:37956) 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.764+0000 7fbb7d800640 1 -- 172.21.15.22:0/1491797299 learned_addr learned my addr 172.21.15.22:0/1491797299 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.764+0000 7fbb7ce00640 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8fa90 0x561847b95bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.764+0000 7fbb7e200640 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b98670 0x561847b960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.765+0000 7fbb7d800640 1 -- 172.21.15.22:0/1491797299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b98670 msgr2=0x561847b960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.765+0000 7fbb7d800640 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b98670 0x561847b960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.765+0000 7fbb7d800640 1 -- 172.21.15.22:0/1491797299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8fa90 msgr2=0x561847b95bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.765+0000 7fbb7d800640 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8fa90 0x561847b95bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.765+0000 7fbb7d800640 1 -- 172.21.15.22:0/1491797299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561847ab1950 con 0x561847b8f170 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.765+0000 7fbb7d800640 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8f170 0x561847b95670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c000c00 tx=0x7fbb6c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9153880540c282b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:27.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.765+0000 7fbb5f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:34:27.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.765+0000 7fbb5f400640 1 -- 172.21.15.22:0/1491797299 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c029070 con 0x561847b8f170 2024-04-20T21:34:27.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.765+0000 7fbb5f400640 1 -- 172.21.15.22:0/1491797299 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fbb6c0037f0 con 0x561847b8f170 2024-04-20T21:34:27.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.765+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1491797299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb6c009e30 con 0x561847b8f170 2024-04-20T21:34:27.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.766+0000 7fbb5f400640 1 -- 172.21.15.22:0/1491797299 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbb6c035050 con 0x561847b8f170 2024-04-20T21:34:27.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.766+0000 7fbb82e0b740 10 client.15291 fetch_fsmap learned FSMap version 20 2024-04-20T21:34:27.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.766+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1491797299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561847a7f820 con 0x561847b8f170 2024-04-20T21:34:27.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.766+0000 7fbb5f400640 1 -- 172.21.15.22:0/1491797299 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb6c009e30 con 0x561847b8f170 2024-04-20T21:34:27.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.766+0000 7fbb5f400640 1 -- 172.21.15.22:0/1491797299 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fbb6c005330 con 0x561847b8f170 2024-04-20T21:34:27.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.766+0000 7fbb82e0b740 10 client.15291 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:34:27.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.766+0000 7fbb82e0b740 10 client.15291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:27.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.766+0000 7fbb82e0b740 20 client.15291 populate_metadata read hostname 'smithi022' 2024-04-20T21:34:27.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.766+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561847ba8510 0x561847baa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.766+0000 7fbb82e0b740 4 client.15291 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.766+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1491797299 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561847b01a70 con 0x561847ba8510 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.767+0000 7fbb7ce00640 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561847ba8510 0x561847baa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.767+0000 7fbb7ce00640 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561847ba8510 0x561847baa900 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f8cf93b624a31c6 server_cookie=4ec71374248152ef in_seq=0 out_seq=0 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.767+0000 7fbb5f400640 10 client.15291 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.768+0000 7fbb5f400640 1 -- 172.21.15.22:0/1491797299 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561847b01a70 con 0x561847ba8510 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.768+0000 7fbb5f400640 10 client.15291 handle_command_reply: tid=0 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.768+0000 7fbb82e0b740 1 client.15291 shutdown 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.768+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1491797299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8f170 msgr2=0x561847b95670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.768+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8f170 0x561847b95670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c000c00 tx=0x7fbb6c009700 comp rx=0 tx=0).stop 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.768+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1491797299 shutdown_connections 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.768+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b98670 0x561847b960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.768+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561847ba8510 0x561847baa900 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.768+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8fa90 0x561847b95bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.768+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/1491797299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b8f170 0x561847b95670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.769+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1491797299 >> 172.21.15.22:0/1491797299 conn(0x561847b898a0 msgr2=0x561847b8b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.769+0000 7fbb5f400640 0 client.15291 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.769+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1491797299 shutdown_connections 2024-04-20T21:34:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.769+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1491797299 wait complete. 2024-04-20T21:34:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.769+0000 7fbb82e0b740 20 client.15291 trim_cache size 0 max 16384 2024-04-20T21:34:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.784+0000 7fbb82e0b740 1 Processor -- start 2024-04-20T21:34:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.784+0000 7fbb82e0b740 1 -- start start 2024-04-20T21:34:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.784+0000 7fbb82e0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8f170 0x561847ba4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.784+0000 7fbb82e0b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8fa90 0x561847b880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.784+0000 7fbb82e0b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 0x561847b94e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.784+0000 7fbb82e0b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb781b2150 con 0x561847b98670 2024-04-20T21:34:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.784+0000 7fbb82e0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb781b49d0 con 0x561847b8f170 2024-04-20T21:34:27.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.784+0000 7fbb82e0b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb781b27c0 con 0x561847b8fa90 2024-04-20T21:34:27.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.784+0000 7fbb7e200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 0x561847b94e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.784+0000 7fbb7e200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 0x561847b94e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37958/0 (socket says 172.21.15.22:37958) 2024-04-20T21:34:27.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.784+0000 7fbb7ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8fa90 0x561847b880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.784+0000 7fbb7e200640 1 -- 172.21.15.22:0/3063375733 learned_addr learned my addr 172.21.15.22:0/3063375733 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.785+0000 7fbb7d800640 1 --2- 172.21.15.22:0/3063375733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8f170 0x561847ba4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.785+0000 7fbb7e200640 1 -- 172.21.15.22:0/3063375733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8fa90 msgr2=0x561847b880d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.785+0000 7fbb7e200640 1 --2- 172.21.15.22:0/3063375733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8fa90 0x561847b880d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.785+0000 7fbb7e200640 1 -- 172.21.15.22:0/3063375733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8f170 msgr2=0x561847ba4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.785+0000 7fbb7e200640 1 --2- 172.21.15.22:0/3063375733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8f170 0x561847ba4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.785+0000 7fbb7e200640 1 -- 172.21.15.22:0/3063375733 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618478063f0 con 0x561847b98670 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.785+0000 7fbb7ce00640 1 --2- 172.21.15.22:0/3063375733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8fa90 0x561847b880d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.785+0000 7fbb7d800640 1 --2- 172.21.15.22:0/3063375733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8f170 0x561847ba4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.785+0000 7fbb7e200640 1 --2- 172.21.15.22:0/3063375733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 0x561847b94e20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb700047d0 tx=0x7fbb7009cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa409f85c85228ea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.785+0000 7fbb6a000640 1 -- 172.21.15.22:0/3063375733 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb700bb070 con 0x561847b98670 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.785+0000 7fbb6a000640 1 -- 172.21.15.22:0/3063375733 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb7009d3c0 con 0x561847b98670 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.786+0000 7fbb6a000640 1 -- 172.21.15.22:0/3063375733 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb700bd0b0 con 0x561847b98670 2024-04-20T21:34:27.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.786+0000 7fbb82e0b740 1 -- 172.21.15.22:0/3063375733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 msgr2=0x561847b94e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.786+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/3063375733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 0x561847b94e20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb700047d0 tx=0x7fbb7009cb50 comp rx=0 tx=0).stop 2024-04-20T21:34:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.786+0000 7fbb82e0b740 1 -- 172.21.15.22:0/3063375733 shutdown_connections 2024-04-20T21:34:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.786+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/3063375733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 0x561847b94e20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.786+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/3063375733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8fa90 0x561847b880d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.786+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/3063375733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8f170 0x561847ba4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.786+0000 7fbb82e0b740 1 -- 172.21.15.22:0/3063375733 >> 172.21.15.22:0/3063375733 conn(0x561847b898a0 msgr2=0x561847b95ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.786+0000 7fbb82e0b740 1 -- 172.21.15.22:0/3063375733 shutdown_connections 2024-04-20T21:34:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.786+0000 7fbb82e0b740 1 -- 172.21.15.22:0/3063375733 wait complete. 2024-04-20T21:34:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.787+0000 7fbb82e0b740 1 -- start start 2024-04-20T21:34:27.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.787+0000 7fbb82e0b740 1 Processor -- start 2024-04-20T21:34:27.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.787+0000 7fbb82e0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8f170 0x561847ba0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.787+0000 7fbb82e0b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8fa90 0x561847b9bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.787+0000 7fbb82e0b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 0x561847b9c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.787+0000 7fbb82e0b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb781b2e90 con 0x561847b98670 2024-04-20T21:34:27.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.787+0000 7fbb82e0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb781b4300 con 0x561847b8f170 2024-04-20T21:34:27.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.787+0000 7fbb82e0b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb781b3560 con 0x561847b8fa90 2024-04-20T21:34:27.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.787+0000 7fbb82e0b740 10 client.0 _ll_register_callbacks cb 0x7fbb6c009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:27.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb7e200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 0x561847b9c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb7e200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 0x561847b9c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37960/0 (socket says 172.21.15.22:37960) 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb7e200640 1 -- 172.21.15.22:0/1011669061 learned_addr learned my addr 172.21.15.22:0/1011669061 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb7d800640 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8f170 0x561847ba0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb7ce00640 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8fa90 0x561847b9bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb7e200640 1 -- 172.21.15.22:0/1011669061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8fa90 msgr2=0x561847b9bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb7e200640 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8fa90 0x561847b9bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb7e200640 1 -- 172.21.15.22:0/1011669061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8f170 msgr2=0x561847ba0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb7e200640 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8f170 0x561847ba0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb7e200640 1 -- 172.21.15.22:0/1011669061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618478d07d0 con 0x561847b98670 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb7d800640 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8f170 0x561847ba0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb7e200640 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 0x561847b9c470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000cba0 tx=0x7fbb7000ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6c47da5e12ecba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb5f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.788+0000 7fbb5f400640 1 -- 172.21.15.22:0/1011669061 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7007f030 con 0x561847b98670 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1011669061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb7009d3c0 con 0x561847b98670 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb5f400640 1 -- 172.21.15.22:0/1011669061 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fbb700a56b0 con 0x561847b98670 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb5f400640 1 -- 172.21.15.22:0/1011669061 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbb700bb040 con 0x561847b98670 2024-04-20T21:34:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb82e0b740 10 client.15297 fetch_fsmap learned FSMap version 20 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1011669061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561847c0e700 con 0x561847b98670 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb5f400640 1 -- 172.21.15.22:0/1011669061 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb7009d3c0 con 0x561847b98670 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb5f400640 1 -- 172.21.15.22:0/1011669061 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fbb70004b40 con 0x561847b98670 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb82e0b740 10 client.15297 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb82e0b740 10 client.15297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb82e0b740 20 client.15297 populate_metadata read hostname 'smithi022' 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561847c0edf0 0x561847c11210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb82e0b740 4 client.15297 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.789+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1011669061 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561847afb8e0 con 0x561847c0edf0 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.790+0000 7fbb7d800640 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561847c0edf0 0x561847c11210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.790+0000 7fbb7d800640 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561847c0edf0 0x561847c11210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83dfbf029d6dd109 server_cookie=ecb32096957833ef in_seq=0 out_seq=0 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.790+0000 7fbb5f400640 10 client.15297 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.790+0000 7fbb5f400640 1 -- 172.21.15.22:0/1011669061 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561847afb8e0 con 0x561847c0edf0 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.790+0000 7fbb5f400640 10 client.15297 handle_command_reply: tid=0 2024-04-20T21:34:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.790+0000 7fbb82e0b740 1 client.15297 shutdown 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.790+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1011669061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 msgr2=0x561847b9c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.790+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 0x561847b9c470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000cba0 tx=0x7fbb7000ce70 comp rx=0 tx=0).stop 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.791+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1011669061 shutdown_connections 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.791+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561847b98670 0x561847b9c470 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.791+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561847b8fa90 0x561847b9bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.791+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561847c0edf0 0x561847c11210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.791+0000 7fbb82e0b740 1 --2- 172.21.15.22:0/1011669061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561847b8f170 0x561847ba0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.791+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1011669061 >> 172.21.15.22:0/1011669061 conn(0x561847b898a0 msgr2=0x561847ba3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.791+0000 7fbb5f400640 0 client.15297 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.791+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1011669061 shutdown_connections 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.791+0000 7fbb82e0b740 1 -- 172.21.15.22:0/1011669061 wait complete. 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.791+0000 7fbb82e0b740 20 client.15297 trim_cache size 0 max 16384 2024-04-20T21:34:27.793 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:34:27.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.791+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2524139562 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbb600a3ec0 msgr2=0x7fbb600a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.791+0000 7fbb7ec00640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbb600a3ec0 0x7fbb600a6380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb64005ab0 tx=0x7fbb64005a20 comp rx=0 tx=0).stop 2024-04-20T21:34:27.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.792+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2524139562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb78112ca0 msgr2=0x7fbb781a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:27.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.792+0000 7fbb7ec00640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb78112ca0 0x7fbb781a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbb70005fd0 tx=0x7fbb7000c6a0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.792+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2524139562 shutdown_connections 2024-04-20T21:34:27.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.792+0000 7fbb7ec00640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbb600a3ec0 0x7fbb600a6380 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.792+0000 7fbb7ec00640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb78112ca0 0x7fbb781a3600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.792+0000 7fbb7ec00640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb78076db0 0x7fbb7819f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.792+0000 7fbb7ec00640 1 --2- 172.21.15.22:0/2524139562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb78076490 0x7fbb7819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:27.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.792+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2524139562 >> 172.21.15.22:0/2524139562 conn(0x7fbb780ffa70 msgr2=0x7fbb781142c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:27.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.792+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2524139562 shutdown_connections 2024-04-20T21:34:27.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:27.792+0000 7fbb7ec00640 1 -- 172.21.15.22:0/2524139562 wait complete. 2024-04-20T21:34:28.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:28 smithi022 ceph-mon[16843]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:34:28.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:28 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/57666885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:34:28.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:28 smithi122 ceph-mon[22147]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:34:28.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:28 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/57666885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:34:28.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:28 smithi033 ceph-mon[21626]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:34:28.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:28 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/57666885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:34:28.809 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:34:28.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:34:28.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:34:29.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.040+0000 7f8b6be00640 1 -- 172.21.15.22:0/1090665104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b6c104270 msgr2=0x7f8b6c104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.040+0000 7f8b6be00640 1 --2- 172.21.15.22:0/1090665104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b6c104270 0x7f8b6c104650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009860 tx=0x7f8b5c02bb70 comp rx=0 tx=0).stop 2024-04-20T21:34:29.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.040+0000 7f8b6be00640 1 -- 172.21.15.22:0/1090665104 shutdown_connections 2024-04-20T21:34:29.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.040+0000 7f8b6be00640 1 --2- 172.21.15.22:0/1090665104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b6c109340 0x7f8b6c110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.040+0000 7f8b6be00640 1 --2- 172.21.15.22:0/1090665104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b6c104c20 0x7f8b6c108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.040+0000 7f8b6be00640 1 --2- 172.21.15.22:0/1090665104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b6c104270 0x7f8b6c104650 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.040+0000 7f8b6be00640 1 -- 172.21.15.22:0/1090665104 >> 172.21.15.22:0/1090665104 conn(0x7f8b6c0ffad0 msgr2=0x7f8b6c101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:29.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.041+0000 7f8b6be00640 1 -- 172.21.15.22:0/1090665104 shutdown_connections 2024-04-20T21:34:29.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.041+0000 7f8b6be00640 1 -- 172.21.15.22:0/1090665104 wait complete. 2024-04-20T21:34:29.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.042+0000 7f8b6be00640 1 Processor -- start 2024-04-20T21:34:29.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.042+0000 7f8b6be00640 1 -- start start 2024-04-20T21:34:29.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.042+0000 7f8b6be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b6c104270 0x7f8b6c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b6c109340 0x7f8b6c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b6c19a730 0x7f8b6c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1143d0 con 0x7f8b6c19a730 2024-04-20T21:34:29.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6be00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c114250 con 0x7f8b6c109340 2024-04-20T21:34:29.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c114550 con 0x7f8b6c104270 2024-04-20T21:34:29.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b6c19a730 0x7f8b6c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b6c104270 0x7f8b6c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b6c104270 0x7f8b6c1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36818/0 (socket says 172.21.15.22:36818) 2024-04-20T21:34:29.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b6c19a730 0x7f8b6c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37980/0 (socket says 172.21.15.22:37980) 2024-04-20T21:34:29.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b63e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b6c109340 0x7f8b6c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6b400640 1 -- 172.21.15.22:0/3252045135 learned_addr learned my addr 172.21.15.22:0/3252045135 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6aa00640 1 -- 172.21.15.22:0/3252045135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b6c109340 msgr2=0x7f8b6c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6aa00640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b6c109340 0x7f8b6c1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6aa00640 1 -- 172.21.15.22:0/3252045135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b6c19a730 msgr2=0x7f8b6c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6aa00640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b6c19a730 0x7f8b6c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b6aa00640 1 -- 172.21.15.22:0/3252045135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6c19b330 con 0x7f8b6c104270 2024-04-20T21:34:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.043+0000 7f8b63e00640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b6c109340 0x7f8b6c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.044+0000 7f8b6aa00640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b6c104270 0x7f8b6c1a0670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c037040 tx=0x7f8b5c0045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.044+0000 7f8b63400640 1 -- 172.21.15.22:0/3252045135 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c041020 con 0x7f8b6c104270 2024-04-20T21:34:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.044+0000 7f8b6be00640 1 -- 172.21.15.22:0/3252045135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6c19b5c0 con 0x7f8b6c104270 2024-04-20T21:34:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.044+0000 7f8b63400640 1 -- 172.21.15.22:0/3252045135 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5c02c9f0 con 0x7f8b6c104270 2024-04-20T21:34:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.044+0000 7f8b63400640 1 -- 172.21.15.22:0/3252045135 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c039040 con 0x7f8b6c104270 2024-04-20T21:34:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.044+0000 7f8b6be00640 1 -- 172.21.15.22:0/3252045135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6c1a7750 con 0x7f8b6c104270 2024-04-20T21:34:29.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.048+0000 7f8b745d1740 1 Processor -- start 2024-04-20T21:34:29.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.049+0000 7f8b745d1740 1 -- start start 2024-04-20T21:34:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.049+0000 7f8b745d1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e830f0 0x555fd1e834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.049+0000 7f8b745d1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e83a10 0x555fd1e8c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.049+0000 7f8b745d1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e8c5f0 0x555fd1e8e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.049+0000 7f8b745d1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555fd1de7d20 con 0x555fd1e8c5f0 2024-04-20T21:34:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.049+0000 7f8b745d1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555fd1d8e3e0 con 0x555fd1e830f0 2024-04-20T21:34:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.049+0000 7f8b745d1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555fd1de3ce0 con 0x555fd1e83a10 2024-04-20T21:34:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.049+0000 7f8b63400640 1 -- 172.21.15.22:0/3252045135 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7f8b5c044020 con 0x7f8b6c104270 2024-04-20T21:34:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.050+0000 7f8b6b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e830f0 0x555fd1e834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.050+0000 7f8b63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e8c5f0 0x555fd1e8e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.050+0000 7f8b6b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e830f0 0x555fd1e834d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33746/0 (socket says 172.21.15.22:33746) 2024-04-20T21:34:29.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.050+0000 7f8b6b400640 1 -- 172.21.15.22:0/378604175 learned_addr learned my addr 172.21.15.22:0/378604175 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:29.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.050+0000 7f8b6aa00640 1 --2- 172.21.15.22:0/378604175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e83a10 0x555fd1e8c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.050+0000 7f8b6b400640 1 -- 172.21.15.22:0/378604175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e83a10 msgr2=0x555fd1e8c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.050+0000 7f8b6b400640 1 --2- 172.21.15.22:0/378604175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e83a10 0x555fd1e8c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.050+0000 7f8b6b400640 1 -- 172.21.15.22:0/378604175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e8c5f0 msgr2=0x555fd1e8e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.050+0000 7f8b6b400640 1 --2- 172.21.15.22:0/378604175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e8c5f0 0x555fd1e8e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.050+0000 7f8b6b400640 1 -- 172.21.15.22:0/378604175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd1dad980 con 0x555fd1e830f0 2024-04-20T21:34:29.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.050+0000 7f8b6b400640 1 --2- 172.21.15.22:0/378604175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e830f0 0x555fd1e834d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400e5e0 tx=0x7f8b640095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=980996145a9446ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:29.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b5be00640 1 -- 172.21.15.22:0/378604175 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64003970 con 0x555fd1e830f0 2024-04-20T21:34:29.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b5be00640 1 -- 172.21.15.22:0/378604175 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6400d870 con 0x555fd1e830f0 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b5be00640 1 -- 172.21.15.22:0/378604175 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b640045c0 con 0x555fd1e830f0 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b63400640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f8b500a3ec0 0x7f8b500a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b745d1740 1 -- 172.21.15.22:0/378604175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e830f0 msgr2=0x555fd1e834d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b745d1740 1 --2- 172.21.15.22:0/378604175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e830f0 0x555fd1e834d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400e5e0 tx=0x7f8b640095c0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b63400640 1 -- 172.21.15.22:0/3252045135 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f8b5c0f2070 con 0x7f8b6c104270 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b745d1740 1 -- 172.21.15.22:0/378604175 shutdown_connections 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b745d1740 1 --2- 172.21.15.22:0/378604175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e8c5f0 0x555fd1e8e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b745d1740 1 --2- 172.21.15.22:0/378604175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e83a10 0x555fd1e8c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b745d1740 1 --2- 172.21.15.22:0/378604175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e830f0 0x555fd1e834d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b745d1740 1 -- 172.21.15.22:0/378604175 >> 172.21.15.22:0/378604175 conn(0x555fd1e7d820 msgr2=0x555fd1e7dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.051+0000 7f8b6aa00640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f8b500a3ec0 0x7f8b500a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.052+0000 7f8b745d1740 1 -- 172.21.15.22:0/378604175 shutdown_connections 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.052+0000 7f8b745d1740 1 -- 172.21.15.22:0/378604175 wait complete. 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.052+0000 7f8b745d1740 1 -- start start 2024-04-20T21:34:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.052+0000 7f8b6aa00640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f8b500a3ec0 0x7f8b500a6380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0ef790 tx=0x7f8b5c02bb70 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:29.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.052+0000 7f8b745d1740 1 Processor -- start 2024-04-20T21:34:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b745d1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e830f0 0x555fd1e895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b745d1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 0x555fd1e89b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b745d1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e8c5f0 0x555fd1e8a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b745d1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555fd1e8f3e0 con 0x555fd1e83a10 2024-04-20T21:34:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b745d1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555fd1da17a0 con 0x555fd1e8c5f0 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b745d1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555fd1e8f560 con 0x555fd1e830f0 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b745d1740 10 client.0 _ll_register_callbacks cb 0x555fd1da3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 0x555fd1e89b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b6b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e830f0 0x555fd1e895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 0x555fd1e89b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38002/0 (socket says 172.21.15.22:38002) 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b6b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e830f0 0x555fd1e895d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36836/0 (socket says 172.21.15.22:36836) 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b63e00640 1 -- 172.21.15.22:0/2838820896 learned_addr learned my addr 172.21.15.22:0/2838820896 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.053+0000 7f8b6b400640 1 --2- 172.21.15.22:0/2838820896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e8c5f0 0x555fd1e8a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.054+0000 7f8b63e00640 1 -- 172.21.15.22:0/2838820896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e830f0 msgr2=0x555fd1e895d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.054+0000 7f8b63e00640 1 --2- 172.21.15.22:0/2838820896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e830f0 0x555fd1e895d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.054+0000 7f8b63e00640 1 -- 172.21.15.22:0/2838820896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e8c5f0 msgr2=0x555fd1e8a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.054+0000 7f8b63e00640 1 --2- 172.21.15.22:0/2838820896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e8c5f0 0x555fd1e8a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.054+0000 7f8b63e00640 1 -- 172.21.15.22:0/2838820896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd1bc27e0 con 0x555fd1e83a10 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.054+0000 7f8b63e00640 1 --2- 172.21.15.22:0/2838820896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 0x555fd1e89b10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8b54012d50 tx=0x7f8b5400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b615ab2a38dd01c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.054+0000 7f8b58c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:34:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.054+0000 7f8b58c00640 1 -- 172.21.15.22:0/2838820896 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54012110 con 0x555fd1e83a10 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.054+0000 7f8b58c00640 1 -- 172.21.15.22:0/2838820896 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f8b54004050 con 0x555fd1e83a10 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.054+0000 7f8b745d1740 1 -- 172.21.15.22:0/2838820896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b6400d870 con 0x555fd1e83a10 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.055+0000 7f8b58c00640 1 -- 172.21.15.22:0/2838820896 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8b540122b0 con 0x555fd1e83a10 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.055+0000 7f8b745d1740 10 client.15309 fetch_fsmap learned FSMap version 20 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.055+0000 7f8b745d1740 1 -- 172.21.15.22:0/2838820896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555fd1c22f70 con 0x555fd1e83a10 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.055+0000 7f8b58c00640 1 -- 172.21.15.22:0/2838820896 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6400d870 con 0x555fd1e83a10 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.055+0000 7f8b58c00640 1 -- 172.21.15.22:0/2838820896 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f8b54010430 con 0x555fd1e83a10 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.055+0000 7f8b745d1740 10 client.15309 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.055+0000 7f8b745d1740 10 client.15309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.055+0000 7f8b745d1740 20 client.15309 populate_metadata read hostname 'smithi022' 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.055+0000 7f8b745d1740 1 --2- 172.21.15.22:0/2838820896 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555fd1e9c9d0 0x555fd1e9edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.055+0000 7f8b745d1740 4 client.15309 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.055+0000 7f8b745d1740 1 -- 172.21.15.22:0/2838820896 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fd1df5740 con 0x555fd1e9c9d0 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.056+0000 7f8b6b400640 1 --2- 172.21.15.22:0/2838820896 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555fd1e9c9d0 0x555fd1e9edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.056+0000 7f8b6b400640 1 --2- 172.21.15.22:0/2838820896 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555fd1e9c9d0 0x555fd1e9edc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6da42652b143ca8 server_cookie=1d746c9b176ea2 in_seq=0 out_seq=0 2024-04-20T21:34:29.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.056+0000 7f8b58c00640 10 client.15309 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.057+0000 7f8b58c00640 1 -- 172.21.15.22:0/2838820896 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fd1df5740 con 0x555fd1e9c9d0 2024-04-20T21:34:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.057+0000 7f8b58c00640 10 client.15309 handle_command_reply: tid=0 2024-04-20T21:34:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.057+0000 7f8b745d1740 1 client.15309 shutdown 2024-04-20T21:34:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.057+0000 7f8b745d1740 1 -- 172.21.15.22:0/2838820896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 msgr2=0x555fd1e89b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.057+0000 7f8b745d1740 1 --2- 172.21.15.22:0/2838820896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 0x555fd1e89b10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8b54012d50 tx=0x7f8b5400f790 comp rx=0 tx=0).stop 2024-04-20T21:34:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.057+0000 7f8b745d1740 1 -- 172.21.15.22:0/2838820896 shutdown_connections 2024-04-20T21:34:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.057+0000 7f8b745d1740 1 --2- 172.21.15.22:0/2838820896 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555fd1e9c9d0 0x555fd1e9edc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.057+0000 7f8b745d1740 1 --2- 172.21.15.22:0/2838820896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e8c5f0 0x555fd1e8a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.057+0000 7f8b745d1740 1 --2- 172.21.15.22:0/2838820896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 0x555fd1e89b10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.057+0000 7f8b745d1740 1 --2- 172.21.15.22:0/2838820896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e830f0 0x555fd1e895d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.057+0000 7f8b745d1740 1 -- 172.21.15.22:0/2838820896 >> 172.21.15.22:0/2838820896 conn(0x555fd1e7d820 msgr2=0x555fd1e7f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.057+0000 7f8b58c00640 0 client.15309 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:29.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.058+0000 7f8b745d1740 1 -- 172.21.15.22:0/2838820896 shutdown_connections 2024-04-20T21:34:29.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.058+0000 7f8b745d1740 1 -- 172.21.15.22:0/2838820896 wait complete. 2024-04-20T21:34:29.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.058+0000 7f8b745d1740 20 client.15309 trim_cache size 0 max 16384 2024-04-20T21:34:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.077+0000 7f8b745d1740 1 Processor -- start 2024-04-20T21:34:29.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.077+0000 7f8b745d1740 1 -- start start 2024-04-20T21:34:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.077+0000 7f8b745d1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e830f0 0x555fd1e982a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.077+0000 7f8b745d1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 0x555fd1e89e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.077+0000 7f8b745d1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e8c5f0 0x555fd1ea46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.077+0000 7f8b745d1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1adda0 con 0x555fd1e83a10 2024-04-20T21:34:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.077+0000 7f8b745d1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1b0620 con 0x555fd1e830f0 2024-04-20T21:34:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.077+0000 7f8b745d1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1ae410 con 0x555fd1e8c5f0 2024-04-20T21:34:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.078+0000 7f8b63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 0x555fd1e89e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.078+0000 7f8b63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 0x555fd1e89e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38016/0 (socket says 172.21.15.22:38016) 2024-04-20T21:34:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.078+0000 7f8b63e00640 1 -- 172.21.15.22:0/1522155842 learned_addr learned my addr 172.21.15.22:0/1522155842 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.078+0000 7f8b6b400640 1 --2- 172.21.15.22:0/1522155842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e830f0 0x555fd1e982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.078+0000 7f8b6b400640 1 --2- 172.21.15.22:0/1522155842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e8c5f0 0x555fd1ea46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.078+0000 7f8b63e00640 1 -- 172.21.15.22:0/1522155842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e8c5f0 msgr2=0x555fd1ea46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.078+0000 7f8b63e00640 1 --2- 172.21.15.22:0/1522155842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e8c5f0 0x555fd1ea46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.078+0000 7f8b63e00640 1 -- 172.21.15.22:0/1522155842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e830f0 msgr2=0x555fd1e982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.078+0000 7f8b63e00640 1 --2- 172.21.15.22:0/1522155842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e830f0 0x555fd1e982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.078+0000 7f8b63e00640 1 -- 172.21.15.22:0/1522155842 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd1e843c0 con 0x555fd1e83a10 2024-04-20T21:34:29.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.078+0000 7f8b6b400640 1 --2- 172.21.15.22:0/1522155842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e8c5f0 0x555fd1ea46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.078+0000 7f8b63e00640 1 --2- 172.21.15.22:0/1522155842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 0x555fd1e89e70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8b540108d0 tx=0x7f8b540127f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b95a9d54f5419d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.079+0000 7f8b5b400640 1 -- 172.21.15.22:0/1522155842 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54028030 con 0x555fd1e83a10 2024-04-20T21:34:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.079+0000 7f8b5b400640 1 -- 172.21.15.22:0/1522155842 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5400fe80 con 0x555fd1e83a10 2024-04-20T21:34:29.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.079+0000 7f8b5b400640 1 -- 172.21.15.22:0/1522155842 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b540035f0 con 0x555fd1e83a10 2024-04-20T21:34:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.079+0000 7f8b745d1740 1 -- 172.21.15.22:0/1522155842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 msgr2=0x555fd1e89e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.079+0000 7f8b745d1740 1 --2- 172.21.15.22:0/1522155842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 0x555fd1e89e70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8b540108d0 tx=0x7f8b540127f0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.080+0000 7f8b745d1740 1 -- 172.21.15.22:0/1522155842 shutdown_connections 2024-04-20T21:34:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.080+0000 7f8b745d1740 1 --2- 172.21.15.22:0/1522155842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e8c5f0 0x555fd1ea46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.080+0000 7f8b745d1740 1 --2- 172.21.15.22:0/1522155842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e83a10 0x555fd1e89e70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.080+0000 7f8b745d1740 1 --2- 172.21.15.22:0/1522155842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e830f0 0x555fd1e982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.080+0000 7f8b745d1740 1 -- 172.21.15.22:0/1522155842 >> 172.21.15.22:0/1522155842 conn(0x555fd1e7d820 msgr2=0x555fd1e7f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.080+0000 7f8b745d1740 1 -- 172.21.15.22:0/1522155842 shutdown_connections 2024-04-20T21:34:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.080+0000 7f8b745d1740 1 -- 172.21.15.22:0/1522155842 wait complete. 2024-04-20T21:34:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.080+0000 7f8b745d1740 1 -- start start 2024-04-20T21:34:29.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.080+0000 7f8b745d1740 1 Processor -- start 2024-04-20T21:34:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.081+0000 7f8b745d1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e830f0 0x555fd1e866f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.081+0000 7f8b745d1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e83a10 0x555fd1e86c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.081+0000 7f8b745d1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e8c5f0 0x555fd1f0a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.081+0000 7f8b745d1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1aeae0 con 0x555fd1e830f0 2024-04-20T21:34:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.081+0000 7f8b745d1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1aff50 con 0x555fd1e83a10 2024-04-20T21:34:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.081+0000 7f8b745d1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c1af1b0 con 0x555fd1e8c5f0 2024-04-20T21:34:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.081+0000 7f8b745d1740 10 client.0 _ll_register_callbacks cb 0x7f8b54012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.081+0000 7f8b6b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e830f0 0x555fd1e866f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.081+0000 7f8b6b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e830f0 0x555fd1e866f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38020/0 (socket says 172.21.15.22:38020) 2024-04-20T21:34:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.081+0000 7f8b6b400640 1 -- 172.21.15.22:0/4062674340 learned_addr learned my addr 172.21.15.22:0/4062674340 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.081+0000 7f8b63e00640 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e83a10 0x555fd1e86c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.081+0000 7f8b6b400640 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e8c5f0 0x555fd1f0a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.082+0000 7f8b6b400640 1 -- 172.21.15.22:0/4062674340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e8c5f0 msgr2=0x555fd1f0a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.082+0000 7f8b6b400640 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e8c5f0 0x555fd1f0a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.082+0000 7f8b6b400640 1 -- 172.21.15.22:0/4062674340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e83a10 msgr2=0x555fd1e86c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.082+0000 7f8b6b400640 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e83a10 0x555fd1e86c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.082+0000 7f8b6b400640 1 -- 172.21.15.22:0/4062674340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd1aba170 con 0x555fd1e830f0 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.082+0000 7f8b6b400640 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e830f0 0x555fd1e866f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8b64010010 tx=0x7f8b64026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b034823dcdd935b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.082+0000 7f8b58c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.082+0000 7f8b58c00640 1 -- 172.21.15.22:0/4062674340 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6403a070 con 0x555fd1e830f0 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.082+0000 7f8b58c00640 1 -- 172.21.15.22:0/4062674340 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f8b640044e0 con 0x555fd1e830f0 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.082+0000 7f8b745d1740 1 -- 172.21.15.22:0/4062674340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555fd1def5b0 con 0x555fd1e830f0 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.083+0000 7f8b58c00640 1 -- 172.21.15.22:0/4062674340 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8b640046e0 con 0x555fd1e830f0 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.083+0000 7f8b745d1740 10 client.15315 fetch_fsmap learned FSMap version 20 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.083+0000 7f8b745d1740 1 -- 172.21.15.22:0/4062674340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555fd1f0bd20 con 0x555fd1e830f0 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.083+0000 7f8b58c00640 1 -- 172.21.15.22:0/4062674340 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555fd1def5b0 con 0x555fd1e830f0 2024-04-20T21:34:29.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.083+0000 7f8b58c00640 1 -- 172.21.15.22:0/4062674340 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f8b6403d020 con 0x555fd1e830f0 2024-04-20T21:34:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.083+0000 7f8b745d1740 10 client.15315 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:34:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.083+0000 7f8b745d1740 10 client.15315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.083+0000 7f8b745d1740 20 client.15315 populate_metadata read hostname 'smithi022' 2024-04-20T21:34:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.083+0000 7f8b745d1740 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555fd1f0c240 0x555fd1f0e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.083+0000 7f8b745d1740 4 client.15315 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:34:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.083+0000 7f8b745d1740 1 -- 172.21.15.22:0/4062674340 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555fd1d6bb10 con 0x555fd1f0c240 2024-04-20T21:34:29.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.084+0000 7f8b63e00640 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555fd1f0c240 0x555fd1f0e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.084+0000 7f8b63e00640 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555fd1f0c240 0x555fd1f0e660 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=807b6ef54d77c127 server_cookie=fd322df35e08b8bd in_seq=0 out_seq=0 2024-04-20T21:34:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.084+0000 7f8b58c00640 10 client.15315 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.085+0000 7f8b58c00640 1 -- 172.21.15.22:0/4062674340 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555fd1d6bb10 con 0x555fd1f0c240 2024-04-20T21:34:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.085+0000 7f8b58c00640 10 client.15315 handle_command_reply: tid=0 2024-04-20T21:34:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.085+0000 7f8b745d1740 1 client.15315 shutdown 2024-04-20T21:34:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.086+0000 7f8b745d1740 1 -- 172.21.15.22:0/4062674340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e830f0 msgr2=0x555fd1e866f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.086+0000 7f8b745d1740 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e830f0 0x555fd1e866f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8b64010010 tx=0x7f8b64026040 comp rx=0 tx=0).stop 2024-04-20T21:34:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.086+0000 7f8b745d1740 1 -- 172.21.15.22:0/4062674340 shutdown_connections 2024-04-20T21:34:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.086+0000 7f8b745d1740 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555fd1e8c5f0 0x555fd1f0a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.086+0000 7f8b745d1740 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555fd1f0c240 0x555fd1f0e660 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.086+0000 7f8b745d1740 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fd1e83a10 0x555fd1e86c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.086+0000 7f8b745d1740 1 --2- 172.21.15.22:0/4062674340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555fd1e830f0 0x555fd1e866f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.086+0000 7f8b58c00640 0 client.15315 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.086+0000 7f8b745d1740 1 -- 172.21.15.22:0/4062674340 >> 172.21.15.22:0/4062674340 conn(0x555fd1e7d820 msgr2=0x555fd1e7c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.086+0000 7f8b745d1740 1 -- 172.21.15.22:0/4062674340 shutdown_connections 2024-04-20T21:34:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.086+0000 7f8b745d1740 1 -- 172.21.15.22:0/4062674340 wait complete. 2024-04-20T21:34:29.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.086+0000 7f8b745d1740 20 client.15315 trim_cache size 0 max 16384 2024-04-20T21:34:29.087 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:34:29.088 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:34:29.088 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "55acd099-1bf0-466e-9f8e-8dde97f0bf9d", 2024-04-20T21:34:29.088 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:34:29.088 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:34:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.087+0000 7f8b6be00640 1 -- 172.21.15.22:0/3252045135 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f8b500a3ec0 msgr2=0x7f8b500a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.087+0000 7f8b6be00640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f8b500a3ec0 0x7f8b500a6380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0ef790 tx=0x7f8b5c02bb70 comp rx=0 tx=0).stop 2024-04-20T21:34:29.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.087+0000 7f8b6be00640 1 -- 172.21.15.22:0/3252045135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b6c104270 msgr2=0x7f8b6c1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.087+0000 7f8b6be00640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b6c104270 0x7f8b6c1a0670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c037040 tx=0x7f8b5c0045b0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.087+0000 7f8b6be00640 1 -- 172.21.15.22:0/3252045135 shutdown_connections 2024-04-20T21:34:29.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.087+0000 7f8b6be00640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f8b500a3ec0 0x7f8b500a6380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.087+0000 7f8b6be00640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b6c19a730 0x7f8b6c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.087+0000 7f8b6be00640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b6c109340 0x7f8b6c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.087+0000 7f8b6be00640 1 --2- 172.21.15.22:0/3252045135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b6c104270 0x7f8b6c1a0670 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.087+0000 7f8b6be00640 1 -- 172.21.15.22:0/3252045135 >> 172.21.15.22:0/3252045135 conn(0x7f8b6c0ffad0 msgr2=0x7f8b6c100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:29.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.087+0000 7f8b6be00640 1 -- 172.21.15.22:0/3252045135 shutdown_connections 2024-04-20T21:34:29.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.087+0000 7f8b6be00640 1 -- 172.21.15.22:0/3252045135 wait complete. 2024-04-20T21:34:29.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:34:29.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:34:29.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.319+0000 7f0322400640 1 -- 172.21.15.22:0/2313864010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f031c102a20 msgr2=0x7f031c106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.319+0000 7f0322400640 1 --2- 172.21.15.22:0/2313864010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f031c102a20 0x7f031c106a60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0304009900 tx=0x7f030402bc30 comp rx=0 tx=0).stop 2024-04-20T21:34:29.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.319+0000 7f0322400640 1 -- 172.21.15.22:0/2313864010 shutdown_connections 2024-04-20T21:34:29.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.319+0000 7f0322400640 1 --2- 172.21.15.22:0/2313864010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f031c107140 0x7f031c10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.319+0000 7f0322400640 1 --2- 172.21.15.22:0/2313864010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f031c102a20 0x7f031c106a60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.319+0000 7f0322400640 1 --2- 172.21.15.22:0/2313864010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c102070 0x7f031c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.319+0000 7f0322400640 1 -- 172.21.15.22:0/2313864010 >> 172.21.15.22:0/2313864010 conn(0x7f031c0fd930 msgr2=0x7f031c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:29.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.319+0000 7f0322400640 1 -- 172.21.15.22:0/2313864010 shutdown_connections 2024-04-20T21:34:29.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.319+0000 7f0322400640 1 -- 172.21.15.22:0/2313864010 wait complete. 2024-04-20T21:34:29.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.320+0000 7f0322400640 1 Processor -- start 2024-04-20T21:34:29.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.320+0000 7f0322400640 1 -- start start 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.321+0000 7f0322400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c102070 0x7f031c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.321+0000 7f0322400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f031c107140 0x7f031c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.321+0000 7f0322400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f031c19a730 0x7f031c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.321+0000 7f0322400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f031c112220 con 0x7f031c107140 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.321+0000 7f0322400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1120a0 con 0x7f031c102070 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.321+0000 7f0322400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1123a0 con 0x7f031c19a730 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.321+0000 7f031be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f031c107140 0x7f031c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.321+0000 7f0321000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c102070 0x7f031c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.321+0000 7f0321000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c102070 0x7f031c1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33790/0 (socket says 172.21.15.22:33790) 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.321+0000 7f0321000640 1 -- 172.21.15.22:0/3178024646 learned_addr learned my addr 172.21.15.22:0/3178024646 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.321+0000 7f0321a00640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f031c19a730 0x7f031c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.322+0000 7f0321000640 1 -- 172.21.15.22:0/3178024646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f031c19a730 msgr2=0x7f031c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.322+0000 7f0321000640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f031c19a730 0x7f031c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.322+0000 7f0321000640 1 -- 172.21.15.22:0/3178024646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f031c107140 msgr2=0x7f031c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.322+0000 7f0321000640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f031c107140 0x7f031c1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.322+0000 7f0321000640 1 -- 172.21.15.22:0/3178024646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c19b330 con 0x7f031c102070 2024-04-20T21:34:29.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.322+0000 7f0321000640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c102070 0x7f031c1a0670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f030c00eb90 tx=0x7f030c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:29.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.322+0000 7f0319600640 1 -- 172.21.15.22:0/3178024646 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c004d20 con 0x7f031c102070 2024-04-20T21:34:29.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.322+0000 7f0319600640 1 -- 172.21.15.22:0/3178024646 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030c00ee90 con 0x7f031c102070 2024-04-20T21:34:29.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.322+0000 7f0319600640 1 -- 172.21.15.22:0/3178024646 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c010830 con 0x7f031c102070 2024-04-20T21:34:29.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.322+0000 7f0322400640 1 -- 172.21.15.22:0/3178024646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031c1a7410 con 0x7f031c102070 2024-04-20T21:34:29.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.322+0000 7f0322400640 1 -- 172.21.15.22:0/3178024646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031c1a77c0 con 0x7f031c102070 2024-04-20T21:34:29.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.326+0000 7f0319600640 1 -- 172.21.15.22:0/3178024646 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7f030c005020 con 0x7f031c102070 2024-04-20T21:34:29.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.326+0000 7f03267ec740 1 Processor -- start 2024-04-20T21:34:29.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.326+0000 7f03267ec740 1 -- start start 2024-04-20T21:34:29.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.326+0000 7f03267ec740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e260983380 0x55e260983760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.326+0000 7f03267ec740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e26098c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.326+0000 7f03267ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e26098c880 0x55e26098ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.326+0000 7f03267ec740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e2608e9150 con 0x55e260983380 2024-04-20T21:34:29.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.326+0000 7f03267ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e26088f370 con 0x55e26098c880 2024-04-20T21:34:29.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.326+0000 7f03267ec740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e2608e5110 con 0x55e260983ca0 2024-04-20T21:34:29.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.327+0000 7f0321000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e26098c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.327+0000 7f031be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e26098c880 0x55e26098ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.327+0000 7f031be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e26098c880 0x55e26098ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33792/0 (socket says 172.21.15.22:33792) 2024-04-20T21:34:29.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.327+0000 7f031be00640 1 -- 172.21.15.22:0/642343760 learned_addr learned my addr 172.21.15.22:0/642343760 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:29.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.327+0000 7f0321a00640 1 --2- 172.21.15.22:0/642343760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e260983380 0x55e260983760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.327+0000 7f031be00640 1 -- 172.21.15.22:0/642343760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 msgr2=0x55e26098c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.327+0000 7f031be00640 1 --2- 172.21.15.22:0/642343760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e26098c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.327+0000 7f031be00640 1 -- 172.21.15.22:0/642343760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e260983380 msgr2=0x55e260983760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.327+0000 7f031be00640 1 --2- 172.21.15.22:0/642343760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e260983380 0x55e260983760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.327+0000 7f031be00640 1 -- 172.21.15.22:0/642343760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2606c57d0 con 0x55e26098c880 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.327+0000 7f0321000640 1 --2- 172.21.15.22:0/642343760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e26098c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.327+0000 7f0319600640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f03000a3e40 0x7f03000a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.328+0000 7f031be00640 1 --2- 172.21.15.22:0/642343760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e26098c880 0x55e26098ec70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0304000c00 tx=0x7f03040095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39258b1f3b0c1386 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.328+0000 7f0319600640 1 -- 172.21.15.22:0/3178024646 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f030c0d1140 con 0x7f031c102070 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.328+0000 7f0321a00640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f03000a3e40 0x7f03000a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.328+0000 7f0312000640 1 -- 172.21.15.22:0/642343760 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304045070 con 0x55e26098c880 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.328+0000 7f0312000640 1 -- 172.21.15.22:0/642343760 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030402c880 con 0x55e26098c880 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.328+0000 7f0312000640 1 -- 172.21.15.22:0/642343760 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304004b60 con 0x55e26098c880 2024-04-20T21:34:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.329+0000 7f0321a00640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f03000a3e40 0x7f03000a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f031400b6c0 tx=0x7f031400b2e0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:29.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.329+0000 7f03267ec740 1 -- 172.21.15.22:0/642343760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e26098c880 msgr2=0x55e26098ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.329+0000 7f03267ec740 1 --2- 172.21.15.22:0/642343760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e26098c880 0x55e26098ec70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0304000c00 tx=0x7f03040095e0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.329+0000 7f03267ec740 1 -- 172.21.15.22:0/642343760 shutdown_connections 2024-04-20T21:34:29.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.329+0000 7f03267ec740 1 --2- 172.21.15.22:0/642343760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e26098c880 0x55e26098ec70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.329+0000 7f03267ec740 1 --2- 172.21.15.22:0/642343760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e26098c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.329+0000 7f03267ec740 1 --2- 172.21.15.22:0/642343760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e260983380 0x55e260983760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.329+0000 7f03267ec740 1 -- 172.21.15.22:0/642343760 >> 172.21.15.22:0/642343760 conn(0x55e26097dab0 msgr2=0x55e26097de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:29.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.329+0000 7f03267ec740 1 -- 172.21.15.22:0/642343760 shutdown_connections 2024-04-20T21:34:29.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.329+0000 7f03267ec740 1 -- 172.21.15.22:0/642343760 wait complete. 2024-04-20T21:34:29.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.329+0000 7f03267ec740 1 -- start start 2024-04-20T21:34:29.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.330+0000 7f03267ec740 1 Processor -- start 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.330+0000 7f03267ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e260989860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.330+0000 7f03267ec740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e260989da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.330+0000 7f03267ec740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 0x55e26098a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.330+0000 7f03267ec740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e26098f610 con 0x55e26098c880 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.330+0000 7f03267ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e2608a2740 con 0x55e260983380 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.330+0000 7f03267ec740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e26098f790 con 0x55e260983ca0 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.330+0000 7f03267ec740 10 client.0 _ll_register_callbacks cb 0x55e26089db20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.331+0000 7f031be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e260989da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.331+0000 7f0321a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 0x55e26098a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.331+0000 7f0321000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e260989860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.331+0000 7f031be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e260989da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36872/0 (socket says 172.21.15.22:36872) 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.331+0000 7f031be00640 1 -- 172.21.15.22:0/2006439501 learned_addr learned my addr 172.21.15.22:0/2006439501 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.331+0000 7f031be00640 1 -- 172.21.15.22:0/2006439501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 msgr2=0x55e260989860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.331+0000 7f031be00640 1 --2- 172.21.15.22:0/2006439501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e260989860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.331+0000 7f031be00640 1 -- 172.21.15.22:0/2006439501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 msgr2=0x55e26098a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.331+0000 7f031be00640 1 --2- 172.21.15.22:0/2006439501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 0x55e26098a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.331+0000 7f031be00640 1 -- 172.21.15.22:0/2006439501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2605fb3f0 con 0x55e260983ca0 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.331+0000 7f031be00640 1 --2- 172.21.15.22:0/2006439501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e260989da0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0304006010 tx=0x7f03040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0229d9ca0ab250 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.331+0000 7f030aa00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.332+0000 7f030aa00640 1 -- 172.21.15.22:0/2006439501 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304045040 con 0x55e260983ca0 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.332+0000 7f030aa00640 1 -- 172.21.15.22:0/2006439501 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f03040052f0 con 0x55e260983ca0 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.332+0000 7f03267ec740 1 -- 172.21.15.22:0/2006439501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f030402c880 con 0x55e260983ca0 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.332+0000 7f030aa00640 1 -- 172.21.15.22:0/2006439501 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f030405b050 con 0x55e260983ca0 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.332+0000 7f03267ec740 10 client.25468 fetch_fsmap learned FSMap version 20 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.332+0000 7f03267ec740 1 -- 172.21.15.22:0/2006439501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e2605bb0f0 con 0x55e260983ca0 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.332+0000 7f030aa00640 1 -- 172.21.15.22:0/2006439501 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030402c880 con 0x55e260983ca0 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.333+0000 7f030aa00640 1 -- 172.21.15.22:0/2006439501 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f030404ac10 con 0x55e260983ca0 2024-04-20T21:34:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.333+0000 7f03267ec740 10 client.25468 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:34:29.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.333+0000 7f03267ec740 10 client.25468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:29.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.333+0000 7f03267ec740 20 client.25468 populate_metadata read hostname 'smithi022' 2024-04-20T21:34:29.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.333+0000 7f03267ec740 1 --2- 172.21.15.22:0/2006439501 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e26099c9e0 0x55e26099edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.333+0000 7f03267ec740 4 client.25468 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:29.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.333+0000 7f03267ec740 1 -- 172.21.15.22:0/2006439501 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2608f6b70 con 0x55e26099c9e0 2024-04-20T21:34:29.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.333+0000 7f0321000640 1 --2- 172.21.15.22:0/2006439501 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e26099c9e0 0x55e26099edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.334+0000 7f0321000640 1 --2- 172.21.15.22:0/2006439501 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e26099c9e0 0x55e26099edd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abb0f80bb0185320 server_cookie=92bc3a2ebc3140a5 in_seq=0 out_seq=0 2024-04-20T21:34:29.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.334+0000 7f030aa00640 10 client.25468 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f030aa00640 1 -- 172.21.15.22:0/2006439501 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2608f6b70 con 0x55e26099c9e0 2024-04-20T21:34:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f030aa00640 10 client.25468 handle_command_reply: tid=0 2024-04-20T21:34:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f03267ec740 1 client.25468 shutdown 2024-04-20T21:34:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f03267ec740 1 -- 172.21.15.22:0/2006439501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 msgr2=0x55e260989da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f03267ec740 1 --2- 172.21.15.22:0/2006439501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e260989da0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0304006010 tx=0x7f03040046c0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f03267ec740 1 -- 172.21.15.22:0/2006439501 shutdown_connections 2024-04-20T21:34:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f03267ec740 1 --2- 172.21.15.22:0/2006439501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 0x55e26098a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f03267ec740 1 --2- 172.21.15.22:0/2006439501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e260989da0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f03267ec740 1 --2- 172.21.15.22:0/2006439501 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e26099c9e0 0x55e26099edd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f03267ec740 1 --2- 172.21.15.22:0/2006439501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e260989860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f030aa00640 0 client.25468 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f03267ec740 1 -- 172.21.15.22:0/2006439501 >> 172.21.15.22:0/2006439501 conn(0x55e26097dab0 msgr2=0x55e26097f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f03267ec740 1 -- 172.21.15.22:0/2006439501 shutdown_connections 2024-04-20T21:34:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f03267ec740 1 -- 172.21.15.22:0/2006439501 wait complete. 2024-04-20T21:34:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.335+0000 7f03267ec740 20 client.25468 trim_cache size 0 max 16384 2024-04-20T21:34:29.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.355+0000 7f03267ec740 1 Processor -- start 2024-04-20T21:34:29.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.355+0000 7f03267ec740 1 -- start start 2024-04-20T21:34:29.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.355+0000 7f03267ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e26098bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.355+0000 7f03267ec740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e2609984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.355+0000 7f03267ec740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 0x55e260989f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.355+0000 7f03267ec740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1adc80 con 0x55e26098c880 2024-04-20T21:34:29.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.355+0000 7f03267ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1b0500 con 0x55e260983380 2024-04-20T21:34:29.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.355+0000 7f03267ec740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1ae2f0 con 0x55e260983ca0 2024-04-20T21:34:29.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.356+0000 7f031be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e2609984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.356+0000 7f0321a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 0x55e260989f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.356+0000 7f031be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e2609984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36874/0 (socket says 172.21.15.22:36874) 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.356+0000 7f031be00640 1 -- 172.21.15.22:0/1406062709 learned_addr learned my addr 172.21.15.22:0/1406062709 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.356+0000 7f0321000640 1 --2- 172.21.15.22:0/1406062709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e26098bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.356+0000 7f031be00640 1 -- 172.21.15.22:0/1406062709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 msgr2=0x55e26098bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.356+0000 7f031be00640 1 --2- 172.21.15.22:0/1406062709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e26098bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.356+0000 7f031be00640 1 -- 172.21.15.22:0/1406062709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 msgr2=0x55e260989f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.356+0000 7f031be00640 1 --2- 172.21.15.22:0/1406062709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 0x55e260989f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.356+0000 7f031be00640 1 -- 172.21.15.22:0/1406062709 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e260987db0 con 0x55e260983ca0 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.356+0000 7f031be00640 1 --2- 172.21.15.22:0/1406062709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e2609984d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f030403f440 tx=0x7f03040097c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f319571dbceb542b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.356+0000 7f0311600640 1 -- 172.21.15.22:0/1406062709 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304045070 con 0x55e260983ca0 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.357+0000 7f0311600640 1 -- 172.21.15.22:0/1406062709 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030402c880 con 0x55e260983ca0 2024-04-20T21:34:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.357+0000 7f0311600640 1 -- 172.21.15.22:0/1406062709 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030403f970 con 0x55e260983ca0 2024-04-20T21:34:29.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.357+0000 7f03267ec740 1 -- 172.21.15.22:0/1406062709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 msgr2=0x55e2609984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.357+0000 7f03267ec740 1 --2- 172.21.15.22:0/1406062709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e2609984d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f030403f440 tx=0x7f03040097c0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.357+0000 7f03267ec740 1 -- 172.21.15.22:0/1406062709 shutdown_connections 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.357+0000 7f03267ec740 1 --2- 172.21.15.22:0/1406062709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 0x55e260989f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.357+0000 7f03267ec740 1 --2- 172.21.15.22:0/1406062709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e2609984d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.357+0000 7f03267ec740 1 --2- 172.21.15.22:0/1406062709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e26098bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.357+0000 7f03267ec740 1 -- 172.21.15.22:0/1406062709 >> 172.21.15.22:0/1406062709 conn(0x55e26097dab0 msgr2=0x55e260989820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.357+0000 7f03267ec740 1 -- 172.21.15.22:0/1406062709 shutdown_connections 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.357+0000 7f03267ec740 1 -- 172.21.15.22:0/1406062709 wait complete. 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.358+0000 7f03267ec740 1 -- start start 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.358+0000 7f03267ec740 1 Processor -- start 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.358+0000 7f03267ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e260986960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.358+0000 7f03267ec740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e260986ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.358+0000 7f03267ec740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 0x55e260a01d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.358+0000 7f03267ec740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1ae9c0 con 0x55e26098c880 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.358+0000 7f03267ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1afe30 con 0x55e260983380 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.358+0000 7f03267ec740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1af090 con 0x55e260983ca0 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.358+0000 7f03267ec740 10 client.0 _ll_register_callbacks cb 0x7f0314004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f031be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e260986ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f031be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e260986ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:36878/0 (socket says 172.21.15.22:36878) 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f031be00640 1 -- 172.21.15.22:0/2085483632 learned_addr learned my addr 172.21.15.22:0/2085483632 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f0321a00640 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 0x55e260a01d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f0321000640 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e260986960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f0321000640 1 -- 172.21.15.22:0/2085483632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 msgr2=0x55e260986ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f0321000640 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e260986ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f0321000640 1 -- 172.21.15.22:0/2085483632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 msgr2=0x55e260a01d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f0321000640 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 0x55e260a01d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f0321000640 1 -- 172.21.15.22:0/2085483632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e260a022c0 con 0x55e260983380 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f031be00640 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e260986ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f0321000640 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e260986960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f030c0ce690 tx=0x7f030c005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d335c911939514b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.359+0000 7f030aa00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.360+0000 7f030aa00640 1 -- 172.21.15.22:0/2085483632 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c09c770 con 0x55e260983380 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.360+0000 7f03267ec740 1 -- 172.21.15.22:0/2085483632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f030402c880 con 0x55e260983380 2024-04-20T21:34:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.360+0000 7f030aa00640 1 -- 172.21.15.22:0/2085483632 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f030c005590 con 0x55e260983380 2024-04-20T21:34:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.360+0000 7f030aa00640 1 -- 172.21.15.22:0/2085483632 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f030c09d510 con 0x55e260983380 2024-04-20T21:34:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.360+0000 7f03267ec740 10 client.25433 fetch_fsmap learned FSMap version 20 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.360+0000 7f03267ec740 1 -- 172.21.15.22:0/2085483632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e260a034d0 con 0x55e260983380 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.360+0000 7f030aa00640 1 -- 172.21.15.22:0/2085483632 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030402c880 con 0x55e260983380 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.360+0000 7f030aa00640 1 -- 172.21.15.22:0/2085483632 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f030c0d4c40 con 0x55e260983380 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.360+0000 7f03267ec740 10 client.25433 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.360+0000 7f03267ec740 10 client.25433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.360+0000 7f03267ec740 20 client.25433 populate_metadata read hostname 'smithi022' 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.360+0000 7f03267ec740 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e260a03b50 0x55e260a05f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.361+0000 7f03267ec740 4 client.25433 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.361+0000 7f03267ec740 1 -- 172.21.15.22:0/2085483632 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e2608f09e0 con 0x55e260a03b50 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.361+0000 7f031be00640 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e260a03b50 0x55e260a05f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.361+0000 7f031be00640 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e260a03b50 0x55e260a05f70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e20fde78777170b4 server_cookie=d1a07dafb8b88abe in_seq=0 out_seq=0 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f030aa00640 10 client.25433 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f030aa00640 1 -- 172.21.15.22:0/2085483632 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e2608f09e0 con 0x55e260a03b50 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f030aa00640 10 client.25433 handle_command_reply: tid=0 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f03267ec740 1 client.25433 shutdown 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f03267ec740 1 -- 172.21.15.22:0/2085483632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 msgr2=0x55e260986960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f03267ec740 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e260986960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f030c0ce690 tx=0x7f030c005960 comp rx=0 tx=0).stop 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f03267ec740 1 -- 172.21.15.22:0/2085483632 shutdown_connections 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f03267ec740 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e26098c880 0x55e260a01d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f03267ec740 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e260983ca0 0x55e260986ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f03267ec740 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e260a03b50 0x55e260a05f70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f03267ec740 1 --2- 172.21.15.22:0/2085483632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e260983380 0x55e260986960 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f03267ec740 1 -- 172.21.15.22:0/2085483632 >> 172.21.15.22:0/2085483632 conn(0x55e26097dab0 msgr2=0x55e26097cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f030aa00640 0 client.25433 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f03267ec740 1 -- 172.21.15.22:0/2085483632 shutdown_connections 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f03267ec740 1 -- 172.21.15.22:0/2085483632 wait complete. 2024-04-20T21:34:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.362+0000 7f03267ec740 20 client.25433 trim_cache size 0 max 16384 2024-04-20T21:34:29.365 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:34:29.365 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:34:29.365 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:34:29.365 INFO:teuthology.orchestra.run.smithi022.stdout: "55acd099-1bf0-466e-9f8e-8dde97f0bf9d": { 2024-04-20T21:34:29.365 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:34:29.365 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "55acd099-1bf0-466e-9f8e-8dde97f0bf9d", 2024-04-20T21:34:29.365 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:34:29.365 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:34:29.365 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:34:29.365 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:34:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.363+0000 7f0322400640 1 -- 172.21.15.22:0/3178024646 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f03000a3e40 msgr2=0x7f03000a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.363+0000 7f0322400640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f03000a3e40 0x7f03000a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f031400b6c0 tx=0x7f031400b2e0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.363+0000 7f0322400640 1 -- 172.21.15.22:0/3178024646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c102070 msgr2=0x7f031c1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.363+0000 7f0322400640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c102070 0x7f031c1a0670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f030c00eb90 tx=0x7f030c00c6a0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.363+0000 7f0322400640 1 -- 172.21.15.22:0/3178024646 shutdown_connections 2024-04-20T21:34:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.363+0000 7f0322400640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f03000a3e40 0x7f03000a6300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.363+0000 7f0322400640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f031c19a730 0x7f031c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.363+0000 7f0322400640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f031c107140 0x7f031c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.363+0000 7f0322400640 1 --2- 172.21.15.22:0/3178024646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c102070 0x7f031c1a0670 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.363+0000 7f0322400640 1 -- 172.21.15.22:0/3178024646 >> 172.21.15.22:0/3178024646 conn(0x7f031c0fd930 msgr2=0x7f031c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.363+0000 7f0322400640 1 -- 172.21.15.22:0/3178024646 shutdown_connections 2024-04-20T21:34:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:29.363+0000 7f0322400640 1 -- 172.21.15.22:0/3178024646 wait complete. 2024-04-20T21:34:29.380 INFO:tasks.cephfs.filesystem:scrub status for tag:55acd099-1bf0-466e-9f8e-8dde97f0bf9d - {'path': '/', 'tag': '55acd099-1bf0-466e-9f8e-8dde97f0bf9d', 'options': 'recursive,force'} 2024-04-20T21:34:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:30 smithi022 ceph-mon[16843]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:30 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:34:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:30 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:30 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:34:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:30 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:30.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:30 smithi122 ceph-mon[22147]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:30.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:30 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:34:30.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:30 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:30.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:30 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:34:30.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:30 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:30.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:30 smithi033 ceph-mon[21626]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:30.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:30 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:34:30.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:30 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:30.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:30 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:34:30.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:30 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:32.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:32 smithi022 ceph-mon[16843]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:34:32.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:32 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:34:32.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:32 smithi122 ceph-mon[22147]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:34:32.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:32 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:34:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:32 smithi033 ceph-mon[21626]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:34:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:32 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:34:34.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:34 smithi022 ceph-mon[16843]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:34:34.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:34 smithi122 ceph-mon[22147]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:34:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:34 smithi033 ceph-mon[21626]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:34:36.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:36 smithi022 ceph-mon[16843]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:34:36.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:36 smithi122 ceph-mon[22147]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:34:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:36 smithi033 ceph-mon[21626]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:34:38.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:38 smithi022 ceph-mon[16843]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 938 B/s wr, 3 op/s 2024-04-20T21:34:38.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:38 smithi122 ceph-mon[22147]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 938 B/s wr, 3 op/s 2024-04-20T21:34:38.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:38 smithi033 ceph-mon[21626]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 938 B/s wr, 3 op/s 2024-04-20T21:34:40.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:40 smithi022 ceph-mon[16843]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 2 op/s 2024-04-20T21:34:40.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:40 smithi122 ceph-mon[22147]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 2 op/s 2024-04-20T21:34:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:40 smithi033 ceph-mon[21626]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 2 op/s 2024-04-20T21:34:42.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:42 smithi022 ceph-mon[16843]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 938 B/s wr, 3 op/s 2024-04-20T21:34:42.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:42 smithi122 ceph-mon[22147]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 938 B/s wr, 3 op/s 2024-04-20T21:34:42.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:42 smithi033 ceph-mon[21626]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 938 B/s wr, 3 op/s 2024-04-20T21:34:44.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:44 smithi022 ceph-mon[16843]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-04-20T21:34:44.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:44 smithi122 ceph-mon[22147]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-04-20T21:34:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:44 smithi033 ceph-mon[21626]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-04-20T21:34:46.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:46 smithi022 ceph-mon[16843]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-04-20T21:34:46.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:46 smithi122 ceph-mon[22147]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-04-20T21:34:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:46 smithi033 ceph-mon[21626]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-04-20T21:34:48.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:48 smithi022 ceph-mon[16843]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-04-20T21:34:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:48 smithi122 ceph-mon[22147]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-04-20T21:34:48.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:48 smithi033 ceph-mon[21626]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-04-20T21:34:50.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:50 smithi022 ceph-mon[16843]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:50.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:50 smithi122 ceph-mon[22147]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:50.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:50 smithi033 ceph-mon[21626]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:52 smithi122 ceph-mon[22147]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:34:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:52 smithi033 ceph-mon[21626]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:34:52.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:52 smithi022 ceph-mon[16843]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:34:54.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:54 smithi122 ceph-mon[22147]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:54.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:54 smithi033 ceph-mon[21626]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:54.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:54 smithi022 ceph-mon[16843]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:34:56.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:56 smithi122 ceph-mon[22147]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:34:56.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:56 smithi033 ceph-mon[21626]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:34:56.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:56 smithi022 ceph-mon[16843]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:34:58.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:58 smithi122 ceph-mon[22147]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:34:58.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:58 smithi033 ceph-mon[21626]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:34:58.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:58 smithi022 ceph-mon[16843]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:34:59.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:34:59.381 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:34:59.442 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:34:59 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:34:59.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:34:59 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:34:59.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:34:59 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:34:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.572+0000 7f1435200640 1 -- 172.21.15.22:0/23502813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1430074310 msgr2=0x7f14300746f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.572+0000 7f1435200640 1 --2- 172.21.15.22:0/23502813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1430074310 0x7f14300746f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f142002bcc0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.572+0000 7f1435200640 1 -- 172.21.15.22:0/23502813 shutdown_connections 2024-04-20T21:34:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.572+0000 7f1435200640 1 --2- 172.21.15.22:0/23502813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f143010e6c0 0x7f1430110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.572+0000 7f1435200640 1 --2- 172.21.15.22:0/23502813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430074c30 0x7f143010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.572+0000 7f1435200640 1 --2- 172.21.15.22:0/23502813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1430074310 0x7f14300746f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.572+0000 7f1435200640 1 -- 172.21.15.22:0/23502813 >> 172.21.15.22:0/23502813 conn(0x7f14300fd930 msgr2=0x7f14300ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.572+0000 7f1435200640 1 -- 172.21.15.22:0/23502813 shutdown_connections 2024-04-20T21:34:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.572+0000 7f1435200640 1 -- 172.21.15.22:0/23502813 wait complete. 2024-04-20T21:34:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.573+0000 7f1435200640 1 Processor -- start 2024-04-20T21:34:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.573+0000 7f1435200640 1 -- start start 2024-04-20T21:34:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.573+0000 7f1435200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430074310 0x7f14301a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.573+0000 7f1435200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1430074c30 0x7f14301a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f1435200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f143010e6c0 0x7f143019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f1435200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14301143d0 con 0x7f1430074310 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f1435200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1430114250 con 0x7f143010e6c0 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f1435200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1430114550 con 0x7f1430074c30 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f142f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430074310 0x7f14301a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f142ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1430074c30 0x7f14301a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f142ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1430074c30 0x7f14301a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:56686/0 (socket says 172.21.15.22:56686) 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f142ea00640 1 -- 172.21.15.22:0/119663555 learned_addr learned my addr 172.21.15.22:0/119663555 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f142fe00640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f143010e6c0 0x7f143019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f142ea00640 1 -- 172.21.15.22:0/119663555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f143010e6c0 msgr2=0x7f143019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f142ea00640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f143010e6c0 0x7f143019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f142ea00640 1 -- 172.21.15.22:0/119663555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430074310 msgr2=0x7f14301a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f142ea00640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430074310 0x7f14301a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.574+0000 7f142ea00640 1 -- 172.21.15.22:0/119663555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143019afb0 con 0x7f1430074c30 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.575+0000 7f142ea00640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1430074c30 0x7f14301a0b70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f141c00e9f0 tx=0x7f141c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.575+0000 7f1427e00640 1 -- 172.21.15.22:0/119663555 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c00cd80 con 0x7f1430074c30 2024-04-20T21:34:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.575+0000 7f1427e00640 1 -- 172.21.15.22:0/119663555 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141c010910 con 0x7f1430074c30 2024-04-20T21:34:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.575+0000 7f1427e00640 1 -- 172.21.15.22:0/119663555 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c0188f0 con 0x7f1430074c30 2024-04-20T21:34:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.575+0000 7f1435200640 1 -- 172.21.15.22:0/119663555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143019b2a0 con 0x7f1430074c30 2024-04-20T21:34:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.575+0000 7f1435200640 1 -- 172.21.15.22:0/119663555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14301a75b0 con 0x7f1430074c30 2024-04-20T21:34:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.580+0000 7f1439583740 1 Processor -- start 2024-04-20T21:34:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.580+0000 7f1439583740 1 -- start start 2024-04-20T21:34:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.581+0000 7f1439583740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331ea1190 0x55b331ea1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.581+0000 7f1439583740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1b40 0x55b331eb2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.581+0000 7f1439583740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331eb3090 0x55b331eb5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.581+0000 7f1439583740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b331e07150 con 0x55b331ea1190 2024-04-20T21:34:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.581+0000 7f1439583740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b331dad370 con 0x55b331ea1b40 2024-04-20T21:34:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.581+0000 7f1439583740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b331e03110 con 0x55b331eb3090 2024-04-20T21:34:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.581+0000 7f1427e00640 1 -- 172.21.15.22:0/119663555 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7f141c010ac0 con 0x7f1430074c30 2024-04-20T21:34:59.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.581+0000 7f142fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331ea1190 0x55b331ea1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.581+0000 7f142ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331eb3090 0x55b331eb5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.581+0000 7f142ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331eb3090 0x55b331eb5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:56690/0 (socket says 172.21.15.22:56690) 2024-04-20T21:34:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.582+0000 7f142ea00640 1 -- 172.21.15.22:0/4061303360 learned_addr learned my addr 172.21.15.22:0/4061303360 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.582+0000 7f142f400640 1 --2- 172.21.15.22:0/4061303360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1b40 0x55b331eb2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.582+0000 7f142ea00640 1 -- 172.21.15.22:0/4061303360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1b40 msgr2=0x55b331eb2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.582+0000 7f142ea00640 1 --2- 172.21.15.22:0/4061303360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1b40 0x55b331eb2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.582+0000 7f142ea00640 1 -- 172.21.15.22:0/4061303360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331ea1190 msgr2=0x55b331ea1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.582+0000 7f142ea00640 1 --2- 172.21.15.22:0/4061303360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331ea1190 0x55b331ea1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.582+0000 7f142ea00640 1 -- 172.21.15.22:0/4061303360 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b331be37d0 con 0x55b331eb3090 2024-04-20T21:34:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.582+0000 7f142f400640 1 --2- 172.21.15.22:0/4061303360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1b40 0x55b331eb2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.582+0000 7f142ea00640 1 --2- 172.21.15.22:0/4061303360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331eb3090 0x55b331eb5480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f141c0ce380 tx=0x7f141c0d9570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71449284d4ce25e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.583+0000 7f1424c00640 1 -- 172.21.15.22:0/4061303360 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c0e4070 con 0x55b331eb3090 2024-04-20T21:34:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.583+0000 7f1424c00640 1 -- 172.21.15.22:0/4061303360 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141c0dfae0 con 0x55b331eb3090 2024-04-20T21:34:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.583+0000 7f1424c00640 1 -- 172.21.15.22:0/4061303360 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c0debb0 con 0x55b331eb3090 2024-04-20T21:34:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.583+0000 7f1427e00640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f14140a3e40 0x7f14140a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.583+0000 7f1427e00640 1 -- 172.21.15.22:0/119663555 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f141c0d0ec0 con 0x7f1430074c30 2024-04-20T21:34:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.583+0000 7f142fe00640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f14140a3e40 0x7f14140a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.584+0000 7f1439583740 1 -- 172.21.15.22:0/4061303360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331eb3090 msgr2=0x55b331eb5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.584+0000 7f1439583740 1 --2- 172.21.15.22:0/4061303360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331eb3090 0x55b331eb5480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f141c0ce380 tx=0x7f141c0d9570 comp rx=0 tx=0).stop 2024-04-20T21:34:59.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.585+0000 7f1439583740 1 -- 172.21.15.22:0/4061303360 shutdown_connections 2024-04-20T21:34:59.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.585+0000 7f142fe00640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f14140a3e40 0x7f14140a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1428005fd0 tx=0x7f1428009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:59.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.585+0000 7f1439583740 1 --2- 172.21.15.22:0/4061303360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331eb3090 0x55b331eb5480 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.585+0000 7f1439583740 1 --2- 172.21.15.22:0/4061303360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1b40 0x55b331eb2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.585+0000 7f1439583740 1 --2- 172.21.15.22:0/4061303360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331ea1190 0x55b331ea1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.585+0000 7f1439583740 1 -- 172.21.15.22:0/4061303360 >> 172.21.15.22:0/4061303360 conn(0x55b331e9bab0 msgr2=0x55b331e9be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.585+0000 7f1439583740 1 -- 172.21.15.22:0/4061303360 shutdown_connections 2024-04-20T21:34:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.585+0000 7f1439583740 1 -- 172.21.15.22:0/4061303360 wait complete. 2024-04-20T21:34:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.586+0000 7f1439583740 1 -- start start 2024-04-20T21:34:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.586+0000 7f1439583740 1 Processor -- start 2024-04-20T21:34:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.586+0000 7f1439583740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1190 0x55b331eb00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.586+0000 7f1439583740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 0x55b331eb0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.586+0000 7f1439583740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331eb3090 0x55b331eb0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.586+0000 7f1439583740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b331eb5e50 con 0x55b331eb3090 2024-04-20T21:34:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f1439583740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b331dc0740 con 0x55b331ea1190 2024-04-20T21:34:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f1439583740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b331eb5fd0 con 0x55b331ea1b40 2024-04-20T21:34:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f1439583740 10 client.0 _ll_register_callbacks cb 0x55b331dbbb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f142f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1190 0x55b331eb00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f142ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 0x55b331eb0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f142ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 0x55b331eb0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:56704/0 (socket says 172.21.15.22:56704) 2024-04-20T21:34:59.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f142ea00640 1 -- 172.21.15.22:0/577440343 learned_addr learned my addr 172.21.15.22:0/577440343 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:59.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f142fe00640 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331eb3090 0x55b331eb0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f142ea00640 1 -- 172.21.15.22:0/577440343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1190 msgr2=0x55b331eb00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f142ea00640 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1190 0x55b331eb00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f142ea00640 1 -- 172.21.15.22:0/577440343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331eb3090 msgr2=0x55b331eb0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f142ea00640 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331eb3090 0x55b331eb0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f142ea00640 1 -- 172.21.15.22:0/577440343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b331b193f0 con 0x55b331ea1b40 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.587+0000 7f142fe00640 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331eb3090 0x55b331eb0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.588+0000 7f142ea00640 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 0x55b331eb0630 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f141c09cbd0 tx=0x7f141c0de450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c53edb6e2e923ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.588+0000 7f1419600640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.588+0000 7f1419600640 1 -- 172.21.15.22:0/577440343 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c098ee0 con 0x55b331ea1b40 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.588+0000 7f1419600640 1 -- 172.21.15.22:0/577440343 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f141c09cea0 con 0x55b331ea1b40 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.588+0000 7f1439583740 1 -- 172.21.15.22:0/577440343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f141c0dfae0 con 0x55b331ea1b40 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.588+0000 7f1419600640 1 -- 172.21.15.22:0/577440343 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f141c0ce3b0 con 0x55b331ea1b40 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.588+0000 7f1439583740 10 client.25486 fetch_fsmap learned FSMap version 20 2024-04-20T21:34:59.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.588+0000 7f1439583740 1 -- 172.21.15.22:0/577440343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b331ad90f0 con 0x55b331ea1b40 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.588+0000 7f1419600640 1 -- 172.21.15.22:0/577440343 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141c0dfae0 con 0x55b331ea1b40 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.589+0000 7f1419600640 1 -- 172.21.15.22:0/577440343 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f141c0cece0 con 0x55b331ea1b40 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.589+0000 7f1439583740 10 client.25486 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.589+0000 7f1439583740 10 client.25486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.589+0000 7f1439583740 20 client.25486 populate_metadata read hostname 'smithi022' 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.589+0000 7f1439583740 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b331ec32e0 0x55b331ec56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.589+0000 7f1439583740 4 client.25486 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.589+0000 7f1439583740 1 -- 172.21.15.22:0/577440343 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b331e14b70 con 0x55b331ec32e0 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.589+0000 7f142f400640 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b331ec32e0 0x55b331ec56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.590+0000 7f142f400640 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b331ec32e0 0x55b331ec56d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=905f4e2d80904149 server_cookie=efec80c860479bb2 in_seq=0 out_seq=0 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.590+0000 7f1419600640 10 client.25486 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.591+0000 7f1419600640 1 -- 172.21.15.22:0/577440343 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b331e14b70 con 0x55b331ec32e0 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.591+0000 7f1419600640 10 client.25486 handle_command_reply: tid=0 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.591+0000 7f1439583740 1 client.25486 shutdown 2024-04-20T21:34:59.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.592+0000 7f1439583740 1 -- 172.21.15.22:0/577440343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 msgr2=0x55b331eb0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.592+0000 7f1439583740 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 0x55b331eb0630 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f141c09cbd0 tx=0x7f141c0de450 comp rx=0 tx=0).stop 2024-04-20T21:34:59.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.592+0000 7f1439583740 1 -- 172.21.15.22:0/577440343 shutdown_connections 2024-04-20T21:34:59.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.592+0000 7f1439583740 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331eb3090 0x55b331eb0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.592+0000 7f1439583740 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 0x55b331eb0630 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.592+0000 7f1439583740 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b331ec32e0 0x55b331ec56d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.592+0000 7f1439583740 1 --2- 172.21.15.22:0/577440343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1190 0x55b331eb00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.592+0000 7f1419600640 0 client.25486 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:59.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.592+0000 7f1439583740 1 -- 172.21.15.22:0/577440343 >> 172.21.15.22:0/577440343 conn(0x55b331e9bab0 msgr2=0x55b331e9d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:59.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.592+0000 7f1439583740 1 -- 172.21.15.22:0/577440343 shutdown_connections 2024-04-20T21:34:59.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.592+0000 7f1439583740 1 -- 172.21.15.22:0/577440343 wait complete. 2024-04-20T21:34:59.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.592+0000 7f1439583740 20 client.25486 trim_cache size 0 max 16384 2024-04-20T21:34:59.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.615+0000 7f1439583740 1 Processor -- start 2024-04-20T21:34:59.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.615+0000 7f1439583740 1 -- start start 2024-04-20T21:34:59.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.615+0000 7f1439583740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1190 0x55b331eb2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f1439583740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 0x55b331e9d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f1439583740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331eb3090 0x55b331ec86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f1439583740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14301a99e0 con 0x55b331eb3090 2024-04-20T21:34:59.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f1439583740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14301ac240 con 0x55b331ea1190 2024-04-20T21:34:59.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f1439583740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14301aa030 con 0x55b331ea1b40 2024-04-20T21:34:59.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f142f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1190 0x55b331eb2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f142f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1190 0x55b331eb2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45474/0 (socket says 172.21.15.22:45474) 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f142f400640 1 -- 172.21.15.22:0/2391957500 learned_addr learned my addr 172.21.15.22:0/2391957500 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f142fe00640 1 --2- 172.21.15.22:0/2391957500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331eb3090 0x55b331ec86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f142ea00640 1 --2- 172.21.15.22:0/2391957500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 0x55b331e9d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f142f400640 1 -- 172.21.15.22:0/2391957500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 msgr2=0x55b331e9d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f142f400640 1 --2- 172.21.15.22:0/2391957500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 0x55b331e9d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.616+0000 7f142f400640 1 -- 172.21.15.22:0/2391957500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331eb3090 msgr2=0x55b331ec86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.617+0000 7f142f400640 1 --2- 172.21.15.22:0/2391957500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331eb3090 0x55b331ec86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.617+0000 7f142f400640 1 -- 172.21.15.22:0/2391957500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b331e9b5e0 con 0x55b331ea1190 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.617+0000 7f142ea00640 1 --2- 172.21.15.22:0/2391957500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 0x55b331e9d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.617+0000 7f142fe00640 1 --2- 172.21.15.22:0/2391957500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331eb3090 0x55b331ec86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.617+0000 7f142f400640 1 --2- 172.21.15.22:0/2391957500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1190 0x55b331eb2820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f142002da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a226abe9a928e183 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.617+0000 7f141be00640 1 -- 172.21.15.22:0/2391957500 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142002db80 con 0x55b331ea1190 2024-04-20T21:34:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.617+0000 7f141be00640 1 -- 172.21.15.22:0/2391957500 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142002dd20 con 0x55b331ea1190 2024-04-20T21:34:59.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.617+0000 7f141be00640 1 -- 172.21.15.22:0/2391957500 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14200049b0 con 0x55b331ea1190 2024-04-20T21:34:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.618+0000 7f1439583740 1 -- 172.21.15.22:0/2391957500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1190 msgr2=0x55b331eb2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.618+0000 7f1439583740 1 --2- 172.21.15.22:0/2391957500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1190 0x55b331eb2820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f142002da50 comp rx=0 tx=0).stop 2024-04-20T21:34:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.618+0000 7f1439583740 1 -- 172.21.15.22:0/2391957500 shutdown_connections 2024-04-20T21:34:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.618+0000 7f1439583740 1 --2- 172.21.15.22:0/2391957500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331eb3090 0x55b331ec86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.618+0000 7f1439583740 1 --2- 172.21.15.22:0/2391957500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1b40 0x55b331e9d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.618+0000 7f1439583740 1 --2- 172.21.15.22:0/2391957500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331ea1190 0x55b331eb2820 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.618+0000 7f1439583740 1 -- 172.21.15.22:0/2391957500 >> 172.21.15.22:0/2391957500 conn(0x55b331e9bab0 msgr2=0x55b331e9ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.619+0000 7f1439583740 1 -- 172.21.15.22:0/2391957500 shutdown_connections 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.619+0000 7f1439583740 1 -- 172.21.15.22:0/2391957500 wait complete. 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.619+0000 7f1439583740 1 -- start start 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.619+0000 7f1439583740 1 Processor -- start 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.620+0000 7f1439583740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1190 0x55b331ec61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.620+0000 7f1439583740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331ea1b40 0x55b331ec66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.620+0000 7f1439583740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331eb3090 0x55b331ec6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.620+0000 7f1439583740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14301aa700 con 0x55b331ea1b40 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.620+0000 7f1439583740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14301abb70 con 0x55b331eb3090 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.620+0000 7f1439583740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14301aadd0 con 0x55b331ea1190 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.620+0000 7f1439583740 10 client.0 _ll_register_callbacks cb 0x7f141c0e2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.620+0000 7f142fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331eb3090 0x55b331ec6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f142fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331eb3090 0x55b331ec6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45490/0 (socket says 172.21.15.22:45490) 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f142fe00640 1 -- 172.21.15.22:0/3916102731 learned_addr learned my addr 172.21.15.22:0/3916102731 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f142f400640 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1190 0x55b331ec61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f142ea00640 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331ea1b40 0x55b331ec66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f142fe00640 1 -- 172.21.15.22:0/3916102731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1190 msgr2=0x55b331ec61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f142fe00640 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1190 0x55b331ec61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f142fe00640 1 -- 172.21.15.22:0/3916102731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331ea1b40 msgr2=0x55b331ec66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f142fe00640 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331ea1b40 0x55b331ec66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f142fe00640 1 -- 172.21.15.22:0/3916102731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b331f28540 con 0x55b331eb3090 2024-04-20T21:34:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f142ea00640 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331ea1b40 0x55b331ec66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:34:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f142f400640 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1190 0x55b331ec61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:34:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f142fe00640 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331eb3090 0x55b331ec6c30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1428003f90 tx=0x7f1428004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ffb01f730909a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.621+0000 7f1419600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:34:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.622+0000 7f1419600640 1 -- 172.21.15.22:0/3916102731 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142802c060 con 0x55b331eb3090 2024-04-20T21:34:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.622+0000 7f1439583740 1 -- 172.21.15.22:0/3916102731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f142002dd20 con 0x55b331eb3090 2024-04-20T21:34:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.622+0000 7f1419600640 1 -- 172.21.15.22:0/3916102731 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f1428004800 con 0x55b331eb3090 2024-04-20T21:34:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.622+0000 7f1419600640 1 -- 172.21.15.22:0/3916102731 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1428004b20 con 0x55b331eb3090 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.622+0000 7f1439583740 10 client.25451 fetch_fsmap learned FSMap version 20 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.622+0000 7f1439583740 1 -- 172.21.15.22:0/3916102731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b331f29850 con 0x55b331eb3090 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.623+0000 7f1419600640 1 -- 172.21.15.22:0/3916102731 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142002dd20 con 0x55b331eb3090 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.623+0000 7f1419600640 1 -- 172.21.15.22:0/3916102731 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f14280050d0 con 0x55b331eb3090 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.623+0000 7f1439583740 10 client.25451 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.623+0000 7f1439583740 10 client.25451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.623+0000 7f1439583740 20 client.25451 populate_metadata read hostname 'smithi022' 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.623+0000 7f1439583740 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b331f29fb0 0x55b331f2c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.623+0000 7f1439583740 4 client.25451 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.623+0000 7f1439583740 1 -- 172.21.15.22:0/3916102731 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b331e0e9e0 con 0x55b331f29fb0 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.623+0000 7f142f400640 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b331f29fb0 0x55b331f2c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f142f400640 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b331f29fb0 0x55b331f2c3d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f977d82270577e0 server_cookie=8e8f3f5202d1a17a in_seq=0 out_seq=0 2024-04-20T21:34:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1419600640 10 client.25451 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1419600640 1 -- 172.21.15.22:0/3916102731 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b331e0e9e0 con 0x55b331f29fb0 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1419600640 10 client.25451 handle_command_reply: tid=0 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1439583740 1 client.25451 shutdown 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1439583740 1 -- 172.21.15.22:0/3916102731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331eb3090 msgr2=0x55b331ec6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1439583740 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331eb3090 0x55b331ec6c30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1428003f90 tx=0x7f1428004a40 comp rx=0 tx=0).stop 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1439583740 1 -- 172.21.15.22:0/3916102731 shutdown_connections 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1439583740 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b331f29fb0 0x55b331f2c3d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1439583740 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b331eb3090 0x55b331ec6c30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1439583740 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b331ea1b40 0x55b331ec66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1439583740 1 --2- 172.21.15.22:0/3916102731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b331ea1190 0x55b331ec61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1439583740 1 -- 172.21.15.22:0/3916102731 >> 172.21.15.22:0/3916102731 conn(0x55b331e9bab0 msgr2=0x55b331ebe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.624+0000 7f1419600640 0 client.25451 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.625+0000 7f1439583740 1 -- 172.21.15.22:0/3916102731 shutdown_connections 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.625+0000 7f1439583740 1 -- 172.21.15.22:0/3916102731 wait complete. 2024-04-20T21:34:59.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.625+0000 7f1439583740 20 client.25451 trim_cache size 0 max 16384 2024-04-20T21:34:59.627 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:34:59.627 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:34:59.627 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:34:59.627 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:34:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.625+0000 7f1435200640 1 -- 172.21.15.22:0/119663555 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f14140a3e40 msgr2=0x7f14140a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.625+0000 7f1435200640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f14140a3e40 0x7f14140a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1428005fd0 tx=0x7f1428009500 comp rx=0 tx=0).stop 2024-04-20T21:34:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.625+0000 7f1435200640 1 -- 172.21.15.22:0/119663555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1430074c30 msgr2=0x7f14301a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.625+0000 7f1435200640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1430074c30 0x7f14301a0b70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f141c00e9f0 tx=0x7f141c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.626+0000 7f1435200640 1 -- 172.21.15.22:0/119663555 shutdown_connections 2024-04-20T21:34:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.626+0000 7f1435200640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f14140a3e40 0x7f14140a6300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.626+0000 7f1435200640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f143010e6c0 0x7f143019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.626+0000 7f1435200640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1430074c30 0x7f14301a0b70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.626+0000 7f1435200640 1 --2- 172.21.15.22:0/119663555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430074310 0x7f14301a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.626+0000 7f1435200640 1 -- 172.21.15.22:0/119663555 >> 172.21.15.22:0/119663555 conn(0x7f14300fd930 msgr2=0x7f14300fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.626+0000 7f1435200640 1 -- 172.21.15.22:0/119663555 shutdown_connections 2024-04-20T21:34:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.626+0000 7f1435200640 1 -- 172.21.15.22:0/119663555 wait complete. 2024-04-20T21:34:59.649 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:34:59.649 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:59.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.898+0000 7efcea200640 1 -- 172.21.15.22:0/3468188128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efce4101ff0 msgr2=0x7efce41023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.898+0000 7efcea200640 1 --2- 172.21.15.22:0/3468188128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efce4101ff0 0x7efce41023d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009900 tx=0x7efcd802bc30 comp rx=0 tx=0).stop 2024-04-20T21:34:59.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.898+0000 7efcea200640 1 -- 172.21.15.22:0/3468188128 shutdown_connections 2024-04-20T21:34:59.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.898+0000 7efcea200640 1 --2- 172.21.15.22:0/3468188128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efce41070c0 0x7efce410ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.898+0000 7efcea200640 1 --2- 172.21.15.22:0/3468188128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efce41029a0 0x7efce41069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.898+0000 7efcea200640 1 --2- 172.21.15.22:0/3468188128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efce4101ff0 0x7efce41023d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.898+0000 7efcea200640 1 -- 172.21.15.22:0/3468188128 >> 172.21.15.22:0/3468188128 conn(0x7efce40fd890 msgr2=0x7efce40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:59.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.898+0000 7efcea200640 1 -- 172.21.15.22:0/3468188128 shutdown_connections 2024-04-20T21:34:59.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.898+0000 7efcea200640 1 -- 172.21.15.22:0/3468188128 wait complete. 2024-04-20T21:34:59.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.900+0000 7efcea200640 1 Processor -- start 2024-04-20T21:34:59.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.900+0000 7efcea200640 1 -- start start 2024-04-20T21:34:59.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.900+0000 7efcea200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efce4101ff0 0x7efce4108cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.900+0000 7efcea200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efce41070c0 0x7efce41091f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.900+0000 7efcea200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efce4109730 0x7efce41aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.901+0000 7efce8e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efce4101ff0 0x7efce4108cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.901+0000 7efce8e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efce4101ff0 0x7efce4108cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45512/0 (socket says 172.21.15.22:45512) 2024-04-20T21:34:59.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.901+0000 7efce3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efce41070c0 0x7efce41091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.901+0000 7efce3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efce41070c0 0x7efce41091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:56752/0 (socket says 172.21.15.22:56752) 2024-04-20T21:34:59.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.901+0000 7efce9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efce4109730 0x7efce41aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.901+0000 7efce9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efce4109730 0x7efce41aaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45148/0 (socket says 172.21.15.22:45148) 2024-04-20T21:34:59.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.901+0000 7efce8e00640 1 -- 172.21.15.22:0/3486682749 learned_addr learned my addr 172.21.15.22:0/3486682749 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:34:59.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.901+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efce41121a0 con 0x7efce4109730 2024-04-20T21:34:59.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.901+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efce4112020 con 0x7efce4101ff0 2024-04-20T21:34:59.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.901+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efce4112320 con 0x7efce41070c0 2024-04-20T21:34:59.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.902+0000 7efce3e00640 1 -- 172.21.15.22:0/3486682749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efce4101ff0 msgr2=0x7efce4108cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.902+0000 7efce3e00640 1 --2- 172.21.15.22:0/3486682749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efce4101ff0 0x7efce4108cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.902+0000 7efce3e00640 1 -- 172.21.15.22:0/3486682749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efce4109730 msgr2=0x7efce41aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:59.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.902+0000 7efce3e00640 1 --2- 172.21.15.22:0/3486682749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efce4109730 0x7efce41aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:59.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.902+0000 7efce3e00640 1 -- 172.21.15.22:0/3486682749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce41ab3f0 con 0x7efce41070c0 2024-04-20T21:34:59.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.902+0000 7efce3e00640 1 --2- 172.21.15.22:0/3486682749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efce41070c0 0x7efce41091f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efcd000d6a0 tx=0x7efcd000db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:59.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.902+0000 7efce1600640 1 -- 172.21.15.22:0/3486682749 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0017070 con 0x7efce41070c0 2024-04-20T21:34:59.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.903+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce41ab680 con 0x7efce41070c0 2024-04-20T21:34:59.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.903+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce41abbf0 con 0x7efce41070c0 2024-04-20T21:34:59.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.903+0000 7efce1600640 1 -- 172.21.15.22:0/3486682749 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd0009820 con 0x7efce41070c0 2024-04-20T21:34:59.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.903+0000 7efce1600640 1 -- 172.21.15.22:0/3486682749 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0004ce0 con 0x7efce41070c0 2024-04-20T21:34:59.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.905+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc4005290 con 0x7efce41070c0 2024-04-20T21:34:59.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.907+0000 7efce1600640 1 -- 172.21.15.22:0/3486682749 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7efcd0005220 con 0x7efce41070c0 2024-04-20T21:34:59.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.909+0000 7efce1600640 1 --2- 172.21.15.22:0/3486682749 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efccc0a3f10 0x7efccc0a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:59.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.910+0000 7efce1600640 1 -- 172.21.15.22:0/3486682749 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7efcd00d0260 con 0x7efce41070c0 2024-04-20T21:34:59.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.910+0000 7efce8e00640 1 --2- 172.21.15.22:0/3486682749 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efccc0a3f10 0x7efccc0a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:59.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.911+0000 7efce8e00640 1 --2- 172.21.15.22:0/3486682749 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efccc0a3f10 0x7efccc0a63d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efcd8006010 tx=0x7efcd802df90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:59.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:34:59.913+0000 7efce1600640 1 -- 172.21.15.22:0/3486682749 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efcd009bfe0 con 0x7efce41070c0 2024-04-20T21:35:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.307+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcc4005850 con 0x7efce41070c0 2024-04-20T21:35:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.308+0000 7efce1600640 1 -- 172.21.15.22:0/3486682749 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7efcd00a02e0 con 0x7efce41070c0 2024-04-20T21:35:00.307 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:35:00.307 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:35:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:35:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.311+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efccc0a3f10 msgr2=0x7efccc0a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.311+0000 7efcea200640 1 --2- 172.21.15.22:0/3486682749 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efccc0a3f10 0x7efccc0a63d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efcd8006010 tx=0x7efcd802df90 comp rx=0 tx=0).stop 2024-04-20T21:35:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.311+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efce41070c0 msgr2=0x7efce41091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:00.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.311+0000 7efcea200640 1 --2- 172.21.15.22:0/3486682749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efce41070c0 0x7efce41091f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efcd000d6a0 tx=0x7efcd000db60 comp rx=0 tx=0).stop 2024-04-20T21:35:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.311+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 shutdown_connections 2024-04-20T21:35:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.311+0000 7efcea200640 1 --2- 172.21.15.22:0/3486682749 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efccc0a3f10 0x7efccc0a63d0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efcd8006010 tx=0x7efcd802df90 comp rx=0 tx=0).stop 2024-04-20T21:35:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.311+0000 7efcea200640 1 --2- 172.21.15.22:0/3486682749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efce4109730 0x7efce41aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.311+0000 7efcea200640 1 --2- 172.21.15.22:0/3486682749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efce41070c0 0x7efce41091f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.311+0000 7efcea200640 1 --2- 172.21.15.22:0/3486682749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efce4101ff0 0x7efce4108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.311+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 >> 172.21.15.22:0/3486682749 conn(0x7efce40fd890 msgr2=0x7efce40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.311+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 shutdown_connections 2024-04-20T21:35:00.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.311+0000 7efcea200640 1 -- 172.21.15.22:0/3486682749 wait complete. 2024-04-20T21:35:00.332 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:35:00.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.505+0000 7f3713e00640 1 -- 172.21.15.22:0/3821937127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3714074310 msgr2=0x7f37140746f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:00.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.505+0000 7f3713e00640 1 --2- 172.21.15.22:0/3821937127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3714074310 0x7f37140746f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3704009900 tx=0x7f370402bcc0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.505+0000 7f3713e00640 1 -- 172.21.15.22:0/3821937127 shutdown_connections 2024-04-20T21:35:00.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.505+0000 7f3713e00640 1 --2- 172.21.15.22:0/3821937127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f371410e6c0 0x7f3714110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.505+0000 7f3713e00640 1 --2- 172.21.15.22:0/3821937127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3714074c30 0x7f371410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.505+0000 7f3713e00640 1 --2- 172.21.15.22:0/3821937127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3714074310 0x7f37140746f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.505+0000 7f3713e00640 1 -- 172.21.15.22:0/3821937127 >> 172.21.15.22:0/3821937127 conn(0x7f37140fd930 msgr2=0x7f37140ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:00.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.505+0000 7f3713e00640 1 -- 172.21.15.22:0/3821937127 shutdown_connections 2024-04-20T21:35:00.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.506+0000 7f3713e00640 1 -- 172.21.15.22:0/3821937127 wait complete. 2024-04-20T21:35:00.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.507+0000 7f3713e00640 1 Processor -- start 2024-04-20T21:35:00.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.508+0000 7f3713e00640 1 -- start start 2024-04-20T21:35:00.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.508+0000 7f3713e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3714074c30 0x7f3714101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.508+0000 7f3713e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f371410e6c0 0x7f3714102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.508+0000 7f3713e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714102860 0x7f37141ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.508+0000 7f3713e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3714114410 con 0x7f371410e6c0 2024-04-20T21:35:00.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.508+0000 7f3713e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3714114290 con 0x7f3714102860 2024-04-20T21:35:00.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.508+0000 7f3713e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3714114590 con 0x7f3714074c30 2024-04-20T21:35:00.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.509+0000 7f3712a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3714074c30 0x7f3714101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.509+0000 7f3713400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714102860 0x7f37141ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.509+0000 7f3713400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714102860 0x7f37141ab120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45544/0 (socket says 172.21.15.22:45544) 2024-04-20T21:35:00.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.509+0000 7f3713400640 1 -- 172.21.15.22:0/549866685 learned_addr learned my addr 172.21.15.22:0/549866685 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:00.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.509+0000 7f3712a00640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3714074c30 0x7f3714101de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:56772/0 (socket says 172.21.15.22:56772) 2024-04-20T21:35:00.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.509+0000 7f3712000640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f371410e6c0 0x7f3714102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.509+0000 7f3712000640 1 -- 172.21.15.22:0/549866685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3714074c30 msgr2=0x7f3714101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:00.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.509+0000 7f3712000640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3714074c30 0x7f3714101de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.510+0000 7f3712000640 1 -- 172.21.15.22:0/549866685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714102860 msgr2=0x7f37141ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:00.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.510+0000 7f3712000640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714102860 0x7f37141ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.510+0000 7f3712000640 1 -- 172.21.15.22:0/549866685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37141ab660 con 0x7f371410e6c0 2024-04-20T21:35:00.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.510+0000 7f3713400640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714102860 0x7f37141ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:00.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.510+0000 7f3712000640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f371410e6c0 0x7f3714102320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f370000e9f0 tx=0x7f370000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:00.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.510+0000 7f370b400640 1 -- 172.21.15.22:0/549866685 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370000ccc0 con 0x7f371410e6c0 2024-04-20T21:35:00.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.511+0000 7f370b400640 1 -- 172.21.15.22:0/549866685 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700010910 con 0x7f371410e6c0 2024-04-20T21:35:00.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.511+0000 7f3713e00640 1 -- 172.21.15.22:0/549866685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37141ab950 con 0x7f371410e6c0 2024-04-20T21:35:00.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.511+0000 7f370b400640 1 -- 172.21.15.22:0/549866685 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700018960 con 0x7f371410e6c0 2024-04-20T21:35:00.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.511+0000 7f3713e00640 1 -- 172.21.15.22:0/549866685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37141abe70 con 0x7f371410e6c0 2024-04-20T21:35:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.515+0000 7f371c973740 1 Processor -- start 2024-04-20T21:35:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.515+0000 7f371c973740 1 -- start start 2024-04-20T21:35:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f370b400640 1 -- 172.21.15.22:0/549866685 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7f3700010ac0 con 0x7f371410e6c0 2024-04-20T21:35:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f371c973740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 0x55639847f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f371c973740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847fa90 0x556398488130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f371c973740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 0x55639848aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f371c973740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5563983e4050 con 0x55639847f170 2024-04-20T21:35:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f371c973740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55639838a490 con 0x55639847fa90 2024-04-20T21:35:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f371c973740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5563983e0010 con 0x556398488670 2024-04-20T21:35:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f3712a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847fa90 0x556398488130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f3713400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 0x55639847f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f3712a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847fa90 0x556398488130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45554/0 (socket says 172.21.15.22:45554) 2024-04-20T21:35:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f3713400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 0x55639847f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45170/0 (socket says 172.21.15.22:45170) 2024-04-20T21:35:00.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f3712000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 0x55639848aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.516+0000 7f3712a00640 1 -- 172.21.15.22:0/3613728834 learned_addr learned my addr 172.21.15.22:0/3613728834 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:00.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.517+0000 7f3713400640 1 -- 172.21.15.22:0/3613728834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 msgr2=0x55639848aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:00.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.517+0000 7f3713400640 1 --2- 172.21.15.22:0/3613728834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 0x55639848aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.517+0000 7f3713400640 1 -- 172.21.15.22:0/3613728834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847fa90 msgr2=0x556398488130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:00.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.517+0000 7f3713400640 1 --2- 172.21.15.22:0/3613728834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847fa90 0x556398488130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.517+0000 7f3713400640 1 -- 172.21.15.22:0/3613728834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556398392070 con 0x55639847f170 2024-04-20T21:35:00.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.517+0000 7f3712000640 1 --2- 172.21.15.22:0/3613728834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 0x55639848aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.517+0000 7f3712a00640 1 --2- 172.21.15.22:0/3613728834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847fa90 0x556398488130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.517+0000 7f3713400640 1 --2- 172.21.15.22:0/3613728834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 0x55639847f550 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f370c011760 tx=0x7f370c011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169f5900949b5480 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.518+0000 7f370b400640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f36f80a3f90 0x7f36f80a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.518+0000 7f36ffe00640 1 -- 172.21.15.22:0/3613728834 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c004d20 con 0x55639847f170 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.518+0000 7f36ffe00640 1 -- 172.21.15.22:0/3613728834 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370c005020 con 0x55639847f170 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.518+0000 7f36ffe00640 1 -- 172.21.15.22:0/3613728834 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c019690 con 0x55639847f170 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.518+0000 7f370b400640 1 -- 172.21.15.22:0/549866685 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f37000d0d20 con 0x7f371410e6c0 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.518+0000 7f3712a00640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f36f80a3f90 0x7f36f80a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.518+0000 7f371c973740 1 -- 172.21.15.22:0/3613728834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 msgr2=0x55639847f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.518+0000 7f371c973740 1 --2- 172.21.15.22:0/3613728834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 0x55639847f550 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f370c011760 tx=0x7f370c011c20 comp rx=0 tx=0).stop 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.519+0000 7f371c973740 1 -- 172.21.15.22:0/3613728834 shutdown_connections 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.519+0000 7f371c973740 1 --2- 172.21.15.22:0/3613728834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 0x55639848aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.519+0000 7f371c973740 1 --2- 172.21.15.22:0/3613728834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847fa90 0x556398488130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.519+0000 7f371c973740 1 --2- 172.21.15.22:0/3613728834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 0x55639847f550 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.519+0000 7f371c973740 1 -- 172.21.15.22:0/3613728834 >> 172.21.15.22:0/3613728834 conn(0x5563984798a0 msgr2=0x556398479c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.519+0000 7f371c973740 1 -- 172.21.15.22:0/3613728834 shutdown_connections 2024-04-20T21:35:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.519+0000 7f371c973740 1 -- 172.21.15.22:0/3613728834 wait complete. 2024-04-20T21:35:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.519+0000 7f3712a00640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f36f80a3f90 0x7f36f80a6450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3704006010 tx=0x7f370402e040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.519+0000 7f371c973740 1 -- start start 2024-04-20T21:35:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.520+0000 7f371c973740 1 Processor -- start 2024-04-20T21:35:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.520+0000 7f371c973740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 0x5563984855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.520+0000 7f371c973740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847fa90 0x556398485b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.520+0000 7f371c973740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 0x556398486050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.520+0000 7f371c973740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55639848b460 con 0x55639847fa90 2024-04-20T21:35:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.520+0000 7f371c973740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55639839d860 con 0x55639847f170 2024-04-20T21:35:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.520+0000 7f371c973740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55639848b5e0 con 0x556398488670 2024-04-20T21:35:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.520+0000 7f371c973740 10 client.0 _ll_register_callbacks cb 0x55639836bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.521+0000 7f3713400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 0x556398486050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.521+0000 7f3713400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 0x556398486050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:56796/0 (socket says 172.21.15.22:56796) 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.521+0000 7f3713400640 1 -- 172.21.15.22:0/3496947779 learned_addr learned my addr 172.21.15.22:0/3496947779 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.521+0000 7f3712000640 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847fa90 0x556398485b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.521+0000 7f3712a00640 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 0x5563984855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.521+0000 7f3713400640 1 -- 172.21.15.22:0/3496947779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 msgr2=0x5563984855d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.521+0000 7f3713400640 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 0x5563984855d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.521+0000 7f3713400640 1 -- 172.21.15.22:0/3496947779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847fa90 msgr2=0x556398485b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.521+0000 7f3713400640 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847fa90 0x556398485b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.521+0000 7f3713400640 1 -- 172.21.15.22:0/3496947779 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563983a1950 con 0x556398488670 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.521+0000 7f3712a00640 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 0x5563984855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.521+0000 7f3712000640 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847fa90 0x556398485b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.522+0000 7f3713400640 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 0x556398486050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f370c01b040 tx=0x7f370c004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c1aeaf5e605322f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.522+0000 7f36fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.522+0000 7f36fcc00640 1 -- 172.21.15.22:0/3496947779 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c028070 con 0x556398488670 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.522+0000 7f371c973740 1 -- 172.21.15.22:0/3496947779 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f370c005020 con 0x556398488670 2024-04-20T21:35:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.522+0000 7f36fcc00640 1 -- 172.21.15.22:0/3496947779 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f370c00fc70 con 0x556398488670 2024-04-20T21:35:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.522+0000 7f36fcc00640 1 -- 172.21.15.22:0/3496947779 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f370c034050 con 0x556398488670 2024-04-20T21:35:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.523+0000 7f371c973740 10 client.25507 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.523+0000 7f371c973740 1 -- 172.21.15.22:0/3496947779 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55639836f820 con 0x556398488670 2024-04-20T21:35:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.523+0000 7f36fcc00640 1 -- 172.21.15.22:0/3496947779 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370c005020 con 0x556398488670 2024-04-20T21:35:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.523+0000 7f36fcc00640 1 -- 172.21.15.22:0/3496947779 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f370c010730 con 0x556398488670 2024-04-20T21:35:00.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:00 smithi122 ceph-mon[22147]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:35:00.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:00 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:35:00.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:00 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:35:00.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:00 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:35:00.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:00 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:35:00.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:00 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:35:00.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.523+0000 7f371c973740 10 client.25507 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:00.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.523+0000 7f371c973740 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:00.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.523+0000 7f371c973740 20 client.25507 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:00.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.523+0000 7f371c973740 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556398498530 0x55639849a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.523+0000 7f371c973740 4 client.25507 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:00.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.523+0000 7f371c973740 1 -- 172.21.15.22:0/3496947779 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563983f1a70 con 0x556398498530 2024-04-20T21:35:00.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.524+0000 7f3712a00640 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556398498530 0x55639849a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.525+0000 7f3712a00640 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556398498530 0x55639849a920 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea0cc0942e75c82 server_cookie=17d0a248255a5c8a in_seq=0 out_seq=0 2024-04-20T21:35:00.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.525+0000 7f36fcc00640 10 client.25507 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:00.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.525+0000 7f36fcc00640 1 -- 172.21.15.22:0/3496947779 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5563983f1a70 con 0x556398498530 2024-04-20T21:35:00.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.525+0000 7f36fcc00640 10 client.25507 handle_command_reply: tid=0 2024-04-20T21:35:00.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f371c973740 1 client.25507 shutdown 2024-04-20T21:35:00.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f371c973740 1 -- 172.21.15.22:0/3496947779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 msgr2=0x556398486050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:00.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f371c973740 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 0x556398486050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f370c01b040 tx=0x7f370c004cf0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f371c973740 1 -- 172.21.15.22:0/3496947779 shutdown_connections 2024-04-20T21:35:00.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f371c973740 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556398488670 0x556398486050 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f371c973740 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847fa90 0x556398485b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f371c973740 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556398498530 0x55639849a920 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f36fcc00640 0 client.25507 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:00.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f371c973740 1 --2- 172.21.15.22:0/3496947779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 0x5563984855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f371c973740 1 -- 172.21.15.22:0/3496947779 >> 172.21.15.22:0/3496947779 conn(0x5563984798a0 msgr2=0x55639847b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:00.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f371c973740 1 -- 172.21.15.22:0/3496947779 shutdown_connections 2024-04-20T21:35:00.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f371c973740 1 -- 172.21.15.22:0/3496947779 wait complete. 2024-04-20T21:35:00.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.526+0000 7f371c973740 20 client.25507 trim_cache size 0 max 16384 2024-04-20T21:35:00.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.551+0000 7f371c973740 1 Processor -- start 2024-04-20T21:35:00.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.551+0000 7f371c973740 1 -- start start 2024-04-20T21:35:00.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.551+0000 7f371c973740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 0x556398494320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.551+0000 7f371c973740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55639847fa90 0x556398484eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.551+0000 7f371c973740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556398488670 0x5563984780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.551+0000 7f371c973740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f37141b2830 con 0x55639847f170 2024-04-20T21:35:00.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.551+0000 7f371c973740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37141b50b0 con 0x556398488670 2024-04-20T21:35:00.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.551+0000 7f371c973740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37141b2ea0 con 0x55639847fa90 2024-04-20T21:35:00.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.552+0000 7f3712000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55639847fa90 0x556398484eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.552+0000 7f3712000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55639847fa90 0x556398484eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:56808/0 (socket says 172.21.15.22:56808) 2024-04-20T21:35:00.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.552+0000 7f3712000640 1 -- 172.21.15.22:0/3638798571 learned_addr learned my addr 172.21.15.22:0/3638798571 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:00.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.552+0000 7f3713400640 1 --2- 172.21.15.22:0/3638798571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556398488670 0x5563984780d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.552+0000 7f3712a00640 1 --2- 172.21.15.22:0/3638798571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 0x556398494320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.552+0000 7f3713400640 1 -- 172.21.15.22:0/3638798571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55639847fa90 msgr2=0x556398484eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:00.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.552+0000 7f3713400640 1 --2- 172.21.15.22:0/3638798571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55639847fa90 0x556398484eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.552+0000 7f3713400640 1 -- 172.21.15.22:0/3638798571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 msgr2=0x556398494320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:00.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.552+0000 7f3713400640 1 --2- 172.21.15.22:0/3638798571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 0x556398494320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.552+0000 7f3713400640 1 -- 172.21.15.22:0/3638798571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556398478f80 con 0x556398488670 2024-04-20T21:35:00.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.552+0000 7f3712a00640 1 --2- 172.21.15.22:0/3638798571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 0x556398494320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:00.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.553+0000 7f3713400640 1 --2- 172.21.15.22:0/3638798571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556398488670 0x5563984780d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f370c005c40 tx=0x7f370c0049d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=571993d4440bd8de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:00.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.553+0000 7f36ff400640 1 -- 172.21.15.22:0/3638798571 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c031070 con 0x556398488670 2024-04-20T21:35:00.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.553+0000 7f36ff400640 1 -- 172.21.15.22:0/3638798571 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370c010910 con 0x556398488670 2024-04-20T21:35:00.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.553+0000 7f36ff400640 1 -- 172.21.15.22:0/3638798571 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c028070 con 0x556398488670 2024-04-20T21:35:00.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:00 smithi033 ceph-mon[21626]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:35:00.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:00 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:35:00.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:00 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:35:00.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:00 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:35:00.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:00 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:35:00.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:00 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:35:00.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.554+0000 7f371c973740 1 -- 172.21.15.22:0/3638798571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556398488670 msgr2=0x5563984780d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:00.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.554+0000 7f371c973740 1 --2- 172.21.15.22:0/3638798571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556398488670 0x5563984780d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f370c005c40 tx=0x7f370c0049d0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.555+0000 7f371c973740 1 -- 172.21.15.22:0/3638798571 shutdown_connections 2024-04-20T21:35:00.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.555+0000 7f371c973740 1 --2- 172.21.15.22:0/3638798571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556398488670 0x5563984780d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.555+0000 7f371c973740 1 --2- 172.21.15.22:0/3638798571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55639847fa90 0x556398484eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.555+0000 7f371c973740 1 --2- 172.21.15.22:0/3638798571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55639847f170 0x556398494320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.555+0000 7f371c973740 1 -- 172.21.15.22:0/3638798571 >> 172.21.15.22:0/3638798571 conn(0x5563984798a0 msgr2=0x556398485e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:00.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.555+0000 7f371c973740 1 -- 172.21.15.22:0/3638798571 shutdown_connections 2024-04-20T21:35:00.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.555+0000 7f371c973740 1 -- 172.21.15.22:0/3638798571 wait complete. 2024-04-20T21:35:00.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.555+0000 7f371c973740 1 -- start start 2024-04-20T21:35:00.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.556+0000 7f371c973740 1 Processor -- start 2024-04-20T21:35:00.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.556+0000 7f371c973740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 0x5563984823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.556+0000 7f371c973740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55639847fa90 0x5563984828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.556+0000 7f371c973740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556398488670 0x556398482e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.556+0000 7f371c973740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f37141b3570 con 0x556398488670 2024-04-20T21:35:00.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.556+0000 7f371c973740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37141b49e0 con 0x55639847f170 2024-04-20T21:35:00.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.556+0000 7f371c973740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37141b3c40 con 0x55639847fa90 2024-04-20T21:35:00.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.556+0000 7f371c973740 10 client.0 _ll_register_callbacks cb 0x7f37000d4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:00.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f3712a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 0x5563984823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f3712a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 0x5563984823a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45584/0 (socket says 172.21.15.22:45584) 2024-04-20T21:35:00.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f3712a00640 1 -- 172.21.15.22:0/283619991 learned_addr learned my addr 172.21.15.22:0/283619991 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:00.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f3712000640 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55639847fa90 0x5563984828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f3713400640 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556398488670 0x556398482e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f3712a00640 1 -- 172.21.15.22:0/283619991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55639847fa90 msgr2=0x5563984828e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:00.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f3712a00640 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55639847fa90 0x5563984828e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f3712a00640 1 -- 172.21.15.22:0/283619991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556398488670 msgr2=0x556398482e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:00.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f3712a00640 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556398488670 0x556398482e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f3712a00640 1 -- 172.21.15.22:0/283619991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563980f63f0 con 0x55639847f170 2024-04-20T21:35:00.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f3713400640 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556398488670 0x556398482e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:00.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f3712a00640 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 0x5563984823a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3704009230 tx=0x7f370402bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dfab85e96d25947 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:00.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.557+0000 7f36fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:35:00.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.558+0000 7f36fcc00640 1 -- 172.21.15.22:0/283619991 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37040046c0 con 0x55639847f170 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.558+0000 7f36fcc00640 1 -- 172.21.15.22:0/283619991 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f3704004860 con 0x55639847f170 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.558+0000 7f371c973740 1 -- 172.21.15.22:0/283619991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f370c010910 con 0x55639847f170 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.558+0000 7f36fcc00640 1 -- 172.21.15.22:0/283619991 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3704004a60 con 0x55639847f170 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.558+0000 7f371c973740 10 client.25472 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.558+0000 7f371c973740 1 -- 172.21.15.22:0/283619991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5563981c07d0 con 0x55639847f170 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.558+0000 7f36fcc00640 1 -- 172.21.15.22:0/283619991 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370c010910 con 0x55639847f170 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.559+0000 7f36fcc00640 1 -- 172.21.15.22:0/283619991 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f3704053470 con 0x55639847f170 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.559+0000 7f371c973740 10 client.25472 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.559+0000 7f371c973740 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.559+0000 7f371c973740 20 client.25472 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.559+0000 7f371c973740 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5563984ff070 0x556398501490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.559+0000 7f371c973740 4 client.25472 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:35:00.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.559+0000 7f371c973740 1 -- 172.21.15.22:0/283619991 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563983eb8e0 con 0x5563984ff070 2024-04-20T21:35:00.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.559+0000 7f3712000640 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5563984ff070 0x556398501490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:00.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.560+0000 7f36fcc00640 10 client.25472 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:00.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.560+0000 7f3712000640 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5563984ff070 0x556398501490 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329b685fa755c150 server_cookie=d1104201c07d8184 in_seq=0 out_seq=0 2024-04-20T21:35:00.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.560+0000 7f36fcc00640 1 -- 172.21.15.22:0/283619991 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563983eb8e0 con 0x5563984ff070 2024-04-20T21:35:00.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.560+0000 7f36fcc00640 10 client.25472 handle_command_reply: tid=0 2024-04-20T21:35:00.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.560+0000 7f371c973740 1 client.25472 shutdown 2024-04-20T21:35:00.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.561+0000 7f371c973740 1 -- 172.21.15.22:0/283619991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 msgr2=0x5563984823a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:00.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.561+0000 7f371c973740 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 0x5563984823a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3704009230 tx=0x7f370402bcc0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.561+0000 7f371c973740 1 -- 172.21.15.22:0/283619991 shutdown_connections 2024-04-20T21:35:00.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.561+0000 7f371c973740 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556398488670 0x556398482e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.561+0000 7f371c973740 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55639847fa90 0x5563984828e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.561+0000 7f371c973740 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5563984ff070 0x556398501490 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.561+0000 7f371c973740 1 --2- 172.21.15.22:0/283619991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55639847f170 0x5563984823a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.561+0000 7f371c973740 1 -- 172.21.15.22:0/283619991 >> 172.21.15.22:0/283619991 conn(0x5563984798a0 msgr2=0x556398485590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:00.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.561+0000 7f36fcc00640 0 client.25472 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:00.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.561+0000 7f371c973740 1 -- 172.21.15.22:0/283619991 shutdown_connections 2024-04-20T21:35:00.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.561+0000 7f371c973740 1 -- 172.21.15.22:0/283619991 wait complete. 2024-04-20T21:35:00.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.561+0000 7f371c973740 20 client.25472 trim_cache size 0 max 16384 2024-04-20T21:35:00.563 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:35:00.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.562+0000 7f3713e00640 1 -- 172.21.15.22:0/549866685 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f36f80a3f90 msgr2=0x7f36f80a6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:00.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.562+0000 7f3713e00640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f36f80a3f90 0x7f36f80a6450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3704006010 tx=0x7f370402e040 comp rx=0 tx=0).stop 2024-04-20T21:35:00.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.562+0000 7f3713e00640 1 -- 172.21.15.22:0/549866685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f371410e6c0 msgr2=0x7f3714102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:00.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.562+0000 7f3713e00640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f371410e6c0 0x7f3714102320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f370000e9f0 tx=0x7f370000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.562+0000 7f3713e00640 1 -- 172.21.15.22:0/549866685 shutdown_connections 2024-04-20T21:35:00.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.562+0000 7f3713e00640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f36f80a3f90 0x7f36f80a6450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.562+0000 7f3713e00640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3714102860 0x7f37141ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.562+0000 7f3713e00640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f371410e6c0 0x7f3714102320 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.562+0000 7f3713e00640 1 --2- 172.21.15.22:0/549866685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3714074c30 0x7f3714101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:00.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.562+0000 7f3713e00640 1 -- 172.21.15.22:0/549866685 >> 172.21.15.22:0/549866685 conn(0x7f37140fd930 msgr2=0x7f37140fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:00.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.563+0000 7f3713e00640 1 -- 172.21.15.22:0/549866685 shutdown_connections 2024-04-20T21:35:00.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:00.563+0000 7f3713e00640 1 -- 172.21.15.22:0/549866685 wait complete. 2024-04-20T21:35:00.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:00 smithi022 ceph-mon[16843]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:35:00.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:00 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:35:00.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:00 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:35:00.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:00 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:35:00.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:00 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:35:00.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:00 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:35:01.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:01 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3486682749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:35:01.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:01 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3486682749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:35:01.584 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:35:01.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:35:01.584 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:35:01.647 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:01 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3486682749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:35:01.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.786+0000 7efcc6200640 1 -- 172.21.15.22:0/1071060124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcc01038e0 msgr2=0x7efcc010bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.786+0000 7efcc6200640 1 --2- 172.21.15.22:0/1071060124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcc01038e0 0x7efcc010bdc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efcac009860 tx=0x7efcac02bc10 comp rx=0 tx=0).stop 2024-04-20T21:35:01.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.786+0000 7efcc6200640 1 -- 172.21.15.22:0/1071060124 shutdown_connections 2024-04-20T21:35:01.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.786+0000 7efcc6200640 1 --2- 172.21.15.22:0/1071060124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc010c3e0 0x7efcc010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.786+0000 7efcc6200640 1 --2- 172.21.15.22:0/1071060124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcc01038e0 0x7efcc010bdc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.786+0000 7efcc6200640 1 --2- 172.21.15.22:0/1071060124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc0102fc0 0x7efcc01033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.786+0000 7efcc6200640 1 -- 172.21.15.22:0/1071060124 >> 172.21.15.22:0/1071060124 conn(0x7efcc00fc7e0 msgr2=0x7efcc00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:01.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.786+0000 7efcc6200640 1 -- 172.21.15.22:0/1071060124 shutdown_connections 2024-04-20T21:35:01.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.786+0000 7efcc6200640 1 -- 172.21.15.22:0/1071060124 wait complete. 2024-04-20T21:35:01.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.787+0000 7efcc6200640 1 Processor -- start 2024-04-20T21:35:01.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.787+0000 7efcc6200640 1 -- start start 2024-04-20T21:35:01.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.788+0000 7efcc6200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc0102fc0 0x7efcc01a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.788+0000 7efcc6200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01038e0 0x7efcc01a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.788+0000 7efcc6200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcc010c3e0 0x7efcc019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.788+0000 7efcc6200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01121c0 con 0x7efcc01038e0 2024-04-20T21:35:01.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.788+0000 7efcc6200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0112040 con 0x7efcc010c3e0 2024-04-20T21:35:01.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.788+0000 7efcc6200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0112340 con 0x7efcc0102fc0 2024-04-20T21:35:01.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.788+0000 7efcbfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01038e0 0x7efcc01a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.788+0000 7efcc4e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc0102fc0 0x7efcc01a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.788+0000 7efcbfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01038e0 0x7efcc01a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45236/0 (socket says 172.21.15.22:45236) 2024-04-20T21:35:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.788+0000 7efcbfe00640 1 -- 172.21.15.22:0/3283972846 learned_addr learned my addr 172.21.15.22:0/3283972846 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.788+0000 7efcc5800640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcc010c3e0 0x7efcc019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.789+0000 7efcc4e00640 1 -- 172.21.15.22:0/3283972846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcc010c3e0 msgr2=0x7efcc019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.789+0000 7efcc4e00640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcc010c3e0 0x7efcc019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.789+0000 7efcc4e00640 1 -- 172.21.15.22:0/3283972846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01038e0 msgr2=0x7efcc01a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.789+0000 7efcc4e00640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01038e0 0x7efcc01a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.789+0000 7efcc4e00640 1 -- 172.21.15.22:0/3283972846 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc019af70 con 0x7efcc0102fc0 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.789+0000 7efcc5800640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcc010c3e0 0x7efcc019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.789+0000 7efcc4e00640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc0102fc0 0x7efcc01a0590 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e9f0 tx=0x7efcb400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.789+0000 7efcbd600640 1 -- 172.21.15.22:0/3283972846 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb400ccc0 con 0x7efcc0102fc0 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.789+0000 7efcc6200640 1 -- 172.21.15.22:0/3283972846 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc019b260 con 0x7efcc0102fc0 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.789+0000 7efcbd600640 1 -- 172.21.15.22:0/3283972846 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb4010910 con 0x7efcc0102fc0 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.789+0000 7efcbd600640 1 -- 172.21.15.22:0/3283972846 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4018960 con 0x7efcc0102fc0 2024-04-20T21:35:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.789+0000 7efcc6200640 1 -- 172.21.15.22:0/3283972846 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc01a73e0 con 0x7efcc0102fc0 2024-04-20T21:35:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.794+0000 7efcca40f740 1 Processor -- start 2024-04-20T21:35:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.794+0000 7efcbd600640 1 -- 172.21.15.22:0/3283972846 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7efcb4010ac0 con 0x7efcc0102fc0 2024-04-20T21:35:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.794+0000 7efcca40f740 1 -- start start 2024-04-20T21:35:01.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.794+0000 7efcca40f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52190f0 0x55dcf52194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.794+0000 7efcca40f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf5219a10 0x55dcf52220b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.794+0000 7efcca40f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52225f0 0x55dcf52249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.794+0000 7efcca40f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dcf517dd20 con 0x55dcf52225f0 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.794+0000 7efcca40f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dcf51243e0 con 0x55dcf52190f0 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.794+0000 7efcca40f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dcf5179ce0 con 0x55dcf5219a10 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.795+0000 7efcc5800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52190f0 0x55dcf52194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.795+0000 7efcc5800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52190f0 0x55dcf52194d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45612/0 (socket says 172.21.15.22:45612) 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.795+0000 7efcc5800640 1 -- 172.21.15.22:0/3840821436 learned_addr learned my addr 172.21.15.22:0/3840821436 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.795+0000 7efcc4e00640 1 --2- 172.21.15.22:0/3840821436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf5219a10 0x55dcf52220b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.795+0000 7efcbfe00640 1 --2- 172.21.15.22:0/3840821436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52225f0 0x55dcf52249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.795+0000 7efcc5800640 1 -- 172.21.15.22:0/3840821436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf5219a10 msgr2=0x55dcf52220b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.795+0000 7efcc5800640 1 --2- 172.21.15.22:0/3840821436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf5219a10 0x55dcf52220b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.795+0000 7efcc5800640 1 -- 172.21.15.22:0/3840821436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52225f0 msgr2=0x55dcf52249e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.795+0000 7efcc5800640 1 --2- 172.21.15.22:0/3840821436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52225f0 0x55dcf52249e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.795+0000 7efcc5800640 1 -- 172.21.15.22:0/3840821436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf5143980 con 0x55dcf52190f0 2024-04-20T21:35:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.796+0000 7efcc5800640 1 --2- 172.21.15.22:0/3840821436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52190f0 0x55dcf52194d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efcb800f400 tx=0x7efcb800f8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a92007a8c7302ae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.796+0000 7efcbd600640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efca80a3e40 0x7efca80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.796+0000 7efcb2000640 1 -- 172.21.15.22:0/3840821436 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb801d070 con 0x55dcf52190f0 2024-04-20T21:35:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.796+0000 7efcb2000640 1 -- 172.21.15.22:0/3840821436 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb800cb80 con 0x55dcf52190f0 2024-04-20T21:35:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.796+0000 7efcb2000640 1 -- 172.21.15.22:0/3840821436 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8018c80 con 0x55dcf52190f0 2024-04-20T21:35:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.796+0000 7efcbd600640 1 -- 172.21.15.22:0/3283972846 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7efcb40d0f30 con 0x7efcc0102fc0 2024-04-20T21:35:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.796+0000 7efcbfe00640 1 --2- 172.21.15.22:0/3840821436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52225f0 0x55dcf52249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.797+0000 7efcc4e00640 1 --2- 172.21.15.22:0/3840821436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf5219a10 0x55dcf52220b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.797+0000 7efcca40f740 1 -- 172.21.15.22:0/3840821436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52190f0 msgr2=0x55dcf52194d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.797+0000 7efcca40f740 1 --2- 172.21.15.22:0/3840821436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52190f0 0x55dcf52194d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efcb800f400 tx=0x7efcb800f8c0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.797+0000 7efcc4e00640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efca80a3e40 0x7efca80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.798+0000 7efcca40f740 1 -- 172.21.15.22:0/3840821436 shutdown_connections 2024-04-20T21:35:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.798+0000 7efcca40f740 1 --2- 172.21.15.22:0/3840821436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52225f0 0x55dcf52249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.798+0000 7efcca40f740 1 --2- 172.21.15.22:0/3840821436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf5219a10 0x55dcf52220b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.798+0000 7efcca40f740 1 --2- 172.21.15.22:0/3840821436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52190f0 0x55dcf52194d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.798+0000 7efcca40f740 1 -- 172.21.15.22:0/3840821436 >> 172.21.15.22:0/3840821436 conn(0x55dcf5213820 msgr2=0x55dcf5213c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.798+0000 7efcc4e00640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efca80a3e40 0x7efca80a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efcb40ce4e0 tx=0x7efcb4009700 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.798+0000 7efcca40f740 1 -- 172.21.15.22:0/3840821436 shutdown_connections 2024-04-20T21:35:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.798+0000 7efcca40f740 1 -- 172.21.15.22:0/3840821436 wait complete. 2024-04-20T21:35:01.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.798+0000 7efcca40f740 1 -- start start 2024-04-20T21:35:01.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.799+0000 7efcca40f740 1 Processor -- start 2024-04-20T21:35:01.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.799+0000 7efcca40f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52190f0 0x55dcf521f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.799+0000 7efcca40f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf5219a10 0x55dcf521faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.799+0000 7efcca40f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52225f0 0x55dcf5220030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.799+0000 7efcca40f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dcf52253e0 con 0x55dcf52190f0 2024-04-20T21:35:01.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.799+0000 7efcca40f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dcf51377a0 con 0x55dcf52225f0 2024-04-20T21:35:01.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.799+0000 7efcca40f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dcf5225560 con 0x55dcf5219a10 2024-04-20T21:35:01.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.799+0000 7efcca40f740 10 client.0 _ll_register_callbacks cb 0x55dcf5139b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:01.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.800+0000 7efcc5800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52190f0 0x55dcf521f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.800+0000 7efcbfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf5219a10 0x55dcf521faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.800+0000 7efcc5800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52190f0 0x55dcf521f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45256/0 (socket says 172.21.15.22:45256) 2024-04-20T21:35:01.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.800+0000 7efcc5800640 1 -- 172.21.15.22:0/758347211 learned_addr learned my addr 172.21.15.22:0/758347211 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:01.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.800+0000 7efcc5800640 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52225f0 0x55dcf5220030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.800+0000 7efcbfe00640 1 -- 172.21.15.22:0/758347211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52225f0 msgr2=0x55dcf5220030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:01.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.800+0000 7efcbfe00640 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52225f0 0x55dcf5220030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.800+0000 7efcbfe00640 1 -- 172.21.15.22:0/758347211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52190f0 msgr2=0x55dcf521f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:01.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.800+0000 7efcbfe00640 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52190f0 0x55dcf521f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.800+0000 7efcbfe00640 1 -- 172.21.15.22:0/758347211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf4f587e0 con 0x55dcf5219a10 2024-04-20T21:35:01.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.800+0000 7efcc5800640 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52190f0 0x55dcf521f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:01.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.801+0000 7efcbfe00640 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf5219a10 0x55dcf521faf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efcac009830 tx=0x7efcac004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6050e6b23d71707e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:01.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.801+0000 7efca6a00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:35:01.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.801+0000 7efca6a00640 1 -- 172.21.15.22:0/758347211 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac044070 con 0x55dcf5219a10 2024-04-20T21:35:01.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.802+0000 7efca6a00640 1 -- 172.21.15.22:0/758347211 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7efcac034440 con 0x55dcf5219a10 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.802+0000 7efcca40f740 1 -- 172.21.15.22:0/758347211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb800cb80 con 0x55dcf5219a10 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.802+0000 7efca6a00640 1 -- 172.21.15.22:0/758347211 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7efcac04e050 con 0x55dcf5219a10 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.802+0000 7efcca40f740 10 client.25522 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.802+0000 7efcca40f740 1 -- 172.21.15.22:0/758347211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dcf4fb8f70 con 0x55dcf5219a10 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.803+0000 7efca6a00640 1 -- 172.21.15.22:0/758347211 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb800cb80 con 0x55dcf5219a10 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.803+0000 7efca6a00640 1 -- 172.21.15.22:0/758347211 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7efcac003680 con 0x55dcf5219a10 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.803+0000 7efcca40f740 10 client.25522 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.803+0000 7efcca40f740 10 client.25522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.803+0000 7efcca40f740 20 client.25522 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.803+0000 7efcca40f740 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dcf5232880 0x55dcf5234c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.803+0000 7efcca40f740 4 client.25522 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.803+0000 7efcca40f740 1 -- 172.21.15.22:0/758347211 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcf518b740 con 0x55dcf5232880 2024-04-20T21:35:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.804+0000 7efcc5800640 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dcf5232880 0x55dcf5234c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.804+0000 7efcc5800640 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dcf5232880 0x55dcf5234c70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d36b04086002ed4f server_cookie=e8506eead005cfb7 in_seq=0 out_seq=0 2024-04-20T21:35:01.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.804+0000 7efca6a00640 10 client.25522 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:01.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.805+0000 7efca6a00640 1 -- 172.21.15.22:0/758347211 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dcf518b740 con 0x55dcf5232880 2024-04-20T21:35:01.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.805+0000 7efca6a00640 10 client.25522 handle_command_reply: tid=0 2024-04-20T21:35:01.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.805+0000 7efcca40f740 1 client.25522 shutdown 2024-04-20T21:35:01.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.805+0000 7efcca40f740 1 -- 172.21.15.22:0/758347211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf5219a10 msgr2=0x55dcf521faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.805+0000 7efcca40f740 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf5219a10 0x55dcf521faf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efcac009830 tx=0x7efcac004040 comp rx=0 tx=0).stop 2024-04-20T21:35:01.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.805+0000 7efcca40f740 1 -- 172.21.15.22:0/758347211 shutdown_connections 2024-04-20T21:35:01.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.806+0000 7efcca40f740 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dcf5232880 0x55dcf5234c70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.806+0000 7efca6a00640 0 client.25522 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:01.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.806+0000 7efcca40f740 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52225f0 0x55dcf5220030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.806+0000 7efcca40f740 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf5219a10 0x55dcf521faf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.806+0000 7efcca40f740 1 --2- 172.21.15.22:0/758347211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52190f0 0x55dcf521f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.806+0000 7efcca40f740 1 -- 172.21.15.22:0/758347211 >> 172.21.15.22:0/758347211 conn(0x55dcf5213820 msgr2=0x55dcf5215570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:01.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.806+0000 7efcca40f740 1 -- 172.21.15.22:0/758347211 shutdown_connections 2024-04-20T21:35:01.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.806+0000 7efcca40f740 1 -- 172.21.15.22:0/758347211 wait complete. 2024-04-20T21:35:01.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.806+0000 7efcca40f740 20 client.25522 trim_cache size 0 max 16384 2024-04-20T21:35:01.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.842+0000 7efcca40f740 1 Processor -- start 2024-04-20T21:35:01.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.842+0000 7efcca40f740 1 -- start start 2024-04-20T21:35:01.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.842+0000 7efcca40f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 0x55dcf5221ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.842+0000 7efcca40f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf5219a10 0x55dcf522e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.842+0000 7efcca40f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52225f0 0x55dcf5212880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.842+0000 7efcca40f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01ade70 con 0x55dcf5219a10 2024-04-20T21:35:01.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.842+0000 7efcca40f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01b06f0 con 0x55dcf52225f0 2024-04-20T21:35:01.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.842+0000 7efcca40f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01ae4e0 con 0x55dcf52190f0 2024-04-20T21:35:01.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.843+0000 7efcc5800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 0x55dcf5221ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.843+0000 7efcbfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf5219a10 0x55dcf522e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.843+0000 7efcc5800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 0x55dcf5221ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:56874/0 (socket says 172.21.15.22:56874) 2024-04-20T21:35:01.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.843+0000 7efcc5800640 1 -- 172.21.15.22:0/4134621648 learned_addr learned my addr 172.21.15.22:0/4134621648 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:01.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.843+0000 7efcc5800640 1 --2- 172.21.15.22:0/4134621648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52225f0 0x55dcf5212880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.843+0000 7efcc5800640 1 -- 172.21.15.22:0/4134621648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52225f0 msgr2=0x55dcf5212880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:01.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.843+0000 7efcc5800640 1 --2- 172.21.15.22:0/4134621648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52225f0 0x55dcf5212880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.843+0000 7efcc5800640 1 -- 172.21.15.22:0/4134621648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf5219a10 msgr2=0x55dcf522e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:01.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.843+0000 7efcc5800640 1 --2- 172.21.15.22:0/4134621648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf5219a10 0x55dcf522e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.843+0000 7efcc5800640 1 -- 172.21.15.22:0/4134621648 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf4e50170 con 0x55dcf52190f0 2024-04-20T21:35:01.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.844+0000 7efcc5800640 1 --2- 172.21.15.22:0/4134621648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 0x55dcf5221ce0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efcb800a010 tx=0x7efcb80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1afb5fa06ecefd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:01.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.844+0000 7efcb1600640 1 -- 172.21.15.22:0/4134621648 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb801d040 con 0x55dcf52190f0 2024-04-20T21:35:01.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.844+0000 7efcb1600640 1 -- 172.21.15.22:0/4134621648 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb8018490 con 0x55dcf52190f0 2024-04-20T21:35:01.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.844+0000 7efcb1600640 1 -- 172.21.15.22:0/4134621648 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8018d50 con 0x55dcf52190f0 2024-04-20T21:35:01.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.845+0000 7efcca40f740 1 -- 172.21.15.22:0/4134621648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 msgr2=0x55dcf5221ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.845+0000 7efcca40f740 1 --2- 172.21.15.22:0/4134621648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 0x55dcf5221ce0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efcb800a010 tx=0x7efcb80095c0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.845+0000 7efcca40f740 1 -- 172.21.15.22:0/4134621648 shutdown_connections 2024-04-20T21:35:01.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.845+0000 7efcca40f740 1 --2- 172.21.15.22:0/4134621648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf52225f0 0x55dcf5212880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.845+0000 7efcca40f740 1 --2- 172.21.15.22:0/4134621648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf5219a10 0x55dcf522e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.845+0000 7efcca40f740 1 --2- 172.21.15.22:0/4134621648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 0x55dcf5221ce0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.845+0000 7efcca40f740 1 -- 172.21.15.22:0/4134621648 >> 172.21.15.22:0/4134621648 conn(0x55dcf5213820 msgr2=0x55dcf5212170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:01.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.845+0000 7efcca40f740 1 -- 172.21.15.22:0/4134621648 shutdown_connections 2024-04-20T21:35:01.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.845+0000 7efcca40f740 1 -- 172.21.15.22:0/4134621648 wait complete. 2024-04-20T21:35:01.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.846+0000 7efcca40f740 1 -- start start 2024-04-20T21:35:01.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.846+0000 7efcca40f740 1 Processor -- start 2024-04-20T21:35:01.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.847+0000 7efcca40f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 0x55dcf521c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.847+0000 7efcca40f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf5219a10 0x55dcf521c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.847+0000 7efcca40f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52225f0 0x55dcf521cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.847+0000 7efcca40f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01aebb0 con 0x55dcf52225f0 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.847+0000 7efcca40f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01b0020 con 0x55dcf5219a10 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.847+0000 7efcca40f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01af280 con 0x55dcf52190f0 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.847+0000 7efcca40f740 10 client.0 _ll_register_callbacks cb 0x7efcac037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.847+0000 7efcbfe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf5219a10 0x55dcf521c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.847+0000 7efcc5800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 0x55dcf521c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.847+0000 7efcbfe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf5219a10 0x55dcf521c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45652/0 (socket says 172.21.15.22:45652) 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.847+0000 7efcbfe00640 1 -- 172.21.15.22:0/2533903392 learned_addr learned my addr 172.21.15.22:0/2533903392 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.847+0000 7efcc5800640 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52225f0 0x55dcf521cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.848+0000 7efcbfe00640 1 -- 172.21.15.22:0/2533903392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 msgr2=0x55dcf521c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.848+0000 7efcbfe00640 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 0x55dcf521c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.848+0000 7efcbfe00640 1 -- 172.21.15.22:0/2533903392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52225f0 msgr2=0x55dcf521cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.848+0000 7efcbfe00640 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52225f0 0x55dcf521cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.848+0000 7efcbfe00640 1 -- 172.21.15.22:0/2533903392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf5297db0 con 0x55dcf5219a10 2024-04-20T21:35:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.848+0000 7efcc5800640 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 0x55dcf521c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.848+0000 7efcbfe00640 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf5219a10 0x55dcf521c7f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efcac009980 tx=0x7efcac02c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f14a5cddcd083bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.848+0000 7efca6a00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:35:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.848+0000 7efca6a00640 1 -- 172.21.15.22:0/2533903392 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac057020 con 0x55dcf5219a10 2024-04-20T21:35:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.848+0000 7efcca40f740 1 -- 172.21.15.22:0/2533903392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb8018490 con 0x55dcf5219a10 2024-04-20T21:35:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.848+0000 7efca6a00640 1 -- 172.21.15.22:0/2533903392 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7efcac034440 con 0x55dcf5219a10 2024-04-20T21:35:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.849+0000 7efca6a00640 1 -- 172.21.15.22:0/2533903392 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7efcac062050 con 0x55dcf5219a10 2024-04-20T21:35:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.849+0000 7efcca40f740 10 client.25490 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.849+0000 7efcca40f740 1 -- 172.21.15.22:0/2533903392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dcf5299090 con 0x55dcf5219a10 2024-04-20T21:35:01.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.849+0000 7efca6a00640 1 -- 172.21.15.22:0/2533903392 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb8018490 con 0x55dcf5219a10 2024-04-20T21:35:01.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.849+0000 7efca6a00640 1 -- 172.21.15.22:0/2533903392 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7efcac02c5b0 con 0x55dcf5219a10 2024-04-20T21:35:01.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.849+0000 7efcca40f740 10 client.25490 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.849+0000 7efcca40f740 10 client.25490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.849+0000 7efcca40f740 20 client.25490 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.849+0000 7efcca40f740 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dcf5232880 0x55dcf529b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.849+0000 7efcca40f740 4 client.25490 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.849+0000 7efcca40f740 1 -- 172.21.15.22:0/2533903392 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dcf51855b0 con 0x55dcf5232880 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.850+0000 7efcc5800640 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dcf5232880 0x55dcf529b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.850+0000 7efcc5800640 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dcf5232880 0x55dcf529b8a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f246005ddcf15b server_cookie=33d0f9586a011588 in_seq=0 out_seq=0 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.850+0000 7efca6a00640 10 client.25490 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.851+0000 7efca6a00640 1 -- 172.21.15.22:0/2533903392 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dcf51855b0 con 0x55dcf5232880 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.851+0000 7efca6a00640 10 client.25490 handle_command_reply: tid=0 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.851+0000 7efcca40f740 1 client.25490 shutdown 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.852+0000 7efcca40f740 1 -- 172.21.15.22:0/2533903392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf5219a10 msgr2=0x55dcf521c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.852+0000 7efcca40f740 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf5219a10 0x55dcf521c7f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efcac009980 tx=0x7efcac02c3a0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.852+0000 7efcca40f740 1 -- 172.21.15.22:0/2533903392 shutdown_connections 2024-04-20T21:35:01.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.852+0000 7efcca40f740 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf52225f0 0x55dcf521cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.852+0000 7efcca40f740 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dcf5232880 0x55dcf529b8a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.852+0000 7efcca40f740 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcf5219a10 0x55dcf521c7f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.852+0000 7efcca40f740 1 --2- 172.21.15.22:0/2533903392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dcf52190f0 0x55dcf521c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.852+0000 7efcca40f740 1 -- 172.21.15.22:0/2533903392 >> 172.21.15.22:0/2533903392 conn(0x55dcf5213820 msgr2=0x55dcf5212a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.852+0000 7efca6a00640 0 client.25490 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.852+0000 7efcca40f740 1 -- 172.21.15.22:0/2533903392 shutdown_connections 2024-04-20T21:35:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.852+0000 7efcca40f740 1 -- 172.21.15.22:0/2533903392 wait complete. 2024-04-20T21:35:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.852+0000 7efcca40f740 20 client.25490 trim_cache size 0 max 16384 2024-04-20T21:35:01.853 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:35:01.853 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:35:01.853 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "b348bb2f-5094-4140-9b00-41dff5d99eb8", 2024-04-20T21:35:01.853 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:35:01.853 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:35:01.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.853+0000 7efcc6200640 1 -- 172.21.15.22:0/3283972846 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efca80a3e40 msgr2=0x7efca80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.853+0000 7efcc6200640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efca80a3e40 0x7efca80a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efcb40ce4e0 tx=0x7efcb4009700 comp rx=0 tx=0).stop 2024-04-20T21:35:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.853+0000 7efcc6200640 1 -- 172.21.15.22:0/3283972846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc0102fc0 msgr2=0x7efcc01a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.853+0000 7efcc6200640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc0102fc0 0x7efcc01a0590 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e9f0 tx=0x7efcb400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.853+0000 7efcc6200640 1 -- 172.21.15.22:0/3283972846 shutdown_connections 2024-04-20T21:35:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.853+0000 7efcc6200640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efca80a3e40 0x7efca80a6300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.853+0000 7efcc6200640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcc010c3e0 0x7efcc019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.853+0000 7efcc6200640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01038e0 0x7efcc01a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.853+0000 7efcc6200640 1 --2- 172.21.15.22:0/3283972846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc0102fc0 0x7efcc01a0590 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.853+0000 7efcc6200640 1 -- 172.21.15.22:0/3283972846 >> 172.21.15.22:0/3283972846 conn(0x7efcc00fc7e0 msgr2=0x7efcc0101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.854+0000 7efcc6200640 1 -- 172.21.15.22:0/3283972846 shutdown_connections 2024-04-20T21:35:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:01.854+0000 7efcc6200640 1 -- 172.21.15.22:0/3283972846 wait complete. 2024-04-20T21:35:01.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:35:01.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:35:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.071+0000 7f55a4c00640 1 -- 172.21.15.22:0/3739211932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a010e6a0 msgr2=0x7f55a0110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:02.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.071+0000 7f55a4c00640 1 --2- 172.21.15.22:0/3739211932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a010e6a0 0x7f55a0110ab0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5598009ca0 tx=0x7f559802c180 comp rx=0 tx=0).stop 2024-04-20T21:35:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.071+0000 7f55a4c00640 1 -- 172.21.15.22:0/3739211932 shutdown_connections 2024-04-20T21:35:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.071+0000 7f55a4c00640 1 --2- 172.21.15.22:0/3739211932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a010e6a0 0x7f55a0110ab0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.071+0000 7f55a4c00640 1 --2- 172.21.15.22:0/3739211932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a0074c50 0x7f55a010e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.071+0000 7f55a4c00640 1 --2- 172.21.15.22:0/3739211932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55a0074330 0x7f55a0074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.071+0000 7f55a4c00640 1 -- 172.21.15.22:0/3739211932 >> 172.21.15.22:0/3739211932 conn(0x7f55a00fd930 msgr2=0x7f55a00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.071+0000 7f55a4c00640 1 -- 172.21.15.22:0/3739211932 shutdown_connections 2024-04-20T21:35:02.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.071+0000 7f55a4c00640 1 -- 172.21.15.22:0/3739211932 wait complete. 2024-04-20T21:35:02.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.072+0000 7f55a4c00640 1 Processor -- start 2024-04-20T21:35:02.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.072+0000 7f55a4c00640 1 -- start start 2024-04-20T21:35:02.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.072+0000 7f55a4c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a0074330 0x7f55a01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.072+0000 7f55a4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55a0074c50 0x7f55a01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f55a4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a010e6a0 0x7f55a019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f55a4c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01143e0 con 0x7f55a0074c50 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f55a4c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0114260 con 0x7f55a0074330 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f55a4c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0114560 con 0x7f55a010e6a0 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f559f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a0074330 0x7f55a01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f559f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a0074330 0x7f55a01a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45672/0 (socket says 172.21.15.22:45672) 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f559f400640 1 -- 172.21.15.22:0/644873972 learned_addr learned my addr 172.21.15.22:0/644873972 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f559ea00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55a0074c50 0x7f55a01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f559fe00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a010e6a0 0x7f55a019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f559fe00640 1 -- 172.21.15.22:0/644873972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a0074330 msgr2=0x7f55a01a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f559fe00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a0074330 0x7f55a01a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f559fe00640 1 -- 172.21.15.22:0/644873972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55a0074c50 msgr2=0x7f55a01a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f559fe00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55a0074c50 0x7f55a01a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f559fe00640 1 -- 172.21.15.22:0/644873972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a019b020 con 0x7f55a010e6a0 2024-04-20T21:35:02.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.073+0000 7f559f400640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a0074330 0x7f55a01a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:02.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.074+0000 7f559ea00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55a0074c50 0x7f55a01a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:02.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.074+0000 7f559fe00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a010e6a0 0x7f55a019a760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f559802c790 tx=0x7f5598035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:02.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.074+0000 7f5597e00640 1 -- 172.21.15.22:0/644873972 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598039040 con 0x7f55a010e6a0 2024-04-20T21:35:02.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.074+0000 7f55a4c00640 1 -- 172.21.15.22:0/644873972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a019b2b0 con 0x7f55a010e6a0 2024-04-20T21:35:02.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.074+0000 7f5597e00640 1 -- 172.21.15.22:0/644873972 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5598044070 con 0x7f55a010e6a0 2024-04-20T21:35:02.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.074+0000 7f5597e00640 1 -- 172.21.15.22:0/644873972 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980328d0 con 0x7f55a010e6a0 2024-04-20T21:35:02.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.075+0000 7f55a4c00640 1 -- 172.21.15.22:0/644873972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a01a73f0 con 0x7f55a010e6a0 2024-04-20T21:35:02.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.079+0000 7f55a8fce740 1 Processor -- start 2024-04-20T21:35:02.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.080+0000 7f5597e00640 1 -- 172.21.15.22:0/644873972 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7f5598003d00 con 0x7f55a010e6a0 2024-04-20T21:35:02.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.080+0000 7f55a8fce740 1 -- start start 2024-04-20T21:35:02.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.080+0000 7f55a8fce740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 0x55c28ece2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.080+0000 7f55a8fce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28eceb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.080+0000 7f55a8fce740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28eceb880 0x55c28ecedc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.080+0000 7f55a8fce740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c28ec48150 con 0x55c28ece2380 2024-04-20T21:35:02.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.080+0000 7f55a8fce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c28ebee370 con 0x55c28ece2ca0 2024-04-20T21:35:02.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.080+0000 7f55a8fce740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c28ec44110 con 0x55c28eceb880 2024-04-20T21:35:02.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.081+0000 7f559f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28eceb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.081+0000 7f559ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 0x55c28ece2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.081+0000 7f559ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 0x55c28ece2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45296/0 (socket says 172.21.15.22:45296) 2024-04-20T21:35:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.081+0000 7f559ea00640 1 -- 172.21.15.22:0/2669859589 learned_addr learned my addr 172.21.15.22:0/2669859589 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.081+0000 7f559ea00640 1 --2- 172.21.15.22:0/2669859589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28eceb880 0x55c28ecedc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.081+0000 7f5597e00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f55840a3e40 0x7f55840a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.081+0000 7f559f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28eceb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45686/0 (socket says 172.21.15.22:45686) 2024-04-20T21:35:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.082+0000 7f559ea00640 1 -- 172.21.15.22:0/2669859589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28eceb880 msgr2=0x55c28ecedc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.082+0000 7f559ea00640 1 --2- 172.21.15.22:0/2669859589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28eceb880 0x55c28ecedc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.082+0000 7f559ea00640 1 -- 172.21.15.22:0/2669859589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 msgr2=0x55c28eceb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.082+0000 7f559ea00640 1 --2- 172.21.15.22:0/2669859589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28eceb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.082+0000 7f5597e00640 1 -- 172.21.15.22:0/644873972 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f55980f2d50 con 0x7f55a010e6a0 2024-04-20T21:35:02.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.082+0000 7f559ea00640 1 -- 172.21.15.22:0/2669859589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28ea247d0 con 0x55c28ece2380 2024-04-20T21:35:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.082+0000 7f559f400640 1 --2- 172.21.15.22:0/2669859589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28eceb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.082+0000 7f559fe00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f55840a3e40 0x7f55840a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.082+0000 7f559ea00640 1 --2- 172.21.15.22:0/2669859589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 0x55c28ece2760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f558c011b50 tx=0x7f558c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d308cce601685571 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:02.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.082+0000 7f558be00640 1 -- 172.21.15.22:0/2669859589 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c00fd10 con 0x55c28ece2380 2024-04-20T21:35:02.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.083+0000 7f558be00640 1 -- 172.21.15.22:0/2669859589 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558c0035f0 con 0x55c28ece2380 2024-04-20T21:35:02.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.083+0000 7f558be00640 1 -- 172.21.15.22:0/2669859589 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c00e640 con 0x55c28ece2380 2024-04-20T21:35:02.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.083+0000 7f55a8fce740 1 -- 172.21.15.22:0/2669859589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 msgr2=0x55c28ece2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:02.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.083+0000 7f559fe00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f55840a3e40 0x7f55840a6300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55980f0470 tx=0x7f559802c180 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:02.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.083+0000 7f55a8fce740 1 --2- 172.21.15.22:0/2669859589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 0x55c28ece2760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f558c011b50 tx=0x7f558c009b30 comp rx=0 tx=0).stop 2024-04-20T21:35:02.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.084+0000 7f55a8fce740 1 -- 172.21.15.22:0/2669859589 shutdown_connections 2024-04-20T21:35:02.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.084+0000 7f55a8fce740 1 --2- 172.21.15.22:0/2669859589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28eceb880 0x55c28ecedc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.084+0000 7f55a8fce740 1 --2- 172.21.15.22:0/2669859589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28eceb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.084+0000 7f55a8fce740 1 --2- 172.21.15.22:0/2669859589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 0x55c28ece2760 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.084+0000 7f55a8fce740 1 -- 172.21.15.22:0/2669859589 >> 172.21.15.22:0/2669859589 conn(0x55c28ecdcab0 msgr2=0x55c28ecdce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:02.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.084+0000 7f55a8fce740 1 -- 172.21.15.22:0/2669859589 shutdown_connections 2024-04-20T21:35:02.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.084+0000 7f55a8fce740 1 -- 172.21.15.22:0/2669859589 wait complete. 2024-04-20T21:35:02.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.084+0000 7f55a8fce740 1 -- start start 2024-04-20T21:35:02.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.085+0000 7f55a8fce740 1 Processor -- start 2024-04-20T21:35:02.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.085+0000 7f55a8fce740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28ece2380 0x55c28ece8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.085+0000 7f55a8fce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28ece8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.085+0000 7f55a8fce740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28eceb880 0x55c28ece93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.085+0000 7f55a8fce740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c28ecee670 con 0x55c28eceb880 2024-04-20T21:35:02.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.085+0000 7f55a8fce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c28ec01740 con 0x55c28ece2ca0 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.085+0000 7f55a8fce740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c28ecee7f0 con 0x55c28ece2380 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.085+0000 7f55a8fce740 10 client.0 _ll_register_callbacks cb 0x55c28ebfcb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.086+0000 7f559ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28eceb880 0x55c28ece93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.086+0000 7f559ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28ece2380 0x55c28ece8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.086+0000 7f559ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28ece2380 0x55c28ece8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:56906/0 (socket says 172.21.15.22:56906) 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.086+0000 7f559ea00640 1 -- 172.21.15.22:0/3115211399 learned_addr learned my addr 172.21.15.22:0/3115211399 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.086+0000 7f559f400640 1 --2- 172.21.15.22:0/3115211399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28ece8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.086+0000 7f559ea00640 1 -- 172.21.15.22:0/3115211399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 msgr2=0x55c28ece8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.086+0000 7f559ea00640 1 --2- 172.21.15.22:0/3115211399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28ece8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.086+0000 7f559ea00640 1 -- 172.21.15.22:0/3115211399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28eceb880 msgr2=0x55c28ece93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.086+0000 7f559ea00640 1 --2- 172.21.15.22:0/3115211399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28eceb880 0x55c28ece93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.086+0000 7f559ea00640 1 -- 172.21.15.22:0/3115211399 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c28e95a3f0 con 0x55c28ece2380 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.087+0000 7f559ea00640 1 --2- 172.21.15.22:0/3115211399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28ece2380 0x55c28ece8950 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f558c011b20 tx=0x7f558c003f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d58a0cfd5e94be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.087+0000 7f5588c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:35:02.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.087+0000 7f5588c00640 1 -- 172.21.15.22:0/3115211399 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c025030 con 0x55c28ece2380 2024-04-20T21:35:02.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.087+0000 7f55a8fce740 1 -- 172.21.15.22:0/3115211399 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f558c0035f0 con 0x55c28ece2380 2024-04-20T21:35:02.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.087+0000 7f5588c00640 1 -- 172.21.15.22:0/3115211399 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f558c004240 con 0x55c28ece2380 2024-04-20T21:35:02.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.087+0000 7f5588c00640 1 -- 172.21.15.22:0/3115211399 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f558c028040 con 0x55c28ece2380 2024-04-20T21:35:02.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.087+0000 7f55a8fce740 10 client.25540 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:02.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.087+0000 7f55a8fce740 1 -- 172.21.15.22:0/3115211399 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c28e91a0f0 con 0x55c28ece2380 2024-04-20T21:35:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.088+0000 7f5588c00640 1 -- 172.21.15.22:0/3115211399 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558c0035f0 con 0x55c28ece2380 2024-04-20T21:35:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.088+0000 7f5588c00640 1 -- 172.21.15.22:0/3115211399 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f558c01fca0 con 0x55c28ece2380 2024-04-20T21:35:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.088+0000 7f55a8fce740 10 client.25540 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.088+0000 7f55a8fce740 10 client.25540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.088+0000 7f55a8fce740 20 client.25540 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.088+0000 7f55a8fce740 1 --2- 172.21.15.22:0/3115211399 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c28ecfbb20 0x55c28ecfdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.088+0000 7f55a8fce740 4 client.25540 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.088+0000 7f55a8fce740 1 -- 172.21.15.22:0/3115211399 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c28ec55b70 con 0x55c28ecfbb20 2024-04-20T21:35:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.089+0000 7f559f400640 1 --2- 172.21.15.22:0/3115211399 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c28ecfbb20 0x55c28ecfdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.089+0000 7f559f400640 1 --2- 172.21.15.22:0/3115211399 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c28ecfbb20 0x55c28ecfdf10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0454344657f483 server_cookie=c9cd1497c6d4d9c7 in_seq=0 out_seq=0 2024-04-20T21:35:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.089+0000 7f5588c00640 10 client.25540 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.090+0000 7f5588c00640 1 -- 172.21.15.22:0/3115211399 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c28ec55b70 con 0x55c28ecfbb20 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.090+0000 7f5588c00640 10 client.25540 handle_command_reply: tid=0 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.090+0000 7f55a8fce740 1 client.25540 shutdown 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.090+0000 7f55a8fce740 1 -- 172.21.15.22:0/3115211399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28ece2380 msgr2=0x55c28ece8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.090+0000 7f55a8fce740 1 --2- 172.21.15.22:0/3115211399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28ece2380 0x55c28ece8950 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f558c011b20 tx=0x7f558c003f40 comp rx=0 tx=0).stop 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.091+0000 7f55a8fce740 1 -- 172.21.15.22:0/3115211399 shutdown_connections 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.091+0000 7f55a8fce740 1 --2- 172.21.15.22:0/3115211399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28eceb880 0x55c28ece93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.091+0000 7f55a8fce740 1 --2- 172.21.15.22:0/3115211399 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c28ecfbb20 0x55c28ecfdf10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.091+0000 7f55a8fce740 1 --2- 172.21.15.22:0/3115211399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28ece8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.091+0000 7f55a8fce740 1 --2- 172.21.15.22:0/3115211399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28ece2380 0x55c28ece8950 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.091+0000 7f5588c00640 0 client.25540 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.091+0000 7f55a8fce740 1 -- 172.21.15.22:0/3115211399 >> 172.21.15.22:0/3115211399 conn(0x55c28ecdcab0 msgr2=0x55c28ecde800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.091+0000 7f55a8fce740 1 -- 172.21.15.22:0/3115211399 shutdown_connections 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.091+0000 7f55a8fce740 1 -- 172.21.15.22:0/3115211399 wait complete. 2024-04-20T21:35:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.091+0000 7f55a8fce740 20 client.25540 trim_cache size 0 max 16384 2024-04-20T21:35:02.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.127+0000 7f55a8fce740 1 Processor -- start 2024-04-20T21:35:02.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.127+0000 7f55a8fce740 1 -- start start 2024-04-20T21:35:02.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.128+0000 7f55a8fce740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 0x55c28eceb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.128+0000 7f55a8fce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28ecf7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.128+0000 7f55a8fce740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28eceb880 0x55c28ed03650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.128+0000 7f55a8fce740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01adcc0 con 0x55c28ece2380 2024-04-20T21:35:02.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.128+0000 7f55a8fce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01b0540 con 0x55c28ece2ca0 2024-04-20T21:35:02.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.128+0000 7f55a8fce740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01ae330 con 0x55c28eceb880 2024-04-20T21:35:02.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.128+0000 7f559ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 0x55c28eceb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.128+0000 7f559f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28ecf7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.128+0000 7f559f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28ecf7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45698/0 (socket says 172.21.15.22:45698) 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.128+0000 7f559f400640 1 -- 172.21.15.22:0/1303355005 learned_addr learned my addr 172.21.15.22:0/1303355005 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.128+0000 7f559ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 0x55c28eceb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45306/0 (socket says 172.21.15.22:45306) 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.129+0000 7f559f400640 1 -- 172.21.15.22:0/1303355005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28eceb880 msgr2=0x55c28ed03650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.129+0000 7f559ea00640 1 --2- 172.21.15.22:0/1303355005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28eceb880 0x55c28ed03650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.129+0000 7f559f400640 1 --2- 172.21.15.22:0/1303355005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28eceb880 0x55c28ed03650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.129+0000 7f559f400640 1 -- 172.21.15.22:0/1303355005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 msgr2=0x55c28eceb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.129+0000 7f559f400640 1 --2- 172.21.15.22:0/1303355005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 0x55c28eceb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.129+0000 7f559f400640 1 -- 172.21.15.22:0/1303355005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28ece6ea0 con 0x55c28ece2ca0 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.129+0000 7f559ea00640 1 --2- 172.21.15.22:0/1303355005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28eceb880 0x55c28ed03650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.129+0000 7f559f400640 1 --2- 172.21.15.22:0/1303355005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28ecf7530 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5590005d00 tx=0x7f5590005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b44744a094c2b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.129+0000 7f558b400640 1 -- 172.21.15.22:0/1303355005 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590023020 con 0x55c28ece2ca0 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.129+0000 7f558b400640 1 -- 172.21.15.22:0/1303355005 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5590010dc0 con 0x55c28ece2ca0 2024-04-20T21:35:02.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.129+0000 7f558b400640 1 -- 172.21.15.22:0/1303355005 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590005650 con 0x55c28ece2ca0 2024-04-20T21:35:02.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.130+0000 7f55a8fce740 1 -- 172.21.15.22:0/1303355005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 msgr2=0x55c28ecf7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:02.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.130+0000 7f55a8fce740 1 --2- 172.21.15.22:0/1303355005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28ecf7530 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5590005d00 tx=0x7f5590005f90 comp rx=0 tx=0).stop 2024-04-20T21:35:02.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.130+0000 7f55a8fce740 1 -- 172.21.15.22:0/1303355005 shutdown_connections 2024-04-20T21:35:02.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.130+0000 7f55a8fce740 1 --2- 172.21.15.22:0/1303355005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28eceb880 0x55c28ed03650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.130+0000 7f55a8fce740 1 --2- 172.21.15.22:0/1303355005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2ca0 0x55c28ecf7530 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.130+0000 7f55a8fce740 1 --2- 172.21.15.22:0/1303355005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28ece2380 0x55c28eceb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.130+0000 7f55a8fce740 1 -- 172.21.15.22:0/1303355005 >> 172.21.15.22:0/1303355005 conn(0x55c28ecdcab0 msgr2=0x55c28ece7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.130+0000 7f55a8fce740 1 -- 172.21.15.22:0/1303355005 shutdown_connections 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.131+0000 7f55a8fce740 1 -- 172.21.15.22:0/1303355005 wait complete. 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.131+0000 7f55a8fce740 1 -- start start 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.131+0000 7f55a8fce740 1 Processor -- start 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.132+0000 7f55a8fce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2380 0x55c28ecf3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.132+0000 7f55a8fce740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28ece2ca0 0x55c28ecef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.132+0000 7f55a8fce740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28eceb880 0x55c28ecef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.132+0000 7f55a8fce740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01aea00 con 0x55c28eceb880 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.132+0000 7f55a8fce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01afe70 con 0x55c28ece2380 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.132+0000 7f55a8fce740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01af0d0 con 0x55c28ece2ca0 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.132+0000 7f55a8fce740 10 client.0 _ll_register_callbacks cb 0x7f558c011610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.132+0000 7f559ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28eceb880 0x55c28ecef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.132+0000 7f559ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28eceb880 0x55c28ecef5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45308/0 (socket says 172.21.15.22:45308) 2024-04-20T21:35:02.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.132+0000 7f559ea00640 1 -- 172.21.15.22:0/1681205442 learned_addr learned my addr 172.21.15.22:0/1681205442 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:02.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.132+0000 7f559ea00640 1 --2- 172.21.15.22:0/1681205442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2380 0x55c28ecf3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.133+0000 7f559ea00640 1 -- 172.21.15.22:0/1681205442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28ece2ca0 msgr2=0x55c28ecef080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.133+0000 7f559ea00640 1 --2- 172.21.15.22:0/1681205442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28ece2ca0 0x55c28ecef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.133+0000 7f559ea00640 1 -- 172.21.15.22:0/1681205442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2380 msgr2=0x55c28ecf3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.133+0000 7f559ea00640 1 --2- 172.21.15.22:0/1681205442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2380 0x55c28ecf3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.133+0000 7f559ea00640 1 -- 172.21.15.22:0/1681205442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c28ecefcb0 con 0x55c28eceb880 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.133+0000 7f559ea00640 1 --2- 172.21.15.22:0/1681205442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28eceb880 0x55c28ecef5c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f558c009250 tx=0x7f558c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12f73b99ff2b6c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.133+0000 7f5588c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.133+0000 7f5588c00640 1 -- 172.21.15.22:0/1681205442 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c025070 con 0x55c28eceb880 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.133+0000 7f5588c00640 1 -- 172.21.15.22:0/1681205442 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f558c009c80 con 0x55c28eceb880 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.134+0000 7f55a8fce740 1 -- 172.21.15.22:0/1681205442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5590010dc0 con 0x55c28eceb880 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.134+0000 7f5588c00640 1 -- 172.21.15.22:0/1681205442 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f558c03a040 con 0x55c28eceb880 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.134+0000 7f55a8fce740 10 client.15396 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.134+0000 7f55a8fce740 1 -- 172.21.15.22:0/1681205442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c28ed61d40 con 0x55c28eceb880 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.134+0000 7f5588c00640 1 -- 172.21.15.22:0/1681205442 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5590010dc0 con 0x55c28eceb880 2024-04-20T21:35:02.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.134+0000 7f5588c00640 1 -- 172.21.15.22:0/1681205442 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f558c040070 con 0x55c28eceb880 2024-04-20T21:35:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.134+0000 7f55a8fce740 10 client.15396 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.134+0000 7f55a8fce740 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.134+0000 7f55a8fce740 20 client.15396 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.135+0000 7f55a8fce740 1 --2- 172.21.15.22:0/1681205442 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c28ed62400 0x55c28ed64840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.135+0000 7f55a8fce740 4 client.15396 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:35:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.135+0000 7f55a8fce740 1 -- 172.21.15.22:0/1681205442 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c28ec4f9e0 con 0x55c28ed62400 2024-04-20T21:35:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.135+0000 7f559f400640 1 --2- 172.21.15.22:0/1681205442 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c28ed62400 0x55c28ed64840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f559f400640 1 --2- 172.21.15.22:0/1681205442 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c28ed62400 0x55c28ed64840 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd7765107d3e0b84 server_cookie=5d0a81d145792118 in_seq=0 out_seq=0 2024-04-20T21:35:02.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f5588c00640 10 client.15396 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f5588c00640 1 -- 172.21.15.22:0/1681205442 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c28ec4f9e0 con 0x55c28ed62400 2024-04-20T21:35:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f5588c00640 10 client.15396 handle_command_reply: tid=0 2024-04-20T21:35:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f55a8fce740 1 client.15396 shutdown 2024-04-20T21:35:02.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f55a8fce740 1 -- 172.21.15.22:0/1681205442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28eceb880 msgr2=0x55c28ecef5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f55a8fce740 1 --2- 172.21.15.22:0/1681205442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28eceb880 0x55c28ecef5c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f558c009250 tx=0x7f558c009b30 comp rx=0 tx=0).stop 2024-04-20T21:35:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f55a8fce740 1 -- 172.21.15.22:0/1681205442 shutdown_connections 2024-04-20T21:35:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f55a8fce740 1 --2- 172.21.15.22:0/1681205442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c28eceb880 0x55c28ecef5c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f55a8fce740 1 --2- 172.21.15.22:0/1681205442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c28ece2ca0 0x55c28ecef080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f55a8fce740 1 --2- 172.21.15.22:0/1681205442 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c28ed62400 0x55c28ed64840 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f55a8fce740 1 --2- 172.21.15.22:0/1681205442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c28ece2380 0x55c28ecf3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.136+0000 7f55a8fce740 1 -- 172.21.15.22:0/1681205442 >> 172.21.15.22:0/1681205442 conn(0x55c28ecdcab0 msgr2=0x55c28ece70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.137+0000 7f5588c00640 0 client.15396 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.137+0000 7f55a8fce740 1 -- 172.21.15.22:0/1681205442 shutdown_connections 2024-04-20T21:35:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.137+0000 7f55a8fce740 1 -- 172.21.15.22:0/1681205442 wait complete. 2024-04-20T21:35:02.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.137+0000 7f55a8fce740 20 client.15396 trim_cache size 0 max 16384 2024-04-20T21:35:02.139 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:35:02.139 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:35:02.139 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:35:02.139 INFO:teuthology.orchestra.run.smithi022.stdout: "b348bb2f-5094-4140-9b00-41dff5d99eb8": { 2024-04-20T21:35:02.139 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:35:02.139 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "b348bb2f-5094-4140-9b00-41dff5d99eb8", 2024-04-20T21:35:02.139 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:35:02.139 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:35:02.139 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:35:02.139 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:35:02.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.138+0000 7f55a4c00640 1 -- 172.21.15.22:0/644873972 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f55840a3e40 msgr2=0x7f55840a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:02.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.138+0000 7f55a4c00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f55840a3e40 0x7f55840a6300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55980f0470 tx=0x7f559802c180 comp rx=0 tx=0).stop 2024-04-20T21:35:02.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.138+0000 7f55a4c00640 1 -- 172.21.15.22:0/644873972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a010e6a0 msgr2=0x7f55a019a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:02.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.138+0000 7f55a4c00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a010e6a0 0x7f55a019a760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f559802c790 tx=0x7f5598035d10 comp rx=0 tx=0).stop 2024-04-20T21:35:02.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.138+0000 7f55a4c00640 1 -- 172.21.15.22:0/644873972 shutdown_connections 2024-04-20T21:35:02.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.138+0000 7f55a4c00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f55840a3e40 0x7f55840a6300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.138+0000 7f55a4c00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a010e6a0 0x7f55a019a760 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.138+0000 7f55a4c00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55a0074c50 0x7f55a01a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.138+0000 7f55a4c00640 1 --2- 172.21.15.22:0/644873972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a0074330 0x7f55a01a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:02.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.138+0000 7f55a4c00640 1 -- 172.21.15.22:0/644873972 >> 172.21.15.22:0/644873972 conn(0x7f55a00fd930 msgr2=0x7f55a00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:02.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.138+0000 7f55a4c00640 1 -- 172.21.15.22:0/644873972 shutdown_connections 2024-04-20T21:35:02.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:02.138+0000 7f55a4c00640 1 -- 172.21.15.22:0/644873972 wait complete. 2024-04-20T21:35:02.154 INFO:tasks.cephfs.filesystem:scrub status for tag:b348bb2f-5094-4140-9b00-41dff5d99eb8 - {'path': '/', 'tag': 'b348bb2f-5094-4140-9b00-41dff5d99eb8', 'options': 'recursive,force'} 2024-04-20T21:35:02.275 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:02 smithi022 ceph-mon[16843]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:35:02.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:02 smithi122 ceph-mon[22147]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:35:02.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:02 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:35:02.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:02 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:02.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:02 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:35:02.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:02 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:02 smithi033 ceph-mon[21626]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:35:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:02 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:35:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:02 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:02 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:35:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:02 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:02.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:02 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:35:02.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:02 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:02.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:02 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:35:02.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:02 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:04.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:04 smithi033 ceph-mon[21626]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:35:04.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:04 smithi022 ceph-mon[16843]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:35:04.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:04 smithi122 ceph-mon[22147]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:35:06.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:06 smithi033 ceph-mon[21626]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 12 KiB/s wr, 4 op/s 2024-04-20T21:35:06.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:06 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:35:06.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:06 smithi022 ceph-mon[16843]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 12 KiB/s wr, 4 op/s 2024-04-20T21:35:06.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:06 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:35:06.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:06 smithi122 ceph-mon[22147]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 12 KiB/s wr, 4 op/s 2024-04-20T21:35:06.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:06 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:35:08.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:08 smithi033 ceph-mon[21626]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T21:35:08.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:08 smithi022 ceph-mon[16843]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T21:35:08.778 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:08 smithi122 ceph-mon[22147]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T21:35:10.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:10 smithi033 ceph-mon[21626]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T21:35:10.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:10 smithi022 ceph-mon[16843]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T21:35:10.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:10 smithi122 ceph-mon[22147]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T21:35:12.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:12 smithi033 ceph-mon[21626]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T21:35:12.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:12 smithi022 ceph-mon[16843]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T21:35:12.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:12 smithi122 ceph-mon[22147]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T21:35:14.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:14 smithi033 ceph-mon[21626]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T21:35:14.577 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:14 smithi122 ceph-mon[22147]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T21:35:14.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:14 smithi022 ceph-mon[16843]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T21:35:16.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:16 smithi033 ceph-mon[21626]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T21:35:16.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:16 smithi022 ceph-mon[16843]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T21:35:16.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:16 smithi122 ceph-mon[22147]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T21:35:18.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:18 smithi033 ceph-mon[21626]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-20T21:35:18.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:18 smithi022 ceph-mon[16843]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-20T21:35:18.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:18 smithi122 ceph-mon[22147]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-20T21:35:20.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:20 smithi022 ceph-mon[16843]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-20T21:35:20.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:20 smithi122 ceph-mon[22147]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-20T21:35:20.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:20 smithi033 ceph-mon[21626]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-20T21:35:22.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:22 smithi022 ceph-mon[16843]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 426 B/s wr, 3 op/s 2024-04-20T21:35:22.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:22 smithi022 ceph-mon[16843]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-04-20T21:35:22.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:22 smithi122 ceph-mon[22147]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 426 B/s wr, 3 op/s 2024-04-20T21:35:22.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:22 smithi122 ceph-mon[22147]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-04-20T21:35:22.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:22 smithi033 ceph-mon[21626]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 426 B/s wr, 3 op/s 2024-04-20T21:35:22.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:22 smithi033 ceph-mon[21626]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-04-20T21:35:24.606 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:24 smithi033 ceph-mon[21626]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:24.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:24 smithi022 ceph-mon[16843]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:24.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:24 smithi122 ceph-mon[22147]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:26.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:26 smithi022 ceph-mon[16843]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:26 smithi122 ceph-mon[22147]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:26 smithi033 ceph-mon[21626]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:28.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:28 smithi022 ceph-mon[16843]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:28.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:28 smithi122 ceph-mon[22147]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:28.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:28 smithi033 ceph-mon[21626]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:29.655 INFO:tasks.workunit.client.0.smithi022.stderr:Updating files: 59% (7243/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 96% (11871/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-20T21:35:30.603 INFO:tasks.workunit.client.0.smithi022.stderr:Note: switching to 'a8c84f5997950fd7d008204839253291d52e6eda'. 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr:state without impacting any branches by switching back to a branch. 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr: git switch -c 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr:Or undo this operation with: 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr: git switch - 2024-04-20T21:35:30.604 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-04-20T21:35:30.605 INFO:tasks.workunit.client.0.smithi022.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-20T21:35:30.605 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-04-20T21:35:30.605 INFO:tasks.workunit.client.0.smithi022.stderr:HEAD is now at a8c84f59979 Merge PR #51659 into wip-rishabh-testing-20240416.193735 2024-04-20T21:35:30.620 DEBUG:teuthology.orchestra.run.smithi022:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-20T21:35:30.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:30 smithi022 ceph-mon[16843]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:30.683 INFO:tasks.workunit.client.0.smithi022.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-20T21:35:30.690 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T21:35:30.691 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-20T21:35:30.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:30 smithi122 ceph-mon[22147]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:30.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:30 smithi033 ceph-mon[21626]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:31.548 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-20T21:35:31.663 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-20T21:35:31.758 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T21:35:31.764 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T21:35:31.764 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-20T21:35:31.848 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T21:35:31.862 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:35:31.862 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-20T21:35:31.964 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-20T21:35:31.965 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-20T21:35:31.965 DEBUG:teuthology.orchestra.run.smithi022:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-20T21:35:32.044 INFO:tasks.workunit.client.0.smithi022.stderr:+ dbench 1 2024-04-20T21:35:32.056 INFO:tasks.workunit.client.0.smithi022.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-20T21:35:32.056 INFO:tasks.workunit.client.0.smithi022.stdout: 2024-04-20T21:35:32.056 INFO:tasks.workunit.client.0.smithi022.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-20T21:35:32.060 INFO:tasks.workunit.client.0.smithi022.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-20T21:35:32.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:35:32.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:35:32.163 INFO:tasks.workunit.client.0.smithi022.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-20T21:35:32.163 INFO:tasks.workunit.client.0.smithi022.stdout:releasing clients 2024-04-20T21:35:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.383+0000 7f7f15c00640 1 -- 172.21.15.22:0/2696883216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f10104c30 msgr2=0x7f7f10108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.383+0000 7f7f15c00640 1 --2- 172.21.15.22:0/2696883216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f10104c30 0x7f7f10108c70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009900 tx=0x7f7ef802bc40 comp rx=0 tx=0).stop 2024-04-20T21:35:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.383+0000 7f7f15c00640 1 -- 172.21.15.22:0/2696883216 shutdown_connections 2024-04-20T21:35:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.383+0000 7f7f15c00640 1 --2- 172.21.15.22:0/2696883216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f10109350 0x7f7f10110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.383+0000 7f7f15c00640 1 --2- 172.21.15.22:0/2696883216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f10104c30 0x7f7f10108c70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.383+0000 7f7f15c00640 1 --2- 172.21.15.22:0/2696883216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f10104280 0x7f7f10104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.384+0000 7f7f15c00640 1 -- 172.21.15.22:0/2696883216 >> 172.21.15.22:0/2696883216 conn(0x7f7f100ffaa0 msgr2=0x7f7f10101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.384+0000 7f7f15c00640 1 -- 172.21.15.22:0/2696883216 shutdown_connections 2024-04-20T21:35:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.384+0000 7f7f15c00640 1 -- 172.21.15.22:0/2696883216 wait complete. 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f15c00640 1 Processor -- start 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f15c00640 1 -- start start 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f15c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f10104280 0x7f7f101a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f15c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f10109350 0x7f7f101a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f15c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1019a7d0 0x7f7f1019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f15c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f10114430 con 0x7f7f10104280 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f15c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f101142b0 con 0x7f7f1019a7d0 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f15c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f101145b0 con 0x7f7f10109350 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f15200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1019a7d0 0x7f7f1019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f0fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f10104280 0x7f7f101a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f15200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1019a7d0 0x7f7f1019ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38880/0 (socket says 172.21.15.22:38880) 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f15200640 1 -- 172.21.15.22:0/2321948279 learned_addr learned my addr 172.21.15.22:0/2321948279 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.385+0000 7f7f07e00640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f10109350 0x7f7f101a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.386+0000 7f7f15200640 1 -- 172.21.15.22:0/2321948279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f10109350 msgr2=0x7f7f101a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.386+0000 7f7f15200640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f10109350 0x7f7f101a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.386+0000 7f7f15200640 1 -- 172.21.15.22:0/2321948279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f10104280 msgr2=0x7f7f101a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.386+0000 7f7f15200640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f10104280 0x7f7f101a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.386+0000 7f7f15200640 1 -- 172.21.15.22:0/2321948279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1019b360 con 0x7f7f1019a7d0 2024-04-20T21:35:32.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.386+0000 7f7f15200640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1019a7d0 0x7f7f1019ac80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009fd0 tx=0x7f7f0800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:32.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.386+0000 7f7f0d600640 1 -- 172.21.15.22:0/2321948279 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08017070 con 0x7f7f1019a7d0 2024-04-20T21:35:32.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.387+0000 7f7f0d600640 1 -- 172.21.15.22:0/2321948279 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f08003e50 con 0x7f7f1019a7d0 2024-04-20T21:35:32.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.387+0000 7f7f15c00640 1 -- 172.21.15.22:0/2321948279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1019b650 con 0x7f7f1019a7d0 2024-04-20T21:35:32.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.387+0000 7f7f0d600640 1 -- 172.21.15.22:0/2321948279 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080106c0 con 0x7f7f1019a7d0 2024-04-20T21:35:32.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.387+0000 7f7f15c00640 1 -- 172.21.15.22:0/2321948279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f101a77b0 con 0x7f7f1019a7d0 2024-04-20T21:35:32.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.390+0000 7f7f19e44740 1 Processor -- start 2024-04-20T21:35:32.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.393+0000 7f7f0d600640 1 -- 172.21.15.22:0/2321948279 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7f08003970 con 0x7f7f1019a7d0 2024-04-20T21:35:32.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.393+0000 7f7f19e44740 1 -- start start 2024-04-20T21:35:32.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f19e44740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f19e44740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 0x5652696ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f19e44740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696ce880 0x5652696d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f19e44740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56526962b150 con 0x5652696c5ca0 2024-04-20T21:35:32.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f19e44740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5652695d1370 con 0x5652696ce880 2024-04-20T21:35:32.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f19e44740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565269627110 con 0x5652696c5380 2024-04-20T21:35:32.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f07e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f0fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 0x5652696ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f0fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 0x5652696ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57228/0 (socket says 172.21.15.22:57228) 2024-04-20T21:35:32.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f07e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:57352/0 (socket says 172.21.15.22:57352) 2024-04-20T21:35:32.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f0fe00640 1 -- 172.21.15.22:0/1836667272 learned_addr learned my addr 172.21.15.22:0/1836667272 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:32.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f07e00640 1 --2- 172.21.15.22:0/1836667272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696ce880 0x5652696d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f0fe00640 1 -- 172.21.15.22:0/1836667272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 msgr2=0x5652696c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:32.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f0fe00640 1 --2- 172.21.15.22:0/1836667272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f0fe00640 1 -- 172.21.15.22:0/1836667272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696ce880 msgr2=0x5652696d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:32.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f0fe00640 1 --2- 172.21.15.22:0/1836667272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696ce880 0x5652696d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.394+0000 7f7f0fe00640 1 -- 172.21.15.22:0/1836667272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652694077d0 con 0x5652696c5ca0 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.395+0000 7f7f0fe00640 1 --2- 172.21.15.22:0/1836667272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 0x5652696ce340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009d20 tx=0x7f7f00011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107ece96a85f0d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.395+0000 7f7f05600640 1 -- 172.21.15.22:0/1836667272 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0001e070 con 0x5652696c5ca0 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.395+0000 7f7f0d600640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7ef40a3da0 0x7f7ef40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.395+0000 7f7f05600640 1 -- 172.21.15.22:0/1836667272 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f00019b10 con 0x5652696c5ca0 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.395+0000 7f7f05600640 1 -- 172.21.15.22:0/1836667272 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000eb20 con 0x5652696c5ca0 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.395+0000 7f7f0d600640 1 -- 172.21.15.22:0/2321948279 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7f080d0410 con 0x7f7f1019a7d0 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.396+0000 7f7f07e00640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7ef40a3da0 0x7f7ef40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.396+0000 7f7f19e44740 1 -- 172.21.15.22:0/1836667272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 msgr2=0x5652696ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.396+0000 7f7f19e44740 1 --2- 172.21.15.22:0/1836667272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 0x5652696ce340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009d20 tx=0x7f7f00011810 comp rx=0 tx=0).stop 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.396+0000 7f7f19e44740 1 -- 172.21.15.22:0/1836667272 shutdown_connections 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.396+0000 7f7f19e44740 1 --2- 172.21.15.22:0/1836667272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696ce880 0x5652696d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.396+0000 7f7f19e44740 1 --2- 172.21.15.22:0/1836667272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 0x5652696ce340 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.396+0000 7f7f19e44740 1 --2- 172.21.15.22:0/1836667272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.396+0000 7f7f19e44740 1 -- 172.21.15.22:0/1836667272 >> 172.21.15.22:0/1836667272 conn(0x5652696bfab0 msgr2=0x5652696bfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:32.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.396+0000 7f7f07e00640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7ef40a3da0 0x7f7ef40a6260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ef802c250 tx=0x7f7ef8000bc0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:32.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.396+0000 7f7f19e44740 1 -- 172.21.15.22:0/1836667272 shutdown_connections 2024-04-20T21:35:32.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.396+0000 7f7f19e44740 1 -- 172.21.15.22:0/1836667272 wait complete. 2024-04-20T21:35:32.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.397+0000 7f7f19e44740 1 -- start start 2024-04-20T21:35:32.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.397+0000 7f7f19e44740 1 Processor -- start 2024-04-20T21:35:32.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.398+0000 7f7f19e44740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.398+0000 7f7f19e44740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 0x5652696cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.398+0000 7f7f19e44740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 0x5652696cc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.398+0000 7f7f19e44740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5652696d1670 con 0x5652696ce880 2024-04-20T21:35:32.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.398+0000 7f7f19e44740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5652695e4740 con 0x5652696c5ca0 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.398+0000 7f7f19e44740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5652696d17f0 con 0x5652696c5380 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.398+0000 7f7f19e44740 10 client.0 _ll_register_callbacks cb 0x5652695dfb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.398+0000 7f7f07e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 0x5652696cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.398+0000 7f7f15200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 0x5652696cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.398+0000 7f7f07e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 0x5652696cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38892/0 (socket says 172.21.15.22:38892) 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.398+0000 7f7f07e00640 1 -- 172.21.15.22:0/2041147722 learned_addr learned my addr 172.21.15.22:0/2041147722 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.398+0000 7f7f0fe00640 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.399+0000 7f7f07e00640 1 -- 172.21.15.22:0/2041147722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 msgr2=0x5652696cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.399+0000 7f7f07e00640 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.399+0000 7f7f07e00640 1 -- 172.21.15.22:0/2041147722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 msgr2=0x5652696cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.399+0000 7f7f07e00640 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 0x5652696cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.399+0000 7f7f07e00640 1 -- 172.21.15.22:0/2041147722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56526933d3f0 con 0x5652696c5ca0 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.399+0000 7f7f15200640 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 0x5652696cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.399+0000 7f7f07e00640 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 0x5652696cbe00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009900 tx=0x7f7ef8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f39c8c0d86ef306c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.399+0000 7f7efe000640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:35:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.399+0000 7f7efe000640 1 -- 172.21.15.22:0/2041147722 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8053070 con 0x5652696c5ca0 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.399+0000 7f7f19e44740 1 -- 172.21.15.22:0/2041147722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f00019b10 con 0x5652696c5ca0 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.399+0000 7f7efe000640 1 -- 172.21.15.22:0/2041147722 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7ef8004ad0 con 0x5652696c5ca0 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.400+0000 7f7efe000640 1 -- 172.21.15.22:0/2041147722 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7ef8004f40 con 0x5652696c5ca0 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.400+0000 7f7f19e44740 10 client.25520 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.400+0000 7f7f19e44740 1 -- 172.21.15.22:0/2041147722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5652692fd0f0 con 0x5652696c5ca0 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.400+0000 7f7efe000640 1 -- 172.21.15.22:0/2041147722 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f00019b10 con 0x5652696c5ca0 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.400+0000 7f7efe000640 1 -- 172.21.15.22:0/2041147722 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f7ef804d040 con 0x5652696c5ca0 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.400+0000 7f7f19e44740 10 client.25520 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.400+0000 7f7f19e44740 10 client.25520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.400+0000 7f7f19e44740 20 client.25520 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.400+0000 7f7f19e44740 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5652696debf0 0x5652696e1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.400+0000 7f7f19e44740 4 client.25520 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.400+0000 7f7f19e44740 1 -- 172.21.15.22:0/2041147722 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565269638b70 con 0x5652696debf0 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.401+0000 7f7f0fe00640 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5652696debf0 0x5652696e1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.401+0000 7f7f0fe00640 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5652696debf0 0x5652696e1030 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6559ffc3d9e6a937 server_cookie=11a00a4f7d89fd36 in_seq=0 out_seq=0 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.401+0000 7f7efe000640 10 client.25520 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.402+0000 7f7efe000640 1 -- 172.21.15.22:0/2041147722 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565269638b70 con 0x5652696debf0 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.402+0000 7f7efe000640 10 client.25520 handle_command_reply: tid=0 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.402+0000 7f7f19e44740 1 client.25520 shutdown 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.403+0000 7f7f19e44740 1 -- 172.21.15.22:0/2041147722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 msgr2=0x5652696cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.403+0000 7f7f19e44740 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 0x5652696cbe00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009900 tx=0x7f7ef8009260 comp rx=0 tx=0).stop 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.403+0000 7f7f19e44740 1 -- 172.21.15.22:0/2041147722 shutdown_connections 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.403+0000 7f7f19e44740 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 0x5652696cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.403+0000 7f7f19e44740 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5652696debf0 0x5652696e1030 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.403+0000 7f7f19e44740 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 0x5652696cbe00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.403+0000 7f7f19e44740 1 --2- 172.21.15.22:0/2041147722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.403+0000 7f7f19e44740 1 -- 172.21.15.22:0/2041147722 >> 172.21.15.22:0/2041147722 conn(0x5652696bfab0 msgr2=0x5652696c1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:32.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.403+0000 7f7efe000640 0 client.25520 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:32.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.403+0000 7f7f19e44740 1 -- 172.21.15.22:0/2041147722 shutdown_connections 2024-04-20T21:35:32.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.403+0000 7f7f19e44740 1 -- 172.21.15.22:0/2041147722 wait complete. 2024-04-20T21:35:32.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.403+0000 7f7f19e44740 20 client.25520 trim_cache size 0 max 16384 2024-04-20T21:35:32.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.426+0000 7f7f19e44740 1 Processor -- start 2024-04-20T21:35:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.426+0000 7f7f19e44740 1 -- start start 2024-04-20T21:35:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f19e44740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5380 0x5652696da530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f19e44740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 0x5652696cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f19e44740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696ce880 0x5652696e6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f19e44740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f101adde0 con 0x5652696c5ca0 2024-04-20T21:35:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f19e44740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f101b0660 con 0x5652696c5380 2024-04-20T21:35:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f19e44740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f101ae450 con 0x5652696ce880 2024-04-20T21:35:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f0fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5380 0x5652696da530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f15200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696ce880 0x5652696e6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f0fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5380 0x5652696da530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38904/0 (socket says 172.21.15.22:38904) 2024-04-20T21:35:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f15200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696ce880 0x5652696e6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:57380/0 (socket says 172.21.15.22:57380) 2024-04-20T21:35:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f0fe00640 1 -- 172.21.15.22:0/3330733405 learned_addr learned my addr 172.21.15.22:0/3330733405 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f07e00640 1 --2- 172.21.15.22:0/3330733405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 0x5652696cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f15200640 1 -- 172.21.15.22:0/3330733405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5380 msgr2=0x5652696da530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f15200640 1 --2- 172.21.15.22:0/3330733405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5380 0x5652696da530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f15200640 1 -- 172.21.15.22:0/3330733405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 msgr2=0x5652696cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f15200640 1 --2- 172.21.15.22:0/3330733405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 0x5652696cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f15200640 1 -- 172.21.15.22:0/3330733405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652696b9f20 con 0x5652696ce880 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f0fe00640 1 --2- 172.21.15.22:0/3330733405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5380 0x5652696da530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.427+0000 7f7f07e00640 1 --2- 172.21.15.22:0/3330733405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 0x5652696cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.428+0000 7f7f15200640 1 --2- 172.21.15.22:0/3330733405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696ce880 0x5652696e6900 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f080cdb40 tx=0x7f7f0809c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523cf976f9495c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.428+0000 7f7f04c00640 1 -- 172.21.15.22:0/3330733405 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08020070 con 0x5652696ce880 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.428+0000 7f7f04c00640 1 -- 172.21.15.22:0/3330733405 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f080a5000 con 0x5652696ce880 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.428+0000 7f7f04c00640 1 -- 172.21.15.22:0/3330733405 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0809b890 con 0x5652696ce880 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.429+0000 7f7f19e44740 1 -- 172.21.15.22:0/3330733405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696ce880 msgr2=0x5652696e6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.429+0000 7f7f19e44740 1 --2- 172.21.15.22:0/3330733405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696ce880 0x5652696e6900 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f080cdb40 tx=0x7f7f0809c040 comp rx=0 tx=0).stop 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.429+0000 7f7f19e44740 1 -- 172.21.15.22:0/3330733405 shutdown_connections 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.429+0000 7f7f19e44740 1 --2- 172.21.15.22:0/3330733405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696ce880 0x5652696e6900 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.429+0000 7f7f19e44740 1 --2- 172.21.15.22:0/3330733405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696c5ca0 0x5652696cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.429+0000 7f7f19e44740 1 --2- 172.21.15.22:0/3330733405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5380 0x5652696da530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.429+0000 7f7f19e44740 1 -- 172.21.15.22:0/3330733405 >> 172.21.15.22:0/3330733405 conn(0x5652696bfab0 msgr2=0x5652696cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.429+0000 7f7f19e44740 1 -- 172.21.15.22:0/3330733405 shutdown_connections 2024-04-20T21:35:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.429+0000 7f7f19e44740 1 -- 172.21.15.22:0/3330733405 wait complete. 2024-04-20T21:35:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.429+0000 7f7f19e44740 1 -- start start 2024-04-20T21:35:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.430+0000 7f7f19e44740 1 Processor -- start 2024-04-20T21:35:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.430+0000 7f7f19e44740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.430+0000 7f7f19e44740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 0x5652696d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.430+0000 7f7f19e44740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 0x56526973d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.430+0000 7f7f19e44740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f101aeb20 con 0x5652696ce880 2024-04-20T21:35:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.430+0000 7f7f19e44740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f101aff90 con 0x5652696c5ca0 2024-04-20T21:35:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.430+0000 7f7f19e44740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f101af1f0 con 0x5652696c5380 2024-04-20T21:35:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.430+0000 7f7f19e44740 10 client.0 _ll_register_callbacks cb 0x7f7ef802cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.430+0000 7f7f07e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 0x5652696d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.430+0000 7f7f15200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 0x56526973d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7f15200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 0x56526973d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57262/0 (socket says 172.21.15.22:57262) 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7f15200640 1 -- 172.21.15.22:0/2832802702 learned_addr learned my addr 172.21.15.22:0/2832802702 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7f0fe00640 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7f07e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 0x5652696d21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38912/0 (socket says 172.21.15.22:38912) 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7f15200640 1 -- 172.21.15.22:0/2832802702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 msgr2=0x5652696d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7f15200640 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7f15200640 1 -- 172.21.15.22:0/2832802702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 msgr2=0x5652696d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7f15200640 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 0x5652696d21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7f15200640 1 -- 172.21.15.22:0/2832802702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56526973d770 con 0x5652696ce880 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7f07e00640 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 0x5652696d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7f15200640 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 0x56526973d130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7f080ce140 tx=0x7f7f0800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ebae80126f59d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7efe000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.431+0000 7f7efe000640 1 -- 172.21.15.22:0/2832802702 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080b9020 con 0x5652696ce880 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7efe000640 1 -- 172.21.15.22:0/2832802702 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7f0809b890 con 0x5652696ce880 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7f19e44740 1 -- 172.21.15.22:0/2832802702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f080a5000 con 0x5652696ce880 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7efe000640 1 -- 172.21.15.22:0/2832802702 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7f08084050 con 0x5652696ce880 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7f19e44740 10 client.15414 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7f19e44740 1 -- 172.21.15.22:0/2832802702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x565269744b30 con 0x5652696ce880 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7efe000640 1 -- 172.21.15.22:0/2832802702 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f080a5000 con 0x5652696ce880 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7efe000640 1 -- 172.21.15.22:0/2832802702 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f7f0809b300 con 0x5652696ce880 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7f19e44740 10 client.15414 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7f19e44740 10 client.15414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7f19e44740 20 client.15414 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7f19e44740 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x565269745240 0x565269747630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7f19e44740 4 client.15414 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:35:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.432+0000 7f7f19e44740 1 -- 172.21.15.22:0/2832802702 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652696329e0 con 0x565269745240 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.433+0000 7f7f0fe00640 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x565269745240 0x565269747630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.433+0000 7f7f0fe00640 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x565269745240 0x565269747630 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e4e50f0560fde5a server_cookie=5d584aa029b71a3e in_seq=0 out_seq=0 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.433+0000 7f7efe000640 10 client.15414 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.433+0000 7f7efe000640 1 -- 172.21.15.22:0/2832802702 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5652696329e0 con 0x565269745240 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.433+0000 7f7efe000640 10 client.15414 handle_command_reply: tid=0 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.433+0000 7f7f19e44740 1 client.15414 shutdown 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.434+0000 7f7f19e44740 1 -- 172.21.15.22:0/2832802702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 msgr2=0x56526973d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.434+0000 7f7f19e44740 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 0x56526973d130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7f080ce140 tx=0x7f7f0800bf10 comp rx=0 tx=0).stop 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.434+0000 7f7f19e44740 1 -- 172.21.15.22:0/2832802702 shutdown_connections 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.434+0000 7f7f19e44740 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652696ce880 0x56526973d130 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.434+0000 7f7f19e44740 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x565269745240 0x565269747630 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.434+0000 7f7f19e44740 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652696c5ca0 0x5652696d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.434+0000 7f7f19e44740 1 --2- 172.21.15.22:0/2832802702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5652696c5380 0x5652696d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.434+0000 7f7efe000640 0 client.15414 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.434+0000 7f7f19e44740 1 -- 172.21.15.22:0/2832802702 >> 172.21.15.22:0/2832802702 conn(0x5652696bfab0 msgr2=0x5652696cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:32.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.434+0000 7f7f19e44740 1 -- 172.21.15.22:0/2832802702 shutdown_connections 2024-04-20T21:35:32.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.434+0000 7f7f19e44740 1 -- 172.21.15.22:0/2832802702 wait complete. 2024-04-20T21:35:32.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.434+0000 7f7f19e44740 20 client.15414 trim_cache size 0 max 16384 2024-04-20T21:35:32.437 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:35:32.437 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:35:32.437 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:35:32.437 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:35:32.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.435+0000 7f7f15c00640 1 -- 172.21.15.22:0/2321948279 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7ef40a3da0 msgr2=0x7f7ef40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.435+0000 7f7f15c00640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7ef40a3da0 0x7f7ef40a6260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ef802c250 tx=0x7f7ef8000bc0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.435+0000 7f7f15c00640 1 -- 172.21.15.22:0/2321948279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1019a7d0 msgr2=0x7f7f1019ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.435+0000 7f7f15c00640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1019a7d0 0x7f7f1019ac80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009fd0 tx=0x7f7f0800b260 comp rx=0 tx=0).stop 2024-04-20T21:35:32.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.435+0000 7f7f15c00640 1 -- 172.21.15.22:0/2321948279 shutdown_connections 2024-04-20T21:35:32.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.435+0000 7f7f15c00640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7ef40a3da0 0x7f7ef40a6260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.435+0000 7f7f15c00640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1019a7d0 0x7f7f1019ac80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.435+0000 7f7f15c00640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f10109350 0x7f7f101a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.435+0000 7f7f15c00640 1 --2- 172.21.15.22:0/2321948279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f10104280 0x7f7f101a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.435+0000 7f7f15c00640 1 -- 172.21.15.22:0/2321948279 >> 172.21.15.22:0/2321948279 conn(0x7f7f100ffaa0 msgr2=0x7f7f10100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:32.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.435+0000 7f7f15c00640 1 -- 172.21.15.22:0/2321948279 shutdown_connections 2024-04-20T21:35:32.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.435+0000 7f7f15c00640 1 -- 172.21.15.22:0/2321948279 wait complete. 2024-04-20T21:35:32.452 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:35:32.452 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:32.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:32 smithi022 ceph-mon[16843]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:32.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.682+0000 7f40d1a00640 1 -- 172.21.15.22:0/44783460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc1070c0 msgr2=0x7f40cc10ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.682+0000 7f40d1a00640 1 --2- 172.21.15.22:0/44783460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc1070c0 0x7f40cc10ec90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009ca0 tx=0x7f40c402c100 comp rx=0 tx=0).stop 2024-04-20T21:35:32.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.682+0000 7f40d1a00640 1 -- 172.21.15.22:0/44783460 shutdown_connections 2024-04-20T21:35:32.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.682+0000 7f40d1a00640 1 --2- 172.21.15.22:0/44783460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc1070c0 0x7f40cc10ec90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.682+0000 7f40d1a00640 1 --2- 172.21.15.22:0/44783460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40cc1029a0 0x7f40cc1069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.682+0000 7f40d1a00640 1 --2- 172.21.15.22:0/44783460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40cc101ff0 0x7f40cc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.682+0000 7f40d1a00640 1 -- 172.21.15.22:0/44783460 >> 172.21.15.22:0/44783460 conn(0x7f40cc0fd890 msgr2=0x7f40cc0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:32.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.682+0000 7f40d1a00640 1 -- 172.21.15.22:0/44783460 shutdown_connections 2024-04-20T21:35:32.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.682+0000 7f40d1a00640 1 -- 172.21.15.22:0/44783460 wait complete. 2024-04-20T21:35:32.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.683+0000 7f40d1a00640 1 Processor -- start 2024-04-20T21:35:32.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.684+0000 7f40d1a00640 1 -- start start 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.684+0000 7f40d1a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc101ff0 0x7f40cc198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.684+0000 7f40d1a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40cc1070c0 0x7f40cc198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.684+0000 7f40d1a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40cc19cf20 0x7f40cc19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.684+0000 7f40d1a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc112180 con 0x7f40cc19cf20 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.684+0000 7f40d1a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc112000 con 0x7f40cc101ff0 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.684+0000 7f40d1a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc112300 con 0x7f40cc1070c0 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc101ff0 0x7f40cc198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc101ff0 0x7f40cc198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38922/0 (socket says 172.21.15.22:38922) 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40cbe00640 1 -- 172.21.15.22:0/2456364541 learned_addr learned my addr 172.21.15.22:0/2456364541 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40c3400640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40cc1070c0 0x7f40cc198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40d1000640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40cc19cf20 0x7f40cc19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40cbe00640 1 -- 172.21.15.22:0/2456364541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40cc1070c0 msgr2=0x7f40cc198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40cbe00640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40cc1070c0 0x7f40cc198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40cbe00640 1 -- 172.21.15.22:0/2456364541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40cc19cf20 msgr2=0x7f40cc19d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40cbe00640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40cc19cf20 0x7f40cc19d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40cbe00640 1 -- 172.21.15.22:0/2456364541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc19da10 con 0x7f40cc101ff0 2024-04-20T21:35:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40d1000640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40cc19cf20 0x7f40cc19d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40c3400640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40cc1070c0 0x7f40cc198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.685+0000 7f40cbe00640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc101ff0 0x7f40cc198660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00ea60 tx=0x7f40bc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.686+0000 7f40c9600640 1 -- 172.21.15.22:0/2456364541 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00cde0 con 0x7f40cc101ff0 2024-04-20T21:35:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.686+0000 7f40c9600640 1 -- 172.21.15.22:0/2456364541 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40bc0035f0 con 0x7f40cc101ff0 2024-04-20T21:35:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.686+0000 7f40c9600640 1 -- 172.21.15.22:0/2456364541 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc010670 con 0x7f40cc101ff0 2024-04-20T21:35:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.686+0000 7f40d1a00640 1 -- 172.21.15.22:0/2456364541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc19dd00 con 0x7f40cc101ff0 2024-04-20T21:35:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.686+0000 7f40d1a00640 1 -- 172.21.15.22:0/2456364541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc1a54a0 con 0x7f40cc101ff0 2024-04-20T21:35:32.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.687+0000 7f40d1a00640 1 -- 172.21.15.22:0/2456364541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40cc068ca0 con 0x7f40cc101ff0 2024-04-20T21:35:32.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.689+0000 7f40c9600640 1 -- 172.21.15.22:0/2456364541 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f40bc0037a0 con 0x7f40cc101ff0 2024-04-20T21:35:32.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.691+0000 7f40c9600640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f40b00a3da0 0x7f40b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:32.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.691+0000 7f40c9600640 1 -- 172.21.15.22:0/2456364541 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f40bc0d0b20 con 0x7f40cc101ff0 2024-04-20T21:35:32.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.691+0000 7f40c3400640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f40b00a3da0 0x7f40b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:32.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.692+0000 7f40c3400640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f40b00a3da0 0x7f40b00a6260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f40cc108670 tx=0x7f40b4005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:32.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.696+0000 7f40c9600640 1 -- 172.21.15.22:0/2456364541 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f40bc09d1a0 con 0x7f40cc101ff0 2024-04-20T21:35:32.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:32 smithi122 ceph-mon[22147]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:32.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:32 smithi033 ceph-mon[21626]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T21:35:32.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.956+0000 7f40d1a00640 1 -- 172.21.15.22:0/2456364541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40cc1999b0 con 0x7f40cc101ff0 2024-04-20T21:35:32.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.957+0000 7f40c9600640 1 -- 172.21.15.22:0/2456364541 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f40bc09caa0 con 0x7f40cc101ff0 2024-04-20T21:35:32.956 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:35:32.956 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:35:32.956 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:35:32.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.960+0000 7f40d1a00640 1 -- 172.21.15.22:0/2456364541 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f40b00a3da0 msgr2=0x7f40b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.960+0000 7f40d1a00640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f40b00a3da0 0x7f40b00a6260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f40cc108670 tx=0x7f40b4005f90 comp rx=0 tx=0).stop 2024-04-20T21:35:32.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.960+0000 7f40d1a00640 1 -- 172.21.15.22:0/2456364541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc101ff0 msgr2=0x7f40cc198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:32.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.960+0000 7f40d1a00640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc101ff0 0x7f40cc198660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00ea60 tx=0x7f40bc00ef20 comp rx=0 tx=0).stop 2024-04-20T21:35:32.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.960+0000 7f40d1a00640 1 -- 172.21.15.22:0/2456364541 shutdown_connections 2024-04-20T21:35:32.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.960+0000 7f40d1a00640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f40b00a3da0 0x7f40b00a6260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.960+0000 7f40d1a00640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40cc19cf20 0x7f40cc19d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.960+0000 7f40d1a00640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40cc1070c0 0x7f40cc198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.960+0000 7f40d1a00640 1 --2- 172.21.15.22:0/2456364541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc101ff0 0x7f40cc198660 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:32.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.960+0000 7f40d1a00640 1 -- 172.21.15.22:0/2456364541 >> 172.21.15.22:0/2456364541 conn(0x7f40cc0fd890 msgr2=0x7f40cc0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:32.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.960+0000 7f40d1a00640 1 -- 172.21.15.22:0/2456364541 shutdown_connections 2024-04-20T21:35:32.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:32.960+0000 7f40d1a00640 1 -- 172.21.15.22:0/2456364541 wait complete. 2024-04-20T21:35:32.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:35:33.161 INFO:tasks.workunit.client.0.smithi022.stdout: 1 2399 27.97 MB/sec warmup 1 sec latency 10.282 ms 2024-04-20T21:35:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.202+0000 7ff2ba400640 1 -- 172.21.15.22:0/585465927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b4112e20 msgr2=0x7ff2b4115210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.202+0000 7ff2ba400640 1 --2- 172.21.15.22:0/585465927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b4112e20 0x7ff2b4115210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009ca0 tx=0x7ff2ac02c190 comp rx=0 tx=0).stop 2024-04-20T21:35:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.202+0000 7ff2ba400640 1 -- 172.21.15.22:0/585465927 shutdown_connections 2024-04-20T21:35:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.202+0000 7ff2ba400640 1 --2- 172.21.15.22:0/585465927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b4112e20 0x7ff2b4115210 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.202+0000 7ff2ba400640 1 --2- 172.21.15.22:0/585465927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b4074c20 0x7ff2b41128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.202+0000 7ff2ba400640 1 --2- 172.21.15.22:0/585465927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2b4074300 0x7ff2b40746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.202+0000 7ff2ba400640 1 -- 172.21.15.22:0/585465927 >> 172.21.15.22:0/585465927 conn(0x7ff2b40fd900 msgr2=0x7ff2b40ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.202+0000 7ff2ba400640 1 -- 172.21.15.22:0/585465927 shutdown_connections 2024-04-20T21:35:33.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.203+0000 7ff2ba400640 1 -- 172.21.15.22:0/585465927 wait complete. 2024-04-20T21:35:33.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.204+0000 7ff2ba400640 1 Processor -- start 2024-04-20T21:35:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.204+0000 7ff2ba400640 1 -- start start 2024-04-20T21:35:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.204+0000 7ff2ba400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b4074300 0x7ff2b41a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.204+0000 7ff2ba400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b4074c20 0x7ff2b41a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.204+0000 7ff2ba400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2b4112e20 0x7ff2b419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.204+0000 7ff2ba400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41187c0 con 0x7ff2b4112e20 2024-04-20T21:35:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.204+0000 7ff2ba400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4118640 con 0x7ff2b4074c20 2024-04-20T21:35:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.204+0000 7ff2ba400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4118940 con 0x7ff2b4074300 2024-04-20T21:35:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b9000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b4074300 0x7ff2b41a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b9000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b4074300 0x7ff2b41a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:57434/0 (socket says 172.21.15.22:57434) 2024-04-20T21:35:33.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b9000640 1 -- 172.21.15.22:0/4066986488 learned_addr learned my addr 172.21.15.22:0/4066986488 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:33.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b3e00640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b4074c20 0x7ff2b41a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2b4112e20 0x7ff2b419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b9000640 1 -- 172.21.15.22:0/4066986488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b4074c20 msgr2=0x7ff2b41a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b9000640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b4074c20 0x7ff2b41a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b9000640 1 -- 172.21.15.22:0/4066986488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2b4112e20 msgr2=0x7ff2b419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b9000640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2b4112e20 0x7ff2b419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b9000640 1 -- 172.21.15.22:0/4066986488 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b419f280 con 0x7ff2b4074300 2024-04-20T21:35:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b3e00640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b4074c20 0x7ff2b41a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2b4112e20 0x7ff2b419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.205+0000 7ff2b9000640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b4074300 0x7ff2b41a4950 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800e9f0 tx=0x7ff2a800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.206+0000 7ff2b1600640 1 -- 172.21.15.22:0/4066986488 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a800cd80 con 0x7ff2b4074300 2024-04-20T21:35:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.206+0000 7ff2b1600640 1 -- 172.21.15.22:0/4066986488 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a8010910 con 0x7ff2b4074300 2024-04-20T21:35:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.206+0000 7ff2b1600640 1 -- 172.21.15.22:0/4066986488 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8018990 con 0x7ff2b4074300 2024-04-20T21:35:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.206+0000 7ff2ba400640 1 -- 172.21.15.22:0/4066986488 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b419f570 con 0x7ff2b4074300 2024-04-20T21:35:33.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.206+0000 7ff2ba400640 1 -- 172.21.15.22:0/4066986488 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b41ab810 con 0x7ff2b4074300 2024-04-20T21:35:33.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.209+0000 7ff2be794740 1 Processor -- start 2024-04-20T21:35:33.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.209+0000 7ff2be794740 1 -- start start 2024-04-20T21:35:33.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2be794740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9dbd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2be794740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dbda90 0x561de9dc6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2be794740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dc6670 0x561de9dc8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2be794740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561de9d22050 con 0x561de9dc6670 2024-04-20T21:35:33.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2be794740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561de9cc8490 con 0x561de9dbda90 2024-04-20T21:35:33.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2be794740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561de9d1e010 con 0x561de9dbd170 2024-04-20T21:35:33.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2b1600640 1 -- 172.21.15.22:0/4066986488 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff2a8010ac0 con 0x7ff2b4074300 2024-04-20T21:35:33.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2b9a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9dbd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2b9a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9dbd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:57444/0 (socket says 172.21.15.22:57444) 2024-04-20T21:35:33.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2b9a00640 1 -- 172.21.15.22:0/569694567 learned_addr learned my addr 172.21.15.22:0/569694567 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:33.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2b9000640 1 --2- 172.21.15.22:0/569694567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dbda90 0x561de9dc6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2b9a00640 1 -- 172.21.15.22:0/569694567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dbda90 msgr2=0x561de9dc6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.210+0000 7ff2b3e00640 1 --2- 172.21.15.22:0/569694567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dc6670 0x561de9dc8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.211+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/569694567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dbda90 0x561de9dc6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.211+0000 7ff2b9a00640 1 -- 172.21.15.22:0/569694567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dc6670 msgr2=0x561de9dc8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.211+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/569694567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dc6670 0x561de9dc8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.211+0000 7ff2b9a00640 1 -- 172.21.15.22:0/569694567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de9cd0070 con 0x561de9dbd170 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.211+0000 7ff2b9000640 1 --2- 172.21.15.22:0/569694567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dbda90 0x561de9dc6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.211+0000 7ff2b3e00640 1 --2- 172.21.15.22:0/569694567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dc6670 0x561de9dc8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.211+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/569694567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9dbd550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac004af0 tx=0x7ff2ac009890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a82c67eb560caf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.211+0000 7ff2a6000640 1 -- 172.21.15.22:0/569694567 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac039050 con 0x561de9dbd170 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.211+0000 7ff2a6000640 1 -- 172.21.15.22:0/569694567 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ac045070 con 0x561de9dbd170 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.211+0000 7ff2a6000640 1 -- 172.21.15.22:0/569694567 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac032670 con 0x561de9dbd170 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.212+0000 7ff2b1600640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff29c0a3d20 0x7ff29c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.212+0000 7ff2be794740 1 -- 172.21.15.22:0/569694567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 msgr2=0x561de9dbd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:33.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.212+0000 7ff2be794740 1 --2- 172.21.15.22:0/569694567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9dbd550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac004af0 tx=0x7ff2ac009890 comp rx=0 tx=0).stop 2024-04-20T21:35:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.212+0000 7ff2b1600640 1 -- 172.21.15.22:0/4066986488 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7ff2a80d0c50 con 0x7ff2b4074300 2024-04-20T21:35:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.212+0000 7ff2be794740 1 -- 172.21.15.22:0/569694567 shutdown_connections 2024-04-20T21:35:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.212+0000 7ff2be794740 1 --2- 172.21.15.22:0/569694567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dc6670 0x561de9dc8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.212+0000 7ff2be794740 1 --2- 172.21.15.22:0/569694567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dbda90 0x561de9dc6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.212+0000 7ff2be794740 1 --2- 172.21.15.22:0/569694567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9dbd550 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.212+0000 7ff2be794740 1 -- 172.21.15.22:0/569694567 >> 172.21.15.22:0/569694567 conn(0x561de9db78a0 msgr2=0x561de9db7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.213+0000 7ff2be794740 1 -- 172.21.15.22:0/569694567 shutdown_connections 2024-04-20T21:35:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.213+0000 7ff2b9000640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff29c0a3d20 0x7ff29c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.213+0000 7ff2be794740 1 -- 172.21.15.22:0/569694567 wait complete. 2024-04-20T21:35:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.213+0000 7ff2be794740 1 -- start start 2024-04-20T21:35:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.213+0000 7ff2b9000640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff29c0a3d20 0x7ff29c0a61e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x561de9dbcc40 tx=0x7ff2a8000bc0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:33.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.213+0000 7ff2be794740 1 Processor -- start 2024-04-20T21:35:33.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.214+0000 7ff2be794740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dbd170 0x561de9dc3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.214+0000 7ff2be794740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbda90 0x561de9dc3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.214+0000 7ff2be794740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dc6670 0x561de9dc40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.214+0000 7ff2be794740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561de9dc9400 con 0x561de9dbd170 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.214+0000 7ff2be794740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561de9cdb860 con 0x561de9dc6670 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.214+0000 7ff2be794740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561de9dc9580 con 0x561de9dbda90 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.214+0000 7ff2be794740 10 client.0 _ll_register_callbacks cb 0x561de9ca9fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.214+0000 7ff2b3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbda90 0x561de9dc3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.214+0000 7ff2b9a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dbd170 0x561de9dc3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.214+0000 7ff2b3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbda90 0x561de9dc3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:57448/0 (socket says 172.21.15.22:57448) 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.214+0000 7ff2b3e00640 1 -- 172.21.15.22:0/2780436682 learned_addr learned my addr 172.21.15.22:0/2780436682 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.214+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dc6670 0x561de9dc40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.215+0000 7ff2b3e00640 1 -- 172.21.15.22:0/2780436682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dc6670 msgr2=0x561de9dc40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:33.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.215+0000 7ff2b3e00640 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dc6670 0x561de9dc40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.215+0000 7ff2b3e00640 1 -- 172.21.15.22:0/2780436682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dbd170 msgr2=0x561de9dc3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.215+0000 7ff2b3e00640 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dbd170 0x561de9dc3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.215+0000 7ff2b3e00640 1 -- 172.21.15.22:0/2780436682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de9cdf950 con 0x561de9dbda90 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.215+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dc6670 0x561de9dc40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.215+0000 7ff2b3e00640 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbda90 0x561de9dc3bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00139f0 tx=0x7ff2a0013eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a41a3ef6ddeb761 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.215+0000 7ff29aa00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.215+0000 7ff29aa00640 1 -- 172.21.15.22:0/2780436682 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0020060 con 0x561de9dbda90 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.215+0000 7ff29aa00640 1 -- 172.21.15.22:0/2780436682 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7ff2a0011cf0 con 0x561de9dbda90 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.215+0000 7ff2be794740 1 -- 172.21.15.22:0/2780436682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2ac045070 con 0x561de9dbda90 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.216+0000 7ff29aa00640 1 -- 172.21.15.22:0/2780436682 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff2a0024430 con 0x561de9dbda90 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.216+0000 7ff2be794740 10 client.25576 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.216+0000 7ff2be794740 1 -- 172.21.15.22:0/2780436682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561de9cad820 con 0x561de9dbda90 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.216+0000 7ff29aa00640 1 -- 172.21.15.22:0/2780436682 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ac045070 con 0x561de9dbda90 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.216+0000 7ff29aa00640 1 -- 172.21.15.22:0/2780436682 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7ff2a000f580 con 0x561de9dbda90 2024-04-20T21:35:33.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.216+0000 7ff2be794740 10 client.25576 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.216+0000 7ff2be794740 10 client.25576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.216+0000 7ff2be794740 20 client.25576 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.216+0000 7ff2be794740 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561de9dd6610 0x561de9dd8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.216+0000 7ff2be794740 4 client.25576 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.216+0000 7ff2be794740 1 -- 172.21.15.22:0/2780436682 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561de9d2fa70 con 0x561de9dd6610 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.217+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561de9dd6610 0x561de9dd8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.217+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561de9dd6610 0x561de9dd8a20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=357f88d0b9e13997 server_cookie=5277883f81473c0b in_seq=0 out_seq=0 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.217+0000 7ff29aa00640 10 client.25576 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff29aa00640 1 -- 172.21.15.22:0/2780436682 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561de9d2fa70 con 0x561de9dd6610 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff29aa00640 10 client.25576 handle_command_reply: tid=0 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff2be794740 1 client.25576 shutdown 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff2be794740 1 -- 172.21.15.22:0/2780436682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbda90 msgr2=0x561de9dc3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff2be794740 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbda90 0x561de9dc3bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00139f0 tx=0x7ff2a0013eb0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff2be794740 1 -- 172.21.15.22:0/2780436682 shutdown_connections 2024-04-20T21:35:33.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff2be794740 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561de9dd6610 0x561de9dd8a20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff2be794740 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dc6670 0x561de9dc40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff2be794740 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbda90 0x561de9dc3bb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff2be794740 1 --2- 172.21.15.22:0/2780436682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dbd170 0x561de9dc3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff29aa00640 0 client.25576 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:33.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff2be794740 1 -- 172.21.15.22:0/2780436682 >> 172.21.15.22:0/2780436682 conn(0x561de9db78a0 msgr2=0x561de9db95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:33.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff2be794740 1 -- 172.21.15.22:0/2780436682 shutdown_connections 2024-04-20T21:35:33.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.218+0000 7ff2be794740 1 -- 172.21.15.22:0/2780436682 wait complete. 2024-04-20T21:35:33.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.219+0000 7ff2be794740 20 client.25576 trim_cache size 0 max 16384 2024-04-20T21:35:33.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.234+0000 7ff2be794740 1 Processor -- start 2024-04-20T21:35:33.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.234+0000 7ff2be794740 1 -- start start 2024-04-20T21:35:33.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.234+0000 7ff2be794740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9db78a0 0x561de9dd22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.234+0000 7ff2be794740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9db93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.234+0000 7ff2be794740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dbda90 0x561de9dde330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.234+0000 7ff2be794740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41b2110 con 0x561de9dbda90 2024-04-20T21:35:33.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2be794740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41b4990 con 0x561de9db78a0 2024-04-20T21:35:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2be794740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41b2780 con 0x561de9dbd170 2024-04-20T21:35:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2b9a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9db78a0 0x561de9dd22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2b9a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9db78a0 0x561de9dd22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38966/0 (socket says 172.21.15.22:38966) 2024-04-20T21:35:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2b9a00640 1 -- 172.21.15.22:0/2757491177 learned_addr learned my addr 172.21.15.22:0/2757491177 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2b3e00640 1 --2- 172.21.15.22:0/2757491177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9db93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/2757491177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dbda90 0x561de9dde330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2b9a00640 1 -- 172.21.15.22:0/2757491177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 msgr2=0x561de9db93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/2757491177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9db93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2b9a00640 1 -- 172.21.15.22:0/2757491177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dbda90 msgr2=0x561de9dde330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/2757491177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dbda90 0x561de9dde330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2b9a00640 1 -- 172.21.15.22:0/2757491177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de9a343f0 con 0x561de9db78a0 2024-04-20T21:35:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.235+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/2757491177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9db78a0 0x561de9dd22f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0049d0 tx=0x7ff2ac0045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dec87b627d1a86fb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.236+0000 7ff2a5600640 1 -- 172.21.15.22:0/2757491177 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac0328f0 con 0x561de9db78a0 2024-04-20T21:35:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.236+0000 7ff2a5600640 1 -- 172.21.15.22:0/2757491177 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ac045070 con 0x561de9db78a0 2024-04-20T21:35:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.236+0000 7ff2a5600640 1 -- 172.21.15.22:0/2757491177 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac004180 con 0x561de9db78a0 2024-04-20T21:35:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.236+0000 7ff2be794740 1 -- 172.21.15.22:0/2757491177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9db78a0 msgr2=0x561de9dd22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.236+0000 7ff2be794740 1 --2- 172.21.15.22:0/2757491177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9db78a0 0x561de9dd22f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0049d0 tx=0x7ff2ac0045b0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.236+0000 7ff2be794740 1 -- 172.21.15.22:0/2757491177 shutdown_connections 2024-04-20T21:35:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.236+0000 7ff2be794740 1 --2- 172.21.15.22:0/2757491177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9dbda90 0x561de9dde330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.236+0000 7ff2be794740 1 --2- 172.21.15.22:0/2757491177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9db93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.236+0000 7ff2be794740 1 --2- 172.21.15.22:0/2757491177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9db78a0 0x561de9dd22f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.236+0000 7ff2be794740 1 -- 172.21.15.22:0/2757491177 >> 172.21.15.22:0/2757491177 conn(0x561de9db2d30 msgr2=0x561de9db8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:33.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.237+0000 7ff2be794740 1 -- 172.21.15.22:0/2757491177 shutdown_connections 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.237+0000 7ff2be794740 1 -- 172.21.15.22:0/2757491177 wait complete. 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.237+0000 7ff2be794740 1 -- start start 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.237+0000 7ff2be794740 1 Processor -- start 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2be794740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9db78a0 0x561de9e35eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2be794740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9e363f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2be794740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dbda90 0x561de9dcaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2be794740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41b2e50 con 0x561de9db78a0 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2be794740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41b42c0 con 0x561de9dbda90 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2be794740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41b3520 con 0x561de9dbd170 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2be794740 10 client.0 _ll_register_callbacks cb 0x7ff2a0013630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2b3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9e363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2b9a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9db78a0 0x561de9e35eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2b3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9e363f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:57464/0 (socket says 172.21.15.22:57464) 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2b3e00640 1 -- 172.21.15.22:0/3727678903 learned_addr learned my addr 172.21.15.22:0/3727678903 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dbda90 0x561de9dcaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2b3e00640 1 -- 172.21.15.22:0/3727678903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dbda90 msgr2=0x561de9dcaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2b3e00640 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dbda90 0x561de9dcaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2b3e00640 1 -- 172.21.15.22:0/3727678903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9db78a0 msgr2=0x561de9e35eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2b3e00640 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9db78a0 0x561de9e35eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2b3e00640 1 -- 172.21.15.22:0/3727678903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de9afe7d0 con 0x561de9dbd170 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.238+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dbda90 0x561de9dcaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.239+0000 7ff2b3e00640 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9e363f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000a010 tx=0x7ff2a000fec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21eded840f51da42 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.239+0000 7ff29aa00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.239+0000 7ff29aa00640 1 -- 172.21.15.22:0/3727678903 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0005040 con 0x561de9dbd170 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.239+0000 7ff29aa00640 1 -- 172.21.15.22:0/3727678903 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7ff2a00051e0 con 0x561de9dbd170 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.239+0000 7ff2be794740 1 -- 172.21.15.22:0/3727678903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2ac045070 con 0x561de9dbd170 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.239+0000 7ff29aa00640 1 -- 172.21.15.22:0/3727678903 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff2a0005400 con 0x561de9dbd170 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.239+0000 7ff2be794740 10 client.25582 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.239+0000 7ff2be794740 1 -- 172.21.15.22:0/3727678903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561de9e3d490 con 0x561de9dbd170 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.239+0000 7ff29aa00640 1 -- 172.21.15.22:0/3727678903 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ac045070 con 0x561de9dbd170 2024-04-20T21:35:33.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.240+0000 7ff29aa00640 1 -- 172.21.15.22:0/3727678903 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7ff2a0024660 con 0x561de9dbd170 2024-04-20T21:35:33.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.240+0000 7ff2be794740 10 client.25582 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:33.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.240+0000 7ff2be794740 10 client.25582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:33.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.240+0000 7ff2be794740 20 client.25582 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:33.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.240+0000 7ff2be794740 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561de9dc6670 0x561de9e3fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:33.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.240+0000 7ff2be794740 4 client.25582 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:35:33.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.240+0000 7ff2be794740 1 -- 172.21.15.22:0/3727678903 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561de9d298e0 con 0x561de9dc6670 2024-04-20T21:35:33.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.240+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561de9dc6670 0x561de9e3fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.240+0000 7ff2b9a00640 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561de9dc6670 0x561de9e3fc10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ceb5508e88ae0fa server_cookie=2c00fdf73a3f7ed5 in_seq=0 out_seq=0 2024-04-20T21:35:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.240+0000 7ff29aa00640 10 client.25582 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff29aa00640 1 -- 172.21.15.22:0/3727678903 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561de9d298e0 con 0x561de9dc6670 2024-04-20T21:35:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff29aa00640 10 client.25582 handle_command_reply: tid=0 2024-04-20T21:35:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff2be794740 1 client.25582 shutdown 2024-04-20T21:35:33.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff2be794740 1 -- 172.21.15.22:0/3727678903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 msgr2=0x561de9e363f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff2be794740 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9e363f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000a010 tx=0x7ff2a000fec0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff2be794740 1 -- 172.21.15.22:0/3727678903 shutdown_connections 2024-04-20T21:35:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff2be794740 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561de9dc6670 0x561de9e3fc10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff2be794740 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561de9dbda90 0x561de9dcaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff2be794740 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561de9dbd170 0x561de9e363f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff29aa00640 0 client.25582 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff2be794740 1 --2- 172.21.15.22:0/3727678903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561de9db78a0 0x561de9e35eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff2be794740 1 -- 172.21.15.22:0/3727678903 >> 172.21.15.22:0/3727678903 conn(0x561de9db2d30 msgr2=0x561de9db8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff2be794740 1 -- 172.21.15.22:0/3727678903 shutdown_connections 2024-04-20T21:35:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff2be794740 1 -- 172.21.15.22:0/3727678903 wait complete. 2024-04-20T21:35:33.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.241+0000 7ff2be794740 20 client.25582 trim_cache size 0 max 16384 2024-04-20T21:35:33.244 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:35:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.242+0000 7ff2ba400640 1 -- 172.21.15.22:0/4066986488 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff29c0a3d20 msgr2=0x7ff29c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.242+0000 7ff2ba400640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff29c0a3d20 0x7ff29c0a61e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x561de9dbcc40 tx=0x7ff2a8000bc0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.242+0000 7ff2ba400640 1 -- 172.21.15.22:0/4066986488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b4074300 msgr2=0x7ff2b41a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.242+0000 7ff2ba400640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b4074300 0x7ff2b41a4950 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800e9f0 tx=0x7ff2a800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.242+0000 7ff2ba400640 1 -- 172.21.15.22:0/4066986488 shutdown_connections 2024-04-20T21:35:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.242+0000 7ff2ba400640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff29c0a3d20 0x7ff29c0a61e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.242+0000 7ff2ba400640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2b4112e20 0x7ff2b419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.242+0000 7ff2ba400640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b4074c20 0x7ff2b41a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.242+0000 7ff2ba400640 1 --2- 172.21.15.22:0/4066986488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b4074300 0x7ff2b41a4950 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:33.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.242+0000 7ff2ba400640 1 -- 172.21.15.22:0/4066986488 >> 172.21.15.22:0/4066986488 conn(0x7ff2b40fd900 msgr2=0x7ff2b40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:33.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.242+0000 7ff2ba400640 1 -- 172.21.15.22:0/4066986488 shutdown_connections 2024-04-20T21:35:33.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:33.242+0000 7ff2ba400640 1 -- 172.21.15.22:0/4066986488 wait complete. 2024-04-20T21:35:33.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:33 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2456364541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:35:33.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:33 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2456364541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:35:33.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:33 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2456364541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:35:34.161 INFO:tasks.workunit.client.0.smithi022.stdout: 1 4369 20.29 MB/sec warmup 2 sec latency 11.320 ms 2024-04-20T21:35:34.260 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:35:34.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:35:34.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:35:34.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.488+0000 7f720ae00640 1 -- 172.21.15.22:0/3511802136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7204104dc0 msgr2=0x7f7204108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.488+0000 7f720ae00640 1 --2- 172.21.15.22:0/3511802136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7204104dc0 0x7f7204108e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009900 tx=0x7f71f002bc30 comp rx=0 tx=0).stop 2024-04-20T21:35:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.488+0000 7f720ae00640 1 -- 172.21.15.22:0/3511802136 shutdown_connections 2024-04-20T21:35:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.488+0000 7f720ae00640 1 --2- 172.21.15.22:0/3511802136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72041094e0 0x7f72041110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.488+0000 7f720ae00640 1 --2- 172.21.15.22:0/3511802136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7204104dc0 0x7f7204108e00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.488+0000 7f720ae00640 1 --2- 172.21.15.22:0/3511802136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7204104410 0x7f72041047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.488+0000 7f720ae00640 1 -- 172.21.15.22:0/3511802136 >> 172.21.15.22:0/3511802136 conn(0x7f72040ffcf0 msgr2=0x7f7204102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.488+0000 7f720ae00640 1 -- 172.21.15.22:0/3511802136 shutdown_connections 2024-04-20T21:35:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.488+0000 7f720ae00640 1 -- 172.21.15.22:0/3511802136 wait complete. 2024-04-20T21:35:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.489+0000 7f720ae00640 1 Processor -- start 2024-04-20T21:35:34.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.490+0000 7f720ae00640 1 -- start start 2024-04-20T21:35:34.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.490+0000 7f720ae00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7204104410 0x7f72041a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.490+0000 7f720ae00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72041094e0 0x7f72041a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.490+0000 7f720ae00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f720419a960 0x7f720419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.490+0000 7f720ae00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f72041145a0 con 0x7f720419a960 2024-04-20T21:35:34.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.490+0000 7f720ae00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7204114420 con 0x7f72041094e0 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.490+0000 7f720ae00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7204114720 con 0x7f7204104410 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.490+0000 7f7203400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72041094e0 0x7f72041a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.491+0000 7f7203400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72041094e0 0x7f72041a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38990/0 (socket says 172.21.15.22:38990) 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.491+0000 7f7203400640 1 -- 172.21.15.22:0/2032110639 learned_addr learned my addr 172.21.15.22:0/2032110639 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.491+0000 7f7203e00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7204104410 0x7f72041a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.491+0000 7f7208e00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f720419a960 0x7f720419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.491+0000 7f7203400640 1 -- 172.21.15.22:0/2032110639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7204104410 msgr2=0x7f72041a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.491+0000 7f7203400640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7204104410 0x7f72041a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.491+0000 7f7203400640 1 -- 172.21.15.22:0/2032110639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f720419a960 msgr2=0x7f720419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.491+0000 7f7203400640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f720419a960 0x7f720419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.491+0000 7f7203400640 1 -- 172.21.15.22:0/2032110639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720419b480 con 0x7f72041094e0 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.491+0000 7f7203e00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7204104410 0x7f72041a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:34.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.491+0000 7f7208e00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f720419a960 0x7f720419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:34.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.491+0000 7f7203400640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72041094e0 0x7f72041a0ca0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f71f002c240 tx=0x7f71f0004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:34.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.492+0000 7f7200c00640 1 -- 172.21.15.22:0/2032110639 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0043020 con 0x7f72041094e0 2024-04-20T21:35:34.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.492+0000 7f720ae00640 1 -- 172.21.15.22:0/2032110639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720419b710 con 0x7f72041094e0 2024-04-20T21:35:34.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.492+0000 7f7200c00640 1 -- 172.21.15.22:0/2032110639 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f00054e0 con 0x7f72041094e0 2024-04-20T21:35:34.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.492+0000 7f7200c00640 1 -- 172.21.15.22:0/2032110639 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0003bb0 con 0x7f72041094e0 2024-04-20T21:35:34.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.492+0000 7f720ae00640 1 -- 172.21.15.22:0/2032110639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72041a7960 con 0x7f72041094e0 2024-04-20T21:35:34.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.495+0000 7f720dc0b740 1 Processor -- start 2024-04-20T21:35:34.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.495+0000 7f720dc0b740 1 -- start start 2024-04-20T21:35:34.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f720dc0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adaf10 0x557606adb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f720dc0b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adb830 0x557606ae3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f7200c00640 1 -- 172.21.15.22:0/2032110639 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f71f0046020 con 0x7f72041094e0 2024-04-20T21:35:34.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f720dc0b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606ae4410 0x557606ae6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f720dc0b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557606a3fd20 con 0x557606adb830 2024-04-20T21:35:34.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f720dc0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5576069e63e0 con 0x557606adaf10 2024-04-20T21:35:34.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f720dc0b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557606a3bce0 con 0x557606ae4410 2024-04-20T21:35:34.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f7208e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adaf10 0x557606adb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f7208e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adaf10 0x557606adb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38998/0 (socket says 172.21.15.22:38998) 2024-04-20T21:35:34.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f7208e00640 1 -- 172.21.15.22:0/3110836841 learned_addr learned my addr 172.21.15.22:0/3110836841 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:34.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f7203400640 1 --2- 172.21.15.22:0/3110836841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606ae4410 0x557606ae6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f7203e00640 1 --2- 172.21.15.22:0/3110836841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adb830 0x557606ae3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f7208e00640 1 -- 172.21.15.22:0/3110836841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606ae4410 msgr2=0x557606ae6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f7208e00640 1 --2- 172.21.15.22:0/3110836841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606ae4410 0x557606ae6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f7208e00640 1 -- 172.21.15.22:0/3110836841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adb830 msgr2=0x557606ae3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f7208e00640 1 --2- 172.21.15.22:0/3110836841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adb830 0x557606ae3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.496+0000 7f7208e00640 1 -- 172.21.15.22:0/3110836841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557606a05980 con 0x557606adaf10 2024-04-20T21:35:34.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.497+0000 7f7208e00640 1 --2- 172.21.15.22:0/3110836841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adaf10 0x557606adb2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009fd0 tx=0x7f71fc010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a17f784ae96997e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:34.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.497+0000 7f71f9600640 1 -- 172.21.15.22:0/3110836841 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc011700 con 0x557606adaf10 2024-04-20T21:35:34.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.497+0000 7f71f9600640 1 -- 172.21.15.22:0/3110836841 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71fc011d80 con 0x557606adaf10 2024-04-20T21:35:34.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.497+0000 7f71f9600640 1 -- 172.21.15.22:0/3110836841 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc00e3c0 con 0x557606adaf10 2024-04-20T21:35:34.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.497+0000 7f7200c00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f71e80a3da0 0x7f71e80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.498+0000 7f7200c00640 1 -- 172.21.15.22:0/2032110639 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f71f00f2770 con 0x7f72041094e0 2024-04-20T21:35:34.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.498+0000 7f720dc0b740 1 -- 172.21.15.22:0/3110836841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adaf10 msgr2=0x557606adb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.498+0000 7f720dc0b740 1 --2- 172.21.15.22:0/3110836841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adaf10 0x557606adb2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009fd0 tx=0x7f71fc010860 comp rx=0 tx=0).stop 2024-04-20T21:35:34.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.498+0000 7f7203e00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f71e80a3da0 0x7f71e80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.498+0000 7f720dc0b740 1 -- 172.21.15.22:0/3110836841 shutdown_connections 2024-04-20T21:35:34.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.498+0000 7f720dc0b740 1 --2- 172.21.15.22:0/3110836841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606ae4410 0x557606ae6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.498+0000 7f720dc0b740 1 --2- 172.21.15.22:0/3110836841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adb830 0x557606ae3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.498+0000 7f720dc0b740 1 --2- 172.21.15.22:0/3110836841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adaf10 0x557606adb2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.498+0000 7f720dc0b740 1 -- 172.21.15.22:0/3110836841 >> 172.21.15.22:0/3110836841 conn(0x557606ad5640 msgr2=0x557606ad5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:34.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.498+0000 7f720dc0b740 1 -- 172.21.15.22:0/3110836841 shutdown_connections 2024-04-20T21:35:34.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.498+0000 7f720dc0b740 1 -- 172.21.15.22:0/3110836841 wait complete. 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.499+0000 7f720dc0b740 1 -- start start 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.499+0000 7f7203e00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f71e80a3da0 0x7f71e80a6260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x557606ada9e0 tx=0x7f71f4005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.499+0000 7f720dc0b740 1 Processor -- start 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.499+0000 7f720dc0b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adaf10 0x557606ae1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.499+0000 7f720dc0b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adb830 0x557606ae1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.499+0000 7f720dc0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606ae4410 0x557606ae1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.499+0000 7f720dc0b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557606ae7200 con 0x557606adaf10 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.499+0000 7f720dc0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5576069f97a0 con 0x557606ae4410 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.499+0000 7f720dc0b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557606ae7380 con 0x557606adb830 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f720dc0b740 10 client.0 _ll_register_callbacks cb 0x5576069fbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f7208e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606ae4410 0x557606ae1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f7203400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adb830 0x557606ae1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f7203e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adaf10 0x557606ae1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f7208e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606ae4410 0x557606ae1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39010/0 (socket says 172.21.15.22:39010) 2024-04-20T21:35:34.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f7203400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adb830 0x557606ae1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:57512/0 (socket says 172.21.15.22:57512) 2024-04-20T21:35:34.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f7203400640 1 -- 172.21.15.22:0/442176598 learned_addr learned my addr 172.21.15.22:0/442176598 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:34.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f7203400640 1 -- 172.21.15.22:0/442176598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606ae4410 msgr2=0x557606ae1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f7203400640 1 --2- 172.21.15.22:0/442176598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606ae4410 0x557606ae1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f7203400640 1 -- 172.21.15.22:0/442176598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adaf10 msgr2=0x557606ae1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f7203400640 1 --2- 172.21.15.22:0/442176598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adaf10 0x557606ae1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f7203400640 1 -- 172.21.15.22:0/442176598 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55760681a7e0 con 0x557606adb830 2024-04-20T21:35:34.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f7203400640 1 --2- 172.21.15.22:0/442176598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adb830 0x557606ae1950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f71f0000c00 tx=0x7f71f0004ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3553be2744afc1d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.500+0000 7f71ee000640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f71ee000640 1 -- 172.21.15.22:0/442176598 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f00be2e0 con 0x557606adb830 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f71ee000640 1 -- 172.21.15.22:0/442176598 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f71f00bae10 con 0x557606adb830 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f720dc0b740 1 -- 172.21.15.22:0/442176598 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71fc011d80 con 0x557606adb830 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f71ee000640 1 -- 172.21.15.22:0/442176598 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f71f00be100 con 0x557606adb830 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f720dc0b740 10 client.25594 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f720dc0b740 1 -- 172.21.15.22:0/442176598 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55760687af70 con 0x557606adb830 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f71ee000640 1 -- 172.21.15.22:0/442176598 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71fc011d80 con 0x557606adb830 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f71ee000640 1 -- 172.21.15.22:0/442176598 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f71f00a7ce0 con 0x557606adb830 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f720dc0b740 10 client.25594 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f720dc0b740 10 client.25594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f720dc0b740 20 client.25594 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:34.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f720dc0b740 1 --2- 172.21.15.22:0/442176598 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x557606af4810 0x557606af6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f720dc0b740 4 client.25594 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:34.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.501+0000 7f720dc0b740 1 -- 172.21.15.22:0/442176598 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557606a4d740 con 0x557606af4810 2024-04-20T21:35:34.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.502+0000 7f7203e00640 1 --2- 172.21.15.22:0/442176598 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x557606af4810 0x557606af6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.502+0000 7f7203e00640 1 --2- 172.21.15.22:0/442176598 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x557606af4810 0x557606af6c40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9ac5ceedf6db46 server_cookie=61f6de502520dfdb in_seq=0 out_seq=0 2024-04-20T21:35:34.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.502+0000 7f71ee000640 10 client.25594 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:34.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f71ee000640 1 -- 172.21.15.22:0/442176598 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557606a4d740 con 0x557606af4810 2024-04-20T21:35:34.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f71ee000640 10 client.25594 handle_command_reply: tid=0 2024-04-20T21:35:34.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f720dc0b740 1 client.25594 shutdown 2024-04-20T21:35:34.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f720dc0b740 1 -- 172.21.15.22:0/442176598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adb830 msgr2=0x557606ae1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f720dc0b740 1 --2- 172.21.15.22:0/442176598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adb830 0x557606ae1950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f71f0000c00 tx=0x7f71f0004ee0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f720dc0b740 1 -- 172.21.15.22:0/442176598 shutdown_connections 2024-04-20T21:35:34.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f720dc0b740 1 --2- 172.21.15.22:0/442176598 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x557606af4810 0x557606af6c40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f720dc0b740 1 --2- 172.21.15.22:0/442176598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606ae4410 0x557606ae1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f720dc0b740 1 --2- 172.21.15.22:0/442176598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adb830 0x557606ae1950 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f71ee000640 0 client.25594 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:34.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f720dc0b740 1 --2- 172.21.15.22:0/442176598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adaf10 0x557606ae1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f720dc0b740 1 -- 172.21.15.22:0/442176598 >> 172.21.15.22:0/442176598 conn(0x557606ad5640 msgr2=0x557606ad7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:34.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f720dc0b740 1 -- 172.21.15.22:0/442176598 shutdown_connections 2024-04-20T21:35:34.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.504+0000 7f720dc0b740 1 -- 172.21.15.22:0/442176598 wait complete. 2024-04-20T21:35:34.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.505+0000 7f720dc0b740 20 client.25594 trim_cache size 0 max 16384 2024-04-20T21:35:34.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.525+0000 7f720dc0b740 1 Processor -- start 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.525+0000 7f720dc0b740 1 -- start start 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.526+0000 7f720dc0b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adaf10 0x557606ae3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.526+0000 7f720dc0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 0x557606af00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.526+0000 7f720dc0b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606ae4410 0x557606ae03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.526+0000 7f720dc0b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f72041ae020 con 0x557606adaf10 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.526+0000 7f720dc0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72041b08a0 con 0x557606adb830 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.526+0000 7f720dc0b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f72041ae690 con 0x557606ae4410 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.526+0000 7f7203400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 0x557606af00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.526+0000 7f7203e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adaf10 0x557606ae3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.526+0000 7f7208e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606ae4410 0x557606ae03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.526+0000 7f7203400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 0x557606af00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39014/0 (socket says 172.21.15.22:39014) 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.526+0000 7f7203400640 1 -- 172.21.15.22:0/2620830222 learned_addr learned my addr 172.21.15.22:0/2620830222 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.527+0000 7f7203400640 1 -- 172.21.15.22:0/2620830222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606ae4410 msgr2=0x557606ae03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.527+0000 7f7203400640 1 --2- 172.21.15.22:0/2620830222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606ae4410 0x557606ae03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.527+0000 7f7203400640 1 -- 172.21.15.22:0/2620830222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adaf10 msgr2=0x557606ae3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.527+0000 7f7203400640 1 --2- 172.21.15.22:0/2620830222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adaf10 0x557606ae3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.527+0000 7f7203400640 1 -- 172.21.15.22:0/2620830222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557606712170 con 0x557606adb830 2024-04-20T21:35:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.527+0000 7f7203400640 1 --2- 172.21.15.22:0/2620830222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 0x557606af00f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f71f00be4a0 tx=0x7f71f00ba440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb9b31e8856a2a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.527+0000 7f71f8c00640 1 -- 172.21.15.22:0/2620830222 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f00be2e0 con 0x557606adb830 2024-04-20T21:35:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.527+0000 7f71f8c00640 1 -- 172.21.15.22:0/2620830222 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0004c80 con 0x557606adb830 2024-04-20T21:35:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.527+0000 7f71f8c00640 1 -- 172.21.15.22:0/2620830222 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f00c2ba0 con 0x557606adb830 2024-04-20T21:35:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.528+0000 7f720dc0b740 1 -- 172.21.15.22:0/2620830222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 msgr2=0x557606af00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.528+0000 7f720dc0b740 1 --2- 172.21.15.22:0/2620830222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 0x557606af00f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f71f00be4a0 tx=0x7f71f00ba440 comp rx=0 tx=0).stop 2024-04-20T21:35:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.528+0000 7f720dc0b740 1 -- 172.21.15.22:0/2620830222 shutdown_connections 2024-04-20T21:35:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.528+0000 7f720dc0b740 1 --2- 172.21.15.22:0/2620830222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606ae4410 0x557606ae03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.528+0000 7f720dc0b740 1 --2- 172.21.15.22:0/2620830222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 0x557606af00f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.528+0000 7f720dc0b740 1 --2- 172.21.15.22:0/2620830222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606adaf10 0x557606ae3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.528+0000 7f720dc0b740 1 -- 172.21.15.22:0/2620830222 >> 172.21.15.22:0/2620830222 conn(0x557606ad5640 msgr2=0x557606ad7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.528+0000 7f720dc0b740 1 -- 172.21.15.22:0/2620830222 shutdown_connections 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.528+0000 7f720dc0b740 1 -- 172.21.15.22:0/2620830222 wait complete. 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.529+0000 7f720dc0b740 1 -- start start 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.529+0000 7f720dc0b740 1 Processor -- start 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.529+0000 7f720dc0b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adaf10 0x557606aec600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.529+0000 7f720dc0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 0x557606aecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.529+0000 7f720dc0b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606ae4410 0x557606ae9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.529+0000 7f720dc0b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f72041aed60 con 0x557606ae4410 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.529+0000 7f720dc0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72041b01d0 con 0x557606adb830 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.529+0000 7f720dc0b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f72041af430 con 0x557606adaf10 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f720dc0b740 10 client.0 _ll_register_callbacks cb 0x7f71f0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7203e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adaf10 0x557606aec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7203e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adaf10 0x557606aec600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:57538/0 (socket says 172.21.15.22:57538) 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7203e00640 1 -- 172.21.15.22:0/2246213131 learned_addr learned my addr 172.21.15.22:0/2246213131 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7208e00640 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606ae4410 0x557606ae9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7203400640 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 0x557606aecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7203e00640 1 -- 172.21.15.22:0/2246213131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 msgr2=0x557606aecb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7203e00640 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 0x557606aecb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7203e00640 1 -- 172.21.15.22:0/2246213131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606ae4410 msgr2=0x557606ae9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7203e00640 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606ae4410 0x557606ae9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7203e00640 1 -- 172.21.15.22:0/2246213131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557606ae9e40 con 0x557606adaf10 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7208e00640 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606ae4410 0x557606ae9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7203400640 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 0x557606aecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f7203e00640 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adaf10 0x557606aec600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x557606af7480 tx=0x7f71f40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50c0302b303546db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.530+0000 7f71ee000640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f71ee000640 1 -- 172.21.15.22:0/2246213131 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4005020 con 0x557606adaf10 2024-04-20T21:35:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f71ee000640 1 -- 172.21.15.22:0/2246213131 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f71f40051c0 con 0x557606adaf10 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f720dc0b740 1 -- 172.21.15.22:0/2246213131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f0004c80 con 0x557606adaf10 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f71ee000640 1 -- 172.21.15.22:0/2246213131 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f71f403b050 con 0x557606adaf10 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f720dc0b740 10 client.25600 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f720dc0b740 1 -- 172.21.15.22:0/2246213131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557606b5b990 con 0x557606adaf10 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f71ee000640 1 -- 172.21.15.22:0/2246213131 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0004c80 con 0x557606adaf10 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f71ee000640 1 -- 172.21.15.22:0/2246213131 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f71f402b2d0 con 0x557606adaf10 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f720dc0b740 10 client.25600 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f720dc0b740 10 client.25600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f720dc0b740 20 client.25600 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f720dc0b740 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x557606b5c060 0x557606b5e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f720dc0b740 4 client.25600 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.531+0000 7f720dc0b740 1 -- 172.21.15.22:0/2246213131 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557606a475b0 con 0x557606b5c060 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.532+0000 7f7203400640 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x557606b5c060 0x557606b5e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.532+0000 7f7203400640 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x557606b5c060 0x557606b5e480 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df41b634293efe2c server_cookie=e445926d960e1870 in_seq=0 out_seq=0 2024-04-20T21:35:34.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.532+0000 7f71ee000640 10 client.25600 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:34.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.534+0000 7f71ee000640 1 -- 172.21.15.22:0/2246213131 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557606a475b0 con 0x557606b5c060 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.534+0000 7f71ee000640 10 client.25600 handle_command_reply: tid=0 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.534+0000 7f720dc0b740 1 client.25600 shutdown 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.534+0000 7f720dc0b740 1 -- 172.21.15.22:0/2246213131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adaf10 msgr2=0x557606aec600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.534+0000 7f720dc0b740 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adaf10 0x557606aec600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x557606af7480 tx=0x7f71f40046c0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.535+0000 7f720dc0b740 1 -- 172.21.15.22:0/2246213131 shutdown_connections 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.535+0000 7f720dc0b740 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557606ae4410 0x557606ae9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.535+0000 7f720dc0b740 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x557606b5c060 0x557606b5e480 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.535+0000 7f720dc0b740 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557606adb830 0x557606aecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.535+0000 7f720dc0b740 1 --2- 172.21.15.22:0/2246213131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557606adaf10 0x557606aec600 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.535+0000 7f71ee000640 0 client.25600 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.535+0000 7f720dc0b740 1 -- 172.21.15.22:0/2246213131 >> 172.21.15.22:0/2246213131 conn(0x557606ad5640 msgr2=0x557606ae13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.535+0000 7f720dc0b740 1 -- 172.21.15.22:0/2246213131 shutdown_connections 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.535+0000 7f720dc0b740 1 -- 172.21.15.22:0/2246213131 wait complete. 2024-04-20T21:35:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.535+0000 7f720dc0b740 20 client.25600 trim_cache size 0 max 16384 2024-04-20T21:35:34.537 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:35:34.537 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:35:34.537 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "4c4bc6d7-2c00-4860-b765-bb70f4e7e07c", 2024-04-20T21:35:34.537 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:35:34.537 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:35:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.536+0000 7f720ae00640 1 -- 172.21.15.22:0/2032110639 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f71e80a3da0 msgr2=0x7f71e80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.536+0000 7f720ae00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f71e80a3da0 0x7f71e80a6260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x557606ada9e0 tx=0x7f71f4005f90 comp rx=0 tx=0).stop 2024-04-20T21:35:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.536+0000 7f720ae00640 1 -- 172.21.15.22:0/2032110639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72041094e0 msgr2=0x7f72041a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.536+0000 7f720ae00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72041094e0 0x7f72041a0ca0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f71f002c240 tx=0x7f71f0004360 comp rx=0 tx=0).stop 2024-04-20T21:35:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.536+0000 7f720ae00640 1 -- 172.21.15.22:0/2032110639 shutdown_connections 2024-04-20T21:35:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.536+0000 7f720ae00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f71e80a3da0 0x7f71e80a6260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.536+0000 7f720ae00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f720419a960 0x7f720419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.536+0000 7f720ae00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72041094e0 0x7f72041a0ca0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.536+0000 7f720ae00640 1 --2- 172.21.15.22:0/2032110639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7204104410 0x7f72041a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.536+0000 7f720ae00640 1 -- 172.21.15.22:0/2032110639 >> 172.21.15.22:0/2032110639 conn(0x7f72040ffcf0 msgr2=0x7f7204101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:34.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.536+0000 7f720ae00640 1 -- 172.21.15.22:0/2032110639 shutdown_connections 2024-04-20T21:35:34.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.536+0000 7f720ae00640 1 -- 172.21.15.22:0/2032110639 wait complete. 2024-04-20T21:35:34.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:35:34.554 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:35:34.622 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:34 smithi022 ceph-mon[16843]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T21:35:34.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.773+0000 7fd68e800640 1 -- 172.21.15.22:0/1688219333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6881038e0 msgr2=0x7fd68810bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.773+0000 7fd68e800640 1 --2- 172.21.15.22:0/1688219333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6881038e0 0x7fd68810bdc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd674009860 tx=0x7fd67402bc10 comp rx=0 tx=0).stop 2024-04-20T21:35:34.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.773+0000 7fd68e800640 1 -- 172.21.15.22:0/1688219333 shutdown_connections 2024-04-20T21:35:34.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.773+0000 7fd68e800640 1 --2- 172.21.15.22:0/1688219333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd68810c3e0 0x7fd68810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.773+0000 7fd68e800640 1 --2- 172.21.15.22:0/1688219333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6881038e0 0x7fd68810bdc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.773+0000 7fd68e800640 1 --2- 172.21.15.22:0/1688219333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd688102fc0 0x7fd6881033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.773+0000 7fd68e800640 1 -- 172.21.15.22:0/1688219333 >> 172.21.15.22:0/1688219333 conn(0x7fd6880fc7e0 msgr2=0x7fd6880fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:34.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.773+0000 7fd68e800640 1 -- 172.21.15.22:0/1688219333 shutdown_connections 2024-04-20T21:35:34.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.773+0000 7fd68e800640 1 -- 172.21.15.22:0/1688219333 wait complete. 2024-04-20T21:35:34.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.774+0000 7fd68e800640 1 Processor -- start 2024-04-20T21:35:34.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.774+0000 7fd68e800640 1 -- start start 2024-04-20T21:35:34.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd688102fc0 0x7fd6881a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68e800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6881038e0 0x7fd6881a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd68810c3e0 0x7fd68819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68e800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881121a0 con 0x7fd688102fc0 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68e800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd688112020 con 0x7fd6881038e0 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68e800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd688112320 con 0x7fd68810c3e0 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd688102fc0 0x7fd6881a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd688102fc0 0x7fd6881a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57390/0 (socket says 172.21.15.22:57390) 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68d400640 1 -- 172.21.15.22:0/2725849283 learned_addr learned my addr 172.21.15.22:0/2725849283 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68ca00640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6881038e0 0x7fd6881a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68de00640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd68810c3e0 0x7fd68819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68ca00640 1 -- 172.21.15.22:0/2725849283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd68810c3e0 msgr2=0x7fd68819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68ca00640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd68810c3e0 0x7fd68819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68ca00640 1 -- 172.21.15.22:0/2725849283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd688102fc0 msgr2=0x7fd6881a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68ca00640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd688102fc0 0x7fd6881a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68ca00640 1 -- 172.21.15.22:0/2725849283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68819af70 con 0x7fd6881038e0 2024-04-20T21:35:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.775+0000 7fd68d400640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd688102fc0 0x7fd6881a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:34.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.776+0000 7fd68ca00640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6881038e0 0x7fd6881a0b70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd67402c150 tx=0x7fd6740344c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:34.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.776+0000 7fd686000640 1 -- 172.21.15.22:0/2725849283 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674041020 con 0x7fd6881038e0 2024-04-20T21:35:34.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.776+0000 7fd686000640 1 -- 172.21.15.22:0/2725849283 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67402ca90 con 0x7fd6881038e0 2024-04-20T21:35:34.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.776+0000 7fd68e800640 1 -- 172.21.15.22:0/2725849283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68819b200 con 0x7fd6881038e0 2024-04-20T21:35:34.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.776+0000 7fd686000640 1 -- 172.21.15.22:0/2725849283 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674039040 con 0x7fd6881038e0 2024-04-20T21:35:34.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.776+0000 7fd68e800640 1 -- 172.21.15.22:0/2725849283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6881a7480 con 0x7fd6881038e0 2024-04-20T21:35:34.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:34 smithi122 ceph-mon[22147]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T21:35:34.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.780+0000 7fd692bc3740 1 Processor -- start 2024-04-20T21:35:34.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.780+0000 7fd686000640 1 -- 172.21.15.22:0/2725849283 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd674044020 con 0x7fd6881038e0 2024-04-20T21:35:34.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.780+0000 7fd692bc3740 1 -- start start 2024-04-20T21:35:34.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd692bc3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 0x55dc7d4c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd692bc3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd692bc3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c9880 0x55dc7d4cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd692bc3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dc7d426150 con 0x55dc7d4c9880 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd692bc3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dc7d3cc370 con 0x55dc7d4c0380 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd692bc3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dc7d422110 con 0x55dc7d4c0ca0 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd68de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 0x55dc7d4c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd68d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd68de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 0x55dc7d4c0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39034/0 (socket says 172.21.15.22:39034) 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd68de00640 1 -- 172.21.15.22:0/1482398368 learned_addr learned my addr 172.21.15.22:0/1482398368 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd68ca00640 1 --2- 172.21.15.22:0/1482398368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c9880 0x55dc7d4cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd68de00640 1 -- 172.21.15.22:0/1482398368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 msgr2=0x55dc7d4c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd68de00640 1 --2- 172.21.15.22:0/1482398368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd68de00640 1 -- 172.21.15.22:0/1482398368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c9880 msgr2=0x55dc7d4cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd68de00640 1 --2- 172.21.15.22:0/1482398368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c9880 0x55dc7d4cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd68de00640 1 -- 172.21.15.22:0/1482398368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7d2027d0 con 0x55dc7d4c0380 2024-04-20T21:35:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.781+0000 7fd68d400640 1 --2- 172.21.15.22:0/1482398368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:34.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.782+0000 7fd686000640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd6700a3d20 0x7fd6700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.782+0000 7fd68de00640 1 --2- 172.21.15.22:0/1482398368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 0x55dc7d4c0760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd680009f50 tx=0x7fd6800118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c78174d0ea4784b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.782+0000 7fd686000640 1 -- 172.21.15.22:0/2725849283 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd6740f1ce0 con 0x7fd6881038e0 2024-04-20T21:35:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.782+0000 7fd67a600640 1 -- 172.21.15.22:0/1482398368 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680004d20 con 0x55dc7d4c0380 2024-04-20T21:35:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.782+0000 7fd68d400640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd6700a3d20 0x7fd6700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.782+0000 7fd67a600640 1 -- 172.21.15.22:0/1482398368 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd680004050 con 0x55dc7d4c0380 2024-04-20T21:35:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.782+0000 7fd67a600640 1 -- 172.21.15.22:0/1482398368 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680005280 con 0x55dc7d4c0380 2024-04-20T21:35:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.783+0000 7fd692bc3740 1 -- 172.21.15.22:0/1482398368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 msgr2=0x55dc7d4c0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.783+0000 7fd692bc3740 1 --2- 172.21.15.22:0/1482398368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 0x55dc7d4c0760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd680009f50 tx=0x7fd6800118d0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.783+0000 7fd68d400640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd6700a3d20 0x7fd6700a61e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55dc7d4bfe50 tx=0x7fd67c009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.783+0000 7fd692bc3740 1 -- 172.21.15.22:0/1482398368 shutdown_connections 2024-04-20T21:35:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.783+0000 7fd692bc3740 1 --2- 172.21.15.22:0/1482398368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c9880 0x55dc7d4cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.784+0000 7fd692bc3740 1 --2- 172.21.15.22:0/1482398368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.784+0000 7fd692bc3740 1 --2- 172.21.15.22:0/1482398368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 0x55dc7d4c0760 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.784+0000 7fd692bc3740 1 -- 172.21.15.22:0/1482398368 >> 172.21.15.22:0/1482398368 conn(0x55dc7d4baab0 msgr2=0x55dc7d4bae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:34.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.784+0000 7fd692bc3740 1 -- 172.21.15.22:0/1482398368 shutdown_connections 2024-04-20T21:35:34.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.784+0000 7fd692bc3740 1 -- 172.21.15.22:0/1482398368 wait complete. 2024-04-20T21:35:34.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.784+0000 7fd692bc3740 1 -- start start 2024-04-20T21:35:34.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.784+0000 7fd692bc3740 1 Processor -- start 2024-04-20T21:35:34.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.785+0000 7fd692bc3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c0380 0x55dc7d4c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.785+0000 7fd692bc3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.785+0000 7fd692bc3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c9880 0x55dc7d4c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.785+0000 7fd692bc3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dc7d4cc670 con 0x55dc7d4c0380 2024-04-20T21:35:34.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.785+0000 7fd692bc3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dc7d3df740 con 0x55dc7d4c9880 2024-04-20T21:35:34.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.785+0000 7fd692bc3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dc7d4cc7f0 con 0x55dc7d4c0ca0 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.785+0000 7fd692bc3740 10 client.0 _ll_register_callbacks cb 0x55dc7d3dab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.785+0000 7fd68ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.785+0000 7fd68de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c9880 0x55dc7d4c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.785+0000 7fd68d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c0380 0x55dc7d4c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.785+0000 7fd68ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:57576/0 (socket says 172.21.15.22:57576) 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.785+0000 7fd68ca00640 1 -- 172.21.15.22:0/2731942027 learned_addr learned my addr 172.21.15.22:0/2731942027 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.786+0000 7fd68de00640 1 -- 172.21.15.22:0/2731942027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 msgr2=0x55dc7d4c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.786+0000 7fd68de00640 1 --2- 172.21.15.22:0/2731942027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.786+0000 7fd68de00640 1 -- 172.21.15.22:0/2731942027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c0380 msgr2=0x55dc7d4c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.786+0000 7fd68de00640 1 --2- 172.21.15.22:0/2731942027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c0380 0x55dc7d4c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.786+0000 7fd68de00640 1 -- 172.21.15.22:0/2731942027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7d1383f0 con 0x55dc7d4c9880 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.786+0000 7fd68ca00640 1 --2- 172.21.15.22:0/2731942027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.786+0000 7fd68de00640 1 --2- 172.21.15.22:0/2731942027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c9880 0x55dc7d4c7470 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd68001b040 tx=0x7fd680009d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=243d9265fbb7af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:34.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.786+0000 7fd66ea00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.786+0000 7fd66ea00640 1 -- 172.21.15.22:0/2731942027 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680029070 con 0x55dc7d4c9880 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.786+0000 7fd692bc3740 1 -- 172.21.15.22:0/2731942027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd680004050 con 0x55dc7d4c9880 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.786+0000 7fd66ea00640 1 -- 172.21.15.22:0/2731942027 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd68000f480 con 0x55dc7d4c9880 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.786+0000 7fd66ea00640 1 -- 172.21.15.22:0/2731942027 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd680032050 con 0x55dc7d4c9880 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.787+0000 7fd692bc3740 10 client.25577 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.787+0000 7fd692bc3740 1 -- 172.21.15.22:0/2731942027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dc7d0f80f0 con 0x55dc7d4c9880 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.787+0000 7fd66ea00640 1 -- 172.21.15.22:0/2731942027 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd680004050 con 0x55dc7d4c9880 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.787+0000 7fd66ea00640 1 -- 172.21.15.22:0/2731942027 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fd68002c020 con 0x55dc7d4c9880 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.787+0000 7fd692bc3740 10 client.25577 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.787+0000 7fd692bc3740 10 client.25577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.787+0000 7fd692bc3740 20 client.25577 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.787+0000 7fd692bc3740 1 --2- 172.21.15.22:0/2731942027 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dc7d4d9c70 0x55dc7d4dc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.787+0000 7fd692bc3740 4 client.25577 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.787+0000 7fd692bc3740 1 -- 172.21.15.22:0/2731942027 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc7d433b70 con 0x55dc7d4d9c70 2024-04-20T21:35:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.788+0000 7fd68d400640 1 --2- 172.21.15.22:0/2731942027 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dc7d4d9c70 0x55dc7d4dc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.788+0000 7fd68d400640 1 --2- 172.21.15.22:0/2731942027 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dc7d4d9c70 0x55dc7d4dc060 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c17f1257f90218 server_cookie=ba0f250508550e7c in_seq=0 out_seq=0 2024-04-20T21:35:34.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.788+0000 7fd66ea00640 10 client.25577 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.789+0000 7fd66ea00640 1 -- 172.21.15.22:0/2731942027 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc7d433b70 con 0x55dc7d4d9c70 2024-04-20T21:35:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.789+0000 7fd66ea00640 10 client.25577 handle_command_reply: tid=0 2024-04-20T21:35:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.789+0000 7fd692bc3740 1 client.25577 shutdown 2024-04-20T21:35:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.789+0000 7fd692bc3740 1 -- 172.21.15.22:0/2731942027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c9880 msgr2=0x55dc7d4c7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.789+0000 7fd692bc3740 1 --2- 172.21.15.22:0/2731942027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c9880 0x55dc7d4c7470 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd68001b040 tx=0x7fd680009d20 comp rx=0 tx=0).stop 2024-04-20T21:35:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.789+0000 7fd692bc3740 1 -- 172.21.15.22:0/2731942027 shutdown_connections 2024-04-20T21:35:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.789+0000 7fd692bc3740 1 --2- 172.21.15.22:0/2731942027 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dc7d4d9c70 0x55dc7d4dc060 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.789+0000 7fd692bc3740 1 --2- 172.21.15.22:0/2731942027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c9880 0x55dc7d4c7470 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.789+0000 7fd692bc3740 1 --2- 172.21.15.22:0/2731942027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.789+0000 7fd692bc3740 1 --2- 172.21.15.22:0/2731942027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c0380 0x55dc7d4c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.789+0000 7fd692bc3740 1 -- 172.21.15.22:0/2731942027 >> 172.21.15.22:0/2731942027 conn(0x55dc7d4baab0 msgr2=0x55dc7d4bc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.789+0000 7fd66ea00640 0 client.25577 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:34.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.790+0000 7fd692bc3740 1 -- 172.21.15.22:0/2731942027 shutdown_connections 2024-04-20T21:35:34.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.790+0000 7fd692bc3740 1 -- 172.21.15.22:0/2731942027 wait complete. 2024-04-20T21:35:34.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.790+0000 7fd692bc3740 20 client.25577 trim_cache size 0 max 16384 2024-04-20T21:35:34.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:34 smithi033 ceph-mon[21626]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T21:35:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.809+0000 7fd692bc3740 1 Processor -- start 2024-04-20T21:35:34.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.810+0000 7fd692bc3740 1 -- start start 2024-04-20T21:35:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.810+0000 7fd692bc3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 0x55dc7d4d5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.810+0000 7fd692bc3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.810+0000 7fd692bc3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c9880 0x55dc7d4c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.810+0000 7fd692bc3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ade00 con 0x55dc7d4c9880 2024-04-20T21:35:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.810+0000 7fd692bc3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881b0680 con 0x55dc7d4c0380 2024-04-20T21:35:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.810+0000 7fd692bc3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ae470 con 0x55dc7d4c0ca0 2024-04-20T21:35:34.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.810+0000 7fd68de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c9880 0x55dc7d4c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.810+0000 7fd68d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 0x55dc7d4d5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.810+0000 7fd68d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 0x55dc7d4d5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39046/0 (socket says 172.21.15.22:39046) 2024-04-20T21:35:34.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.810+0000 7fd68d400640 1 -- 172.21.15.22:0/2567117698 learned_addr learned my addr 172.21.15.22:0/2567117698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:34.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.810+0000 7fd68ca00640 1 --2- 172.21.15.22:0/2567117698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.811+0000 7fd68d400640 1 -- 172.21.15.22:0/2567117698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 msgr2=0x55dc7d4c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.811+0000 7fd68d400640 1 --2- 172.21.15.22:0/2567117698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.811+0000 7fd68d400640 1 -- 172.21.15.22:0/2567117698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c9880 msgr2=0x55dc7d4c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.811+0000 7fd68d400640 1 --2- 172.21.15.22:0/2567117698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c9880 0x55dc7d4c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.811+0000 7fd68d400640 1 -- 172.21.15.22:0/2567117698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7d4cc1d0 con 0x55dc7d4c0380 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.811+0000 7fd68d400640 1 --2- 172.21.15.22:0/2567117698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 0x55dc7d4d5530 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55dc7d4dc6c0 tx=0x7fd67c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dfd455fc5b794c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.811+0000 7fd679c00640 1 -- 172.21.15.22:0/2567117698 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0044c0 con 0x55dc7d4c0380 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.811+0000 7fd679c00640 1 -- 172.21.15.22:0/2567117698 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c00fe20 con 0x55dc7d4c0380 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.811+0000 7fd679c00640 1 -- 172.21.15.22:0/2567117698 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c01e7c0 con 0x55dc7d4c0380 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.812+0000 7fd692bc3740 1 -- 172.21.15.22:0/2567117698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 msgr2=0x55dc7d4d5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.812+0000 7fd692bc3740 1 --2- 172.21.15.22:0/2567117698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 0x55dc7d4d5530 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55dc7d4dc6c0 tx=0x7fd67c01b000 comp rx=0 tx=0).stop 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.812+0000 7fd692bc3740 1 -- 172.21.15.22:0/2567117698 shutdown_connections 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.812+0000 7fd692bc3740 1 --2- 172.21.15.22:0/2567117698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c9880 0x55dc7d4c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.812+0000 7fd692bc3740 1 --2- 172.21.15.22:0/2567117698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.812+0000 7fd692bc3740 1 --2- 172.21.15.22:0/2567117698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0380 0x55dc7d4d5530 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.812+0000 7fd692bc3740 1 -- 172.21.15.22:0/2567117698 >> 172.21.15.22:0/2567117698 conn(0x55dc7d4baab0 msgr2=0x55dc7d4b9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:34.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.812+0000 7fd692bc3740 1 -- 172.21.15.22:0/2567117698 shutdown_connections 2024-04-20T21:35:34.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.812+0000 7fd692bc3740 1 -- 172.21.15.22:0/2567117698 wait complete. 2024-04-20T21:35:34.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.813+0000 7fd692bc3740 1 -- start start 2024-04-20T21:35:34.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.813+0000 7fd692bc3740 1 Processor -- start 2024-04-20T21:35:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.813+0000 7fd692bc3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c0380 0x55dc7d4d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.813+0000 7fd692bc3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.813+0000 7fd692bc3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c9880 0x55dc7d4ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.813+0000 7fd692bc3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881aeb40 con 0x55dc7d4c0380 2024-04-20T21:35:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.813+0000 7fd692bc3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881affb0 con 0x55dc7d4c0ca0 2024-04-20T21:35:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.813+0000 7fd692bc3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881af210 con 0x55dc7d4c9880 2024-04-20T21:35:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.813+0000 7fd692bc3740 10 client.0 _ll_register_callbacks cb 0x7fd67c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.813+0000 7fd68d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c0380 0x55dc7d4d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68ca00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c9880 0x55dc7d4ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68ca00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4d1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39048/0 (socket says 172.21.15.22:39048) 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68ca00640 1 -- 172.21.15.22:0/92440666 learned_addr learned my addr 172.21.15.22:0/92440666 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c0380 0x55dc7d4d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57420/0 (socket says 172.21.15.22:57420) 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68ca00640 1 -- 172.21.15.22:0/92440666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c9880 msgr2=0x55dc7d4ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68ca00640 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c9880 0x55dc7d4ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68ca00640 1 -- 172.21.15.22:0/92440666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c0380 msgr2=0x55dc7d4d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68ca00640 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c0380 0x55dc7d4d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68ca00640 1 -- 172.21.15.22:0/92440666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7d4cd700 con 0x55dc7d4c0ca0 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68de00640 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c9880 0x55dc7d4ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68d400640 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c0380 0x55dc7d4d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd68ca00640 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4d1ee0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd67402c220 tx=0x7fd67402bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd6a58f82c4fa3f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd66ea00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd66ea00640 1 -- 172.21.15.22:0/92440666 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6740ef830 con 0x55dc7d4c0ca0 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd66ea00640 1 -- 172.21.15.22:0/92440666 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd6740c64f0 con 0x55dc7d4c0ca0 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.814+0000 7fd692bc3740 1 -- 172.21.15.22:0/92440666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67c00fe20 con 0x55dc7d4c0ca0 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.815+0000 7fd66ea00640 1 -- 172.21.15.22:0/92440666 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd6740bddc0 con 0x55dc7d4c0ca0 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.815+0000 7fd692bc3740 10 client.25583 fetch_fsmap learned FSMap version 20 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.815+0000 7fd692bc3740 1 -- 172.21.15.22:0/92440666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dc7d53fbb0 con 0x55dc7d4c0ca0 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.815+0000 7fd66ea00640 1 -- 172.21.15.22:0/92440666 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c00fe20 con 0x55dc7d4c0ca0 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.815+0000 7fd66ea00640 1 -- 172.21.15.22:0/92440666 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fd67403c030 con 0x55dc7d4c0ca0 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.815+0000 7fd692bc3740 10 client.25583 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.815+0000 7fd692bc3740 10 client.25583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.815+0000 7fd692bc3740 20 client.25583 populate_metadata read hostname 'smithi022' 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.815+0000 7fd692bc3740 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dc7d540290 0x55dc7d5426b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.815+0000 7fd692bc3740 4 client.25583 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.815+0000 7fd692bc3740 1 -- 172.21.15.22:0/92440666 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc7d42d9e0 con 0x55dc7d540290 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.816+0000 7fd68d400640 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dc7d540290 0x55dc7d5426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.816+0000 7fd68d400640 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dc7d540290 0x55dc7d5426b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d41842cbb95467 server_cookie=2d038dc8d4958893 in_seq=0 out_seq=0 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.816+0000 7fd66ea00640 10 client.25583 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.816+0000 7fd66ea00640 1 -- 172.21.15.22:0/92440666 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dc7d42d9e0 con 0x55dc7d540290 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.816+0000 7fd66ea00640 10 client.25583 handle_command_reply: tid=0 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.816+0000 7fd692bc3740 1 client.25583 shutdown 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.817+0000 7fd692bc3740 1 -- 172.21.15.22:0/92440666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0ca0 msgr2=0x55dc7d4d1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.817+0000 7fd692bc3740 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4d1ee0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd67402c220 tx=0x7fd67402bc10 comp rx=0 tx=0).stop 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.817+0000 7fd692bc3740 1 -- 172.21.15.22:0/92440666 shutdown_connections 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.817+0000 7fd692bc3740 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc7d4c9880 0x55dc7d4ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.817+0000 7fd692bc3740 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dc7d540290 0x55dc7d5426b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.817+0000 7fd692bc3740 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc7d4c0ca0 0x55dc7d4d1ee0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.817+0000 7fd692bc3740 1 --2- 172.21.15.22:0/92440666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc7d4c0380 0x55dc7d4d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.817+0000 7fd692bc3740 1 -- 172.21.15.22:0/92440666 >> 172.21.15.22:0/92440666 conn(0x55dc7d4baab0 msgr2=0x55dc7d4b9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.817+0000 7fd66ea00640 0 client.25583 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:35:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.817+0000 7fd692bc3740 1 -- 172.21.15.22:0/92440666 shutdown_connections 2024-04-20T21:35:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.817+0000 7fd692bc3740 1 -- 172.21.15.22:0/92440666 wait complete. 2024-04-20T21:35:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.817+0000 7fd692bc3740 20 client.25583 trim_cache size 0 max 16384 2024-04-20T21:35:34.820 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:35:34.820 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:35:34.820 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:35:34.820 INFO:teuthology.orchestra.run.smithi022.stdout: "4c4bc6d7-2c00-4860-b765-bb70f4e7e07c": { 2024-04-20T21:35:34.820 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:35:34.821 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4c4bc6d7-2c00-4860-b765-bb70f4e7e07c", 2024-04-20T21:35:34.821 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:35:34.821 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:35:34.821 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:35:34.821 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:35:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.818+0000 7fd68e800640 1 -- 172.21.15.22:0/2725849283 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd6700a3d20 msgr2=0x7fd6700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.818+0000 7fd68e800640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd6700a3d20 0x7fd6700a61e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55dc7d4bfe50 tx=0x7fd67c009260 comp rx=0 tx=0).stop 2024-04-20T21:35:34.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.818+0000 7fd68e800640 1 -- 172.21.15.22:0/2725849283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6881038e0 msgr2=0x7fd6881a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:34.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.818+0000 7fd68e800640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6881038e0 0x7fd6881a0b70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd67402c150 tx=0x7fd6740344c0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.818+0000 7fd68e800640 1 -- 172.21.15.22:0/2725849283 shutdown_connections 2024-04-20T21:35:34.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.818+0000 7fd68e800640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd6700a3d20 0x7fd6700a61e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.818+0000 7fd68e800640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd68810c3e0 0x7fd68819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.818+0000 7fd68e800640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6881038e0 0x7fd6881a0b70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.818+0000 7fd68e800640 1 --2- 172.21.15.22:0/2725849283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd688102fc0 0x7fd6881a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:34.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.818+0000 7fd68e800640 1 -- 172.21.15.22:0/2725849283 >> 172.21.15.22:0/2725849283 conn(0x7fd6880fc7e0 msgr2=0x7fd688100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:34.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.818+0000 7fd68e800640 1 -- 172.21.15.22:0/2725849283 shutdown_connections 2024-04-20T21:35:34.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:35:34.818+0000 7fd68e800640 1 -- 172.21.15.22:0/2725849283 wait complete. 2024-04-20T21:35:34.836 INFO:tasks.cephfs.filesystem:scrub status for tag:4c4bc6d7-2c00-4860-b765-bb70f4e7e07c - {'path': '/', 'tag': '4c4bc6d7-2c00-4860-b765-bb70f4e7e07c', 'options': 'recursive,force'} 2024-04-20T21:35:35.161 INFO:tasks.workunit.client.0.smithi022.stdout: 1 7345 19.63 MB/sec warmup 3 sec latency 10.787 ms 2024-04-20T21:35:35.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:35 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:35:35.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:35 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:35.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:35 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:35:35.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:35 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:35.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:35 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:35:35.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:35 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:35.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:35 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:35:35.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:35 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:35.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:35 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:35:35.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:35 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:35.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:35 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:35:35.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:35 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:36.161 INFO:tasks.workunit.client.0.smithi022.stdout: 1 9441 16.52 MB/sec warmup 4 sec latency 11.340 ms 2024-04-20T21:35:36.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:36 smithi022 ceph-mon[16843]: pgmap v56: 97 pgs: 97 active+clean; 4.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 333 KiB/s wr, 39 op/s 2024-04-20T21:35:36.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:36 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:35:36.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:36 smithi122 ceph-mon[22147]: pgmap v56: 97 pgs: 97 active+clean; 4.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 333 KiB/s wr, 39 op/s 2024-04-20T21:35:36.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:36 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:35:36.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:36 smithi033 ceph-mon[21626]: pgmap v56: 97 pgs: 97 active+clean; 4.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 333 KiB/s wr, 39 op/s 2024-04-20T21:35:36.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:36 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:35:37.162 INFO:tasks.workunit.client.0.smithi022.stdout: 1 11309 15.68 MB/sec warmup 5 sec latency 19.699 ms 2024-04-20T21:35:37.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:37 smithi022 ceph-mon[16843]: pgmap v57: 97 pgs: 97 active+clean; 4.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 333 KiB/s wr, 38 op/s 2024-04-20T21:35:37.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:37 smithi122 ceph-mon[22147]: pgmap v57: 97 pgs: 97 active+clean; 4.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 333 KiB/s wr, 38 op/s 2024-04-20T21:35:37.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:37 smithi033 ceph-mon[21626]: pgmap v57: 97 pgs: 97 active+clean; 4.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 333 KiB/s wr, 38 op/s 2024-04-20T21:35:38.162 INFO:tasks.workunit.client.0.smithi022.stdout: 1 13675 15.15 MB/sec warmup 6 sec latency 21.306 ms 2024-04-20T21:35:39.162 INFO:tasks.workunit.client.0.smithi022.stdout: 1 15375 14.21 MB/sec warmup 7 sec latency 11.452 ms 2024-04-20T21:35:40.162 INFO:tasks.workunit.client.0.smithi022.stdout: 1 18034 14.53 MB/sec warmup 8 sec latency 8.860 ms 2024-04-20T21:35:40.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:39 smithi122 ceph-mon[22147]: pgmap v58: 97 pgs: 97 active+clean; 4.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 333 KiB/s wr, 38 op/s 2024-04-20T21:35:40.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:39 smithi033 ceph-mon[21626]: pgmap v58: 97 pgs: 97 active+clean; 4.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 333 KiB/s wr, 38 op/s 2024-04-20T21:35:40.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:39 smithi022 ceph-mon[16843]: pgmap v58: 97 pgs: 97 active+clean; 4.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 333 KiB/s wr, 38 op/s 2024-04-20T21:35:41.162 INFO:tasks.workunit.client.0.smithi022.stdout: 1 20221 13.75 MB/sec warmup 9 sec latency 20.406 ms 2024-04-20T21:35:42.162 INFO:tasks.workunit.client.0.smithi022.stdout: 1 22046 13.60 MB/sec warmup 10 sec latency 19.284 ms 2024-04-20T21:35:42.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:41 smithi122 ceph-mon[22147]: pgmap v59: 97 pgs: 97 active+clean; 9.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 870 KiB/s wr, 101 op/s 2024-04-20T21:35:42.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:41 smithi033 ceph-mon[21626]: pgmap v59: 97 pgs: 97 active+clean; 9.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 870 KiB/s wr, 101 op/s 2024-04-20T21:35:42.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:41 smithi022 ceph-mon[16843]: pgmap v59: 97 pgs: 97 active+clean; 9.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 870 KiB/s wr, 101 op/s 2024-04-20T21:35:43.162 INFO:tasks.workunit.client.0.smithi022.stdout: 1 24628 13.65 MB/sec warmup 11 sec latency 44.172 ms 2024-04-20T21:35:44.162 INFO:tasks.workunit.client.0.smithi022.stdout: 1 26331 13.33 MB/sec warmup 12 sec latency 12.993 ms 2024-04-20T21:35:44.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:43 smithi122 ceph-mon[22147]: pgmap v60: 97 pgs: 97 active+clean; 9.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 869 KiB/s wr, 101 op/s 2024-04-20T21:35:44.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:43 smithi033 ceph-mon[21626]: pgmap v60: 97 pgs: 97 active+clean; 9.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 869 KiB/s wr, 101 op/s 2024-04-20T21:35:44.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:43 smithi022 ceph-mon[16843]: pgmap v60: 97 pgs: 97 active+clean; 9.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 869 KiB/s wr, 101 op/s 2024-04-20T21:35:45.163 INFO:tasks.workunit.client.0.smithi022.stdout: 1 28807 13.38 MB/sec warmup 13 sec latency 8.726 ms 2024-04-20T21:35:46.163 INFO:tasks.workunit.client.0.smithi022.stdout: 1 31167 13.21 MB/sec warmup 14 sec latency 12.136 ms 2024-04-20T21:35:46.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:45 smithi122 ceph-mon[22147]: pgmap v61: 97 pgs: 97 active+clean; 15 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-04-20T21:35:46.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:45 smithi033 ceph-mon[21626]: pgmap v61: 97 pgs: 97 active+clean; 15 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-04-20T21:35:46.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:45 smithi022 ceph-mon[16843]: pgmap v61: 97 pgs: 97 active+clean; 15 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-04-20T21:35:47.163 INFO:tasks.workunit.client.0.smithi022.stdout: 1 32933 12.93 MB/sec warmup 15 sec latency 20.742 ms 2024-04-20T21:35:48.163 INFO:tasks.workunit.client.0.smithi022.stdout: 1 35626 13.18 MB/sec warmup 16 sec latency 8.567 ms 2024-04-20T21:35:48.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:47 smithi122 ceph-mon[22147]: pgmap v62: 97 pgs: 97 active+clean; 15 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-04-20T21:35:48.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:47 smithi033 ceph-mon[21626]: pgmap v62: 97 pgs: 97 active+clean; 15 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-04-20T21:35:48.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:47 smithi022 ceph-mon[16843]: pgmap v62: 97 pgs: 97 active+clean; 15 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-04-20T21:35:49.163 INFO:tasks.workunit.client.0.smithi022.stdout: 1 37284 12.80 MB/sec warmup 17 sec latency 20.492 ms 2024-04-20T21:35:50.163 INFO:tasks.workunit.client.0.smithi022.stdout: 1 39486 12.85 MB/sec warmup 18 sec latency 9.006 ms 2024-04-20T21:35:50.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:49 smithi122 ceph-mon[22147]: pgmap v63: 97 pgs: 97 active+clean; 15 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-04-20T21:35:50.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:49 smithi033 ceph-mon[21626]: pgmap v63: 97 pgs: 97 active+clean; 15 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-04-20T21:35:50.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:49 smithi022 ceph-mon[16843]: pgmap v63: 97 pgs: 97 active+clean; 15 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-04-20T21:35:51.163 INFO:tasks.workunit.client.0.smithi022.stdout: 1 41536 12.58 MB/sec warmup 19 sec latency 16.372 ms 2024-04-20T21:35:52.163 INFO:tasks.workunit.client.0.smithi022.stdout: 1 43217 12.52 MB/sec warmup 20 sec latency 16.106 ms 2024-04-20T21:35:52.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:51 smithi122 ceph-mon[22147]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-04-20T21:35:52.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:51 smithi033 ceph-mon[21626]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-04-20T21:35:52.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:51 smithi022 ceph-mon[16843]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-04-20T21:35:53.164 INFO:tasks.workunit.client.0.smithi022.stdout: 1 45357 12.45 MB/sec warmup 21 sec latency 15.669 ms 2024-04-20T21:35:54.164 INFO:tasks.workunit.client.0.smithi022.stdout: 1 46831 12.26 MB/sec warmup 22 sec latency 27.722 ms 2024-04-20T21:35:54.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:53 smithi122 ceph-mon[22147]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-04-20T21:35:54.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:53 smithi033 ceph-mon[21626]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-04-20T21:35:54.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:53 smithi022 ceph-mon[16843]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-04-20T21:35:55.164 INFO:tasks.workunit.client.0.smithi022.stdout: 1 49118 12.26 MB/sec warmup 23 sec latency 10.183 ms 2024-04-20T21:35:56.164 INFO:tasks.workunit.client.0.smithi022.stdout: 1 50555 12.05 MB/sec warmup 24 sec latency 17.941 ms 2024-04-20T21:35:56.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:55 smithi122 ceph-mon[22147]: pgmap v66: 97 pgs: 97 active+clean; 24 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-04-20T21:35:56.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:55 smithi033 ceph-mon[21626]: pgmap v66: 97 pgs: 97 active+clean; 24 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-04-20T21:35:56.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:55 smithi022 ceph-mon[16843]: pgmap v66: 97 pgs: 97 active+clean; 24 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-04-20T21:35:57.164 INFO:tasks.workunit.client.0.smithi022.stdout: 1 52891 12.08 MB/sec warmup 25 sec latency 11.015 ms 2024-04-20T21:35:58.164 INFO:tasks.workunit.client.0.smithi022.stdout: 1 54181 11.87 MB/sec warmup 26 sec latency 34.612 ms 2024-04-20T21:35:58.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:57 smithi122 ceph-mon[22147]: pgmap v67: 97 pgs: 97 active+clean; 24 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-04-20T21:35:58.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:57 smithi033 ceph-mon[21626]: pgmap v67: 97 pgs: 97 active+clean; 24 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-04-20T21:35:58.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:57 smithi022 ceph-mon[16843]: pgmap v67: 97 pgs: 97 active+clean; 24 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-04-20T21:35:59.164 INFO:tasks.workunit.client.0.smithi022.stdout: 1 56608 11.94 MB/sec warmup 27 sec latency 10.566 ms 2024-04-20T21:36:00.164 INFO:tasks.workunit.client.0.smithi022.stdout: 1 58031 11.75 MB/sec warmup 28 sec latency 28.549 ms 2024-04-20T21:36:00.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:35:59 smithi122 ceph-mon[22147]: pgmap v68: 97 pgs: 97 active+clean; 24 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-04-20T21:36:00.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:35:59 smithi033 ceph-mon[21626]: pgmap v68: 97 pgs: 97 active+clean; 24 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-04-20T21:36:00.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:35:59 smithi022 ceph-mon[16843]: pgmap v68: 97 pgs: 97 active+clean; 24 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-04-20T21:36:01.165 INFO:tasks.workunit.client.0.smithi022.stdout: 1 60031 11.75 MB/sec warmup 29 sec latency 45.918 ms 2024-04-20T21:36:01.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:01 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:36:01.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:01 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:36:01.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:01 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:36:02.165 INFO:tasks.workunit.client.0.smithi022.stdout: 1 61066 11.55 MB/sec warmup 30 sec latency 43.448 ms 2024-04-20T21:36:02.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:02 smithi122 ceph-mon[22147]: pgmap v69: 97 pgs: 97 active+clean; 28 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-20T21:36:02.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:02 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:36:02.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:02 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:36:02.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:02 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:36:02.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:02 smithi033 ceph-mon[21626]: pgmap v69: 97 pgs: 97 active+clean; 28 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-20T21:36:02.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:02 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:36:02.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:02 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:36:02.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:02 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:36:02.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:02 smithi022 ceph-mon[16843]: pgmap v69: 97 pgs: 97 active+clean; 28 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-20T21:36:02.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:02 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:36:02.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:02 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:36:02.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:02 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:36:03.165 INFO:tasks.workunit.client.0.smithi022.stdout: 1 62808 11.41 MB/sec warmup 31 sec latency 61.114 ms 2024-04-20T21:36:04.165 INFO:tasks.workunit.client.0.smithi022.stdout: 1 64471 11.41 MB/sec warmup 32 sec latency 15.614 ms 2024-04-20T21:36:04.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:04 smithi122 ceph-mon[22147]: pgmap v70: 97 pgs: 97 active+clean; 28 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 950 KiB/s wr, 113 op/s 2024-04-20T21:36:04.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:04 smithi033 ceph-mon[21626]: pgmap v70: 97 pgs: 97 active+clean; 28 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 950 KiB/s wr, 113 op/s 2024-04-20T21:36:04.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:04 smithi022 ceph-mon[16843]: pgmap v70: 97 pgs: 97 active+clean; 28 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 950 KiB/s wr, 113 op/s 2024-04-20T21:36:04.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:36:04.837 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:36:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.071+0000 7fc172800640 1 -- 172.21.15.22:0/1590010346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc16c104dc0 msgr2=0x7fc16c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.071+0000 7fc172800640 1 --2- 172.21.15.22:0/1590010346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc16c104dc0 0x7fc16c108e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc154009900 tx=0x7fc15402bc30 comp rx=0 tx=0).stop 2024-04-20T21:36:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.072+0000 7fc172800640 1 -- 172.21.15.22:0/1590010346 shutdown_connections 2024-04-20T21:36:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.072+0000 7fc172800640 1 --2- 172.21.15.22:0/1590010346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc16c1094e0 0x7fc16c1110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.072+0000 7fc172800640 1 --2- 172.21.15.22:0/1590010346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc16c104dc0 0x7fc16c108e00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.072+0000 7fc172800640 1 --2- 172.21.15.22:0/1590010346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc16c104410 0x7fc16c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.072+0000 7fc172800640 1 -- 172.21.15.22:0/1590010346 >> 172.21.15.22:0/1590010346 conn(0x7fc16c0ffcf0 msgr2=0x7fc16c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.072+0000 7fc172800640 1 -- 172.21.15.22:0/1590010346 shutdown_connections 2024-04-20T21:36:05.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.072+0000 7fc172800640 1 -- 172.21.15.22:0/1590010346 wait complete. 2024-04-20T21:36:05.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.073+0000 7fc172800640 1 Processor -- start 2024-04-20T21:36:05.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.073+0000 7fc172800640 1 -- start start 2024-04-20T21:36:05.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.073+0000 7fc172800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc16c104410 0x7fc16c076470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.073+0000 7fc172800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc16c1094e0 0x7fc16c0769b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.073+0000 7fc172800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc16c076ef0 0x7fc16c1ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.073+0000 7fc172800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1145c0 con 0x7fc16c076ef0 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.073+0000 7fc172800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c114440 con 0x7fc16c104410 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.073+0000 7fc172800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c114740 con 0x7fc16c1094e0 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.074+0000 7fc16b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc16c104410 0x7fc16c076470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.074+0000 7fc16b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc16c104410 0x7fc16c076470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33872/0 (socket says 172.21.15.22:33872) 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.074+0000 7fc16b400640 1 -- 172.21.15.22:0/1059845471 learned_addr learned my addr 172.21.15.22:0/1059845471 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.074+0000 7fc16aa00640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc16c1094e0 0x7fc16c0769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.074+0000 7fc16be00640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc16c076ef0 0x7fc16c1ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.074+0000 7fc16b400640 1 -- 172.21.15.22:0/1059845471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc16c1094e0 msgr2=0x7fc16c0769b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.074+0000 7fc16b400640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc16c1094e0 0x7fc16c0769b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.074+0000 7fc16b400640 1 -- 172.21.15.22:0/1059845471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc16c076ef0 msgr2=0x7fc16c1ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.074+0000 7fc16b400640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc16c076ef0 0x7fc16c1ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.074+0000 7fc16b400640 1 -- 172.21.15.22:0/1059845471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16c1ab610 con 0x7fc16c104410 2024-04-20T21:36:05.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.074+0000 7fc16be00640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc16c076ef0 0x7fc16c1ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:05.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.074+0000 7fc16b400640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc16c104410 0x7fc16c076470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00ebc0 tx=0x7fc15c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.075+0000 7fc163e00640 1 -- 172.21.15.22:0/1059845471 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c010630 con 0x7fc16c104410 2024-04-20T21:36:05.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.075+0000 7fc163e00640 1 -- 172.21.15.22:0/1059845471 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc15c010cb0 con 0x7fc16c104410 2024-04-20T21:36:05.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.075+0000 7fc163e00640 1 -- 172.21.15.22:0/1059845471 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c018bd0 con 0x7fc16c104410 2024-04-20T21:36:05.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.075+0000 7fc172800640 1 -- 172.21.15.22:0/1059845471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16c1ab8a0 con 0x7fc16c104410 2024-04-20T21:36:05.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.075+0000 7fc172800640 1 -- 172.21.15.22:0/1059845471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16c1abd70 con 0x7fc16c104410 2024-04-20T21:36:05.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.080+0000 7fc1755c9740 1 Processor -- start 2024-04-20T21:36:05.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.080+0000 7fc1755c9740 1 -- start start 2024-04-20T21:36:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.080+0000 7fc163e00640 1 -- 172.21.15.22:0/1059845471 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc15c0107d0 con 0x7fc16c104410 2024-04-20T21:36:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.080+0000 7fc1755c9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e7a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.080+0000 7fc1755c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e83160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc1755c9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e85a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc1755c9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560092de0150 con 0x560092e7a1a0 2024-04-20T21:36:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc1755c9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560092d86370 con 0x560092e836a0 2024-04-20T21:36:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc1755c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560092ddc110 con 0x560092e7aac0 2024-04-20T21:36:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc16b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e83160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc16aa00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e85a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc16b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e83160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33224/0 (socket says 172.21.15.22:33224) 2024-04-20T21:36:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc16b400640 1 -- 172.21.15.22:0/3770635987 learned_addr learned my addr 172.21.15.22:0/3770635987 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc16aa00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e85a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33874/0 (socket says 172.21.15.22:33874) 2024-04-20T21:36:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc16be00640 1 --2- 172.21.15.22:0/3770635987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc16b400640 1 -- 172.21.15.22:0/3770635987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 msgr2=0x560092e85a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc16b400640 1 --2- 172.21.15.22:0/3770635987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e85a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc16b400640 1 -- 172.21.15.22:0/3770635987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 msgr2=0x560092e7a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.081+0000 7fc16b400640 1 --2- 172.21.15.22:0/3770635987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e7a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.082+0000 7fc16b400640 1 -- 172.21.15.22:0/3770635987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560092bbc7d0 con 0x560092e7aac0 2024-04-20T21:36:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.082+0000 7fc16aa00640 1 --2- 172.21.15.22:0/3770635987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e85a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.082+0000 7fc16be00640 1 --2- 172.21.15.22:0/3770635987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e7a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.082+0000 7fc16b400640 1 --2- 172.21.15.22:0/3770635987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e83160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0ce530 tx=0x7fc15c0d9570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9012fd54dd2218e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.083+0000 7fc160c00640 1 -- 172.21.15.22:0/3770635987 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c0e4070 con 0x560092e7aac0 2024-04-20T21:36:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.083+0000 7fc160c00640 1 -- 172.21.15.22:0/3770635987 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc15c0dfae0 con 0x560092e7aac0 2024-04-20T21:36:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.083+0000 7fc160c00640 1 -- 172.21.15.22:0/3770635987 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c0dec10 con 0x560092e7aac0 2024-04-20T21:36:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.083+0000 7fc163e00640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc1500a3d20 0x7fc1500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.083+0000 7fc163e00640 1 -- 172.21.15.22:0/1059845471 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc15c0d0f20 con 0x7fc16c104410 2024-04-20T21:36:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.083+0000 7fc16be00640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc1500a3d20 0x7fc1500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.083+0000 7fc1755c9740 1 -- 172.21.15.22:0/3770635987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 msgr2=0x560092e83160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.083+0000 7fc1755c9740 1 --2- 172.21.15.22:0/3770635987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e83160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0ce530 tx=0x7fc15c0d9570 comp rx=0 tx=0).stop 2024-04-20T21:36:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.084+0000 7fc1755c9740 1 -- 172.21.15.22:0/3770635987 shutdown_connections 2024-04-20T21:36:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.084+0000 7fc1755c9740 1 --2- 172.21.15.22:0/3770635987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e85a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.084+0000 7fc1755c9740 1 --2- 172.21.15.22:0/3770635987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e83160 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.084+0000 7fc1755c9740 1 --2- 172.21.15.22:0/3770635987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e7a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.084+0000 7fc1755c9740 1 -- 172.21.15.22:0/3770635987 >> 172.21.15.22:0/3770635987 conn(0x560092e748d0 msgr2=0x560092e74cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.084+0000 7fc1755c9740 1 -- 172.21.15.22:0/3770635987 shutdown_connections 2024-04-20T21:36:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.084+0000 7fc1755c9740 1 -- 172.21.15.22:0/3770635987 wait complete. 2024-04-20T21:36:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.084+0000 7fc16be00640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc1500a3d20 0x7fc1500a61e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc164009cc0 tx=0x7fc164005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.085+0000 7fc1755c9740 1 -- start start 2024-04-20T21:36:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.085+0000 7fc1755c9740 1 Processor -- start 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.085+0000 7fc1755c9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.085+0000 7fc1755c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e80ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.085+0000 7fc1755c9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.085+0000 7fc1755c9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560092e86490 con 0x560092e7a1a0 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.085+0000 7fc1755c9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560092d99740 con 0x560092e836a0 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.085+0000 7fc1755c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560092e86610 con 0x560092e7aac0 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.085+0000 7fc1755c9740 10 client.0 _ll_register_callbacks cb 0x560092d94b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e80ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e811e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33882/0 (socket says 172.21.15.22:33882) 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16be00640 1 -- 172.21.15.22:0/3849320509 learned_addr learned my addr 172.21.15.22:0/3849320509 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e80ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33226/0 (socket says 172.21.15.22:33226) 2024-04-20T21:36:05.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16be00640 1 -- 172.21.15.22:0/3849320509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 msgr2=0x560092e80ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16be00640 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e80ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16be00640 1 -- 172.21.15.22:0/3849320509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 msgr2=0x560092e80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16be00640 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e80760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16be00640 1 -- 172.21.15.22:0/3849320509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560092af23f0 con 0x560092e836a0 2024-04-20T21:36:05.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16aa00640 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e80ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:05.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.086+0000 7fc16b400640 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:05.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.087+0000 7fc16be00640 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e811e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc1640058c0 tx=0x7fc1640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e29cd7aec3b85e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.087+0000 7fc159600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:36:05.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.087+0000 7fc159600640 1 -- 172.21.15.22:0/3849320509 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16402b060 con 0x560092e836a0 2024-04-20T21:36:05.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.087+0000 7fc159600640 1 -- 172.21.15.22:0/3849320509 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc16401ec50 con 0x560092e836a0 2024-04-20T21:36:05.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.087+0000 7fc1755c9740 1 -- 172.21.15.22:0/3849320509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc15c0dfae0 con 0x560092e836a0 2024-04-20T21:36:05.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.087+0000 7fc159600640 1 -- 172.21.15.22:0/3849320509 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc16401ee50 con 0x560092e836a0 2024-04-20T21:36:05.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.087+0000 7fc1755c9740 10 client.25595 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:05.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.087+0000 7fc1755c9740 1 -- 172.21.15.22:0/3849320509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560092ab20f0 con 0x560092e836a0 2024-04-20T21:36:05.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.088+0000 7fc159600640 1 -- 172.21.15.22:0/3849320509 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc15c0dfae0 con 0x560092e836a0 2024-04-20T21:36:05.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.088+0000 7fc159600640 1 -- 172.21.15.22:0/3849320509 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fc16401a040 con 0x560092e836a0 2024-04-20T21:36:05.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.088+0000 7fc1755c9740 10 client.25595 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:05.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.088+0000 7fc1755c9740 10 client.25595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:05.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.088+0000 7fc1755c9740 20 client.25595 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:05.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.088+0000 7fc1755c9740 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560092e93a60 0x560092e95eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.088+0000 7fc1755c9740 4 client.25595 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:05.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.088+0000 7fc1755c9740 1 -- 172.21.15.22:0/3849320509 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560092dedb70 con 0x560092e93a60 2024-04-20T21:36:05.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.088+0000 7fc16b400640 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560092e93a60 0x560092e95eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.089+0000 7fc16b400640 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560092e93a60 0x560092e95eb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5970b1b905866cd1 server_cookie=88b02a6b28a35bae in_seq=0 out_seq=0 2024-04-20T21:36:05.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.089+0000 7fc159600640 10 client.25595 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:05.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc159600640 1 -- 172.21.15.22:0/3849320509 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560092dedb70 con 0x560092e93a60 2024-04-20T21:36:05.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc159600640 10 client.25595 handle_command_reply: tid=0 2024-04-20T21:36:05.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc1755c9740 1 client.25595 shutdown 2024-04-20T21:36:05.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc1755c9740 1 -- 172.21.15.22:0/3849320509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 msgr2=0x560092e811e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc1755c9740 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e811e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc1640058c0 tx=0x7fc1640046c0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc1755c9740 1 -- 172.21.15.22:0/3849320509 shutdown_connections 2024-04-20T21:36:05.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc1755c9740 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560092e93a60 0x560092e95eb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc1755c9740 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e811e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc1755c9740 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e80ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc1755c9740 1 --2- 172.21.15.22:0/3849320509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc159600640 0 client.25595 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:05.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc1755c9740 1 -- 172.21.15.22:0/3849320509 >> 172.21.15.22:0/3849320509 conn(0x560092e748d0 msgr2=0x560092e76620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc1755c9740 1 -- 172.21.15.22:0/3849320509 shutdown_connections 2024-04-20T21:36:05.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.091+0000 7fc1755c9740 1 -- 172.21.15.22:0/3849320509 wait complete. 2024-04-20T21:36:05.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.092+0000 7fc1755c9740 20 client.25595 trim_cache size 0 max 16384 2024-04-20T21:36:05.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.116+0000 7fc1755c9740 1 Processor -- start 2024-04-20T21:36:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.116+0000 7fc1755c9740 1 -- start start 2024-04-20T21:36:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.116+0000 7fc1755c9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e82e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.116+0000 7fc1755c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e8f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc1755c9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e73740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc1755c9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1b24a0 con 0x560092e7a1a0 2024-04-20T21:36:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc1755c9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1b4d20 con 0x560092e836a0 2024-04-20T21:36:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc1755c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1b2b10 con 0x560092e7aac0 2024-04-20T21:36:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc16be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e73740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc16be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e73740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33894/0 (socket says 172.21.15.22:33894) 2024-04-20T21:36:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc16be00640 1 -- 172.21.15.22:0/1935446196 learned_addr learned my addr 172.21.15.22:0/1935446196 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc16aa00640 1 --2- 172.21.15.22:0/1935446196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e8f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc16b400640 1 --2- 172.21.15.22:0/1935446196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e82e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc16be00640 1 -- 172.21.15.22:0/1935446196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 msgr2=0x560092e8f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc16be00640 1 --2- 172.21.15.22:0/1935446196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e8f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc16be00640 1 -- 172.21.15.22:0/1935446196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 msgr2=0x560092e82e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc16be00640 1 --2- 172.21.15.22:0/1935446196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e82e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.117+0000 7fc16be00640 1 -- 172.21.15.22:0/1935446196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560092e79940 con 0x560092e836a0 2024-04-20T21:36:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.118+0000 7fc16be00640 1 --2- 172.21.15.22:0/1935446196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e73740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc164026840 tx=0x7fc1640058f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0e3e7eb2191808 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.118+0000 7fc15be00640 1 -- 172.21.15.22:0/1935446196 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164035080 con 0x560092e836a0 2024-04-20T21:36:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.118+0000 7fc15be00640 1 -- 172.21.15.22:0/1935446196 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc164026410 con 0x560092e836a0 2024-04-20T21:36:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.118+0000 7fc15be00640 1 -- 172.21.15.22:0/1935446196 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164004990 con 0x560092e836a0 2024-04-20T21:36:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.119+0000 7fc1755c9740 1 -- 172.21.15.22:0/1935446196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 msgr2=0x560092e73740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.119+0000 7fc1755c9740 1 --2- 172.21.15.22:0/1935446196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e73740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc164026840 tx=0x7fc1640058f0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.119+0000 7fc1755c9740 1 -- 172.21.15.22:0/1935446196 shutdown_connections 2024-04-20T21:36:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.119+0000 7fc1755c9740 1 --2- 172.21.15.22:0/1935446196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e836a0 0x560092e73740 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.119+0000 7fc1755c9740 1 --2- 172.21.15.22:0/1935446196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e7aac0 0x560092e8f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.119+0000 7fc1755c9740 1 --2- 172.21.15.22:0/1935446196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7a1a0 0x560092e82e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.119+0000 7fc1755c9740 1 -- 172.21.15.22:0/1935446196 >> 172.21.15.22:0/1935446196 conn(0x560092e748d0 msgr2=0x560092e73030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.119+0000 7fc1755c9740 1 -- 172.21.15.22:0/1935446196 shutdown_connections 2024-04-20T21:36:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.119+0000 7fc1755c9740 1 -- 172.21.15.22:0/1935446196 wait complete. 2024-04-20T21:36:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.119+0000 7fc1755c9740 1 -- start start 2024-04-20T21:36:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.120+0000 7fc1755c9740 1 Processor -- start 2024-04-20T21:36:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.120+0000 7fc1755c9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e7a1a0 0x560092e7d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.120+0000 7fc1755c9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7aac0 0x560092e7dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.120+0000 7fc1755c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e836a0 0x560092e7e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.120+0000 7fc1755c9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1b31e0 con 0x560092e7aac0 2024-04-20T21:36:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.120+0000 7fc1755c9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1b4650 con 0x560092e7a1a0 2024-04-20T21:36:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.120+0000 7fc1755c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c1b38b0 con 0x560092e836a0 2024-04-20T21:36:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.120+0000 7fc1755c9740 10 client.0 _ll_register_callbacks cb 0x7fc164009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e7a1a0 0x560092e7d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e836a0 0x560092e7e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7aac0 0x560092e7dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e7a1a0 0x560092e7d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33896/0 (socket says 172.21.15.22:33896) 2024-04-20T21:36:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e836a0 0x560092e7e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33236/0 (socket says 172.21.15.22:33236) 2024-04-20T21:36:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16b400640 1 -- 172.21.15.22:0/868678816 learned_addr learned my addr 172.21.15.22:0/868678816 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16b400640 1 -- 172.21.15.22:0/868678816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e836a0 msgr2=0x560092e7e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16b400640 1 --2- 172.21.15.22:0/868678816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e836a0 0x560092e7e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16b400640 1 -- 172.21.15.22:0/868678816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7aac0 msgr2=0x560092e7dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16b400640 1 --2- 172.21.15.22:0/868678816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7aac0 0x560092e7dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16b400640 1 -- 172.21.15.22:0/868678816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560092ef9290 con 0x560092e7a1a0 2024-04-20T21:36:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16be00640 1 --2- 172.21.15.22:0/868678816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e836a0 0x560092e7e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc16b400640 1 --2- 172.21.15.22:0/868678816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e7a1a0 0x560092e7d6f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0992a0 tx=0x7fc15c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=895b891c8ba1902d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.121+0000 7fc159600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc159600640 1 -- 172.21.15.22:0/868678816 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c09f030 con 0x560092e7a1a0 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc159600640 1 -- 172.21.15.22:0/868678816 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc15c018420 con 0x560092e7a1a0 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc1755c9740 1 -- 172.21.15.22:0/868678816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc164026410 con 0x560092e7a1a0 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc159600640 1 -- 172.21.15.22:0/868678816 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc15c010e60 con 0x560092e7a1a0 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc1755c9740 10 client.25601 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc1755c9740 1 -- 172.21.15.22:0/868678816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560092efa5a0 con 0x560092e7a1a0 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc159600640 1 -- 172.21.15.22:0/868678816 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc164026410 con 0x560092e7a1a0 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc159600640 1 -- 172.21.15.22:0/868678816 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fc15c0de3a0 con 0x560092e7a1a0 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc1755c9740 10 client.25601 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc1755c9740 10 client.25601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc1755c9740 20 client.25601 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc1755c9740 1 --2- 172.21.15.22:0/868678816 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560092efa950 0x560092efcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc1755c9740 4 client.25601 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:36:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.122+0000 7fc1755c9740 1 -- 172.21.15.22:0/868678816 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560092de79e0 con 0x560092efa950 2024-04-20T21:36:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.123+0000 7fc16aa00640 1 --2- 172.21.15.22:0/868678816 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560092efa950 0x560092efcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.123+0000 7fc16aa00640 1 --2- 172.21.15.22:0/868678816 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560092efa950 0x560092efcd40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aaa4d2c3ab925e7 server_cookie=261bf9ee54fbb32f in_seq=0 out_seq=0 2024-04-20T21:36:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.123+0000 7fc159600640 10 client.25601 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc159600640 1 -- 172.21.15.22:0/868678816 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560092de79e0 con 0x560092efa950 2024-04-20T21:36:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc159600640 10 client.25601 handle_command_reply: tid=0 2024-04-20T21:36:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc1755c9740 1 client.25601 shutdown 2024-04-20T21:36:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc1755c9740 1 -- 172.21.15.22:0/868678816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e7a1a0 msgr2=0x560092e7d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc1755c9740 1 --2- 172.21.15.22:0/868678816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e7a1a0 0x560092e7d6f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0992a0 tx=0x7fc15c000f40 comp rx=0 tx=0).stop 2024-04-20T21:36:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc1755c9740 1 -- 172.21.15.22:0/868678816 shutdown_connections 2024-04-20T21:36:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc1755c9740 1 --2- 172.21.15.22:0/868678816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560092e836a0 0x560092e7e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc1755c9740 1 --2- 172.21.15.22:0/868678816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560092e7aac0 0x560092e7dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc1755c9740 1 --2- 172.21.15.22:0/868678816 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560092efa950 0x560092efcd40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc1755c9740 1 --2- 172.21.15.22:0/868678816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560092e7a1a0 0x560092e7d6f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc1755c9740 1 -- 172.21.15.22:0/868678816 >> 172.21.15.22:0/868678816 conn(0x560092e748d0 msgr2=0x560092e76590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc159600640 0 client.25601 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc1755c9740 1 -- 172.21.15.22:0/868678816 shutdown_connections 2024-04-20T21:36:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc1755c9740 1 -- 172.21.15.22:0/868678816 wait complete. 2024-04-20T21:36:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.124+0000 7fc1755c9740 20 client.25601 trim_cache size 0 max 16384 2024-04-20T21:36:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:36:05.129 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:36:05.129 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:36:05.129 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:36:05.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.125+0000 7fc172800640 1 -- 172.21.15.22:0/1059845471 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc1500a3d20 msgr2=0x7fc1500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.125+0000 7fc172800640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc1500a3d20 0x7fc1500a61e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc164009cc0 tx=0x7fc164005f90 comp rx=0 tx=0).stop 2024-04-20T21:36:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.125+0000 7fc172800640 1 -- 172.21.15.22:0/1059845471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc16c104410 msgr2=0x7fc16c076470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.125+0000 7fc172800640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc16c104410 0x7fc16c076470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00ebc0 tx=0x7fc15c00c6a0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.125+0000 7fc172800640 1 -- 172.21.15.22:0/1059845471 shutdown_connections 2024-04-20T21:36:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.125+0000 7fc172800640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc1500a3d20 0x7fc1500a61e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.125+0000 7fc172800640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc16c076ef0 0x7fc16c1ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.125+0000 7fc172800640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc16c1094e0 0x7fc16c0769b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.125+0000 7fc172800640 1 --2- 172.21.15.22:0/1059845471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc16c104410 0x7fc16c076470 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.125+0000 7fc172800640 1 -- 172.21.15.22:0/1059845471 >> 172.21.15.22:0/1059845471 conn(0x7fc16c0ffcf0 msgr2=0x7fc16c101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.125+0000 7fc172800640 1 -- 172.21.15.22:0/1059845471 shutdown_connections 2024-04-20T21:36:05.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.125+0000 7fc172800640 1 -- 172.21.15.22:0/1059845471 wait complete. 2024-04-20T21:36:05.145 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:36:05.146 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:05.165 INFO:tasks.workunit.client.0.smithi022.stdout: 1 66485 11.31 MB/sec warmup 33 sec latency 14.571 ms 2024-04-20T21:36:05.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.360+0000 7f590a800640 1 -- 172.21.15.22:0/12181980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59041043a0 msgr2=0x7f5904104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.360+0000 7f590a800640 1 --2- 172.21.15.22:0/12181980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59041043a0 0x7f5904104780 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009900 tx=0x7f58f402bc40 comp rx=0 tx=0).stop 2024-04-20T21:36:05.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.360+0000 7f590a800640 1 -- 172.21.15.22:0/12181980 shutdown_connections 2024-04-20T21:36:05.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.360+0000 7f590a800640 1 --2- 172.21.15.22:0/12181980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5904109470 0x7f5904111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.360+0000 7f590a800640 1 --2- 172.21.15.22:0/12181980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5904104d50 0x7f5904108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.360+0000 7f590a800640 1 --2- 172.21.15.22:0/12181980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59041043a0 0x7f5904104780 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.360+0000 7f590a800640 1 -- 172.21.15.22:0/12181980 >> 172.21.15.22:0/12181980 conn(0x7f59040ffc60 msgr2=0x7f5904102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.360+0000 7f590a800640 1 -- 172.21.15.22:0/12181980 shutdown_connections 2024-04-20T21:36:05.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.361+0000 7f590a800640 1 -- 172.21.15.22:0/12181980 wait complete. 2024-04-20T21:36:05.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.361+0000 7f590a800640 1 Processor -- start 2024-04-20T21:36:05.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.362+0000 7f590a800640 1 -- start start 2024-04-20T21:36:05.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.362+0000 7f590a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59041043a0 0x7f59041a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.362+0000 7f590a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5904109470 0x7f59041a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.362+0000 7f590a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f590419a870 0x7f590419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.362+0000 7f590a800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5904114550 con 0x7f590419a870 2024-04-20T21:36:05.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.362+0000 7f590a800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59041143d0 con 0x7f59041043a0 2024-04-20T21:36:05.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.362+0000 7f590a800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59041146d0 con 0x7f5904109470 2024-04-20T21:36:05.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.362+0000 7f5903400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59041043a0 0x7f59041a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.362+0000 7f5903400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59041043a0 0x7f59041a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33914/0 (socket says 172.21.15.22:33914) 2024-04-20T21:36:05.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.362+0000 7f5903400640 1 -- 172.21.15.22:0/262906871 learned_addr learned my addr 172.21.15.22:0/262906871 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:05.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.362+0000 7f5902a00640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5904109470 0x7f59041a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.362+0000 7f5903e00640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f590419a870 0x7f590419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.363+0000 7f5903400640 1 -- 172.21.15.22:0/262906871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5904109470 msgr2=0x7f59041a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.363+0000 7f5903400640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5904109470 0x7f59041a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.363+0000 7f5903400640 1 -- 172.21.15.22:0/262906871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f590419a870 msgr2=0x7f590419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.363+0000 7f5903400640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f590419a870 0x7f590419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.363+0000 7f5903400640 1 -- 172.21.15.22:0/262906871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590419b390 con 0x7f59041043a0 2024-04-20T21:36:05.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.363+0000 7f5903400640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59041043a0 0x7f59041a06c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f58f4005ab0 tx=0x7f58f40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.363+0000 7f58fbe00640 1 -- 172.21.15.22:0/262906871 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4041070 con 0x7f59041043a0 2024-04-20T21:36:05.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.363+0000 7f58fbe00640 1 -- 172.21.15.22:0/262906871 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f402cca0 con 0x7f59041043a0 2024-04-20T21:36:05.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.364+0000 7f590a800640 1 -- 172.21.15.22:0/262906871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590419b620 con 0x7f59041043a0 2024-04-20T21:36:05.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.364+0000 7f58fbe00640 1 -- 172.21.15.22:0/262906871 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4003d90 con 0x7f59041043a0 2024-04-20T21:36:05.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.364+0000 7f590a800640 1 -- 172.21.15.22:0/262906871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59041a7870 con 0x7f59041043a0 2024-04-20T21:36:05.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.365+0000 7f590a800640 1 -- 172.21.15.22:0/262906871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59040691d0 con 0x7f59041043a0 2024-04-20T21:36:05.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.367+0000 7f58fbe00640 1 -- 172.21.15.22:0/262906871 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f58f4044070 con 0x7f59041043a0 2024-04-20T21:36:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.369+0000 7f58fbe00640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f58e80a3da0 0x7f58e80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.369+0000 7f58fbe00640 1 -- 172.21.15.22:0/262906871 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f58f40f1ff0 con 0x7f59041043a0 2024-04-20T21:36:05.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.369+0000 7f5902a00640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f58e80a3da0 0x7f58e80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.370+0000 7f5902a00640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f58e80a3da0 0x7f58e80a6260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5904103370 tx=0x7f58f0005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.373+0000 7f58fbe00640 1 -- 172.21.15.22:0/262906871 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f58f40be0f0 con 0x7f59041043a0 2024-04-20T21:36:05.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.651+0000 7f590a800640 1 -- 172.21.15.22:0/262906871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5904104d50 con 0x7f59041043a0 2024-04-20T21:36:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.652+0000 7f58fbe00640 1 -- 172.21.15.22:0/262906871 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f58f40c23f0 con 0x7f59041043a0 2024-04-20T21:36:05.652 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:36:05.652 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:36:05.652 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:36:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.655+0000 7f590a800640 1 -- 172.21.15.22:0/262906871 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f58e80a3da0 msgr2=0x7f58e80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.655+0000 7f590a800640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f58e80a3da0 0x7f58e80a6260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5904103370 tx=0x7f58f0005f90 comp rx=0 tx=0).stop 2024-04-20T21:36:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.655+0000 7f590a800640 1 -- 172.21.15.22:0/262906871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59041043a0 msgr2=0x7f59041a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.655+0000 7f590a800640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59041043a0 0x7f59041a06c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f58f4005ab0 tx=0x7f58f40042f0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.656+0000 7f590a800640 1 -- 172.21.15.22:0/262906871 shutdown_connections 2024-04-20T21:36:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.656+0000 7f590a800640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f58e80a3da0 0x7f58e80a6260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.656+0000 7f590a800640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f590419a870 0x7f590419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.656+0000 7f590a800640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5904109470 0x7f59041a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.656+0000 7f590a800640 1 --2- 172.21.15.22:0/262906871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59041043a0 0x7f59041a06c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.656+0000 7f590a800640 1 -- 172.21.15.22:0/262906871 >> 172.21.15.22:0/262906871 conn(0x7f59040ffc60 msgr2=0x7f5904101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.656+0000 7f590a800640 1 -- 172.21.15.22:0/262906871 shutdown_connections 2024-04-20T21:36:05.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.656+0000 7f590a800640 1 -- 172.21.15.22:0/262906871 wait complete. 2024-04-20T21:36:05.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:36:05.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.953+0000 7f8147a00640 1 -- 172.21.15.22:0/3525983510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8140104410 msgr2=0x7f81401047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.953+0000 7f8147a00640 1 --2- 172.21.15.22:0/3525983510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8140104410 0x7f81401047f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8134009900 tx=0x7f813402bc30 comp rx=0 tx=0).stop 2024-04-20T21:36:05.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.954+0000 7f8147a00640 1 -- 172.21.15.22:0/3525983510 shutdown_connections 2024-04-20T21:36:05.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.954+0000 7f8147a00640 1 --2- 172.21.15.22:0/3525983510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81401094e0 0x7f81401110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.954+0000 7f8147a00640 1 --2- 172.21.15.22:0/3525983510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8140104dc0 0x7f8140108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.954+0000 7f8147a00640 1 --2- 172.21.15.22:0/3525983510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8140104410 0x7f81401047f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.954+0000 7f8147a00640 1 -- 172.21.15.22:0/3525983510 >> 172.21.15.22:0/3525983510 conn(0x7f81400ffcf0 msgr2=0x7f8140102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.954+0000 7f8147a00640 1 -- 172.21.15.22:0/3525983510 shutdown_connections 2024-04-20T21:36:05.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.954+0000 7f8147a00640 1 -- 172.21.15.22:0/3525983510 wait complete. 2024-04-20T21:36:05.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.955+0000 7f8147a00640 1 Processor -- start 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.955+0000 7f8147a00640 1 -- start start 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8147a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8140104410 0x7f81401a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8147a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81401094e0 0x7f81401a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8147a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814019a940 0x7f814019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8147a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f81401145c0 con 0x7f81401094e0 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8147a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8140114440 con 0x7f8140104410 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8147a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8140114740 con 0x7f814019a940 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8145000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8140104410 0x7f81401a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8145000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8140104410 0x7f81401a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33932/0 (socket says 172.21.15.22:33932) 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8145000640 1 -- 172.21.15.22:0/1801066425 learned_addr learned my addr 172.21.15.22:0/1801066425 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8145a00640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814019a940 0x7f814019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f813fe00640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81401094e0 0x7f81401a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8145000640 1 -- 172.21.15.22:0/1801066425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814019a940 msgr2=0x7f814019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8145000640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814019a940 0x7f814019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8145000640 1 -- 172.21.15.22:0/1801066425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81401094e0 msgr2=0x7f81401a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8145000640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81401094e0 0x7f81401a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8145000640 1 -- 172.21.15.22:0/1801066425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814019b460 con 0x7f8140104410 2024-04-20T21:36:05.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.956+0000 7f8145a00640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814019a940 0x7f814019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:05.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.957+0000 7f8145000640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8140104410 0x7f81401a0740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f813402c240 tx=0x7f81340042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.957+0000 7f813d600640 1 -- 172.21.15.22:0/1801066425 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134043020 con 0x7f8140104410 2024-04-20T21:36:05.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.957+0000 7f813d600640 1 -- 172.21.15.22:0/1801066425 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81340054e0 con 0x7f8140104410 2024-04-20T21:36:05.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.957+0000 7f813d600640 1 -- 172.21.15.22:0/1801066425 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134003b30 con 0x7f8140104410 2024-04-20T21:36:05.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.957+0000 7f8147a00640 1 -- 172.21.15.22:0/1801066425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814019b6f0 con 0x7f8140104410 2024-04-20T21:36:05.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.957+0000 7f8147a00640 1 -- 172.21.15.22:0/1801066425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81401a78a0 con 0x7f8140104410 2024-04-20T21:36:05.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.961+0000 7f814a76a740 1 Processor -- start 2024-04-20T21:36:05.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.961+0000 7f814a76a740 1 -- start start 2024-04-20T21:36:05.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.961+0000 7f813d600640 1 -- 172.21.15.22:0/1801066425 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8134046020 con 0x7f8140104410 2024-04-20T21:36:05.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.962+0000 7f814a76a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca7efb0 0x5653eca7f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.962+0000 7f814a76a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca87f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.962+0000 7f814a76a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca884b0 0x5653eca8a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.962+0000 7f814a76a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5653ec9e4050 con 0x5653eca7efb0 2024-04-20T21:36:05.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.962+0000 7f814a76a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653ec98a490 con 0x5653eca7f8d0 2024-04-20T21:36:05.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.962+0000 7f814a76a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5653ec9e0010 con 0x5653eca884b0 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.962+0000 7f8145000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca87f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.962+0000 7f8145000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca87f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33942/0 (socket says 172.21.15.22:33942) 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.962+0000 7f8145000640 1 -- 172.21.15.22:0/1991554147 learned_addr learned my addr 172.21.15.22:0/1991554147 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.962+0000 7f8145a00640 1 --2- 172.21.15.22:0/1991554147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca7efb0 0x5653eca7f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.962+0000 7f813fe00640 1 --2- 172.21.15.22:0/1991554147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca884b0 0x5653eca8a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.962+0000 7f8145000640 1 -- 172.21.15.22:0/1991554147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca884b0 msgr2=0x5653eca8a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.963+0000 7f8145000640 1 --2- 172.21.15.22:0/1991554147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca884b0 0x5653eca8a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.963+0000 7f8145000640 1 -- 172.21.15.22:0/1991554147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca7efb0 msgr2=0x5653eca7f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.963+0000 7f8145000640 1 --2- 172.21.15.22:0/1991554147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca7efb0 0x5653eca7f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.963+0000 7f8145000640 1 -- 172.21.15.22:0/1991554147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ec992070 con 0x5653eca7f8d0 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.963+0000 7f8145a00640 1 --2- 172.21.15.22:0/1991554147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca7efb0 0x5653eca7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.963+0000 7f813fe00640 1 --2- 172.21.15.22:0/1991554147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca884b0 0x5653eca8a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.963+0000 7f8145000640 1 --2- 172.21.15.22:0/1991554147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca87f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f81340efd10 tx=0x7f81340f8660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b8429591592b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.963+0000 7f8132000640 1 -- 172.21.15.22:0/1991554147 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81340f2df0 con 0x5653eca7f8d0 2024-04-20T21:36:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.963+0000 7f8132000640 1 -- 172.21.15.22:0/1991554147 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81340ff030 con 0x5653eca7f8d0 2024-04-20T21:36:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.963+0000 7f8132000640 1 -- 172.21.15.22:0/1991554147 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134104980 con 0x5653eca7f8d0 2024-04-20T21:36:05.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.963+0000 7f813d600640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f81280a3da0 0x7f81280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.964+0000 7f813d600640 1 -- 172.21.15.22:0/1801066425 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f81340f25f0 con 0x7f8140104410 2024-04-20T21:36:05.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.964+0000 7f8145a00640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f81280a3da0 0x7f81280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.964+0000 7f814a76a740 1 -- 172.21.15.22:0/1991554147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 msgr2=0x5653eca87f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.964+0000 7f814a76a740 1 --2- 172.21.15.22:0/1991554147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca87f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f81340efd10 tx=0x7f81340f8660 comp rx=0 tx=0).stop 2024-04-20T21:36:05.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.964+0000 7f814a76a740 1 -- 172.21.15.22:0/1991554147 shutdown_connections 2024-04-20T21:36:05.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.964+0000 7f814a76a740 1 --2- 172.21.15.22:0/1991554147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca884b0 0x5653eca8a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.964+0000 7f814a76a740 1 --2- 172.21.15.22:0/1991554147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca87f70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.964+0000 7f814a76a740 1 --2- 172.21.15.22:0/1991554147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca7efb0 0x5653eca7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.964+0000 7f814a76a740 1 -- 172.21.15.22:0/1991554147 >> 172.21.15.22:0/1991554147 conn(0x5653eca796e0 msgr2=0x5653eca79ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.964+0000 7f814a76a740 1 -- 172.21.15.22:0/1991554147 shutdown_connections 2024-04-20T21:36:05.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.964+0000 7f814a76a740 1 -- 172.21.15.22:0/1991554147 wait complete. 2024-04-20T21:36:05.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.965+0000 7f8145a00640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f81280a3da0 0x7f81280a6260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8138009fd0 tx=0x7f8138009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.965+0000 7f814a76a740 1 -- start start 2024-04-20T21:36:05.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.965+0000 7f814a76a740 1 Processor -- start 2024-04-20T21:36:05.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f814a76a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca7efb0 0x5653eca854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f814a76a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca85a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f814a76a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca884b0 0x5653eca85f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f814a76a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5653eca9c070 con 0x5653eca884b0 2024-04-20T21:36:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f814a76a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653ec99d860 con 0x5653eca7f8d0 2024-04-20T21:36:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f814a76a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5653eca9c1f0 con 0x5653eca7efb0 2024-04-20T21:36:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f814a76a740 10 client.0 _ll_register_callbacks cb 0x5653ec96bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f813fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca85a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f813fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca85a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33948/0 (socket says 172.21.15.22:33948) 2024-04-20T21:36:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f813fe00640 1 -- 172.21.15.22:0/3438402699 learned_addr learned my addr 172.21.15.22:0/3438402699 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f8145000640 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca7efb0 0x5653eca854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f8145a00640 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca884b0 0x5653eca85f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f813fe00640 1 -- 172.21.15.22:0/3438402699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca7efb0 msgr2=0x5653eca854f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f813fe00640 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca7efb0 0x5653eca854f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f813fe00640 1 -- 172.21.15.22:0/3438402699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca884b0 msgr2=0x5653eca85f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f813fe00640 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca884b0 0x5653eca85f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.966+0000 7f813fe00640 1 -- 172.21.15.22:0/3438402699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ec9a1950 con 0x5653eca7f8d0 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.967+0000 7f813fe00640 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca85a30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f812c013710 tx=0x7f812c013bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1aa3d09187496be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.967+0000 7f8126a00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.967+0000 7f8126a00640 1 -- 172.21.15.22:0/3438402699 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c004460 con 0x5653eca7f8d0 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.967+0000 7f814a76a740 1 -- 172.21.15.22:0/3438402699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81340ff030 con 0x5653eca7f8d0 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.967+0000 7f8126a00640 1 -- 172.21.15.22:0/3438402699 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f812c0119c0 con 0x5653eca7f8d0 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.967+0000 7f8126a00640 1 -- 172.21.15.22:0/3438402699 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f812c011bc0 con 0x5653eca7f8d0 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.967+0000 7f814a76a740 10 client.25619 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.967+0000 7f814a76a740 1 -- 172.21.15.22:0/3438402699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5653ec96f820 con 0x5653eca7f8d0 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.967+0000 7f8126a00640 1 -- 172.21.15.22:0/3438402699 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81340ff030 con 0x5653eca7f8d0 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.968+0000 7f8126a00640 1 -- 172.21.15.22:0/3438402699 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f812c01a070 con 0x5653eca7f8d0 2024-04-20T21:36:05.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.968+0000 7f814a76a740 10 client.25619 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:05.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.968+0000 7f814a76a740 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:05.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.968+0000 7f814a76a740 20 client.25619 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:05.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.968+0000 7f814a76a740 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5653ecaa9170 0x5653ecaab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.968+0000 7f814a76a740 4 client.25619 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:05.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.968+0000 7f814a76a740 1 -- 172.21.15.22:0/3438402699 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653ec9f1a70 con 0x5653ecaa9170 2024-04-20T21:36:05.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.968+0000 7f8145000640 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5653ecaa9170 0x5653ecaab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.969+0000 7f8145000640 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5653ecaa9170 0x5653ecaab610 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca2c4dab1cc7b86 server_cookie=80481a7bfa672953 in_seq=0 out_seq=0 2024-04-20T21:36:05.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.969+0000 7f8126a00640 10 client.25619 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:05.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.969+0000 7f8126a00640 1 -- 172.21.15.22:0/3438402699 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653ec9f1a70 con 0x5653ecaa9170 2024-04-20T21:36:05.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.969+0000 7f8126a00640 10 client.25619 handle_command_reply: tid=0 2024-04-20T21:36:05.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.969+0000 7f814a76a740 1 client.25619 shutdown 2024-04-20T21:36:05.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.970+0000 7f814a76a740 1 -- 172.21.15.22:0/3438402699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 msgr2=0x5653eca85a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.970+0000 7f814a76a740 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca85a30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f812c013710 tx=0x7f812c013bd0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.970+0000 7f814a76a740 1 -- 172.21.15.22:0/3438402699 shutdown_connections 2024-04-20T21:36:05.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.970+0000 7f814a76a740 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca884b0 0x5653eca85f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.970+0000 7f814a76a740 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5653ecaa9170 0x5653ecaab610 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.970+0000 7f814a76a740 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca85a30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.970+0000 7f814a76a740 1 --2- 172.21.15.22:0/3438402699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca7efb0 0x5653eca854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.970+0000 7f8126a00640 0 client.25619 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:05.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.970+0000 7f814a76a740 1 -- 172.21.15.22:0/3438402699 >> 172.21.15.22:0/3438402699 conn(0x5653eca796e0 msgr2=0x5653eca7b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.970+0000 7f814a76a740 1 -- 172.21.15.22:0/3438402699 shutdown_connections 2024-04-20T21:36:05.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.970+0000 7f814a76a740 1 -- 172.21.15.22:0/3438402699 wait complete. 2024-04-20T21:36:05.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.970+0000 7f814a76a740 20 client.25619 trim_cache size 0 max 16384 2024-04-20T21:36:05.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.986+0000 7f814a76a740 1 Processor -- start 2024-04-20T21:36:05.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f814a76a740 1 -- start start 2024-04-20T21:36:05.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f814a76a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca7efb0 0x5653eca87c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f814a76a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653ecaa4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f814a76a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca884b0 0x5653ecab0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f814a76a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8140071ed0 con 0x5653eca7efb0 2024-04-20T21:36:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f814a76a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81401a8030 con 0x5653eca7f8d0 2024-04-20T21:36:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f814a76a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8140072540 con 0x5653eca884b0 2024-04-20T21:36:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f8145a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca884b0 0x5653ecab0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f813fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653ecaa4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f813fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653ecaa4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33964/0 (socket says 172.21.15.22:33964) 2024-04-20T21:36:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f813fe00640 1 -- 172.21.15.22:0/315471775 learned_addr learned my addr 172.21.15.22:0/315471775 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f8145000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca7efb0 0x5653eca87c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.987+0000 7f8145a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca884b0 0x5653ecab0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33296/0 (socket says 172.21.15.22:33296) 2024-04-20T21:36:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.988+0000 7f8145000640 1 -- 172.21.15.22:0/315471775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca884b0 msgr2=0x5653ecab0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.988+0000 7f8145000640 1 --2- 172.21.15.22:0/315471775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca884b0 0x5653ecab0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.988+0000 7f8145000640 1 -- 172.21.15.22:0/315471775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 msgr2=0x5653ecaa4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.988+0000 7f8145000640 1 --2- 172.21.15.22:0/315471775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653ecaa4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.988+0000 7f8145000640 1 -- 172.21.15.22:0/315471775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ec6f63f0 con 0x5653eca7efb0 2024-04-20T21:36:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.988+0000 7f813fe00640 1 --2- 172.21.15.22:0/315471775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653ecaa4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.988+0000 7f8145000640 1 --2- 172.21.15.22:0/315471775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca7efb0 0x5653eca87c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f81340be320 tx=0x7f813402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe0d622fc9530bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.988+0000 7f8131600640 1 -- 172.21.15.22:0/315471775 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81340be3f0 con 0x5653eca7efb0 2024-04-20T21:36:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.988+0000 7f8131600640 1 -- 172.21.15.22:0/315471775 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81340ff070 con 0x5653eca7efb0 2024-04-20T21:36:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.989+0000 7f8131600640 1 -- 172.21.15.22:0/315471775 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81340bdf80 con 0x5653eca7efb0 2024-04-20T21:36:05.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.989+0000 7f814a76a740 1 -- 172.21.15.22:0/315471775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca7efb0 msgr2=0x5653eca87c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.989+0000 7f814a76a740 1 --2- 172.21.15.22:0/315471775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca7efb0 0x5653eca87c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f81340be320 tx=0x7f813402bc30 comp rx=0 tx=0).stop 2024-04-20T21:36:05.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.990+0000 7f814a76a740 1 -- 172.21.15.22:0/315471775 shutdown_connections 2024-04-20T21:36:05.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.990+0000 7f814a76a740 1 --2- 172.21.15.22:0/315471775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca884b0 0x5653ecab0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.990+0000 7f814a76a740 1 --2- 172.21.15.22:0/315471775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653ecaa4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.990+0000 7f814a76a740 1 --2- 172.21.15.22:0/315471775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca7efb0 0x5653eca87c20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.990+0000 7f814a76a740 1 -- 172.21.15.22:0/315471775 >> 172.21.15.22:0/315471775 conn(0x5653eca796e0 msgr2=0x5653eca79ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.990+0000 7f814a76a740 1 -- 172.21.15.22:0/315471775 shutdown_connections 2024-04-20T21:36:05.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.990+0000 7f814a76a740 1 -- 172.21.15.22:0/315471775 wait complete. 2024-04-20T21:36:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.990+0000 7f814a76a740 1 -- start start 2024-04-20T21:36:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f814a76a740 1 Processor -- start 2024-04-20T21:36:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f814a76a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca7efb0 0x5653ecaa17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f814a76a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca81e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f814a76a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca884b0 0x5653eca823f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f814a76a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8140072c10 con 0x5653eca884b0 2024-04-20T21:36:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f814a76a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81401a7a70 con 0x5653eca7f8d0 2024-04-20T21:36:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f814a76a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f81400732e0 con 0x5653eca7efb0 2024-04-20T21:36:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f814a76a740 10 client.0 _ll_register_callbacks cb 0x7f812c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f8145000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca7efb0 0x5653ecaa17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f813fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca81e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f8145000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca7efb0 0x5653ecaa17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33302/0 (socket says 172.21.15.22:33302) 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f8145a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca884b0 0x5653eca823f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.991+0000 7f8145000640 1 -- 172.21.15.22:0/1318210337 learned_addr learned my addr 172.21.15.22:0/1318210337 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.992+0000 7f8145000640 1 -- 172.21.15.22:0/1318210337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 msgr2=0x5653eca81e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.992+0000 7f8145000640 1 --2- 172.21.15.22:0/1318210337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca81e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.992+0000 7f8145000640 1 -- 172.21.15.22:0/1318210337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca884b0 msgr2=0x5653eca823f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.992+0000 7f8145000640 1 --2- 172.21.15.22:0/1318210337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca884b0 0x5653eca823f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.992+0000 7f8145000640 1 -- 172.21.15.22:0/1318210337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ec7c07d0 con 0x5653eca7efb0 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.992+0000 7f813fe00640 1 --2- 172.21.15.22:0/1318210337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca81e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.992+0000 7f8145000640 1 --2- 172.21.15.22:0/1318210337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca7efb0 0x5653ecaa17c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8134036040 tx=0x7f81340f8c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bccbc3b7d8108a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.992+0000 7f8126a00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.992+0000 7f8126a00640 1 -- 172.21.15.22:0/1318210337 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81340be160 con 0x5653eca7efb0 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.992+0000 7f8126a00640 1 -- 172.21.15.22:0/1318210337 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f81340bc920 con 0x5653eca7efb0 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.992+0000 7f814a76a740 1 -- 172.21.15.22:0/1318210337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81340ff070 con 0x5653eca7efb0 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.992+0000 7f8126a00640 1 -- 172.21.15.22:0/1318210337 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f81340dc050 con 0x5653eca7efb0 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.993+0000 7f814a76a740 10 client.25657 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.993+0000 7f814a76a740 1 -- 172.21.15.22:0/1318210337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5653ecb0f640 con 0x5653eca7efb0 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.993+0000 7f8126a00640 1 -- 172.21.15.22:0/1318210337 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81340ff070 con 0x5653eca7efb0 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.993+0000 7f8126a00640 1 -- 172.21.15.22:0/1318210337 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f8134003cd0 con 0x5653eca7efb0 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.993+0000 7f814a76a740 10 client.25657 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.993+0000 7f814a76a740 10 client.25657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.993+0000 7f814a76a740 20 client.25657 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.993+0000 7f814a76a740 1 --2- 172.21.15.22:0/1318210337 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5653ecb0fab0 0x5653ecb11ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.993+0000 7f814a76a740 4 client.25657 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.993+0000 7f814a76a740 1 -- 172.21.15.22:0/1318210337 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653ec9eb8e0 con 0x5653ecb0fab0 2024-04-20T21:36:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.993+0000 7f813fe00640 1 --2- 172.21.15.22:0/1318210337 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5653ecb0fab0 0x5653ecb11ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:05.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.994+0000 7f813fe00640 1 --2- 172.21.15.22:0/1318210337 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5653ecb0fab0 0x5653ecb11ea0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b93b47b0e269c1 server_cookie=20a2f1d650db0ea2 in_seq=0 out_seq=0 2024-04-20T21:36:05.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.994+0000 7f8126a00640 10 client.25657 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:05.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f8126a00640 1 -- 172.21.15.22:0/1318210337 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653ec9eb8e0 con 0x5653ecb0fab0 2024-04-20T21:36:05.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f8126a00640 10 client.25657 handle_command_reply: tid=0 2024-04-20T21:36:05.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f814a76a740 1 client.25657 shutdown 2024-04-20T21:36:05.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f814a76a740 1 -- 172.21.15.22:0/1318210337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca7efb0 msgr2=0x5653ecaa17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f814a76a740 1 --2- 172.21.15.22:0/1318210337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca7efb0 0x5653ecaa17c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8134036040 tx=0x7f81340f8c90 comp rx=0 tx=0).stop 2024-04-20T21:36:05.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f814a76a740 1 -- 172.21.15.22:0/1318210337 shutdown_connections 2024-04-20T21:36:05.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f814a76a740 1 --2- 172.21.15.22:0/1318210337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653eca884b0 0x5653eca823f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f814a76a740 1 --2- 172.21.15.22:0/1318210337 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5653ecb0fab0 0x5653ecb11ea0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f814a76a740 1 --2- 172.21.15.22:0/1318210337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653eca7f8d0 0x5653eca81e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f814a76a740 1 --2- 172.21.15.22:0/1318210337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653eca7efb0 0x5653ecaa17c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f814a76a740 1 -- 172.21.15.22:0/1318210337 >> 172.21.15.22:0/1318210337 conn(0x5653eca796e0 msgr2=0x5653eca77eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f8126a00640 0 client.25657 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:05.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f814a76a740 1 -- 172.21.15.22:0/1318210337 shutdown_connections 2024-04-20T21:36:05.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f814a76a740 1 -- 172.21.15.22:0/1318210337 wait complete. 2024-04-20T21:36:05.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.995+0000 7f814a76a740 20 client.25657 trim_cache size 0 max 16384 2024-04-20T21:36:05.998 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:36:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.996+0000 7f8147a00640 1 -- 172.21.15.22:0/1801066425 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f81280a3da0 msgr2=0x7f81280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.996+0000 7f8147a00640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f81280a3da0 0x7f81280a6260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8138009fd0 tx=0x7f8138009500 comp rx=0 tx=0).stop 2024-04-20T21:36:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.996+0000 7f8147a00640 1 -- 172.21.15.22:0/1801066425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8140104410 msgr2=0x7f81401a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.996+0000 7f8147a00640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8140104410 0x7f81401a0740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f813402c240 tx=0x7f81340042f0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.996+0000 7f8147a00640 1 -- 172.21.15.22:0/1801066425 shutdown_connections 2024-04-20T21:36:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.996+0000 7f8147a00640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f81280a3da0 0x7f81280a6260 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.996+0000 7f8147a00640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814019a940 0x7f814019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.996+0000 7f8147a00640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81401094e0 0x7f81401a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.996+0000 7f8147a00640 1 --2- 172.21.15.22:0/1801066425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8140104410 0x7f81401a0740 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.996+0000 7f8147a00640 1 -- 172.21.15.22:0/1801066425 >> 172.21.15.22:0/1801066425 conn(0x7f81400ffcf0 msgr2=0x7f8140101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.996+0000 7f8147a00640 1 -- 172.21.15.22:0/1801066425 shutdown_connections 2024-04-20T21:36:06.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:05.996+0000 7f8147a00640 1 -- 172.21.15.22:0/1801066425 wait complete. 2024-04-20T21:36:06.165 INFO:tasks.workunit.client.0.smithi022.stdout: 1 68102 11.31 MB/sec warmup 34 sec latency 40.393 ms 2024-04-20T21:36:06.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:06 smithi122 ceph-mon[22147]: pgmap v71: 97 pgs: 97 active+clean; 53 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 3.1 MiB/s wr, 207 op/s 2024-04-20T21:36:06.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:06 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/262906871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:36:06.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:06 smithi033 ceph-mon[21626]: pgmap v71: 97 pgs: 97 active+clean; 53 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 3.1 MiB/s wr, 207 op/s 2024-04-20T21:36:06.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:06 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/262906871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:36:06.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:06 smithi022 ceph-mon[16843]: pgmap v71: 97 pgs: 97 active+clean; 53 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 3.1 MiB/s wr, 207 op/s 2024-04-20T21:36:06.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:06 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/262906871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:36:07.015 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:36:07.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:36:07.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:36:07.165 INFO:tasks.workunit.client.0.smithi022.stdout: 1 69996 11.21 MB/sec warmup 35 sec latency 12.210 ms 2024-04-20T21:36:07.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.255+0000 7ff0d3200640 1 -- 172.21.15.22:0/1209304657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc10e820 msgr2=0x7ff0cc110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.255+0000 7ff0d3200640 1 --2- 172.21.15.22:0/1209304657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc10e820 0x7ff0cc110c10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009ca0 tx=0x7ff0c402c100 comp rx=0 tx=0).stop 2024-04-20T21:36:07.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.255+0000 7ff0d3200640 1 -- 172.21.15.22:0/1209304657 shutdown_connections 2024-04-20T21:36:07.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.255+0000 7ff0d3200640 1 --2- 172.21.15.22:0/1209304657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc10e820 0x7ff0cc110c10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.255+0000 7ff0d3200640 1 --2- 172.21.15.22:0/1209304657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0cc06aaa0 0x7ff0cc06aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.255+0000 7ff0d3200640 1 --2- 172.21.15.22:0/1209304657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0cc102e40 0x7ff0cc06a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.255+0000 7ff0d3200640 1 -- 172.21.15.22:0/1209304657 >> 172.21.15.22:0/1209304657 conn(0x7ff0cc076c60 msgr2=0x7ff0cc077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.255+0000 7ff0d3200640 1 -- 172.21.15.22:0/1209304657 shutdown_connections 2024-04-20T21:36:07.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.255+0000 7ff0d3200640 1 -- 172.21.15.22:0/1209304657 wait complete. 2024-04-20T21:36:07.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0d3200640 1 Processor -- start 2024-04-20T21:36:07.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0d3200640 1 -- start start 2024-04-20T21:36:07.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0d3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0cc06aaa0 0x7ff0cc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0d3200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0cc102e40 0x7ff0cc1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0d3200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc10e820 0x7ff0cc19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0d3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1144d0 con 0x7ff0cc06aaa0 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0d3200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc114350 con 0x7ff0cc10e820 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0d3200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc114650 con 0x7ff0cc102e40 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0cc06aaa0 0x7ff0cc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0c3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0cc102e40 0x7ff0cc1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0d1200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc10e820 0x7ff0cc19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0c3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0cc102e40 0x7ff0cc1a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33306/0 (socket says 172.21.15.22:33306) 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0cc06aaa0 0x7ff0cc1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36986/0 (socket says 172.21.15.22:36986) 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.257+0000 7ff0c3e00640 1 -- 172.21.15.22:0/1396635998 learned_addr learned my addr 172.21.15.22:0/1396635998 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.258+0000 7ff0d1200640 1 -- 172.21.15.22:0/1396635998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0cc102e40 msgr2=0x7ff0cc1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.258+0000 7ff0d1200640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0cc102e40 0x7ff0cc1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.258+0000 7ff0d1200640 1 -- 172.21.15.22:0/1396635998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0cc06aaa0 msgr2=0x7ff0cc1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.258+0000 7ff0d1200640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0cc06aaa0 0x7ff0cc1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.258+0000 7ff0d1200640 1 -- 172.21.15.22:0/1396635998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc19af80 con 0x7ff0cc10e820 2024-04-20T21:36:07.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.258+0000 7ff0c3e00640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0cc102e40 0x7ff0cc1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:07.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.258+0000 7ff0d1200640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc10e820 0x7ff0cc19a7b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4037040 tx=0x7ff0c4035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.258+0000 7ff0c9600640 1 -- 172.21.15.22:0/1396635998 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4039040 con 0x7ff0cc10e820 2024-04-20T21:36:07.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.258+0000 7ff0d3200640 1 -- 172.21.15.22:0/1396635998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0cc19b210 con 0x7ff0cc10e820 2024-04-20T21:36:07.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.259+0000 7ff0c9600640 1 -- 172.21.15.22:0/1396635998 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c4044070 con 0x7ff0cc10e820 2024-04-20T21:36:07.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.259+0000 7ff0c9600640 1 -- 172.21.15.22:0/1396635998 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40042a0 con 0x7ff0cc10e820 2024-04-20T21:36:07.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.259+0000 7ff0d3200640 1 -- 172.21.15.22:0/1396635998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0cc1a7490 con 0x7ff0cc10e820 2024-04-20T21:36:07.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.262+0000 7ff0c9600640 1 -- 172.21.15.22:0/1396635998 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff0c4004500 con 0x7ff0cc10e820 2024-04-20T21:36:07.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.262+0000 7ff0d5f7a740 1 Processor -- start 2024-04-20T21:36:07.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.263+0000 7ff0d5f7a740 1 -- start start 2024-04-20T21:36:07.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.263+0000 7ff0d5f7a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 0x563445d072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.263+0000 7ff0d5f7a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d0fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.263+0000 7ff0d5f7a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d10410 0x563445d12800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.263+0000 7ff0d5f7a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563445c6bd20 con 0x563445d07830 2024-04-20T21:36:07.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.263+0000 7ff0d5f7a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563445c123e0 con 0x563445d06f10 2024-04-20T21:36:07.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.263+0000 7ff0d5f7a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563445c67ce0 con 0x563445d10410 2024-04-20T21:36:07.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.263+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d0fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.263+0000 7ff0c3e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 0x563445d072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.263+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d0fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36992/0 (socket says 172.21.15.22:36992) 2024-04-20T21:36:07.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.263+0000 7ff0cbe00640 1 -- 172.21.15.22:0/3699178860 learned_addr learned my addr 172.21.15.22:0/3699178860 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:07.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.264+0000 7ff0c3e00640 1 --2- 172.21.15.22:0/3699178860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d10410 0x563445d12800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.264+0000 7ff0cbe00640 1 -- 172.21.15.22:0/3699178860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d10410 msgr2=0x563445d12800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.264+0000 7ff0cbe00640 1 --2- 172.21.15.22:0/3699178860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d10410 0x563445d12800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.264+0000 7ff0cbe00640 1 -- 172.21.15.22:0/3699178860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 msgr2=0x563445d072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.264+0000 7ff0cbe00640 1 --2- 172.21.15.22:0/3699178860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 0x563445d072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.264+0000 7ff0cbe00640 1 -- 172.21.15.22:0/3699178860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563445c31980 con 0x563445d07830 2024-04-20T21:36:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.264+0000 7ff0c3e00640 1 --2- 172.21.15.22:0/3699178860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 0x563445d072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.264+0000 7ff0c9600640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff0b00a3d20 0x7ff0b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.264+0000 7ff0cbe00640 1 --2- 172.21.15.22:0/3699178860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d0fed0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009c70 tx=0x7ff0bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9305087c12faa76 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.265+0000 7ff0c9600640 1 -- 172.21.15.22:0/1396635998 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7ff0c40f2a50 con 0x7ff0cc10e820 2024-04-20T21:36:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.265+0000 7ff0c3e00640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff0b00a3d20 0x7ff0b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.265+0000 7ff0c1600640 1 -- 172.21.15.22:0/3699178860 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc004230 con 0x563445d07830 2024-04-20T21:36:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.265+0000 7ff0c1600640 1 -- 172.21.15.22:0/3699178860 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0bc0043d0 con 0x563445d07830 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.265+0000 7ff0c1600640 1 -- 172.21.15.22:0/3699178860 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc019740 con 0x563445d07830 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.266+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/3699178860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 msgr2=0x563445d0fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.266+0000 7ff0c3e00640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff0b00a3d20 0x7ff0b00a61e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x563445d027d0 tx=0x7ff0b4009530 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.266+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/3699178860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d0fed0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009c70 tx=0x7ff0bc011810 comp rx=0 tx=0).stop 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.266+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/3699178860 shutdown_connections 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.266+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/3699178860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d10410 0x563445d12800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.266+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/3699178860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d0fed0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.266+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/3699178860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 0x563445d072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.266+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/3699178860 >> 172.21.15.22:0/3699178860 conn(0x563445d01640 msgr2=0x563445d01a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.266+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/3699178860 shutdown_connections 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.266+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/3699178860 wait complete. 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.266+0000 7ff0d5f7a740 1 -- start start 2024-04-20T21:36:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.267+0000 7ff0d5f7a740 1 Processor -- start 2024-04-20T21:36:07.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.267+0000 7ff0d5f7a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d06f10 0x563445d0d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.267+0000 7ff0d5f7a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d0d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.267+0000 7ff0d5f7a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d10410 0x563445d0de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.267+0000 7ff0d5f7a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563445d13200 con 0x563445d07830 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.267+0000 7ff0d5f7a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563445c257a0 con 0x563445d10410 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.267+0000 7ff0d5f7a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563445d13380 con 0x563445d06f10 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.267+0000 7ff0d5f7a740 10 client.0 _ll_register_callbacks cb 0x563445c27b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.268+0000 7ff0d1200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d10410 0x563445d0de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.268+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d06f10 0x563445d0d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.268+0000 7ff0d1200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d10410 0x563445d0de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34016/0 (socket says 172.21.15.22:34016) 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.268+0000 7ff0d1200640 1 -- 172.21.15.22:0/2267092409 learned_addr learned my addr 172.21.15.22:0/2267092409 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.268+0000 7ff0c3e00640 1 --2- 172.21.15.22:0/2267092409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d0d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.268+0000 7ff0d1200640 1 -- 172.21.15.22:0/2267092409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d06f10 msgr2=0x563445d0d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.268+0000 7ff0d1200640 1 --2- 172.21.15.22:0/2267092409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d06f10 0x563445d0d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.268+0000 7ff0d1200640 1 -- 172.21.15.22:0/2267092409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 msgr2=0x563445d0d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.268+0000 7ff0d1200640 1 --2- 172.21.15.22:0/2267092409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d0d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.268+0000 7ff0d1200640 1 -- 172.21.15.22:0/2267092409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563445a467e0 con 0x563445d10410 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.268+0000 7ff0d1200640 1 --2- 172.21.15.22:0/2267092409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d10410 0x563445d0de60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40bea70 tx=0x7ff0c40be970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a195e49d69531f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.269+0000 7ff0ba000640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.269+0000 7ff0ba000640 1 -- 172.21.15.22:0/2267092409 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4032790 con 0x563445d10410 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.269+0000 7ff0ba000640 1 -- 172.21.15.22:0/2267092409 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7ff0c40bb150 con 0x563445d10410 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.269+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/2267092409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0bc0043d0 con 0x563445d10410 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.269+0000 7ff0ba000640 1 -- 172.21.15.22:0/2267092409 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff0c40bf1e0 con 0x563445d10410 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.269+0000 7ff0d5f7a740 10 client.25637 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.269+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/2267092409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563445aa6f70 con 0x563445d10410 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.269+0000 7ff0ba000640 1 -- 172.21.15.22:0/2267092409 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0bc0043d0 con 0x563445d10410 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.270+0000 7ff0ba000640 1 -- 172.21.15.22:0/2267092409 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7ff0c403c030 con 0x563445d10410 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.270+0000 7ff0d5f7a740 10 client.25637 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.270+0000 7ff0d5f7a740 10 client.25637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.270+0000 7ff0d5f7a740 20 client.25637 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.270+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/2267092409 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563445d205c0 0x563445d22a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.270+0000 7ff0d5f7a740 4 client.25637 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.270+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/2267092409 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563445c79740 con 0x563445d205c0 2024-04-20T21:36:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.270+0000 7ff0cbe00640 1 --2- 172.21.15.22:0/2267092409 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563445d205c0 0x563445d22a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.271+0000 7ff0cbe00640 1 --2- 172.21.15.22:0/2267092409 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563445d205c0 0x563445d22a30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919293dc960af41a server_cookie=3df57da688438a21 in_seq=0 out_seq=0 2024-04-20T21:36:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.271+0000 7ff0ba000640 10 client.25637 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.272+0000 7ff0ba000640 1 -- 172.21.15.22:0/2267092409 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563445c79740 con 0x563445d205c0 2024-04-20T21:36:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.272+0000 7ff0ba000640 10 client.25637 handle_command_reply: tid=0 2024-04-20T21:36:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.272+0000 7ff0d5f7a740 1 client.25637 shutdown 2024-04-20T21:36:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.273+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/2267092409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d10410 msgr2=0x563445d0de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.273+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/2267092409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d10410 0x563445d0de60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40bea70 tx=0x7ff0c40be970 comp rx=0 tx=0).stop 2024-04-20T21:36:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.273+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/2267092409 shutdown_connections 2024-04-20T21:36:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.273+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/2267092409 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563445d205c0 0x563445d22a30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.273+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/2267092409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d10410 0x563445d0de60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.273+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/2267092409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d0d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.273+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/2267092409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d06f10 0x563445d0d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.273+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/2267092409 >> 172.21.15.22:0/2267092409 conn(0x563445d01640 msgr2=0x563445d03390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.273+0000 7ff0ba000640 0 client.25637 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.273+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/2267092409 shutdown_connections 2024-04-20T21:36:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.273+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/2267092409 wait complete. 2024-04-20T21:36:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.273+0000 7ff0d5f7a740 20 client.25637 trim_cache size 0 max 16384 2024-04-20T21:36:07.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.296+0000 7ff0d5f7a740 1 Processor -- start 2024-04-20T21:36:07.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.296+0000 7ff0d5f7a740 1 -- start start 2024-04-20T21:36:07.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.296+0000 7ff0d5f7a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 0x563445d0fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.296+0000 7ff0d5f7a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d1c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.296+0000 7ff0d5f7a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d10410 0x563445d004d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.296+0000 7ff0d5f7a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1ade40 con 0x563445d07830 2024-04-20T21:36:07.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.296+0000 7ff0d5f7a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1b06c0 con 0x563445d06f10 2024-04-20T21:36:07.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.296+0000 7ff0d5f7a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1ae4b0 con 0x563445d10410 2024-04-20T21:36:07.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.297+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 0x563445d0fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.297+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 0x563445d0fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34028/0 (socket says 172.21.15.22:34028) 2024-04-20T21:36:07.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.297+0000 7ff0d1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d10410 0x563445d004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.297+0000 7ff0cbe00640 1 -- 172.21.15.22:0/1844950004 learned_addr learned my addr 172.21.15.22:0/1844950004 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:07.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.297+0000 7ff0c3e00640 1 --2- 172.21.15.22:0/1844950004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d1c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.297+0000 7ff0cbe00640 1 -- 172.21.15.22:0/1844950004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d10410 msgr2=0x563445d004d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.297+0000 7ff0cbe00640 1 --2- 172.21.15.22:0/1844950004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d10410 0x563445d004d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.297+0000 7ff0cbe00640 1 -- 172.21.15.22:0/1844950004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 msgr2=0x563445d1c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.297+0000 7ff0cbe00640 1 --2- 172.21.15.22:0/1844950004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d1c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.297+0000 7ff0cbe00640 1 -- 172.21.15.22:0/1844950004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56344593e170 con 0x563445d06f10 2024-04-20T21:36:07.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.297+0000 7ff0d1200640 1 --2- 172.21.15.22:0/1844950004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d10410 0x563445d004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:07.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.298+0000 7ff0cbe00640 1 --2- 172.21.15.22:0/1844950004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 0x563445d0fb10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009bc0 tx=0x7ff0bc003ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=639023060997e01b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.298+0000 7ff0c0c00640 1 -- 172.21.15.22:0/1844950004 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc009e20 con 0x563445d06f10 2024-04-20T21:36:07.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.298+0000 7ff0c0c00640 1 -- 172.21.15.22:0/1844950004 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0bc005510 con 0x563445d06f10 2024-04-20T21:36:07.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.298+0000 7ff0c0c00640 1 -- 172.21.15.22:0/1844950004 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc003850 con 0x563445d06f10 2024-04-20T21:36:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.299+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1844950004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 msgr2=0x563445d0fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.299+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/1844950004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 0x563445d0fb10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009bc0 tx=0x7ff0bc003ed0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.299+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1844950004 shutdown_connections 2024-04-20T21:36:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.299+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/1844950004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d10410 0x563445d004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.299+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/1844950004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d1c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.299+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/1844950004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d06f10 0x563445d0fb10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.299+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1844950004 >> 172.21.15.22:0/1844950004 conn(0x563445d01640 msgr2=0x563445cffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.299+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1844950004 shutdown_connections 2024-04-20T21:36:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.299+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1844950004 wait complete. 2024-04-20T21:36:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.299+0000 7ff0d5f7a740 1 -- start start 2024-04-20T21:36:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.300+0000 7ff0d5f7a740 1 Processor -- start 2024-04-20T21:36:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.300+0000 7ff0d5f7a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d06f10 0x563445d18850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.300+0000 7ff0d5f7a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d13c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.300+0000 7ff0d5f7a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d10410 0x563445d14150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.300+0000 7ff0d5f7a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1aeb80 con 0x563445d07830 2024-04-20T21:36:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.300+0000 7ff0d5f7a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1afff0 con 0x563445d10410 2024-04-20T21:36:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.300+0000 7ff0d5f7a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1af250 con 0x563445d06f10 2024-04-20T21:36:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.300+0000 7ff0d5f7a740 10 client.0 _ll_register_callbacks cb 0x7ff0c40bad80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.301+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d06f10 0x563445d18850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.301+0000 7ff0c3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d13c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.301+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d06f10 0x563445d18850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33342/0 (socket says 172.21.15.22:33342) 2024-04-20T21:36:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.301+0000 7ff0cbe00640 1 -- 172.21.15.22:0/1284296435 learned_addr learned my addr 172.21.15.22:0/1284296435 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.301+0000 7ff0d1200640 1 --2- 172.21.15.22:0/1284296435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d10410 0x563445d14150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.301+0000 7ff0c3e00640 1 -- 172.21.15.22:0/1284296435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d06f10 msgr2=0x563445d18850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.301+0000 7ff0c3e00640 1 --2- 172.21.15.22:0/1284296435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d06f10 0x563445d18850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.301+0000 7ff0c3e00640 1 -- 172.21.15.22:0/1284296435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d10410 msgr2=0x563445d14150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.301+0000 7ff0c3e00640 1 --2- 172.21.15.22:0/1284296435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d10410 0x563445d14150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.301+0000 7ff0c3e00640 1 -- 172.21.15.22:0/1284296435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563445d147f0 con 0x563445d07830 2024-04-20T21:36:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.302+0000 7ff0c3e00640 1 --2- 172.21.15.22:0/1284296435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d13c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x563445d0e9b0 tx=0x7ff0b4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a16bd6a4da558e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.302+0000 7ff0ba000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.302+0000 7ff0ba000640 1 -- 172.21.15.22:0/1284296435 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4004020 con 0x563445d07830 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.302+0000 7ff0ba000640 1 -- 172.21.15.22:0/1284296435 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7ff0b4009040 con 0x563445d07830 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.302+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1284296435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563445c735b0 con 0x563445d07830 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.302+0000 7ff0ba000640 1 -- 172.21.15.22:0/1284296435 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff0b40041c0 con 0x563445d07830 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.302+0000 7ff0d5f7a740 10 client.15528 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.302+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1284296435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563445d8f270 con 0x563445d07830 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.303+0000 7ff0ba000640 1 -- 172.21.15.22:0/1284296435 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563445c735b0 con 0x563445d07830 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.303+0000 7ff0ba000640 1 -- 172.21.15.22:0/1284296435 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7ff0b402b070 con 0x563445d07830 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.303+0000 7ff0d5f7a740 10 client.15528 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.303+0000 7ff0d5f7a740 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.303+0000 7ff0d5f7a740 20 client.15528 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.303+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/1284296435 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563445d8fa40 0x563445d91e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.303+0000 7ff0d5f7a740 4 client.15528 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:36:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.303+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1284296435 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563445befb10 con 0x563445d8fa40 2024-04-20T21:36:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.303+0000 7ff0cbe00640 1 --2- 172.21.15.22:0/1284296435 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563445d8fa40 0x563445d91e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.304+0000 7ff0cbe00640 1 --2- 172.21.15.22:0/1284296435 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563445d8fa40 0x563445d91e60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c70783bc609d42b server_cookie=376d6a49830511e6 in_seq=0 out_seq=0 2024-04-20T21:36:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.304+0000 7ff0ba000640 10 client.15528 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:07.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.307+0000 7ff0ba000640 1 -- 172.21.15.22:0/1284296435 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563445befb10 con 0x563445d8fa40 2024-04-20T21:36:07.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.307+0000 7ff0ba000640 10 client.15528 handle_command_reply: tid=0 2024-04-20T21:36:07.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.307+0000 7ff0d5f7a740 1 client.15528 shutdown 2024-04-20T21:36:07.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.307+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1284296435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 msgr2=0x563445d13c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.307+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/1284296435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d13c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x563445d0e9b0 tx=0x7ff0b4005f90 comp rx=0 tx=0).stop 2024-04-20T21:36:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.308+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1284296435 shutdown_connections 2024-04-20T21:36:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.308+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/1284296435 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563445d8fa40 0x563445d91e60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.308+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/1284296435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563445d10410 0x563445d14150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.308+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/1284296435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563445d07830 0x563445d13c10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.308+0000 7ff0d5f7a740 1 --2- 172.21.15.22:0/1284296435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563445d06f10 0x563445d18850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.308+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1284296435 >> 172.21.15.22:0/1284296435 conn(0x563445d01640 msgr2=0x563445d00650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.308+0000 7ff0ba000640 0 client.15528 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:07.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.308+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1284296435 shutdown_connections 2024-04-20T21:36:07.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.308+0000 7ff0d5f7a740 1 -- 172.21.15.22:0/1284296435 wait complete. 2024-04-20T21:36:07.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.308+0000 7ff0d5f7a740 20 client.15528 trim_cache size 0 max 16384 2024-04-20T21:36:07.308 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:36:07.308 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:36:07.308 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "2c5e9550-35d6-44e0-bc5b-ec4c0088a815", 2024-04-20T21:36:07.309 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:36:07.309 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:36:07.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.309+0000 7ff0d3200640 1 -- 172.21.15.22:0/1396635998 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff0b00a3d20 msgr2=0x7ff0b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.309+0000 7ff0d3200640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff0b00a3d20 0x7ff0b00a61e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x563445d027d0 tx=0x7ff0b4009530 comp rx=0 tx=0).stop 2024-04-20T21:36:07.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.309+0000 7ff0d3200640 1 -- 172.21.15.22:0/1396635998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc10e820 msgr2=0x7ff0cc19a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.309+0000 7ff0d3200640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc10e820 0x7ff0cc19a7b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4037040 tx=0x7ff0c4035d10 comp rx=0 tx=0).stop 2024-04-20T21:36:07.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.309+0000 7ff0d3200640 1 -- 172.21.15.22:0/1396635998 shutdown_connections 2024-04-20T21:36:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.309+0000 7ff0d3200640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff0b00a3d20 0x7ff0b00a61e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.309+0000 7ff0d3200640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc10e820 0x7ff0cc19a7b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.309+0000 7ff0d3200640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0cc102e40 0x7ff0cc1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.309+0000 7ff0d3200640 1 --2- 172.21.15.22:0/1396635998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0cc06aaa0 0x7ff0cc1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.309+0000 7ff0d3200640 1 -- 172.21.15.22:0/1396635998 >> 172.21.15.22:0/1396635998 conn(0x7ff0cc076c60 msgr2=0x7ff0cc10f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.309+0000 7ff0d3200640 1 -- 172.21.15.22:0/1396635998 shutdown_connections 2024-04-20T21:36:07.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.309+0000 7ff0d3200640 1 -- 172.21.15.22:0/1396635998 wait complete. 2024-04-20T21:36:07.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:36:07.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:36:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.547+0000 7fc073200640 1 -- 172.21.15.22:0/103630967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074108500 msgr2=0x7fc074108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.547+0000 7fc073200640 1 --2- 172.21.15.22:0/103630967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074108500 0x7fc074108960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc05c009860 tx=0x7fc05c02bc00 comp rx=0 tx=0).stop 2024-04-20T21:36:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.547+0000 7fc073200640 1 -- 172.21.15.22:0/103630967 shutdown_connections 2024-04-20T21:36:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.547+0000 7fc073200640 1 --2- 172.21.15.22:0/103630967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc074108ea0 0x7fc0741152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.547+0000 7fc073200640 1 --2- 172.21.15.22:0/103630967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074108500 0x7fc074108960 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.547+0000 7fc073200640 1 --2- 172.21.15.22:0/103630967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc07410d3d0 0x7fc07410d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.547+0000 7fc073200640 1 -- 172.21.15.22:0/103630967 >> 172.21.15.22:0/103630967 conn(0x7fc074077d80 msgr2=0x7fc074106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.548+0000 7fc073200640 1 -- 172.21.15.22:0/103630967 shutdown_connections 2024-04-20T21:36:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.548+0000 7fc073200640 1 -- 172.21.15.22:0/103630967 wait complete. 2024-04-20T21:36:07.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.549+0000 7fc073200640 1 Processor -- start 2024-04-20T21:36:07.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.549+0000 7fc073200640 1 -- start start 2024-04-20T21:36:07.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.550+0000 7fc073200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074108500 0x7fc0741a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.550+0000 7fc073200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074108ea0 0x7fc0741a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.550+0000 7fc073200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07410d3d0 0x7fc07419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.550+0000 7fc073200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741187f0 con 0x7fc074108ea0 2024-04-20T21:36:07.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.550+0000 7fc073200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc074118670 con 0x7fc07410d3d0 2024-04-20T21:36:07.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.550+0000 7fc073200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc074118970 con 0x7fc074108500 2024-04-20T21:36:07.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.550+0000 7fc072800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07410d3d0 0x7fc07419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.550+0000 7fc072800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07410d3d0 0x7fc07419eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34070/0 (socket says 172.21.15.22:34070) 2024-04-20T21:36:07.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.550+0000 7fc072800640 1 -- 172.21.15.22:0/3646872223 learned_addr learned my addr 172.21.15.22:0/3646872223 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:07.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.550+0000 7fc071e00640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074108500 0x7fc0741a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.551+0000 7fc06be00640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074108ea0 0x7fc0741a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.551+0000 7fc072800640 1 -- 172.21.15.22:0/3646872223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074108500 msgr2=0x7fc0741a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.551+0000 7fc072800640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074108500 0x7fc0741a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.551+0000 7fc072800640 1 -- 172.21.15.22:0/3646872223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074108ea0 msgr2=0x7fc0741a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.551+0000 7fc072800640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074108ea0 0x7fc0741a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.551+0000 7fc072800640 1 -- 172.21.15.22:0/3646872223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07419f2d0 con 0x7fc07410d3d0 2024-04-20T21:36:07.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.551+0000 7fc06be00640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074108ea0 0x7fc0741a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:07.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.551+0000 7fc072800640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07410d3d0 0x7fc07419eb50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00ef00 tx=0x7fc06c00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.552+0000 7fc06aa00640 1 -- 172.21.15.22:0/3646872223 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c003970 con 0x7fc07410d3d0 2024-04-20T21:36:07.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.552+0000 7fc073200640 1 -- 172.21.15.22:0/3646872223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07419f5c0 con 0x7fc07410d3d0 2024-04-20T21:36:07.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.552+0000 7fc073200640 1 -- 172.21.15.22:0/3646872223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0741ab810 con 0x7fc07410d3d0 2024-04-20T21:36:07.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.552+0000 7fc06aa00640 1 -- 172.21.15.22:0/3646872223 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc06c009880 con 0x7fc07410d3d0 2024-04-20T21:36:07.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.552+0000 7fc06aa00640 1 -- 172.21.15.22:0/3646872223 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00b8a0 con 0x7fc07410d3d0 2024-04-20T21:36:07.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.555+0000 7fc07bfe4740 1 Processor -- start 2024-04-20T21:36:07.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.556+0000 7fc06aa00640 1 -- 172.21.15.22:0/3646872223 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc06c01d020 con 0x7fc07410d3d0 2024-04-20T21:36:07.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.556+0000 7fc07bfe4740 1 -- start start 2024-04-20T21:36:07.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.556+0000 7fc07bfe4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699380 0x5585ae699760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.556+0000 7fc07bfe4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699ca0 0x5585ae6a2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.556+0000 7fc07bfe4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae6a2880 0x5585ae6a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.556+0000 7fc07bfe4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5585ae5ff150 con 0x5585ae699ca0 2024-04-20T21:36:07.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.556+0000 7fc07bfe4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5585ae5a5370 con 0x5585ae6a2880 2024-04-20T21:36:07.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.556+0000 7fc07bfe4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5585ae5fb110 con 0x5585ae699380 2024-04-20T21:36:07.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.557+0000 7fc071e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699ca0 0x5585ae6a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.557+0000 7fc06be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae6a2880 0x5585ae6a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.557+0000 7fc071e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699ca0 0x5585ae6a2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37044/0 (socket says 172.21.15.22:37044) 2024-04-20T21:36:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.557+0000 7fc071e00640 1 -- 172.21.15.22:0/2134076832 learned_addr learned my addr 172.21.15.22:0/2134076832 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.557+0000 7fc06be00640 1 --2- 172.21.15.22:0/2134076832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699380 0x5585ae699760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.557+0000 7fc071e00640 1 -- 172.21.15.22:0/2134076832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699380 msgr2=0x5585ae699760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.557+0000 7fc071e00640 1 --2- 172.21.15.22:0/2134076832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699380 0x5585ae699760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.557+0000 7fc071e00640 1 -- 172.21.15.22:0/2134076832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae6a2880 msgr2=0x5585ae6a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.557+0000 7fc071e00640 1 --2- 172.21.15.22:0/2134076832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae6a2880 0x5585ae6a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.557+0000 7fc071e00640 1 -- 172.21.15.22:0/2134076832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ae3db7d0 con 0x5585ae699ca0 2024-04-20T21:36:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.558+0000 7fc06aa00640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc0580a3da0 0x7fc0580a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.558+0000 7fc071e00640 1 --2- 172.21.15.22:0/2134076832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699ca0 0x5585ae6a2340 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc064009c70 tx=0x7fc064011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92dffc8f1338c6c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.558+0000 7fc06aa00640 1 -- 172.21.15.22:0/3646872223 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc06c0d04b0 con 0x7fc07410d3d0 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.558+0000 7fc06be00640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc0580a3da0 0x7fc0580a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.558+0000 7fc063400640 1 -- 172.21.15.22:0/2134076832 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064004230 con 0x5585ae699ca0 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.558+0000 7fc063400640 1 -- 172.21.15.22:0/2134076832 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0640043d0 con 0x5585ae699ca0 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.558+0000 7fc063400640 1 -- 172.21.15.22:0/2134076832 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064019820 con 0x5585ae699ca0 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.559+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2134076832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699ca0 msgr2=0x5585ae6a2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.559+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2134076832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699ca0 0x5585ae6a2340 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc064009c70 tx=0x7fc064011810 comp rx=0 tx=0).stop 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.559+0000 7fc06be00640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc0580a3da0 0x7fc0580a6260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc05c005b60 tx=0x7fc05c005750 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.559+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2134076832 shutdown_connections 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.559+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2134076832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae6a2880 0x5585ae6a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.559+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2134076832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699ca0 0x5585ae6a2340 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.559+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2134076832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699380 0x5585ae699760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.559+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2134076832 >> 172.21.15.22:0/2134076832 conn(0x5585ae693ab0 msgr2=0x5585ae693e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.560+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2134076832 shutdown_connections 2024-04-20T21:36:07.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.560+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2134076832 wait complete. 2024-04-20T21:36:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.560+0000 7fc07bfe4740 1 -- start start 2024-04-20T21:36:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.560+0000 7fc07bfe4740 1 Processor -- start 2024-04-20T21:36:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.561+0000 7fc07bfe4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699380 0x5585ae69f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.561+0000 7fc07bfe4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699ca0 0x5585ae69fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.561+0000 7fc07bfe4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae6a2880 0x5585ae6a0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.561+0000 7fc07bfe4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5585ae6a5670 con 0x5585ae6a2880 2024-04-20T21:36:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.561+0000 7fc07bfe4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5585ae5b8740 con 0x5585ae699380 2024-04-20T21:36:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.561+0000 7fc07bfe4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5585ae6a57f0 con 0x5585ae699ca0 2024-04-20T21:36:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.561+0000 7fc07bfe4740 10 client.0 _ll_register_callbacks cb 0x5585ae5b3b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.561+0000 7fc071e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699380 0x5585ae69f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.561+0000 7fc071e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699380 0x5585ae69f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34082/0 (socket says 172.21.15.22:34082) 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.561+0000 7fc071e00640 1 -- 172.21.15.22:0/4172729394 learned_addr learned my addr 172.21.15.22:0/4172729394 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.561+0000 7fc072800640 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae6a2880 0x5585ae6a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.561+0000 7fc06be00640 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699ca0 0x5585ae69fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.562+0000 7fc071e00640 1 -- 172.21.15.22:0/4172729394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699ca0 msgr2=0x5585ae69fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.562+0000 7fc071e00640 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699ca0 0x5585ae69fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.562+0000 7fc071e00640 1 -- 172.21.15.22:0/4172729394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae6a2880 msgr2=0x5585ae6a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.562+0000 7fc071e00640 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae6a2880 0x5585ae6a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.562+0000 7fc071e00640 1 -- 172.21.15.22:0/4172729394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ae3113f0 con 0x5585ae699380 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.562+0000 7fc06be00640 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699ca0 0x5585ae69fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.562+0000 7fc071e00640 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699380 0x5585ae69f880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc064009560 tx=0x7fc064005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fab3efd48bd5682 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.562+0000 7fc057e00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.562+0000 7fc057e00640 1 -- 172.21.15.22:0/4172729394 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064027070 con 0x5585ae699380 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.562+0000 7fc057e00640 1 -- 172.21.15.22:0/4172729394 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc064003800 con 0x5585ae699380 2024-04-20T21:36:07.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.562+0000 7fc07bfe4740 1 -- 172.21.15.22:0/4172729394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0640043d0 con 0x5585ae699380 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.562+0000 7fc057e00640 1 -- 172.21.15.22:0/4172729394 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc064033050 con 0x5585ae699380 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.563+0000 7fc07bfe4740 10 client.25655 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.563+0000 7fc07bfe4740 1 -- 172.21.15.22:0/4172729394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5585ae2d10f0 con 0x5585ae699380 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.563+0000 7fc057e00640 1 -- 172.21.15.22:0/4172729394 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0640043d0 con 0x5585ae699380 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.563+0000 7fc057e00640 1 -- 172.21.15.22:0/4172729394 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fc064018070 con 0x5585ae699380 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.563+0000 7fc07bfe4740 10 client.25655 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.563+0000 7fc07bfe4740 10 client.25655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.563+0000 7fc07bfe4740 20 client.25655 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.563+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5585ae6b2b10 0x5585ae6b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.563+0000 7fc07bfe4740 4 client.25655 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.563+0000 7fc07bfe4740 1 -- 172.21.15.22:0/4172729394 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585ae60cb70 con 0x5585ae6b2b10 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.563+0000 7fc06be00640 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5585ae6b2b10 0x5585ae6b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.564+0000 7fc06be00640 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5585ae6b2b10 0x5585ae6b4f00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d9838cf73181b8 server_cookie=144c4024e2ddd02b in_seq=0 out_seq=0 2024-04-20T21:36:07.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.564+0000 7fc057e00640 10 client.25655 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.565+0000 7fc057e00640 1 -- 172.21.15.22:0/4172729394 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585ae60cb70 con 0x5585ae6b2b10 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.565+0000 7fc057e00640 10 client.25655 handle_command_reply: tid=0 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.565+0000 7fc07bfe4740 1 client.25655 shutdown 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.565+0000 7fc07bfe4740 1 -- 172.21.15.22:0/4172729394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699380 msgr2=0x5585ae69f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.565+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699380 0x5585ae69f880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc064009560 tx=0x7fc064005760 comp rx=0 tx=0).stop 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.565+0000 7fc07bfe4740 1 -- 172.21.15.22:0/4172729394 shutdown_connections 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.565+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae6a2880 0x5585ae6a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.565+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699ca0 0x5585ae69fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.565+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5585ae6b2b10 0x5585ae6b4f00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.565+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/4172729394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699380 0x5585ae69f880 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.565+0000 7fc07bfe4740 1 -- 172.21.15.22:0/4172729394 >> 172.21.15.22:0/4172729394 conn(0x5585ae693ab0 msgr2=0x5585ae695800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.565+0000 7fc057e00640 0 client.25655 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:07.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.566+0000 7fc07bfe4740 1 -- 172.21.15.22:0/4172729394 shutdown_connections 2024-04-20T21:36:07.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.566+0000 7fc07bfe4740 1 -- 172.21.15.22:0/4172729394 wait complete. 2024-04-20T21:36:07.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.566+0000 7fc07bfe4740 20 client.25655 trim_cache size 0 max 16384 2024-04-20T21:36:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.588+0000 7fc07bfe4740 1 Processor -- start 2024-04-20T21:36:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.588+0000 7fc07bfe4740 1 -- start start 2024-04-20T21:36:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.588+0000 7fc07bfe4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699380 0x5585ae6a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.588+0000 7fc07bfe4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699ca0 0x5585ae6ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.588+0000 7fc07bfe4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae6a2880 0x5585ae6ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.588+0000 7fc07bfe4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc05000c830 con 0x5585ae699380 2024-04-20T21:36:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.588+0000 7fc07bfe4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc05000f0b0 con 0x5585ae699ca0 2024-04-20T21:36:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.588+0000 7fc07bfe4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc05000cea0 con 0x5585ae6a2880 2024-04-20T21:36:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.588+0000 7fc06be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699ca0 0x5585ae6ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.588+0000 7fc06be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699ca0 0x5585ae6ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34094/0 (socket says 172.21.15.22:34094) 2024-04-20T21:36:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.588+0000 7fc06be00640 1 -- 172.21.15.22:0/2498885983 learned_addr learned my addr 172.21.15.22:0/2498885983 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.588+0000 7fc072800640 1 --2- 172.21.15.22:0/2498885983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae6a2880 0x5585ae6ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.589+0000 7fc071e00640 1 --2- 172.21.15.22:0/2498885983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699380 0x5585ae6a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.589+0000 7fc06be00640 1 -- 172.21.15.22:0/2498885983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae6a2880 msgr2=0x5585ae6ba6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.589+0000 7fc06be00640 1 --2- 172.21.15.22:0/2498885983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae6a2880 0x5585ae6ba6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.589+0000 7fc06be00640 1 -- 172.21.15.22:0/2498885983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699380 msgr2=0x5585ae6a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.589+0000 7fc06be00640 1 --2- 172.21.15.22:0/2498885983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699380 0x5585ae6a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.589+0000 7fc06be00640 1 -- 172.21.15.22:0/2498885983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ae699ad0 con 0x5585ae699ca0 2024-04-20T21:36:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.589+0000 7fc072800640 1 --2- 172.21.15.22:0/2498885983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae6a2880 0x5585ae6ba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:07.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.589+0000 7fc06be00640 1 --2- 172.21.15.22:0/2498885983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699ca0 0x5585ae6ae590 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0057c0 tx=0x7fc05c02bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf2ecf01b778ade0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.589+0000 7fc062a00640 1 -- 172.21.15.22:0/2498885983 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c057070 con 0x5585ae699ca0 2024-04-20T21:36:07.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.590+0000 7fc062a00640 1 -- 172.21.15.22:0/2498885983 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc05c03a040 con 0x5585ae699ca0 2024-04-20T21:36:07.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.590+0000 7fc062a00640 1 -- 172.21.15.22:0/2498885983 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c052110 con 0x5585ae699ca0 2024-04-20T21:36:07.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.590+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2498885983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699ca0 msgr2=0x5585ae6ae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.590+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2498885983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699ca0 0x5585ae6ae590 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0057c0 tx=0x7fc05c02bc00 comp rx=0 tx=0).stop 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.590+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2498885983 shutdown_connections 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.590+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2498885983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae6a2880 0x5585ae6ba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.590+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2498885983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae699ca0 0x5585ae6ae590 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.590+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2498885983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699380 0x5585ae6a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.590+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2498885983 >> 172.21.15.22:0/2498885983 conn(0x5585ae693ab0 msgr2=0x5585ae692b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.591+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2498885983 shutdown_connections 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.591+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2498885983 wait complete. 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.591+0000 7fc07bfe4740 1 -- start start 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.591+0000 7fc07bfe4740 1 Processor -- start 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc07bfe4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699380 0x5585ae6aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc07bfe4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699ca0 0x5585ae6a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc07bfe4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae6a2880 0x5585ae6a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc07bfe4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc05000d570 con 0x5585ae699ca0 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc07bfe4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc05000e9e0 con 0x5585ae6a2880 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc07bfe4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc05000dc40 con 0x5585ae699380 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc07bfe4740 10 client.0 _ll_register_callbacks cb 0x7fc05c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc072800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae6a2880 0x5585ae6a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc06be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699ca0 0x5585ae6a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc072800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae6a2880 0x5585ae6a67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34100/0 (socket says 172.21.15.22:34100) 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc072800640 1 -- 172.21.15.22:0/2151153229 learned_addr learned my addr 172.21.15.22:0/2151153229 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc071e00640 1 --2- 172.21.15.22:0/2151153229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699380 0x5585ae6aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc072800640 1 -- 172.21.15.22:0/2151153229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699380 msgr2=0x5585ae6aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc072800640 1 --2- 172.21.15.22:0/2151153229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699380 0x5585ae6aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc072800640 1 -- 172.21.15.22:0/2151153229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699ca0 msgr2=0x5585ae6a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc072800640 1 --2- 172.21.15.22:0/2151153229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699ca0 0x5585ae6a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.592+0000 7fc072800640 1 -- 172.21.15.22:0/2151153229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ae6a6fd0 con 0x5585ae6a2880 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.593+0000 7fc072800640 1 --2- 172.21.15.22:0/2151153229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae6a2880 0x5585ae6a67a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0cdbe0 tx=0x7fc06c0d03d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59f69b9e32ec7a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.593+0000 7fc057e00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.593+0000 7fc057e00640 1 -- 172.21.15.22:0/2151153229 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0ba060 con 0x5585ae6a2880 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.593+0000 7fc057e00640 1 -- 172.21.15.22:0/2151153229 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc06c0b5b50 con 0x5585ae6a2880 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.593+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2151153229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc05c03a040 con 0x5585ae6a2880 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.593+0000 7fc057e00640 1 -- 172.21.15.22:0/2151153229 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc06c0cdf80 con 0x5585ae6a2880 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.593+0000 7fc07bfe4740 10 client.25661 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.593+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2151153229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5585ae719050 con 0x5585ae6a2880 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.593+0000 7fc057e00640 1 -- 172.21.15.22:0/2151153229 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc05c03a040 con 0x5585ae6a2880 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.594+0000 7fc057e00640 1 -- 172.21.15.22:0/2151153229 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fc06c00b040 con 0x5585ae6a2880 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.594+0000 7fc07bfe4740 10 client.25661 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.594+0000 7fc07bfe4740 10 client.25661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.594+0000 7fc07bfe4740 20 client.25661 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:07.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.594+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2151153229 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5585ae719770 0x5585ae71bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:07.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.594+0000 7fc07bfe4740 4 client.25661 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:36:07.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.594+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2151153229 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585ae6069e0 con 0x5585ae719770 2024-04-20T21:36:07.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.594+0000 7fc071e00640 1 --2- 172.21.15.22:0/2151153229 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5585ae719770 0x5585ae71bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:07.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.594+0000 7fc071e00640 1 --2- 172.21.15.22:0/2151153229 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5585ae719770 0x5585ae71bb90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce5b61a66989fa4 server_cookie=a9e0624ced2c8cb3 in_seq=0 out_seq=0 2024-04-20T21:36:07.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.594+0000 7fc057e00640 10 client.25661 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:07.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc057e00640 1 -- 172.21.15.22:0/2151153229 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5585ae6069e0 con 0x5585ae719770 2024-04-20T21:36:07.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc057e00640 10 client.25661 handle_command_reply: tid=0 2024-04-20T21:36:07.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc07bfe4740 1 client.25661 shutdown 2024-04-20T21:36:07.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2151153229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae6a2880 msgr2=0x5585ae6a67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2151153229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae6a2880 0x5585ae6a67a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0cdbe0 tx=0x7fc06c0d03d0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2151153229 shutdown_connections 2024-04-20T21:36:07.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2151153229 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5585ae719770 0x5585ae71bb90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2151153229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ae6a2880 0x5585ae6a67a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2151153229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585ae699ca0 0x5585ae6a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc07bfe4740 1 --2- 172.21.15.22:0/2151153229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ae699380 0x5585ae6aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc057e00640 0 client.25661 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:07.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2151153229 >> 172.21.15.22:0/2151153229 conn(0x5585ae693ab0 msgr2=0x5585ae69f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2151153229 shutdown_connections 2024-04-20T21:36:07.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc07bfe4740 1 -- 172.21.15.22:0/2151153229 wait complete. 2024-04-20T21:36:07.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.596+0000 7fc07bfe4740 20 client.25661 trim_cache size 0 max 16384 2024-04-20T21:36:07.599 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:36:07.599 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:36:07.599 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:36:07.600 INFO:teuthology.orchestra.run.smithi022.stdout: "2c5e9550-35d6-44e0-bc5b-ec4c0088a815": { 2024-04-20T21:36:07.600 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:36:07.600 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "2c5e9550-35d6-44e0-bc5b-ec4c0088a815", 2024-04-20T21:36:07.600 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:36:07.600 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:36:07.600 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:36:07.600 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:36:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.597+0000 7fc073200640 1 -- 172.21.15.22:0/3646872223 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc0580a3da0 msgr2=0x7fc0580a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.597+0000 7fc073200640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc0580a3da0 0x7fc0580a6260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc05c005b60 tx=0x7fc05c005750 comp rx=0 tx=0).stop 2024-04-20T21:36:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.597+0000 7fc073200640 1 -- 172.21.15.22:0/3646872223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07410d3d0 msgr2=0x7fc07419eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.597+0000 7fc073200640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07410d3d0 0x7fc07419eb50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00ef00 tx=0x7fc06c00c740 comp rx=0 tx=0).stop 2024-04-20T21:36:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.598+0000 7fc073200640 1 -- 172.21.15.22:0/3646872223 shutdown_connections 2024-04-20T21:36:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.598+0000 7fc073200640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc0580a3da0 0x7fc0580a6260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.598+0000 7fc073200640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc07410d3d0 0x7fc07419eb50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.598+0000 7fc073200640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc074108ea0 0x7fc0741a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.598+0000 7fc073200640 1 --2- 172.21.15.22:0/3646872223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074108500 0x7fc0741a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.598+0000 7fc073200640 1 -- 172.21.15.22:0/3646872223 >> 172.21.15.22:0/3646872223 conn(0x7fc074077d80 msgr2=0x7fc0740783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:07.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.598+0000 7fc073200640 1 -- 172.21.15.22:0/3646872223 shutdown_connections 2024-04-20T21:36:07.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:07.598+0000 7fc073200640 1 -- 172.21.15.22:0/3646872223 wait complete. 2024-04-20T21:36:07.617 INFO:tasks.cephfs.filesystem:scrub status for tag:2c5e9550-35d6-44e0-bc5b-ec4c0088a815 - {'path': '/', 'tag': '2c5e9550-35d6-44e0-bc5b-ec4c0088a815', 'options': 'recursive,force'} 2024-04-20T21:36:08.165 INFO:tasks.workunit.client.0.smithi022.stdout: 1 71532 11.21 MB/sec warmup 36 sec latency 49.834 ms 2024-04-20T21:36:08.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:08 smithi122 ceph-mon[22147]: pgmap v72: 97 pgs: 97 active+clean; 53 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-04-20T21:36:08.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:08 smithi033 ceph-mon[21626]: pgmap v72: 97 pgs: 97 active+clean; 53 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-04-20T21:36:08.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:08 smithi022 ceph-mon[16843]: pgmap v72: 97 pgs: 97 active+clean; 53 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-04-20T21:36:09.165 INFO:tasks.workunit.client.0.smithi022.stdout: 1 73437 11.10 MB/sec warmup 37 sec latency 47.854 ms 2024-04-20T21:36:09.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:09 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:36:09.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:09 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:09.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:09 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:36:09.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:09 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:09.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:09 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:36:09.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:09 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:09.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:09 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:36:09.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:09 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:09.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:09 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:36:09.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:09 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:09.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:09 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:36:09.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:09 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:10.165 INFO:tasks.workunit.client.0.smithi022.stdout: 1 75276 11.13 MB/sec warmup 38 sec latency 12.479 ms 2024-04-20T21:36:10.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:10 smithi033 ceph-mon[21626]: pgmap v73: 97 pgs: 97 active+clean; 53 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-04-20T21:36:10.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:10 smithi022 ceph-mon[16843]: pgmap v73: 97 pgs: 97 active+clean; 53 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-04-20T21:36:10.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:10 smithi122 ceph-mon[22147]: pgmap v73: 97 pgs: 97 active+clean; 53 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-04-20T21:36:11.166 INFO:tasks.workunit.client.0.smithi022.stdout: 1 77899 11.21 MB/sec warmup 39 sec latency 11.139 ms 2024-04-20T21:36:12.166 INFO:tasks.workunit.client.0.smithi022.stdout: 1 79724 11.17 MB/sec warmup 40 sec latency 20.178 ms 2024-04-20T21:36:12.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:12 smithi033 ceph-mon[21626]: pgmap v74: 97 pgs: 97 active+clean; 56 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 3.9 MiB/s wr, 234 op/s 2024-04-20T21:36:12.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:12 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:36:12.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:12 smithi022 ceph-mon[16843]: pgmap v74: 97 pgs: 97 active+clean; 56 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 3.9 MiB/s wr, 234 op/s 2024-04-20T21:36:12.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:12 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:36:12.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:12 smithi122 ceph-mon[22147]: pgmap v74: 97 pgs: 97 active+clean; 56 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 3.9 MiB/s wr, 234 op/s 2024-04-20T21:36:12.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:12 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:36:13.166 INFO:tasks.workunit.client.0.smithi022.stdout: 1 81945 11.23 MB/sec warmup 41 sec latency 49.295 ms 2024-04-20T21:36:14.166 INFO:tasks.workunit.client.0.smithi022.stdout: 1 83549 11.10 MB/sec warmup 42 sec latency 79.901 ms 2024-04-20T21:36:14.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:14 smithi033 ceph-mon[21626]: pgmap v75: 97 pgs: 97 active+clean; 56 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.4 MiB/s wr, 178 op/s 2024-04-20T21:36:14.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:14 smithi122 ceph-mon[22147]: pgmap v75: 97 pgs: 97 active+clean; 56 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.4 MiB/s wr, 178 op/s 2024-04-20T21:36:14.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:14 smithi022 ceph-mon[16843]: pgmap v75: 97 pgs: 97 active+clean; 56 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.4 MiB/s wr, 178 op/s 2024-04-20T21:36:15.166 INFO:tasks.workunit.client.0.smithi022.stdout: 1 85687 11.16 MB/sec warmup 43 sec latency 8.904 ms 2024-04-20T21:36:16.166 INFO:tasks.workunit.client.0.smithi022.stdout: 1 88071 11.18 MB/sec warmup 44 sec latency 13.242 ms 2024-04-20T21:36:16.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:16 smithi033 ceph-mon[21626]: pgmap v76: 97 pgs: 97 active+clean; 60 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 269 op/s 2024-04-20T21:36:16.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:16 smithi022 ceph-mon[16843]: pgmap v76: 97 pgs: 97 active+clean; 60 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 269 op/s 2024-04-20T21:36:16.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:16 smithi122 ceph-mon[22147]: pgmap v76: 97 pgs: 97 active+clean; 60 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 269 op/s 2024-04-20T21:36:17.166 INFO:tasks.workunit.client.0.smithi022.stdout: 1 89649 11.11 MB/sec warmup 45 sec latency 12.833 ms 2024-04-20T21:36:18.167 INFO:tasks.workunit.client.0.smithi022.stdout: 1 92331 11.23 MB/sec warmup 46 sec latency 10.666 ms 2024-04-20T21:36:18.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:18 smithi033 ceph-mon[21626]: pgmap v77: 97 pgs: 97 active+clean; 60 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-04-20T21:36:18.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:18 smithi022 ceph-mon[16843]: pgmap v77: 97 pgs: 97 active+clean; 60 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-04-20T21:36:18.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:18 smithi122 ceph-mon[22147]: pgmap v77: 97 pgs: 97 active+clean; 60 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-04-20T21:36:19.167 INFO:tasks.workunit.client.0.smithi022.stdout: 1 93679 11.13 MB/sec warmup 47 sec latency 51.798 ms 2024-04-20T21:36:20.167 INFO:tasks.workunit.client.0.smithi022.stdout: 1 96104 11.19 MB/sec warmup 48 sec latency 8.885 ms 2024-04-20T21:36:20.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:20 smithi033 ceph-mon[21626]: pgmap v78: 97 pgs: 97 active+clean; 60 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-04-20T21:36:20.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:20 smithi022 ceph-mon[16843]: pgmap v78: 97 pgs: 97 active+clean; 60 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-04-20T21:36:20.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:20 smithi122 ceph-mon[22147]: pgmap v78: 97 pgs: 97 active+clean; 60 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-04-20T21:36:21.167 INFO:tasks.workunit.client.0.smithi022.stdout: 1 98206 11.11 MB/sec warmup 49 sec latency 13.131 ms 2024-04-20T21:36:22.167 INFO:tasks.workunit.client.0.smithi022.stdout: 1 100047 11.13 MB/sec warmup 50 sec latency 14.463 ms 2024-04-20T21:36:22.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:22 smithi022 ceph-mon[16843]: pgmap v79: 97 pgs: 97 active+clean; 65 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 266 op/s 2024-04-20T21:36:22.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:22 smithi122 ceph-mon[22147]: pgmap v79: 97 pgs: 97 active+clean; 65 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 266 op/s 2024-04-20T21:36:22.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:22 smithi033 ceph-mon[21626]: pgmap v79: 97 pgs: 97 active+clean; 65 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 266 op/s 2024-04-20T21:36:23.167 INFO:tasks.workunit.client.0.smithi022.stdout: 1 102382 11.16 MB/sec warmup 51 sec latency 8.812 ms 2024-04-20T21:36:24.168 INFO:tasks.workunit.client.0.smithi022.stdout: 1 103690 11.08 MB/sec warmup 52 sec latency 44.258 ms 2024-04-20T21:36:24.354 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:24 smithi033 ceph-mon[21626]: pgmap v80: 97 pgs: 97 active+clean; 65 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 2.8 MiB/s wr, 182 op/s 2024-04-20T21:36:24.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:24 smithi022 ceph-mon[16843]: pgmap v80: 97 pgs: 97 active+clean; 65 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 2.8 MiB/s wr, 182 op/s 2024-04-20T21:36:24.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:24 smithi122 ceph-mon[22147]: pgmap v80: 97 pgs: 97 active+clean; 65 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 2.8 MiB/s wr, 182 op/s 2024-04-20T21:36:25.168 INFO:tasks.workunit.client.0.smithi022.stdout: 1 105932 11.10 MB/sec warmup 53 sec latency 11.025 ms 2024-04-20T21:36:26.168 INFO:tasks.workunit.client.0.smithi022.stdout: 1 107394 11.03 MB/sec warmup 54 sec latency 14.009 ms 2024-04-20T21:36:26.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:26 smithi022 ceph-mon[16843]: pgmap v81: 97 pgs: 97 active+clean; 69 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-04-20T21:36:26.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:26 smithi122 ceph-mon[22147]: pgmap v81: 97 pgs: 97 active+clean; 69 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-04-20T21:36:26.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:26 smithi033 ceph-mon[21626]: pgmap v81: 97 pgs: 97 active+clean; 69 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-04-20T21:36:27.168 INFO:tasks.workunit.client.0.smithi022.stdout: 1 109770 11.07 MB/sec warmup 55 sec latency 10.083 ms 2024-04-20T21:36:28.168 INFO:tasks.workunit.client.0.smithi022.stdout: 1 111047 10.99 MB/sec warmup 56 sec latency 36.857 ms 2024-04-20T21:36:28.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:28 smithi022 ceph-mon[16843]: pgmap v82: 97 pgs: 97 active+clean; 69 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-04-20T21:36:28.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:28 smithi122 ceph-mon[22147]: pgmap v82: 97 pgs: 97 active+clean; 69 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-04-20T21:36:28.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:28 smithi033 ceph-mon[21626]: pgmap v82: 97 pgs: 97 active+clean; 69 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-04-20T21:36:29.168 INFO:tasks.workunit.client.0.smithi022.stdout: 1 113020 10.99 MB/sec warmup 57 sec latency 45.611 ms 2024-04-20T21:36:30.168 INFO:tasks.workunit.client.0.smithi022.stdout: 1 114480 10.93 MB/sec warmup 58 sec latency 15.707 ms 2024-04-20T21:36:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:30 smithi022 ceph-mon[16843]: pgmap v83: 97 pgs: 97 active+clean; 69 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-04-20T21:36:30.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:30 smithi122 ceph-mon[22147]: pgmap v83: 97 pgs: 97 active+clean; 69 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-04-20T21:36:30.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:30 smithi033 ceph-mon[21626]: pgmap v83: 97 pgs: 97 active+clean; 69 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-04-20T21:36:31.168 INFO:tasks.workunit.client.0.smithi022.stdout: 1 116879 10.97 MB/sec warmup 59 sec latency 12.010 ms 2024-04-20T21:36:32.168 INFO:tasks.workunit.client.0.smithi022.stdout: 1 118244 10.90 MB/sec warmup 60 sec latency 15.568 ms 2024-04-20T21:36:32.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:32 smithi022 ceph-mon[16843]: pgmap v84: 97 pgs: 97 active+clean; 73 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 249 op/s 2024-04-20T21:36:32.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:32 smithi122 ceph-mon[22147]: pgmap v84: 97 pgs: 97 active+clean; 73 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 249 op/s 2024-04-20T21:36:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:32 smithi033 ceph-mon[21626]: pgmap v84: 97 pgs: 97 active+clean; 73 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 249 op/s 2024-04-20T21:36:33.168 INFO:tasks.workunit.client.0.smithi022.stdout: 1 120723 10.98 MB/sec warmup 61 sec latency 18.578 ms 2024-04-20T21:36:34.168 INFO:tasks.workunit.client.0.smithi022.stdout: 1 121871 10.86 MB/sec warmup 62 sec latency 34.840 ms 2024-04-20T21:36:34.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:34 smithi022 ceph-mon[16843]: pgmap v85: 97 pgs: 97 active+clean; 73 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 2.3 MiB/s wr, 157 op/s 2024-04-20T21:36:34.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:34 smithi122 ceph-mon[22147]: pgmap v85: 97 pgs: 97 active+clean; 73 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 2.3 MiB/s wr, 157 op/s 2024-04-20T21:36:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:34 smithi033 ceph-mon[21626]: pgmap v85: 97 pgs: 97 active+clean; 73 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 2.3 MiB/s wr, 157 op/s 2024-04-20T21:36:35.168 INFO:tasks.workunit.client.0.smithi022.stdout: 1 124469 10.95 MB/sec warmup 63 sec latency 11.635 ms 2024-04-20T21:36:36.169 INFO:tasks.workunit.client.0.smithi022.stdout: 1 126595 10.90 MB/sec warmup 64 sec latency 12.597 ms 2024-04-20T21:36:36.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:36 smithi022 ceph-mon[16843]: pgmap v86: 97 pgs: 97 active+clean; 78 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-04-20T21:36:36.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:36 smithi122 ceph-mon[22147]: pgmap v86: 97 pgs: 97 active+clean; 78 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-04-20T21:36:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:36 smithi033 ceph-mon[21626]: pgmap v86: 97 pgs: 97 active+clean; 78 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-04-20T21:36:37.169 INFO:tasks.workunit.client.0.smithi022.stdout: 1 128405 10.92 MB/sec warmup 65 sec latency 12.815 ms 2024-04-20T21:36:37.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:36:37.618 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:36:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.845+0000 7feb15800640 1 -- 172.21.15.22:0/1852239745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb101038e0 msgr2=0x7feb1010bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.845+0000 7feb15800640 1 --2- 172.21.15.22:0/1852239745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb101038e0 0x7feb1010bdc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009ca0 tx=0x7feaf802c180 comp rx=0 tx=0).stop 2024-04-20T21:36:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.845+0000 7feb15800640 1 -- 172.21.15.22:0/1852239745 shutdown_connections 2024-04-20T21:36:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.845+0000 7feb15800640 1 --2- 172.21.15.22:0/1852239745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb1010c3e0 0x7feb1010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.845+0000 7feb15800640 1 --2- 172.21.15.22:0/1852239745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb101038e0 0x7feb1010bdc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.845+0000 7feb15800640 1 --2- 172.21.15.22:0/1852239745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb10102fc0 0x7feb101033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.845+0000 7feb15800640 1 -- 172.21.15.22:0/1852239745 >> 172.21.15.22:0/1852239745 conn(0x7feb100fc7e0 msgr2=0x7feb100fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.845+0000 7feb15800640 1 -- 172.21.15.22:0/1852239745 shutdown_connections 2024-04-20T21:36:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.845+0000 7feb15800640 1 -- 172.21.15.22:0/1852239745 wait complete. 2024-04-20T21:36:37.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb15800640 1 Processor -- start 2024-04-20T21:36:37.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb15800640 1 -- start start 2024-04-20T21:36:37.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb15800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb10102fc0 0x7feb101a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb15800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb101038e0 0x7feb101a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb15800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb1010c3e0 0x7feb1019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb15800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feb101121a0 con 0x7feb1010c3e0 2024-04-20T21:36:37.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb15800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb10112020 con 0x7feb101038e0 2024-04-20T21:36:37.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb15800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb10112320 con 0x7feb10102fc0 2024-04-20T21:36:37.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb10102fc0 0x7feb101a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb0f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb101038e0 0x7feb101a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb10102fc0 0x7feb101a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33296/0 (socket says 172.21.15.22:33296) 2024-04-20T21:36:37.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb0fe00640 1 -- 172.21.15.22:0/2098439894 learned_addr learned my addr 172.21.15.22:0/2098439894 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:37.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.847+0000 7feb14e00640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb1010c3e0 0x7feb1019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.848+0000 7feb0fe00640 1 -- 172.21.15.22:0/2098439894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb101038e0 msgr2=0x7feb101a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:37.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.848+0000 7feb0fe00640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb101038e0 0x7feb101a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.848+0000 7feb0fe00640 1 -- 172.21.15.22:0/2098439894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb1010c3e0 msgr2=0x7feb1019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:37.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.848+0000 7feb0fe00640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb1010c3e0 0x7feb1019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.848+0000 7feb0fe00640 1 -- 172.21.15.22:0/2098439894 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1019af70 con 0x7feb10102fc0 2024-04-20T21:36:37.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.848+0000 7feb0f400640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb101038e0 0x7feb101a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:37.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.848+0000 7feb0fe00640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb10102fc0 0x7feb101a0630 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feb0000e9f0 tx=0x7feb0000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:37.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.848+0000 7feb0cc00640 1 -- 172.21.15.22:0/2098439894 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0000cd80 con 0x7feb10102fc0 2024-04-20T21:36:37.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.848+0000 7feb15800640 1 -- 172.21.15.22:0/2098439894 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1019b260 con 0x7feb10102fc0 2024-04-20T21:36:37.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.848+0000 7feb0cc00640 1 -- 172.21.15.22:0/2098439894 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb00010910 con 0x7feb10102fc0 2024-04-20T21:36:37.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.848+0000 7feb0cc00640 1 -- 172.21.15.22:0/2098439894 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000188f0 con 0x7feb10102fc0 2024-04-20T21:36:37.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.849+0000 7feb15800640 1 -- 172.21.15.22:0/2098439894 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb101a7480 con 0x7feb10102fc0 2024-04-20T21:36:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.852+0000 7feb19b81740 1 Processor -- start 2024-04-20T21:36:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.852+0000 7feb19b81740 1 -- start start 2024-04-20T21:36:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.852+0000 7feb19b81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 0x558a71e8e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.852+0000 7feb19b81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e97340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.852+0000 7feb19b81740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e99c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.852+0000 7feb19b81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a71df4150 con 0x558a71e8eca0 2024-04-20T21:36:37.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.852+0000 7feb19b81740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558a71d9a370 con 0x558a71e97880 2024-04-20T21:36:37.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.852+0000 7feb19b81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558a71df0110 con 0x558a71e8e380 2024-04-20T21:36:37.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.852+0000 7feb0cc00640 1 -- 172.21.15.22:0/2098439894 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7feb00010ac0 con 0x7feb10102fc0 2024-04-20T21:36:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb0f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e99c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb14e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 0x558a71e8e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb0f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e99c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41154/0 (socket says 172.21.15.22:41154) 2024-04-20T21:36:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb0f400640 1 -- 172.21.15.22:0/2899375866 learned_addr learned my addr 172.21.15.22:0/2899375866 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb14e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 0x558a71e8e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33310/0 (socket says 172.21.15.22:33310) 2024-04-20T21:36:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb0fe00640 1 --2- 172.21.15.22:0/2899375866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e97340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb0f400640 1 -- 172.21.15.22:0/2899375866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 msgr2=0x558a71e8e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb0f400640 1 --2- 172.21.15.22:0/2899375866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 0x558a71e8e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb0f400640 1 -- 172.21.15.22:0/2899375866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 msgr2=0x558a71e97340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb0f400640 1 --2- 172.21.15.22:0/2899375866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e97340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb0f400640 1 -- 172.21.15.22:0/2899375866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a71bd07d0 con 0x558a71e97880 2024-04-20T21:36:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb0fe00640 1 --2- 172.21.15.22:0/2899375866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e97340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.853+0000 7feb0f400640 1 --2- 172.21.15.22:0/2899375866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e99c70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feaf8037040 tx=0x7feaf80098f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec3600b9ea3beee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.854+0000 7feb05600640 1 -- 172.21.15.22:0/2899375866 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8043070 con 0x558a71e97880 2024-04-20T21:36:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.854+0000 7feb05600640 1 -- 172.21.15.22:0/2899375866 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf8003e50 con 0x558a71e97880 2024-04-20T21:36:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.854+0000 7feb05600640 1 -- 172.21.15.22:0/2899375866 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8039040 con 0x558a71e97880 2024-04-20T21:36:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.854+0000 7feb0cc00640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7feaf00a3d20 0x7feaf00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.854+0000 7feb0cc00640 1 -- 172.21.15.22:0/2098439894 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7feb000d0bc0 con 0x7feb10102fc0 2024-04-20T21:36:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.854+0000 7feb14e00640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7feaf00a3d20 0x7feaf00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.855+0000 7feb19b81740 1 -- 172.21.15.22:0/2899375866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 msgr2=0x558a71e99c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.855+0000 7feb19b81740 1 --2- 172.21.15.22:0/2899375866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e99c70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feaf8037040 tx=0x7feaf80098f0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.855+0000 7feb19b81740 1 -- 172.21.15.22:0/2899375866 shutdown_connections 2024-04-20T21:36:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.855+0000 7feb19b81740 1 --2- 172.21.15.22:0/2899375866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e99c70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.855+0000 7feb19b81740 1 --2- 172.21.15.22:0/2899375866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e97340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.855+0000 7feb19b81740 1 --2- 172.21.15.22:0/2899375866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 0x558a71e8e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.855+0000 7feb19b81740 1 -- 172.21.15.22:0/2899375866 >> 172.21.15.22:0/2899375866 conn(0x558a71e88ab0 msgr2=0x558a71e88e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.855+0000 7feb19b81740 1 -- 172.21.15.22:0/2899375866 shutdown_connections 2024-04-20T21:36:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.855+0000 7feb19b81740 1 -- 172.21.15.22:0/2899375866 wait complete. 2024-04-20T21:36:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.855+0000 7feb14e00640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7feaf00a3d20 0x7feaf00a61e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feb08009d20 tx=0x7feb08005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.856+0000 7feb19b81740 1 -- start start 2024-04-20T21:36:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.856+0000 7feb19b81740 1 Processor -- start 2024-04-20T21:36:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.856+0000 7feb19b81740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e8e380 0x558a71e94850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.856+0000 7feb19b81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e94d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.856+0000 7feb19b81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e97880 0x558a71e952d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.856+0000 7feb19b81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a71e9a610 con 0x558a71e8eca0 2024-04-20T21:36:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.856+0000 7feb19b81740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558a71dad740 con 0x558a71e8e380 2024-04-20T21:36:37.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.856+0000 7feb19b81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558a71e9a790 con 0x558a71e97880 2024-04-20T21:36:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb19b81740 10 client.0 _ll_register_callbacks cb 0x558a71da8b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e8e380 0x558a71e94850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e8e380 0x558a71e94850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41170/0 (socket says 172.21.15.22:41170) 2024-04-20T21:36:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb0fe00640 1 -- 172.21.15.22:0/1011366917 learned_addr learned my addr 172.21.15.22:0/1011366917 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb14e00640 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e97880 0x558a71e952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb0f400640 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e94d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb0fe00640 1 -- 172.21.15.22:0/1011366917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e97880 msgr2=0x558a71e952d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb0fe00640 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e97880 0x558a71e952d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb0fe00640 1 -- 172.21.15.22:0/1011366917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 msgr2=0x558a71e94d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb0fe00640 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e94d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb0fe00640 1 -- 172.21.15.22:0/1011366917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a71b063f0 con 0x558a71e8e380 2024-04-20T21:36:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb0f400640 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e94d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feb0fe00640 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e8e380 0x558a71e94850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb0000c6a0 tx=0x7feb0009cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14de632f550a514a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feafe000640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:36:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.857+0000 7feafe000640 1 -- 172.21.15.22:0/1011366917 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0009c240 con 0x558a71e8e380 2024-04-20T21:36:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feafe000640 1 -- 172.21.15.22:0/1011366917 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7feb00010430 con 0x558a71e8e380 2024-04-20T21:36:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feb19b81740 1 -- 172.21.15.22:0/1011366917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf8003e50 con 0x558a71e8e380 2024-04-20T21:36:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feafe000640 1 -- 172.21.15.22:0/1011366917 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7feb000bd1a0 con 0x558a71e8e380 2024-04-20T21:36:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feb19b81740 10 client.25673 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feb19b81740 1 -- 172.21.15.22:0/1011366917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558a71ac60f0 con 0x558a71e8e380 2024-04-20T21:36:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feafe000640 1 -- 172.21.15.22:0/1011366917 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf8003e50 con 0x558a71e8e380 2024-04-20T21:36:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feafe000640 1 -- 172.21.15.22:0/1011366917 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7feb0009bab0 con 0x558a71e8e380 2024-04-20T21:36:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feb19b81740 10 client.25673 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feb19b81740 10 client.25673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feb19b81740 20 client.25673 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feb19b81740 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a71ea7a00 0x558a71ea9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feb19b81740 4 client.25673 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.858+0000 7feb19b81740 1 -- 172.21.15.22:0/1011366917 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a71e01b70 con 0x558a71ea7a00 2024-04-20T21:36:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.859+0000 7feb0f400640 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a71ea7a00 0x558a71ea9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.859+0000 7feb0f400640 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a71ea7a00 0x558a71ea9e90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52acc3844e2b8db server_cookie=6984d09577228032 in_seq=0 out_seq=0 2024-04-20T21:36:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.859+0000 7feafe000640 10 client.25673 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.860+0000 7feafe000640 1 -- 172.21.15.22:0/1011366917 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a71e01b70 con 0x558a71ea7a00 2024-04-20T21:36:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.860+0000 7feafe000640 10 client.25673 handle_command_reply: tid=0 2024-04-20T21:36:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.860+0000 7feb19b81740 1 client.25673 shutdown 2024-04-20T21:36:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.860+0000 7feb19b81740 1 -- 172.21.15.22:0/1011366917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e8e380 msgr2=0x558a71e94850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.860+0000 7feb19b81740 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e8e380 0x558a71e94850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb0000c6a0 tx=0x7feb0009cd30 comp rx=0 tx=0).stop 2024-04-20T21:36:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.860+0000 7feb19b81740 1 -- 172.21.15.22:0/1011366917 shutdown_connections 2024-04-20T21:36:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.860+0000 7feb19b81740 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e97880 0x558a71e952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.860+0000 7feb19b81740 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e94d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.860+0000 7feb19b81740 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a71ea7a00 0x558a71ea9e90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.860+0000 7feb19b81740 1 --2- 172.21.15.22:0/1011366917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e8e380 0x558a71e94850 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.860+0000 7feafe000640 0 client.25673 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.860+0000 7feb19b81740 1 -- 172.21.15.22:0/1011366917 >> 172.21.15.22:0/1011366917 conn(0x558a71e88ab0 msgr2=0x558a71e8a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.861+0000 7feb19b81740 1 -- 172.21.15.22:0/1011366917 shutdown_connections 2024-04-20T21:36:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.861+0000 7feb19b81740 1 -- 172.21.15.22:0/1011366917 wait complete. 2024-04-20T21:36:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.861+0000 7feb19b81740 20 client.25673 trim_cache size 0 max 16384 2024-04-20T21:36:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.881+0000 7feb19b81740 1 Processor -- start 2024-04-20T21:36:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.881+0000 7feb19b81740 1 -- start start 2024-04-20T21:36:37.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.881+0000 7feb19b81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8e380 0x558a71e87d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.881+0000 7feb19b81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8eca0 0x558a71ea34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.881+0000 7feb19b81740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e94130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb19b81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feb101addf0 con 0x558a71e8e380 2024-04-20T21:36:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb19b81740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb101b0670 con 0x558a71e97880 2024-04-20T21:36:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb19b81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb101ae460 con 0x558a71e8eca0 2024-04-20T21:36:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb14e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e94130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb14e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e94130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41186/0 (socket says 172.21.15.22:41186) 2024-04-20T21:36:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb0f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8eca0 0x558a71ea34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb14e00640 1 -- 172.21.15.22:0/4006182048 learned_addr learned my addr 172.21.15.22:0/4006182048 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb0fe00640 1 --2- 172.21.15.22:0/4006182048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8e380 0x558a71e87d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb14e00640 1 -- 172.21.15.22:0/4006182048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8eca0 msgr2=0x558a71ea34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb14e00640 1 --2- 172.21.15.22:0/4006182048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8eca0 0x558a71ea34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb14e00640 1 -- 172.21.15.22:0/4006182048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8e380 msgr2=0x558a71e87d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb14e00640 1 --2- 172.21.15.22:0/4006182048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8e380 0x558a71e87d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb14e00640 1 -- 172.21.15.22:0/4006182048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a71e8ead0 con 0x558a71e97880 2024-04-20T21:36:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.882+0000 7feb0f400640 1 --2- 172.21.15.22:0/4006182048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8eca0 0x558a71ea34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.883+0000 7feb14e00640 1 --2- 172.21.15.22:0/4006182048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e94130 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feb08004380 tx=0x7feb0801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca692f6a55e63a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.883+0000 7feb04c00640 1 -- 172.21.15.22:0/4006182048 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb080043b0 con 0x558a71e97880 2024-04-20T21:36:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.883+0000 7feb04c00640 1 -- 172.21.15.22:0/4006182048 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb08005500 con 0x558a71e97880 2024-04-20T21:36:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.883+0000 7feb04c00640 1 -- 172.21.15.22:0/4006182048 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0801e6d0 con 0x558a71e97880 2024-04-20T21:36:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.884+0000 7feb19b81740 1 -- 172.21.15.22:0/4006182048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 msgr2=0x558a71e94130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.884+0000 7feb19b81740 1 --2- 172.21.15.22:0/4006182048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e94130 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feb08004380 tx=0x7feb0801b000 comp rx=0 tx=0).stop 2024-04-20T21:36:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.884+0000 7feb19b81740 1 -- 172.21.15.22:0/4006182048 shutdown_connections 2024-04-20T21:36:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.884+0000 7feb19b81740 1 --2- 172.21.15.22:0/4006182048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e94130 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.884+0000 7feb19b81740 1 --2- 172.21.15.22:0/4006182048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8eca0 0x558a71ea34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.884+0000 7feb19b81740 1 --2- 172.21.15.22:0/4006182048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8e380 0x558a71e87d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.884+0000 7feb19b81740 1 -- 172.21.15.22:0/4006182048 >> 172.21.15.22:0/4006182048 conn(0x558a71e88ab0 msgr2=0x558a71e8a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.884+0000 7feb19b81740 1 -- 172.21.15.22:0/4006182048 shutdown_connections 2024-04-20T21:36:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.884+0000 7feb19b81740 1 -- 172.21.15.22:0/4006182048 wait complete. 2024-04-20T21:36:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.884+0000 7feb19b81740 1 -- start start 2024-04-20T21:36:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb19b81740 1 Processor -- start 2024-04-20T21:36:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb19b81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 0x558a71e9b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb19b81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e9b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb19b81740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e9bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb19b81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feb101aeb30 con 0x558a71e8eca0 2024-04-20T21:36:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb19b81740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb101affa0 con 0x558a71e97880 2024-04-20T21:36:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb19b81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb101af200 con 0x558a71e8e380 2024-04-20T21:36:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb19b81740 10 client.0 _ll_register_callbacks cb 0x7feb0000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb14e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e9bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 0x558a71e9b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb0f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e9b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 0x558a71e9b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33334/0 (socket says 172.21.15.22:33334) 2024-04-20T21:36:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb0fe00640 1 -- 172.21.15.22:0/3901802208 learned_addr learned my addr 172.21.15.22:0/3901802208 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.885+0000 7feb14e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e9bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41198/0 (socket says 172.21.15.22:41198) 2024-04-20T21:36:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.886+0000 7feb0fe00640 1 -- 172.21.15.22:0/3901802208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 msgr2=0x558a71e9bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.886+0000 7feb0fe00640 1 --2- 172.21.15.22:0/3901802208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e9bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.886+0000 7feb0fe00640 1 -- 172.21.15.22:0/3901802208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 msgr2=0x558a71e9b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.886+0000 7feb0fe00640 1 --2- 172.21.15.22:0/3901802208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e9b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.886+0000 7feb0fe00640 1 -- 172.21.15.22:0/3901802208 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a71f0cff0 con 0x558a71e8e380 2024-04-20T21:36:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.886+0000 7feb0fe00640 1 --2- 172.21.15.22:0/3901802208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 0x558a71e9b310 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7feb000bd0b0 tx=0x7feb00098b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e3ddbc185563d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.886+0000 7feafe000640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:36:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.886+0000 7feafe000640 1 -- 172.21.15.22:0/3901802208 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0009c040 con 0x558a71e8e380 2024-04-20T21:36:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.886+0000 7feafe000640 1 -- 172.21.15.22:0/3901802208 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7feb0009c1e0 con 0x558a71e8e380 2024-04-20T21:36:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.886+0000 7feb19b81740 1 -- 172.21.15.22:0/3901802208 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb08005500 con 0x558a71e8e380 2024-04-20T21:36:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.887+0000 7feafe000640 1 -- 172.21.15.22:0/3901802208 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7feb00098c60 con 0x558a71e8e380 2024-04-20T21:36:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.887+0000 7feb19b81740 10 client.25711 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.887+0000 7feb19b81740 1 -- 172.21.15.22:0/3901802208 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558a71f0e200 con 0x558a71e8e380 2024-04-20T21:36:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.887+0000 7feafe000640 1 -- 172.21.15.22:0/3901802208 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb08005500 con 0x558a71e8e380 2024-04-20T21:36:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.887+0000 7feafe000640 1 -- 172.21.15.22:0/3901802208 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7feb000cea40 con 0x558a71e8e380 2024-04-20T21:36:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.887+0000 7feb19b81740 10 client.25711 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.887+0000 7feb19b81740 10 client.25711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.887+0000 7feb19b81740 20 client.25711 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.887+0000 7feb19b81740 1 --2- 172.21.15.22:0/3901802208 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a71f0e810 0x558a71f10c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.887+0000 7feb19b81740 4 client.25711 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:36:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.887+0000 7feb19b81740 1 -- 172.21.15.22:0/3901802208 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a71dfb9e0 con 0x558a71f0e810 2024-04-20T21:36:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.888+0000 7feb0f400640 1 --2- 172.21.15.22:0/3901802208 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a71f0e810 0x558a71f10c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:37.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.888+0000 7feb0f400640 1 --2- 172.21.15.22:0/3901802208 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a71f0e810 0x558a71f10c30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=313102b1cc8c9336 server_cookie=9a1df7a56265aff8 in_seq=0 out_seq=0 2024-04-20T21:36:37.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.888+0000 7feafe000640 10 client.25711 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:37.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feafe000640 1 -- 172.21.15.22:0/3901802208 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558a71dfb9e0 con 0x558a71f0e810 2024-04-20T21:36:37.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feafe000640 10 client.25711 handle_command_reply: tid=0 2024-04-20T21:36:37.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feb19b81740 1 client.25711 shutdown 2024-04-20T21:36:37.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feb19b81740 1 -- 172.21.15.22:0/3901802208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 msgr2=0x558a71e9b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:37.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feb19b81740 1 --2- 172.21.15.22:0/3901802208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 0x558a71e9b310 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7feb000bd0b0 tx=0x7feb00098b70 comp rx=0 tx=0).stop 2024-04-20T21:36:37.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feb19b81740 1 -- 172.21.15.22:0/3901802208 shutdown_connections 2024-04-20T21:36:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feb19b81740 1 --2- 172.21.15.22:0/3901802208 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a71f0e810 0x558a71f10c30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feb19b81740 1 --2- 172.21.15.22:0/3901802208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a71e97880 0x558a71e9bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feb19b81740 1 --2- 172.21.15.22:0/3901802208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a71e8eca0 0x558a71e9b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feb19b81740 1 --2- 172.21.15.22:0/3901802208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a71e8e380 0x558a71e9b310 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feafe000640 0 client.25711 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feb19b81740 1 -- 172.21.15.22:0/3901802208 >> 172.21.15.22:0/3901802208 conn(0x558a71e88ab0 msgr2=0x558a71e94810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:37.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feb19b81740 1 -- 172.21.15.22:0/3901802208 shutdown_connections 2024-04-20T21:36:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feb19b81740 1 -- 172.21.15.22:0/3901802208 wait complete. 2024-04-20T21:36:37.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.889+0000 7feb19b81740 20 client.25711 trim_cache size 0 max 16384 2024-04-20T21:36:37.903 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:36:37.903 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:36:37.903 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:36:37.904 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:36:37.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.890+0000 7feb15800640 1 -- 172.21.15.22:0/2098439894 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7feaf00a3d20 msgr2=0x7feaf00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:37.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.890+0000 7feb15800640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7feaf00a3d20 0x7feaf00a61e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feb08009d20 tx=0x7feb08005f90 comp rx=0 tx=0).stop 2024-04-20T21:36:37.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.890+0000 7feb15800640 1 -- 172.21.15.22:0/2098439894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb10102fc0 msgr2=0x7feb101a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:37.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.890+0000 7feb15800640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb10102fc0 0x7feb101a0630 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feb0000e9f0 tx=0x7feb0000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.890+0000 7feb15800640 1 -- 172.21.15.22:0/2098439894 shutdown_connections 2024-04-20T21:36:37.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.890+0000 7feb15800640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7feaf00a3d20 0x7feaf00a61e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.890+0000 7feb15800640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb1010c3e0 0x7feb1019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.890+0000 7feb15800640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb101038e0 0x7feb101a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.890+0000 7feb15800640 1 --2- 172.21.15.22:0/2098439894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb10102fc0 0x7feb101a0630 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:37.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.890+0000 7feb15800640 1 -- 172.21.15.22:0/2098439894 >> 172.21.15.22:0/2098439894 conn(0x7feb100fc7e0 msgr2=0x7feb10100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:37.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.891+0000 7feb15800640 1 -- 172.21.15.22:0/2098439894 shutdown_connections 2024-04-20T21:36:37.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:37.891+0000 7feb15800640 1 -- 172.21.15.22:0/2098439894 wait complete. 2024-04-20T21:36:37.907 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:36:37.908 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:38.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.110+0000 7f5e41400640 1 -- 172.21.15.22:0/2969558102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e3c101910 msgr2=0x7f5e3c10db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.110+0000 7f5e41400640 1 --2- 172.21.15.22:0/2969558102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e3c101910 0x7f5e3c10db20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009cd0 tx=0x7f5e3402c170 comp rx=0 tx=0).stop 2024-04-20T21:36:38.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.110+0000 7f5e41400640 1 -- 172.21.15.22:0/2969558102 shutdown_connections 2024-04-20T21:36:38.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.110+0000 7f5e41400640 1 --2- 172.21.15.22:0/2969558102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e3c101910 0x7f5e3c10db20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.110+0000 7f5e41400640 1 --2- 172.21.15.22:0/2969558102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c100f50 0x7f5e3c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.110+0000 7f5e41400640 1 --2- 172.21.15.22:0/2969558102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e3c105c90 0x7f5e3c106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.110+0000 7f5e41400640 1 -- 172.21.15.22:0/2969558102 >> 172.21.15.22:0/2969558102 conn(0x7f5e3c0767a0 msgr2=0x7f5e3c076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:38.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.110+0000 7f5e41400640 1 -- 172.21.15.22:0/2969558102 shutdown_connections 2024-04-20T21:36:38.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.110+0000 7f5e41400640 1 -- 172.21.15.22:0/2969558102 wait complete. 2024-04-20T21:36:38.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.111+0000 7f5e41400640 1 Processor -- start 2024-04-20T21:36:38.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.111+0000 7f5e41400640 1 -- start start 2024-04-20T21:36:38.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e41400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e3c100f50 0x7f5e3c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e41400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e3c101910 0x7f5e3c1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e41400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c105c90 0x7f5e3c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e41400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c111060 con 0x7f5e3c100f50 2024-04-20T21:36:38.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e41400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c110ee0 con 0x7f5e3c101910 2024-04-20T21:36:38.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e41400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1111e0 con 0x7f5e3c105c90 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e3c100f50 0x7f5e3c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e3b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e3c101910 0x7f5e3c1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e3b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e3c101910 0x7f5e3c1a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41220/0 (socket says 172.21.15.22:41220) 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e3b400640 1 -- 172.21.15.22:0/4090847720 learned_addr learned my addr 172.21.15.22:0/4090847720 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e40a00640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c105c90 0x7f5e3c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e3b400640 1 -- 172.21.15.22:0/4090847720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c105c90 msgr2=0x7f5e3c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.112+0000 7f5e3b400640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c105c90 0x7f5e3c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.113+0000 7f5e3b400640 1 -- 172.21.15.22:0/4090847720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e3c100f50 msgr2=0x7f5e3c1a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.113+0000 7f5e3b400640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e3c100f50 0x7f5e3c1a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.113+0000 7f5e3b400640 1 -- 172.21.15.22:0/4090847720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c19ad70 con 0x7f5e3c101910 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.113+0000 7f5e3b400640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e3c101910 0x7f5e3c1a0980 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800e9f0 tx=0x7f5e2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.113+0000 7f5e38c00640 1 -- 172.21.15.22:0/4090847720 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800cd80 con 0x7f5e3c101910 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.113+0000 7f5e38c00640 1 -- 172.21.15.22:0/4090847720 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e28010910 con 0x7f5e3c101910 2024-04-20T21:36:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.113+0000 7f5e41400640 1 -- 172.21.15.22:0/4090847720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3c19b060 con 0x7f5e3c101910 2024-04-20T21:36:38.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.113+0000 7f5e38c00640 1 -- 172.21.15.22:0/4090847720 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28018990 con 0x7f5e3c101910 2024-04-20T21:36:38.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.113+0000 7f5e41400640 1 -- 172.21.15.22:0/4090847720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3c1a7300 con 0x7f5e3c101910 2024-04-20T21:36:38.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.115+0000 7f5e41400640 1 -- 172.21.15.22:0/4090847720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e3c068ca0 con 0x7f5e3c101910 2024-04-20T21:36:38.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.118+0000 7f5e38c00640 1 -- 172.21.15.22:0/4090847720 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5e28010ac0 con 0x7f5e3c101910 2024-04-20T21:36:38.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.119+0000 7f5e38c00640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5e200a3da0 0x7f5e200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.120+0000 7f5e38c00640 1 -- 172.21.15.22:0/4090847720 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5e280d0b00 con 0x7f5e3c101910 2024-04-20T21:36:38.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.120+0000 7f5e3be00640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5e200a3da0 0x7f5e200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.121+0000 7f5e3be00640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5e200a3da0 0x7f5e200a6260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c078fd0 tx=0x7f5e2c00b390 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.125+0000 7f5e38c00640 1 -- 172.21.15.22:0/4090847720 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5e2809cad0 con 0x7f5e3c101910 2024-04-20T21:36:38.169 INFO:tasks.workunit.client.0.smithi022.stdout: 1 131104 10.97 MB/sec warmup 66 sec latency 9.606 ms 2024-04-20T21:36:38.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.377+0000 7f5e41400640 1 -- 172.21.15.22:0/4090847720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e3c1a7690 con 0x7f5e3c101910 2024-04-20T21:36:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.379+0000 7f5e38c00640 1 -- 172.21.15.22:0/4090847720 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f5e280a0dd0 con 0x7f5e3c101910 2024-04-20T21:36:38.378 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:36:38.378 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:36:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:36:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.381+0000 7f5e41400640 1 -- 172.21.15.22:0/4090847720 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5e200a3da0 msgr2=0x7f5e200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.381+0000 7f5e41400640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5e200a3da0 0x7f5e200a6260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c078fd0 tx=0x7f5e2c00b390 comp rx=0 tx=0).stop 2024-04-20T21:36:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.381+0000 7f5e41400640 1 -- 172.21.15.22:0/4090847720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e3c101910 msgr2=0x7f5e3c1a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.381+0000 7f5e41400640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e3c101910 0x7f5e3c1a0980 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800e9f0 tx=0x7f5e2800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.382+0000 7f5e41400640 1 -- 172.21.15.22:0/4090847720 shutdown_connections 2024-04-20T21:36:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.382+0000 7f5e41400640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5e200a3da0 0x7f5e200a6260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.382+0000 7f5e41400640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c105c90 0x7f5e3c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.382+0000 7f5e41400640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e3c101910 0x7f5e3c1a0980 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.382+0000 7f5e41400640 1 --2- 172.21.15.22:0/4090847720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e3c100f50 0x7f5e3c1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.382+0000 7f5e41400640 1 -- 172.21.15.22:0/4090847720 >> 172.21.15.22:0/4090847720 conn(0x7f5e3c0767a0 msgr2=0x7f5e3c0690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.382+0000 7f5e41400640 1 -- 172.21.15.22:0/4090847720 shutdown_connections 2024-04-20T21:36:38.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.382+0000 7f5e41400640 1 -- 172.21.15.22:0/4090847720 wait complete. 2024-04-20T21:36:38.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:38 smithi022 ceph-mon[16843]: pgmap v87: 97 pgs: 97 active+clean; 78 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-04-20T21:36:38.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:36:38.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:38 smithi122 ceph-mon[22147]: pgmap v87: 97 pgs: 97 active+clean; 78 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-04-20T21:36:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:38 smithi033 ceph-mon[21626]: pgmap v87: 97 pgs: 97 active+clean; 78 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-04-20T21:36:38.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.624+0000 7fc433200640 1 -- 172.21.15.22:0/3772575909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341039a0 msgr2=0x7fc43410be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.624+0000 7fc433200640 1 --2- 172.21.15.22:0/3772575909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341039a0 0x7fc43410be80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc424009860 tx=0x7fc42402bc10 comp rx=0 tx=0).stop 2024-04-20T21:36:38.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.625+0000 7fc433200640 1 -- 172.21.15.22:0/3772575909 shutdown_connections 2024-04-20T21:36:38.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.625+0000 7fc433200640 1 --2- 172.21.15.22:0/3772575909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc43410c450 0x7fc43410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.625+0000 7fc433200640 1 --2- 172.21.15.22:0/3772575909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341039a0 0x7fc43410be80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.625+0000 7fc433200640 1 --2- 172.21.15.22:0/3772575909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc434103080 0x7fc434103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.625+0000 7fc433200640 1 -- 172.21.15.22:0/3772575909 >> 172.21.15.22:0/3772575909 conn(0x7fc4340fc7e0 msgr2=0x7fc4340fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:38.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.625+0000 7fc433200640 1 -- 172.21.15.22:0/3772575909 shutdown_connections 2024-04-20T21:36:38.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.625+0000 7fc433200640 1 -- 172.21.15.22:0/3772575909 wait complete. 2024-04-20T21:36:38.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.626+0000 7fc433200640 1 Processor -- start 2024-04-20T21:36:38.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.626+0000 7fc433200640 1 -- start start 2024-04-20T21:36:38.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.626+0000 7fc433200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc434103080 0x7fc4341a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.626+0000 7fc433200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4341039a0 0x7fc4341a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.626+0000 7fc433200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc43410c450 0x7fc43419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.626+0000 7fc433200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc434112230 con 0x7fc43410c450 2024-04-20T21:36:38.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.626+0000 7fc433200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4341120b0 con 0x7fc4341039a0 2024-04-20T21:36:38.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.626+0000 7fc433200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4341123b0 con 0x7fc434103080 2024-04-20T21:36:38.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.627+0000 7fc431400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4341039a0 0x7fc4341a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.627+0000 7fc431400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4341039a0 0x7fc4341a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41236/0 (socket says 172.21.15.22:41236) 2024-04-20T21:36:38.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.627+0000 7fc431400640 1 -- 172.21.15.22:0/3584391016 learned_addr learned my addr 172.21.15.22:0/3584391016 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:38.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.627+0000 7fc431e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc434103080 0x7fc4341a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.627+0000 7fc432800640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc43410c450 0x7fc43419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.627+0000 7fc431400640 1 -- 172.21.15.22:0/3584391016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc434103080 msgr2=0x7fc4341a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.627+0000 7fc431400640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc434103080 0x7fc4341a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.627+0000 7fc431400640 1 -- 172.21.15.22:0/3584391016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc43410c450 msgr2=0x7fc43419a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.627+0000 7fc431400640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc43410c450 0x7fc43419a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.627+0000 7fc431400640 1 -- 172.21.15.22:0/3584391016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43419afe0 con 0x7fc4341039a0 2024-04-20T21:36:38.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.627+0000 7fc431e00640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc434103080 0x7fc4341a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:38.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.627+0000 7fc431400640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4341039a0 0x7fc4341a0b90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc424037040 tx=0x7fc424034440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:38.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.628+0000 7fc42aa00640 1 -- 172.21.15.22:0/3584391016 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc424039040 con 0x7fc4341039a0 2024-04-20T21:36:38.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.628+0000 7fc42aa00640 1 -- 172.21.15.22:0/3584391016 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc424042070 con 0x7fc4341039a0 2024-04-20T21:36:38.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.628+0000 7fc42aa00640 1 -- 172.21.15.22:0/3584391016 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc424003920 con 0x7fc4341039a0 2024-04-20T21:36:38.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.628+0000 7fc433200640 1 -- 172.21.15.22:0/3584391016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43419b270 con 0x7fc4341039a0 2024-04-20T21:36:38.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.628+0000 7fc433200640 1 -- 172.21.15.22:0/3584391016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4341a74a0 con 0x7fc4341039a0 2024-04-20T21:36:38.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.631+0000 7fc43bf84740 1 Processor -- start 2024-04-20T21:36:38.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.631+0000 7fc43bf84740 1 -- start start 2024-04-20T21:36:38.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc43bf84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47b1170 0x55e5c47b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc42aa00640 1 -- 172.21.15.22:0/3584391016 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc42402c6d0 con 0x7fc4341039a0 2024-04-20T21:36:38.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc43bf84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1a90 0x55e5c47ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc43bf84740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47ba670 0x55e5c47bca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc43bf84740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5c4716050 con 0x55e5c47ba670 2024-04-20T21:36:38.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc43bf84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e5c46bc490 con 0x55e5c47b1170 2024-04-20T21:36:38.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc43bf84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e5c4712010 con 0x55e5c47b1a90 2024-04-20T21:36:38.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc432800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47b1170 0x55e5c47b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc432800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47b1170 0x55e5c47b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41244/0 (socket says 172.21.15.22:41244) 2024-04-20T21:36:38.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc431e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1a90 0x55e5c47ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc432800640 1 -- 172.21.15.22:0/865499057 learned_addr learned my addr 172.21.15.22:0/865499057 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:38.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc431400640 1 --2- 172.21.15.22:0/865499057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47ba670 0x55e5c47bca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc431e00640 1 -- 172.21.15.22:0/865499057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47b1170 msgr2=0x55e5c47b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:38.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc431e00640 1 --2- 172.21.15.22:0/865499057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47b1170 0x55e5c47b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc431e00640 1 -- 172.21.15.22:0/865499057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47ba670 msgr2=0x55e5c47bca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:38.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc431e00640 1 --2- 172.21.15.22:0/865499057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47ba670 0x55e5c47bca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc431e00640 1 -- 172.21.15.22:0/865499057 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c46c4070 con 0x55e5c47b1a90 2024-04-20T21:36:38.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.632+0000 7fc432800640 1 --2- 172.21.15.22:0/865499057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47b1170 0x55e5c47b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.633+0000 7fc431400640 1 --2- 172.21.15.22:0/865499057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47ba670 0x55e5c47bca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.633+0000 7fc431e00640 1 --2- 172.21.15.22:0/865499057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1a90 0x55e5c47ba130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009fd0 tx=0x7fc41c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22a5925abdc1447f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.633+0000 7fc423400640 1 -- 172.21.15.22:0/865499057 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c01e070 con 0x55e5c47b1a90 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.633+0000 7fc423400640 1 -- 172.21.15.22:0/865499057 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc41c004310 con 0x55e5c47b1a90 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.633+0000 7fc423400640 1 -- 172.21.15.22:0/865499057 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00f910 con 0x55e5c47b1a90 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.634+0000 7fc42aa00640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc4180a3de0 0x7fc4180a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.634+0000 7fc42aa00640 1 -- 172.21.15.22:0/3584391016 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc4240f1b80 con 0x7fc4341039a0 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.634+0000 7fc43bf84740 1 -- 172.21.15.22:0/865499057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1a90 msgr2=0x55e5c47ba130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.634+0000 7fc43bf84740 1 --2- 172.21.15.22:0/865499057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1a90 0x55e5c47ba130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009fd0 tx=0x7fc41c0118d0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.634+0000 7fc432800640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc4180a3de0 0x7fc4180a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.634+0000 7fc43bf84740 1 -- 172.21.15.22:0/865499057 shutdown_connections 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.634+0000 7fc43bf84740 1 --2- 172.21.15.22:0/865499057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47ba670 0x55e5c47bca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.634+0000 7fc43bf84740 1 --2- 172.21.15.22:0/865499057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1a90 0x55e5c47ba130 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.634+0000 7fc43bf84740 1 --2- 172.21.15.22:0/865499057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47b1170 0x55e5c47b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.634+0000 7fc43bf84740 1 -- 172.21.15.22:0/865499057 >> 172.21.15.22:0/865499057 conn(0x55e5c47ab8a0 msgr2=0x55e5c47abc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:38.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.634+0000 7fc43bf84740 1 -- 172.21.15.22:0/865499057 shutdown_connections 2024-04-20T21:36:38.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.634+0000 7fc43bf84740 1 -- 172.21.15.22:0/865499057 wait complete. 2024-04-20T21:36:38.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.635+0000 7fc43bf84740 1 -- start start 2024-04-20T21:36:38.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.635+0000 7fc432800640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc4180a3de0 0x7fc4180a62a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009fd0 tx=0x7fc42c010040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:38.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.635+0000 7fc43bf84740 1 Processor -- start 2024-04-20T21:36:38.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc43bf84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1170 0x55e5c47b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc43bf84740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1a90 0x55e5c47b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc43bf84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47b8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc43bf84740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5c47bd460 con 0x55e5c47b1a90 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc43bf84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e5c46cf860 con 0x55e5c47ba670 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc43bf84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e5c47bd5e0 con 0x55e5c47b1170 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc43bf84740 10 client.0 _ll_register_callbacks cb 0x55e5c469dfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc432800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc432800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47b8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41256/0 (socket says 172.21.15.22:41256) 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc432800640 1 -- 172.21.15.22:0/3247690825 learned_addr learned my addr 172.21.15.22:0/3247690825 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc431400640 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1a90 0x55e5c47b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc431e00640 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1170 0x55e5c47b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc432800640 1 -- 172.21.15.22:0/3247690825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1170 msgr2=0x55e5c47b76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:38.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc432800640 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1170 0x55e5c47b76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc432800640 1 -- 172.21.15.22:0/3247690825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1a90 msgr2=0x55e5c47b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc432800640 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1a90 0x55e5c47b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.636+0000 7fc432800640 1 -- 172.21.15.22:0/3247690825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c46d3950 con 0x55e5c47ba670 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.637+0000 7fc432800640 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47b8170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00df90 tx=0x7fc42c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6806372b4008a660 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.637+0000 7fc417e00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.637+0000 7fc417e00640 1 -- 172.21.15.22:0/3247690825 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c02a060 con 0x55e5c47ba670 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.637+0000 7fc417e00640 1 -- 172.21.15.22:0/3247690825 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc42c003970 con 0x55e5c47ba670 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.637+0000 7fc43bf84740 1 -- 172.21.15.22:0/3247690825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc41c004310 con 0x55e5c47ba670 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.637+0000 7fc417e00640 1 -- 172.21.15.22:0/3247690825 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc42c003b70 con 0x55e5c47ba670 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.637+0000 7fc43bf84740 10 client.25697 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.637+0000 7fc43bf84740 1 -- 172.21.15.22:0/3247690825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e5c46a1820 con 0x55e5c47ba670 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.638+0000 7fc417e00640 1 -- 172.21.15.22:0/3247690825 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc41c004310 con 0x55e5c47ba670 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.638+0000 7fc417e00640 1 -- 172.21.15.22:0/3247690825 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fc42c024070 con 0x55e5c47ba670 2024-04-20T21:36:38.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.638+0000 7fc43bf84740 10 client.25697 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.638+0000 7fc43bf84740 10 client.25697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.638+0000 7fc43bf84740 20 client.25697 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.638+0000 7fc43bf84740 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e5c47ca6b0 0x55e5c47ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.638+0000 7fc43bf84740 4 client.25697 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.638+0000 7fc43bf84740 1 -- 172.21.15.22:0/3247690825 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5c4723a70 con 0x55e5c47ca6b0 2024-04-20T21:36:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.638+0000 7fc431e00640 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e5c47ca6b0 0x55e5c47ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.639+0000 7fc431e00640 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e5c47ca6b0 0x55e5c47ccac0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c980a92115c64d5 server_cookie=240f1af52729d21b in_seq=0 out_seq=0 2024-04-20T21:36:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.639+0000 7fc417e00640 10 client.25697 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:38.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc417e00640 1 -- 172.21.15.22:0/3247690825 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5c4723a70 con 0x55e5c47ca6b0 2024-04-20T21:36:38.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc417e00640 10 client.25697 handle_command_reply: tid=0 2024-04-20T21:36:38.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc43bf84740 1 client.25697 shutdown 2024-04-20T21:36:38.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc43bf84740 1 -- 172.21.15.22:0/3247690825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 msgr2=0x55e5c47b8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc43bf84740 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47b8170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00df90 tx=0x7fc42c004ab0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc43bf84740 1 -- 172.21.15.22:0/3247690825 shutdown_connections 2024-04-20T21:36:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc43bf84740 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e5c47ca6b0 0x55e5c47ccac0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc43bf84740 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47b8170 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc43bf84740 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1a90 0x55e5c47b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc417e00640 0 client.25697 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc43bf84740 1 --2- 172.21.15.22:0/3247690825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1170 0x55e5c47b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc43bf84740 1 -- 172.21.15.22:0/3247690825 >> 172.21.15.22:0/3247690825 conn(0x55e5c47ab8a0 msgr2=0x55e5c47ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc43bf84740 1 -- 172.21.15.22:0/3247690825 shutdown_connections 2024-04-20T21:36:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc43bf84740 1 -- 172.21.15.22:0/3247690825 wait complete. 2024-04-20T21:36:38.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.641+0000 7fc43bf84740 20 client.25697 trim_cache size 0 max 16384 2024-04-20T21:36:38.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.656+0000 7fc43bf84740 1 Processor -- start 2024-04-20T21:36:38.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.656+0000 7fc43bf84740 1 -- start start 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.657+0000 7fc43bf84740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1170 0x55e5c47b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.657+0000 7fc43bf84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1a90 0x55e5c47c6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.657+0000 7fc43bf84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.657+0000 7fc43bf84740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc4341addd0 con 0x55e5c47b1170 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.657+0000 7fc43bf84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4341b0650 con 0x55e5c47ba670 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.657+0000 7fc43bf84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4341ae440 con 0x55e5c47b1a90 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.657+0000 7fc431e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1170 0x55e5c47b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.657+0000 7fc431e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1170 0x55e5c47b9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40524/0 (socket says 172.21.15.22:40524) 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.657+0000 7fc431e00640 1 -- 172.21.15.22:0/1889567516 learned_addr learned my addr 172.21.15.22:0/1889567516 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.657+0000 7fc431400640 1 --2- 172.21.15.22:0/1889567516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1a90 0x55e5c47c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.657+0000 7fc432800640 1 --2- 172.21.15.22:0/1889567516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.658+0000 7fc431400640 1 -- 172.21.15.22:0/1889567516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 msgr2=0x55e5c47d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.658+0000 7fc431400640 1 --2- 172.21.15.22:0/1889567516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47d2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.658+0000 7fc431400640 1 -- 172.21.15.22:0/1889567516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1170 msgr2=0x55e5c47b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:38.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.658+0000 7fc431400640 1 --2- 172.21.15.22:0/1889567516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1170 0x55e5c47b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.658+0000 7fc431400640 1 -- 172.21.15.22:0/1889567516 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c47bcdd0 con 0x55e5c47b1a90 2024-04-20T21:36:38.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.658+0000 7fc431e00640 1 --2- 172.21.15.22:0/1889567516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1170 0x55e5c47b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:38.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.658+0000 7fc432800640 1 --2- 172.21.15.22:0/1889567516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:38.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.658+0000 7fc431400640 1 --2- 172.21.15.22:0/1889567516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1a90 0x55e5c47c6350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc4240ef360 tx=0x7fc42402c5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66d113ec35a337ed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:38.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.658+0000 7fc422a00640 1 -- 172.21.15.22:0/1889567516 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4240be2b0 con 0x55e5c47b1a90 2024-04-20T21:36:38.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.658+0000 7fc422a00640 1 -- 172.21.15.22:0/1889567516 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4240bdbb0 con 0x55e5c47b1a90 2024-04-20T21:36:38.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.658+0000 7fc422a00640 1 -- 172.21.15.22:0/1889567516 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4240ba220 con 0x55e5c47b1a90 2024-04-20T21:36:38.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.659+0000 7fc43bf84740 1 -- 172.21.15.22:0/1889567516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1a90 msgr2=0x55e5c47c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.659+0000 7fc43bf84740 1 --2- 172.21.15.22:0/1889567516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1a90 0x55e5c47c6350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc4240ef360 tx=0x7fc42402c5b0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.659+0000 7fc43bf84740 1 -- 172.21.15.22:0/1889567516 shutdown_connections 2024-04-20T21:36:38.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.659+0000 7fc43bf84740 1 --2- 172.21.15.22:0/1889567516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.659+0000 7fc43bf84740 1 --2- 172.21.15.22:0/1889567516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1a90 0x55e5c47c6350 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.659+0000 7fc43bf84740 1 --2- 172.21.15.22:0/1889567516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1170 0x55e5c47b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.659+0000 7fc43bf84740 1 -- 172.21.15.22:0/1889567516 >> 172.21.15.22:0/1889567516 conn(0x55e5c47ab8a0 msgr2=0x55e5c47b6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:38.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.659+0000 7fc43bf84740 1 -- 172.21.15.22:0/1889567516 shutdown_connections 2024-04-20T21:36:38.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.660+0000 7fc43bf84740 1 -- 172.21.15.22:0/1889567516 wait complete. 2024-04-20T21:36:38.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.660+0000 7fc43bf84740 1 -- start start 2024-04-20T21:36:38.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.660+0000 7fc43bf84740 1 Processor -- start 2024-04-20T21:36:38.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.661+0000 7fc43bf84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1170 0x55e5c47c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.661+0000 7fc43bf84740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1a90 0x55e5c47b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.661+0000 7fc43bf84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.661+0000 7fc43bf84740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc4341aeb10 con 0x55e5c47b1a90 2024-04-20T21:36:38.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.661+0000 7fc43bf84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4341aff80 con 0x55e5c47ba670 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.661+0000 7fc43bf84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4341af1e0 con 0x55e5c47b1170 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.661+0000 7fc43bf84740 10 client.0 _ll_register_callbacks cb 0x7fc42c00d960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.661+0000 7fc431e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1170 0x55e5c47c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.661+0000 7fc431e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1170 0x55e5c47c2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33416/0 (socket says 172.21.15.22:33416) 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.661+0000 7fc431e00640 1 -- 172.21.15.22:0/443436996 learned_addr learned my addr 172.21.15.22:0/443436996 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.661+0000 7fc432800640 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.662+0000 7fc431e00640 1 -- 172.21.15.22:0/443436996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 msgr2=0x55e5c47b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.662+0000 7fc431400640 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1a90 0x55e5c47b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.662+0000 7fc431e00640 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.662+0000 7fc431e00640 1 -- 172.21.15.22:0/443436996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1a90 msgr2=0x55e5c47b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.662+0000 7fc431e00640 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1a90 0x55e5c47b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.662+0000 7fc431e00640 1 -- 172.21.15.22:0/443436996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c44283f0 con 0x55e5c47b1170 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.662+0000 7fc431e00640 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1170 0x55e5c47c2c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc41c011dd0 tx=0x7fc41c004190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b13bc1ccd2008e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:38.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.662+0000 7fc417e00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:36:38.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.662+0000 7fc417e00640 1 -- 172.21.15.22:0/443436996 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c031060 con 0x55e5c47b1170 2024-04-20T21:36:38.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.662+0000 7fc417e00640 1 -- 172.21.15.22:0/443436996 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc41c005450 con 0x55e5c47b1170 2024-04-20T21:36:38.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.662+0000 7fc43bf84740 1 -- 172.21.15.22:0/443436996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4240bdbb0 con 0x55e5c47b1170 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.663+0000 7fc417e00640 1 -- 172.21.15.22:0/443436996 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc41c01e040 con 0x55e5c47b1170 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.663+0000 7fc43bf84740 10 client.25735 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.663+0000 7fc43bf84740 1 -- 172.21.15.22:0/443436996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e5c44f27d0 con 0x55e5c47b1170 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.663+0000 7fc417e00640 1 -- 172.21.15.22:0/443436996 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4240bdbb0 con 0x55e5c47b1170 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.663+0000 7fc417e00640 1 -- 172.21.15.22:0/443436996 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fc41c0048b0 con 0x55e5c47b1170 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.663+0000 7fc43bf84740 10 client.25735 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.663+0000 7fc43bf84740 10 client.25735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.663+0000 7fc43bf84740 20 client.25735 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.663+0000 7fc43bf84740 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e5c4830de0 0x55e5c4833200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.663+0000 7fc43bf84740 4 client.25735 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.664+0000 7fc43bf84740 1 -- 172.21.15.22:0/443436996 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5c471d8e0 con 0x55e5c4830de0 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.664+0000 7fc431400640 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e5c4830de0 0x55e5c4833200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:38.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.664+0000 7fc431400640 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e5c4830de0 0x55e5c4833200 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dcbfbdefeda4bad server_cookie=18043508185ac699 in_seq=0 out_seq=0 2024-04-20T21:36:38.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.664+0000 7fc417e00640 10 client.25735 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:38.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.668+0000 7fc417e00640 1 -- 172.21.15.22:0/443436996 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5c471d8e0 con 0x55e5c4830de0 2024-04-20T21:36:38.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.668+0000 7fc417e00640 10 client.25735 handle_command_reply: tid=0 2024-04-20T21:36:38.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.668+0000 7fc43bf84740 1 client.25735 shutdown 2024-04-20T21:36:38.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.669+0000 7fc43bf84740 1 -- 172.21.15.22:0/443436996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1170 msgr2=0x55e5c47c2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.669+0000 7fc43bf84740 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1170 0x55e5c47c2c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc41c011dd0 tx=0x7fc41c004190 comp rx=0 tx=0).stop 2024-04-20T21:36:38.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.669+0000 7fc43bf84740 1 -- 172.21.15.22:0/443436996 shutdown_connections 2024-04-20T21:36:38.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.669+0000 7fc43bf84740 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e5c4830de0 0x55e5c4833200 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.669+0000 7fc43bf84740 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5c47ba670 0x55e5c47b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.669+0000 7fc43bf84740 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c47b1a90 0x55e5c47b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.669+0000 7fc43bf84740 1 --2- 172.21.15.22:0/443436996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5c47b1170 0x55e5c47c2c50 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc41c011dd0 tx=0x7fc41c004190 comp rx=0 tx=0).stop 2024-04-20T21:36:38.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.669+0000 7fc43bf84740 1 -- 172.21.15.22:0/443436996 >> 172.21.15.22:0/443436996 conn(0x55e5c47ab8a0 msgr2=0x55e5c47aa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:38.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.669+0000 7fc417e00640 0 client.25735 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:38.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.671+0000 7fc43bf84740 1 -- 172.21.15.22:0/443436996 shutdown_connections 2024-04-20T21:36:38.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.672+0000 7fc43bf84740 1 -- 172.21.15.22:0/443436996 wait complete. 2024-04-20T21:36:38.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.672+0000 7fc43bf84740 20 client.25735 trim_cache size 0 max 16384 2024-04-20T21:36:38.673 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:36:38.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.676+0000 7fc433200640 1 -- 172.21.15.22:0/3584391016 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc4180a3de0 msgr2=0x7fc4180a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.676+0000 7fc433200640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc4180a3de0 0x7fc4180a62a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009fd0 tx=0x7fc42c010040 comp rx=0 tx=0).stop 2024-04-20T21:36:38.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.676+0000 7fc433200640 1 -- 172.21.15.22:0/3584391016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4341039a0 msgr2=0x7fc4341a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:38.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.676+0000 7fc433200640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4341039a0 0x7fc4341a0b90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc424037040 tx=0x7fc424034440 comp rx=0 tx=0).stop 2024-04-20T21:36:38.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.677+0000 7fc433200640 1 -- 172.21.15.22:0/3584391016 shutdown_connections 2024-04-20T21:36:38.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.677+0000 7fc433200640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc4180a3de0 0x7fc4180a62a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.677+0000 7fc433200640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc43410c450 0x7fc43419a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.677+0000 7fc433200640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4341039a0 0x7fc4341a0b90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.677+0000 7fc433200640 1 --2- 172.21.15.22:0/3584391016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc434103080 0x7fc4341a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:38.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.677+0000 7fc433200640 1 -- 172.21.15.22:0/3584391016 >> 172.21.15.22:0/3584391016 conn(0x7fc4340fc7e0 msgr2=0x7fc434101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:38.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.678+0000 7fc433200640 1 -- 172.21.15.22:0/3584391016 shutdown_connections 2024-04-20T21:36:38.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:38.678+0000 7fc433200640 1 -- 172.21.15.22:0/3584391016 wait complete. 2024-04-20T21:36:39.169 INFO:tasks.workunit.client.0.smithi022.stdout: 1 132465 10.90 MB/sec warmup 67 sec latency 59.427 ms 2024-04-20T21:36:39.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:39 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/4090847720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:36:39.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:39 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/4090847720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:36:39.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:39 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/4090847720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:36:39.714 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:36:39.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:36:39.715 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:36:39.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.942+0000 7fb02b200640 1 -- 172.21.15.22:0/120367344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb024101570 msgr2=0x7fb02410e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:39.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.942+0000 7fb02b200640 1 --2- 172.21.15.22:0/120367344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb024101570 0x7fb02410e230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb014009860 tx=0x7fb01402bb70 comp rx=0 tx=0).stop 2024-04-20T21:36:39.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.942+0000 7fb02b200640 1 -- 172.21.15.22:0/120367344 shutdown_connections 2024-04-20T21:36:39.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.942+0000 7fb02b200640 1 --2- 172.21.15.22:0/120367344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb02410e770 0x7fb024110b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.942+0000 7fb02b200640 1 --2- 172.21.15.22:0/120367344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb024101570 0x7fb02410e230 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.942+0000 7fb02b200640 1 --2- 172.21.15.22:0/120367344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb024100bc0 0x7fb024100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.942+0000 7fb02b200640 1 -- 172.21.15.22:0/120367344 >> 172.21.15.22:0/120367344 conn(0x7fb0240fc7e0 msgr2=0x7fb0240fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:39.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.942+0000 7fb02b200640 1 -- 172.21.15.22:0/120367344 shutdown_connections 2024-04-20T21:36:39.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.942+0000 7fb02b200640 1 -- 172.21.15.22:0/120367344 wait complete. 2024-04-20T21:36:39.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.943+0000 7fb02b200640 1 Processor -- start 2024-04-20T21:36:39.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.944+0000 7fb02b200640 1 -- start start 2024-04-20T21:36:39.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.944+0000 7fb02b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb024100bc0 0x7fb024102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.944+0000 7fb02b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb024101570 0x7fb0241033e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.944+0000 7fb02b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb02410e770 0x7fb024103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.944+0000 7fb02b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb024114400 con 0x7fb02410e770 2024-04-20T21:36:39.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.944+0000 7fb02b200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb024114280 con 0x7fb024101570 2024-04-20T21:36:39.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.944+0000 7fb02b200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb024114580 con 0x7fb024100bc0 2024-04-20T21:36:39.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.945+0000 7fb029e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb024100bc0 0x7fb024102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.945+0000 7fb029e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb024100bc0 0x7fb024102ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33424/0 (socket says 172.21.15.22:33424) 2024-04-20T21:36:39.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.945+0000 7fb029e00640 1 -- 172.21.15.22:0/357353931 learned_addr learned my addr 172.21.15.22:0/357353931 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:39.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.945+0000 7fb029400640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb024101570 0x7fb0241033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.945+0000 7fb02a800640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb02410e770 0x7fb024103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.945+0000 7fb029400640 1 -- 172.21.15.22:0/357353931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb024100bc0 msgr2=0x7fb024102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:39.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.945+0000 7fb029400640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb024100bc0 0x7fb024102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.945+0000 7fb029400640 1 -- 172.21.15.22:0/357353931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb02410e770 msgr2=0x7fb024103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:39.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.945+0000 7fb029400640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb02410e770 0x7fb024103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.945+0000 7fb029400640 1 -- 172.21.15.22:0/357353931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0241aaf90 con 0x7fb024101570 2024-04-20T21:36:39.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.946+0000 7fb029400640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb024101570 0x7fb0241033e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb014009780 tx=0x7fb0140045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:39.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.946+0000 7fb01ea00640 1 -- 172.21.15.22:0/357353931 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014041020 con 0x7fb024101570 2024-04-20T21:36:39.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.946+0000 7fb01ea00640 1 -- 172.21.15.22:0/357353931 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb01402c9f0 con 0x7fb024101570 2024-04-20T21:36:39.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.946+0000 7fb02b200640 1 -- 172.21.15.22:0/357353931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0241ab220 con 0x7fb024101570 2024-04-20T21:36:39.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.946+0000 7fb01ea00640 1 -- 172.21.15.22:0/357353931 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014039040 con 0x7fb024101570 2024-04-20T21:36:39.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.946+0000 7fb02b200640 1 -- 172.21.15.22:0/357353931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0241ab760 con 0x7fb024101570 2024-04-20T21:36:39.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.949+0000 7fb02f60c740 1 Processor -- start 2024-04-20T21:36:39.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.950+0000 7fb02f60c740 1 -- start start 2024-04-20T21:36:39.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.950+0000 7fb02f60c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a997350f0 0x558a997354d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.950+0000 7fb02f60c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 0x558a9973e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.950+0000 7fb02f60c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a9973e5f0 0x558a997409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.950+0000 7fb02f60c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a99699d20 con 0x558a997350f0 2024-04-20T21:36:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.950+0000 7fb02f60c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558a996403e0 con 0x558a9973e5f0 2024-04-20T21:36:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.950+0000 7fb02f60c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558a99695ce0 con 0x558a99735a10 2024-04-20T21:36:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.950+0000 7fb02a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a997350f0 0x558a997354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.950+0000 7fb029e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 0x558a9973e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.950+0000 7fb02a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a997350f0 0x558a997354d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40570/0 (socket says 172.21.15.22:40570) 2024-04-20T21:36:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.950+0000 7fb02a800640 1 -- 172.21.15.22:0/4032008233 learned_addr learned my addr 172.21.15.22:0/4032008233 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.950+0000 7fb01ea00640 1 -- 172.21.15.22:0/357353931 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb014044020 con 0x7fb024101570 2024-04-20T21:36:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.951+0000 7fb029e00640 1 -- 172.21.15.22:0/4032008233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a9973e5f0 msgr2=0x558a997409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.951+0000 7fb029e00640 1 --2- 172.21.15.22:0/4032008233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a9973e5f0 0x558a997409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.951+0000 7fb029e00640 1 -- 172.21.15.22:0/4032008233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a997350f0 msgr2=0x558a997354d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.951+0000 7fb029e00640 1 --2- 172.21.15.22:0/4032008233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a997350f0 0x558a997354d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.951+0000 7fb029e00640 1 -- 172.21.15.22:0/4032008233 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9965f980 con 0x558a99735a10 2024-04-20T21:36:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.951+0000 7fb029e00640 1 --2- 172.21.15.22:0/4032008233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 0x558a9973e0b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb018009cc0 tx=0x7fb018011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63ef4e7ed8591ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.951+0000 7fb013000640 1 -- 172.21.15.22:0/4032008233 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018004230 con 0x558a99735a10 2024-04-20T21:36:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.951+0000 7fb013000640 1 -- 172.21.15.22:0/4032008233 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0180043d0 con 0x558a99735a10 2024-04-20T21:36:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.951+0000 7fb013000640 1 -- 172.21.15.22:0/4032008233 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018004e10 con 0x558a99735a10 2024-04-20T21:36:39.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.952+0000 7fb02f60c740 1 -- 172.21.15.22:0/4032008233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 msgr2=0x558a9973e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:39.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.952+0000 7fb02f60c740 1 --2- 172.21.15.22:0/4032008233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 0x558a9973e0b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb018009cc0 tx=0x7fb018011810 comp rx=0 tx=0).stop 2024-04-20T21:36:39.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.952+0000 7fb01ea00640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb00c0a3da0 0x7fb00c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.952+0000 7fb02f60c740 1 -- 172.21.15.22:0/4032008233 shutdown_connections 2024-04-20T21:36:39.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.952+0000 7fb02f60c740 1 --2- 172.21.15.22:0/4032008233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a9973e5f0 0x558a997409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.952+0000 7fb02f60c740 1 --2- 172.21.15.22:0/4032008233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 0x558a9973e0b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.952+0000 7fb02f60c740 1 --2- 172.21.15.22:0/4032008233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a997350f0 0x558a997354d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.952+0000 7fb01ea00640 1 -- 172.21.15.22:0/357353931 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fb0140f1d80 con 0x7fb024101570 2024-04-20T21:36:39.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.952+0000 7fb02f60c740 1 -- 172.21.15.22:0/4032008233 >> 172.21.15.22:0/4032008233 conn(0x558a9972f820 msgr2=0x558a9972fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:39.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.953+0000 7fb02a800640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb00c0a3da0 0x7fb00c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.953+0000 7fb02f60c740 1 -- 172.21.15.22:0/4032008233 shutdown_connections 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.953+0000 7fb02f60c740 1 -- 172.21.15.22:0/4032008233 wait complete. 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.953+0000 7fb02a800640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb00c0a3da0 0x7fb00c0a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb020005fd0 tx=0x7fb020009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.953+0000 7fb02f60c740 1 -- start start 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.954+0000 7fb02f60c740 1 Processor -- start 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.954+0000 7fb02f60c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a997350f0 0x558a9973b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.954+0000 7fb02f60c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 0x558a9973bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.954+0000 7fb02f60c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9973e5f0 0x558a9973c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.954+0000 7fb02f60c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a997413e0 con 0x558a9973e5f0 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.954+0000 7fb02f60c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558a996537a0 con 0x558a997350f0 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.954+0000 7fb02f60c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558a99741560 con 0x558a99735a10 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.954+0000 7fb02f60c740 10 client.0 _ll_register_callbacks cb 0x558a99655b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb029400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 0x558a9973bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb029e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a997350f0 0x558a9973b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb029400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 0x558a9973bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33432/0 (socket says 172.21.15.22:33432) 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb029e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a997350f0 0x558a9973b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41316/0 (socket says 172.21.15.22:41316) 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb029400640 1 -- 172.21.15.22:0/1406412329 learned_addr learned my addr 172.21.15.22:0/1406412329 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb02a800640 1 --2- 172.21.15.22:0/1406412329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9973e5f0 0x558a9973c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb029400640 1 -- 172.21.15.22:0/1406412329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a997350f0 msgr2=0x558a9973b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb029400640 1 --2- 172.21.15.22:0/1406412329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a997350f0 0x558a9973b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb029400640 1 -- 172.21.15.22:0/1406412329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9973e5f0 msgr2=0x558a9973c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb029400640 1 --2- 172.21.15.22:0/1406412329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9973e5f0 0x558a9973c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb029400640 1 -- 172.21.15.22:0/1406412329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a994747e0 con 0x558a99735a10 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb029400640 1 --2- 172.21.15.22:0/1406412329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 0x558a9973bb60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb01402c180 tx=0x7fb0140bd9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee4519dd3c39576c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.955+0000 7fb007400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.956+0000 7fb007400640 1 -- 172.21.15.22:0/1406412329 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140f3070 con 0x558a99735a10 2024-04-20T21:36:39.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.956+0000 7fb007400640 1 -- 172.21.15.22:0/1406412329 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fb0140bc0b0 con 0x558a99735a10 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.956+0000 7fb02f60c740 1 -- 172.21.15.22:0/1406412329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0180043d0 con 0x558a99735a10 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.956+0000 7fb007400640 1 -- 172.21.15.22:0/1406412329 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb0140c2160 con 0x558a99735a10 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.956+0000 7fb02f60c740 10 client.25747 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.956+0000 7fb02f60c740 1 -- 172.21.15.22:0/1406412329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558a994d4f70 con 0x558a99735a10 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.956+0000 7fb007400640 1 -- 172.21.15.22:0/1406412329 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0180043d0 con 0x558a99735a10 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.956+0000 7fb007400640 1 -- 172.21.15.22:0/1406412329 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fb0140a8040 con 0x558a99735a10 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.956+0000 7fb02f60c740 10 client.25747 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.956+0000 7fb02f60c740 10 client.25747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.957+0000 7fb02f60c740 20 client.25747 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.957+0000 7fb02f60c740 1 --2- 172.21.15.22:0/1406412329 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a9974e970 0x558a99750db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.957+0000 7fb02f60c740 4 client.25747 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.957+0000 7fb02f60c740 1 -- 172.21.15.22:0/1406412329 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a996a7740 con 0x558a9974e970 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.957+0000 7fb029e00640 1 --2- 172.21.15.22:0/1406412329 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a9974e970 0x558a99750db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.958+0000 7fb029e00640 1 --2- 172.21.15.22:0/1406412329 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a9974e970 0x558a99750db0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64b6e1f8b2812ae2 server_cookie=da6bc8664ab7d1aa in_seq=0 out_seq=0 2024-04-20T21:36:39.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.958+0000 7fb007400640 10 client.25747 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:39.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.959+0000 7fb007400640 1 -- 172.21.15.22:0/1406412329 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a996a7740 con 0x558a9974e970 2024-04-20T21:36:39.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.959+0000 7fb007400640 10 client.25747 handle_command_reply: tid=0 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.959+0000 7fb02f60c740 1 client.25747 shutdown 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.959+0000 7fb02f60c740 1 -- 172.21.15.22:0/1406412329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 msgr2=0x558a9973bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.959+0000 7fb02f60c740 1 --2- 172.21.15.22:0/1406412329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 0x558a9973bb60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb01402c180 tx=0x7fb0140bd9b0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.960+0000 7fb02f60c740 1 -- 172.21.15.22:0/1406412329 shutdown_connections 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.960+0000 7fb02f60c740 1 --2- 172.21.15.22:0/1406412329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9973e5f0 0x558a9973c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.960+0000 7fb02f60c740 1 --2- 172.21.15.22:0/1406412329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a99735a10 0x558a9973bb60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.960+0000 7fb02f60c740 1 --2- 172.21.15.22:0/1406412329 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a9974e970 0x558a99750db0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.960+0000 7fb02f60c740 1 --2- 172.21.15.22:0/1406412329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a997350f0 0x558a9973b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.960+0000 7fb02f60c740 1 -- 172.21.15.22:0/1406412329 >> 172.21.15.22:0/1406412329 conn(0x558a9972f820 msgr2=0x558a99731570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.960+0000 7fb007400640 0 client.25747 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.960+0000 7fb02f60c740 1 -- 172.21.15.22:0/1406412329 shutdown_connections 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.960+0000 7fb02f60c740 1 -- 172.21.15.22:0/1406412329 wait complete. 2024-04-20T21:36:39.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.960+0000 7fb02f60c740 20 client.25747 trim_cache size 0 max 16384 2024-04-20T21:36:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.981+0000 7fb02f60c740 1 Processor -- start 2024-04-20T21:36:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.981+0000 7fb02f60c740 1 -- start start 2024-04-20T21:36:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.981+0000 7fb02f60c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a997350f0 0x558a9973dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb02f60c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 0x558a9974a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb02f60c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9973e5f0 0x558a99731380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb02f60c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241b2120 con 0x558a9973e5f0 2024-04-20T21:36:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb02f60c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241b49a0 con 0x558a99735a10 2024-04-20T21:36:39.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb02f60c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241b2790 con 0x558a997350f0 2024-04-20T21:36:39.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb029400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 0x558a9974a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb02a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9973e5f0 0x558a99731380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb029400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 0x558a9974a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41322/0 (socket says 172.21.15.22:41322) 2024-04-20T21:36:39.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb029400640 1 -- 172.21.15.22:0/2841776362 learned_addr learned my addr 172.21.15.22:0/2841776362 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:39.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb029e00640 1 --2- 172.21.15.22:0/2841776362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a997350f0 0x558a9973dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb029400640 1 -- 172.21.15.22:0/2841776362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a997350f0 msgr2=0x558a9973dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:39.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb029400640 1 --2- 172.21.15.22:0/2841776362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a997350f0 0x558a9973dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb029400640 1 -- 172.21.15.22:0/2841776362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9973e5f0 msgr2=0x558a99731380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:39.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb029400640 1 --2- 172.21.15.22:0/2841776362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9973e5f0 0x558a99731380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.982+0000 7fb029400640 1 -- 172.21.15.22:0/2841776362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9936c170 con 0x558a99735a10 2024-04-20T21:36:39.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.983+0000 7fb029400640 1 --2- 172.21.15.22:0/2841776362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 0x558a9974a2d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb014000c00 tx=0x7fb014005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1381fa1ca68f3766 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:39.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.983+0000 7fb012600640 1 -- 172.21.15.22:0/2841776362 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140c6a20 con 0x558a99735a10 2024-04-20T21:36:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.983+0000 7fb012600640 1 -- 172.21.15.22:0/2841776362 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0140c6760 con 0x558a99735a10 2024-04-20T21:36:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.983+0000 7fb012600640 1 -- 172.21.15.22:0/2841776362 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140bd460 con 0x558a99735a10 2024-04-20T21:36:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.984+0000 7fb02f60c740 1 -- 172.21.15.22:0/2841776362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 msgr2=0x558a9974a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.984+0000 7fb02f60c740 1 --2- 172.21.15.22:0/2841776362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 0x558a9974a2d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb014000c00 tx=0x7fb014005f90 comp rx=0 tx=0).stop 2024-04-20T21:36:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.984+0000 7fb02f60c740 1 -- 172.21.15.22:0/2841776362 shutdown_connections 2024-04-20T21:36:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.984+0000 7fb02f60c740 1 --2- 172.21.15.22:0/2841776362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9973e5f0 0x558a99731380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.984+0000 7fb02f60c740 1 --2- 172.21.15.22:0/2841776362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 0x558a9974a2d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.984+0000 7fb02f60c740 1 --2- 172.21.15.22:0/2841776362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a997350f0 0x558a9973dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.984+0000 7fb02f60c740 1 -- 172.21.15.22:0/2841776362 >> 172.21.15.22:0/2841776362 conn(0x558a9972f820 msgr2=0x558a99730c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.984+0000 7fb02f60c740 1 -- 172.21.15.22:0/2841776362 shutdown_connections 2024-04-20T21:36:39.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.984+0000 7fb02f60c740 1 -- 172.21.15.22:0/2841776362 wait complete. 2024-04-20T21:36:39.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.985+0000 7fb02f60c740 1 -- start start 2024-04-20T21:36:39.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.985+0000 7fb02f60c740 1 Processor -- start 2024-04-20T21:36:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.985+0000 7fb02f60c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a997350f0 0x558a99746c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.985+0000 7fb02f60c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 0x558a99742000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.985+0000 7fb02f60c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a9973e5f0 0x558a99742610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.985+0000 7fb02f60c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241b2e60 con 0x558a997350f0 2024-04-20T21:36:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.985+0000 7fb02f60c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241b42d0 con 0x558a99735a10 2024-04-20T21:36:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.985+0000 7fb02f60c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241b3530 con 0x558a9973e5f0 2024-04-20T21:36:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.985+0000 7fb02f60c740 10 client.0 _ll_register_callbacks cb 0x7fb0140ba120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb02a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a9973e5f0 0x558a99742610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb029400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 0x558a99742000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb02a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a9973e5f0 0x558a99742610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33448/0 (socket says 172.21.15.22:33448) 2024-04-20T21:36:39.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb029400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 0x558a99742000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41326/0 (socket says 172.21.15.22:41326) 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb02a800640 1 -- 172.21.15.22:0/2274584954 learned_addr learned my addr 172.21.15.22:0/2274584954 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb029e00640 1 --2- 172.21.15.22:0/2274584954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a997350f0 0x558a99746c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb029400640 1 -- 172.21.15.22:0/2274584954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a9973e5f0 msgr2=0x558a99742610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb029400640 1 --2- 172.21.15.22:0/2274584954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a9973e5f0 0x558a99742610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb029400640 1 -- 172.21.15.22:0/2274584954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a997350f0 msgr2=0x558a99746c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb029400640 1 --2- 172.21.15.22:0/2274584954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a997350f0 0x558a99746c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb029400640 1 -- 172.21.15.22:0/2274584954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a99742e40 con 0x558a99735a10 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb029400640 1 --2- 172.21.15.22:0/2274584954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 0x558a99742000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x558a9973cb40 tx=0x7fb0140c6910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e88d121fa1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb007400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.986+0000 7fb007400640 1 -- 172.21.15.22:0/2274584954 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140bc190 con 0x558a99735a10 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb007400640 1 -- 172.21.15.22:0/2274584954 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fb0140ef640 con 0x558a99735a10 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb02f60c740 1 -- 172.21.15.22:0/2274584954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0140c6760 con 0x558a99735a10 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb007400640 1 -- 172.21.15.22:0/2274584954 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb0140c6a20 con 0x558a99735a10 2024-04-20T21:36:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb02f60c740 10 client.25718 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb02f60c740 1 -- 172.21.15.22:0/2274584954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558a99742e40 con 0x558a99735a10 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb007400640 1 -- 172.21.15.22:0/2274584954 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0140c6760 con 0x558a99735a10 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb007400640 1 -- 172.21.15.22:0/2274584954 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fb0140a8040 con 0x558a99735a10 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb02f60c740 10 client.25718 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb02f60c740 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb02f60c740 20 client.25718 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb02f60c740 1 --2- 172.21.15.22:0/2274584954 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a997b52d0 0x558a997b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb02f60c740 4 client.25718 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb02f60c740 1 -- 172.21.15.22:0/2274584954 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558a996a15b0 con 0x558a997b52d0 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.987+0000 7fb029e00640 1 --2- 172.21.15.22:0/2274584954 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a997b52d0 0x558a997b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.988+0000 7fb029e00640 1 --2- 172.21.15.22:0/2274584954 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a997b52d0 0x558a997b76c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=470571e4d0dc28ff server_cookie=c9e7e07684e6e559 in_seq=0 out_seq=0 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.988+0000 7fb007400640 10 client.25718 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb007400640 1 -- 172.21.15.22:0/2274584954 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558a996a15b0 con 0x558a997b52d0 2024-04-20T21:36:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb007400640 10 client.25718 handle_command_reply: tid=0 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb02f60c740 1 client.25718 shutdown 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb02f60c740 1 -- 172.21.15.22:0/2274584954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 msgr2=0x558a99742000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb02f60c740 1 --2- 172.21.15.22:0/2274584954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 0x558a99742000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x558a9973cb40 tx=0x7fb0140c6910 comp rx=0 tx=0).stop 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb02f60c740 1 -- 172.21.15.22:0/2274584954 shutdown_connections 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb02f60c740 1 --2- 172.21.15.22:0/2274584954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a9973e5f0 0x558a99742610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb02f60c740 1 --2- 172.21.15.22:0/2274584954 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x558a997b52d0 0x558a997b76c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb02f60c740 1 --2- 172.21.15.22:0/2274584954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a99735a10 0x558a99742000 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb007400640 0 client.25718 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb02f60c740 1 --2- 172.21.15.22:0/2274584954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a997350f0 0x558a99746c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb02f60c740 1 -- 172.21.15.22:0/2274584954 >> 172.21.15.22:0/2274584954 conn(0x558a9972f820 msgr2=0x558a9972ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb02f60c740 1 -- 172.21.15.22:0/2274584954 shutdown_connections 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb02f60c740 1 -- 172.21.15.22:0/2274584954 wait complete. 2024-04-20T21:36:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.990+0000 7fb02f60c740 20 client.25718 trim_cache size 0 max 16384 2024-04-20T21:36:39.994 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:36:39.994 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:36:39.994 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "906c0cef-e3a7-4aff-b41a-24db6226ac7d", 2024-04-20T21:36:39.994 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:36:39.994 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:36:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.991+0000 7fb02b200640 1 -- 172.21.15.22:0/357353931 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb00c0a3da0 msgr2=0x7fb00c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.991+0000 7fb02b200640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb00c0a3da0 0x7fb00c0a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb020005fd0 tx=0x7fb020009500 comp rx=0 tx=0).stop 2024-04-20T21:36:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.991+0000 7fb02b200640 1 -- 172.21.15.22:0/357353931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb024101570 msgr2=0x7fb0241033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.991+0000 7fb02b200640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb024101570 0x7fb0241033e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb014009780 tx=0x7fb0140045b0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.991+0000 7fb02b200640 1 -- 172.21.15.22:0/357353931 shutdown_connections 2024-04-20T21:36:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.991+0000 7fb02b200640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb00c0a3da0 0x7fb00c0a6260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.991+0000 7fb02b200640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb02410e770 0x7fb024103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.991+0000 7fb02b200640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb024101570 0x7fb0241033e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.991+0000 7fb02b200640 1 --2- 172.21.15.22:0/357353931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb024100bc0 0x7fb024102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.991+0000 7fb02b200640 1 -- 172.21.15.22:0/357353931 >> 172.21.15.22:0/357353931 conn(0x7fb0240fc7e0 msgr2=0x7fb02410faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.992+0000 7fb02b200640 1 -- 172.21.15.22:0/357353931 shutdown_connections 2024-04-20T21:36:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:39.992+0000 7fb02b200640 1 -- 172.21.15.22:0/357353931 wait complete. 2024-04-20T21:36:40.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:36:40.013 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:36:40.169 INFO:tasks.workunit.client.0.smithi022.stdout: 1 134882 10.97 MB/sec warmup 68 sec latency 10.603 ms 2024-04-20T21:36:40.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.230+0000 7f2adfe00640 1 -- 172.21.15.22:0/4196086655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ae01038e0 msgr2=0x7f2ae010bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:40.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.230+0000 7f2adfe00640 1 --2- 172.21.15.22:0/4196086655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ae01038e0 0x7f2ae010bdc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009900 tx=0x7f2ac802bcd0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.230+0000 7f2adfe00640 1 -- 172.21.15.22:0/4196086655 shutdown_connections 2024-04-20T21:36:40.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.230+0000 7f2adfe00640 1 --2- 172.21.15.22:0/4196086655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ae010c3e0 0x7f2ae010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.230+0000 7f2adfe00640 1 --2- 172.21.15.22:0/4196086655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ae01038e0 0x7f2ae010bdc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.230+0000 7f2adfe00640 1 --2- 172.21.15.22:0/4196086655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ae0102fc0 0x7f2ae01033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.230+0000 7f2adfe00640 1 -- 172.21.15.22:0/4196086655 >> 172.21.15.22:0/4196086655 conn(0x7f2ae00fc7e0 msgr2=0x7f2ae00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:40.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.230+0000 7f2adfe00640 1 -- 172.21.15.22:0/4196086655 shutdown_connections 2024-04-20T21:36:40.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.230+0000 7f2adfe00640 1 -- 172.21.15.22:0/4196086655 wait complete. 2024-04-20T21:36:40.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.232+0000 7f2adfe00640 1 Processor -- start 2024-04-20T21:36:40.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.232+0000 7f2adfe00640 1 -- start start 2024-04-20T21:36:40.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.232+0000 7f2adfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ae0102fc0 0x7f2ae0100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.232+0000 7f2adfe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ae01038e0 0x7f2ae0101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2adfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ae010c3e0 0x7f2ae0101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2adfe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae01121f0 con 0x7f2ae0102fc0 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2adfe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae0112070 con 0x7f2ae01038e0 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2adfe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae0112370 con 0x7f2ae010c3e0 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2adea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ae0102fc0 0x7f2ae0100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2ade000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ae01038e0 0x7f2ae0101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2adea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ae0102fc0 0x7f2ae0100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40628/0 (socket says 172.21.15.22:40628) 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2adea00640 1 -- 172.21.15.22:0/1808804797 learned_addr learned my addr 172.21.15.22:0/1808804797 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2adf400640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ae010c3e0 0x7f2ae0101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2ade000640 1 -- 172.21.15.22:0/1808804797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ae010c3e0 msgr2=0x7f2ae0101690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2ade000640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ae010c3e0 0x7f2ae0101690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2ade000640 1 -- 172.21.15.22:0/1808804797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ae0102fc0 msgr2=0x7f2ae0100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2ade000640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ae0102fc0 0x7f2ae0100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2ade000640 1 -- 172.21.15.22:0/1808804797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae01a6c30 con 0x7f2ae01038e0 2024-04-20T21:36:40.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2adea00640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ae0102fc0 0x7f2ae0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:40.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.233+0000 7f2adf400640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ae010c3e0 0x7f2ae0101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:40.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.234+0000 7f2ade000640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ae01038e0 0x7f2ae0101150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2ac802c210 tx=0x7f2ac80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:40.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.234+0000 7f2ae4a00640 1 -- 172.21.15.22:0/1808804797 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8042020 con 0x7f2ae01038e0 2024-04-20T21:36:40.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.234+0000 7f2ae4a00640 1 -- 172.21.15.22:0/1808804797 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ac80054e0 con 0x7f2ae01038e0 2024-04-20T21:36:40.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.234+0000 7f2adfe00640 1 -- 172.21.15.22:0/1808804797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae01a6ec0 con 0x7f2ae01038e0 2024-04-20T21:36:40.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.234+0000 7f2ae4a00640 1 -- 172.21.15.22:0/1808804797 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8003a50 con 0x7f2ae01038e0 2024-04-20T21:36:40.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.234+0000 7f2adfe00640 1 -- 172.21.15.22:0/1808804797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae01a73d0 con 0x7f2ae01038e0 2024-04-20T21:36:40.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.239+0000 7f2ae8bb8740 1 Processor -- start 2024-04-20T21:36:40.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.239+0000 7f2ae8bb8740 1 -- start start 2024-04-20T21:36:40.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.239+0000 7f2ae4a00640 1 -- 172.21.15.22:0/1808804797 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2ac8045020 con 0x7f2ae01038e0 2024-04-20T21:36:40.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.239+0000 7f2ae8bb8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133380 0x55d101133760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.239+0000 7f2ae8bb8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133ca0 0x55d10113c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.239+0000 7f2ae8bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 0x55d10113ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.239+0000 7f2ae8bb8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d101099150 con 0x55d101133380 2024-04-20T21:36:40.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.239+0000 7f2ae8bb8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d10103f370 con 0x55d101133ca0 2024-04-20T21:36:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.239+0000 7f2ae8bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d101095110 con 0x55d10113c880 2024-04-20T21:36:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.240+0000 7f2adea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133ca0 0x55d10113c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.240+0000 7f2adea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133ca0 0x55d10113c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41356/0 (socket says 172.21.15.22:41356) 2024-04-20T21:36:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.240+0000 7f2adea00640 1 -- 172.21.15.22:0/1994719248 learned_addr learned my addr 172.21.15.22:0/1994719248 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.240+0000 7f2ade000640 1 --2- 172.21.15.22:0/1994719248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 0x55d10113ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.240+0000 7f2adf400640 1 --2- 172.21.15.22:0/1994719248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133380 0x55d101133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.240+0000 7f2adea00640 1 -- 172.21.15.22:0/1994719248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 msgr2=0x55d10113ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.240+0000 7f2adea00640 1 --2- 172.21.15.22:0/1994719248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 0x55d10113ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.240+0000 7f2adea00640 1 -- 172.21.15.22:0/1994719248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133380 msgr2=0x55d101133760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.240+0000 7f2adea00640 1 --2- 172.21.15.22:0/1994719248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133380 0x55d101133760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.240+0000 7f2adea00640 1 -- 172.21.15.22:0/1994719248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d100e757d0 con 0x55d101133ca0 2024-04-20T21:36:40.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.241+0000 7f2adea00640 1 --2- 172.21.15.22:0/1994719248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133ca0 0x55d10113c340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000f8d0 tx=0x7f2ad000fd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d04592d6ce43dd46 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:40.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.241+0000 7f2acfe00640 1 -- 172.21.15.22:0/1994719248 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0011c20 con 0x55d101133ca0 2024-04-20T21:36:40.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.241+0000 7f2acfe00640 1 -- 172.21.15.22:0/1994719248 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ad0011dc0 con 0x55d101133ca0 2024-04-20T21:36:40.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.241+0000 7f2acfe00640 1 -- 172.21.15.22:0/1994719248 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad00187d0 con 0x55d101133ca0 2024-04-20T21:36:40.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.241+0000 7f2ae4a00640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2ac40a3da0 0x7f2ac40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.242+0000 7f2ae4a00640 1 -- 172.21.15.22:0/1808804797 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f2ac80f2100 con 0x7f2ae01038e0 2024-04-20T21:36:40.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.242+0000 7f2adf400640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2ac40a3da0 0x7f2ac40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.242+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1994719248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133ca0 msgr2=0x55d10113c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:40.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.242+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/1994719248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133ca0 0x55d10113c340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000f8d0 tx=0x7f2ad000fd90 comp rx=0 tx=0).stop 2024-04-20T21:36:40.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.242+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1994719248 shutdown_connections 2024-04-20T21:36:40.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.242+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/1994719248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 0x55d10113ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.242+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/1994719248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133ca0 0x55d10113c340 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.242+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/1994719248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133380 0x55d101133760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.242+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1994719248 >> 172.21.15.22:0/1994719248 conn(0x55d10112dab0 msgr2=0x55d10112de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:40.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.242+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1994719248 shutdown_connections 2024-04-20T21:36:40.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.243+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1994719248 wait complete. 2024-04-20T21:36:40.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.243+0000 7f2adf400640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2ac40a3da0 0x7f2ac40a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8012560 tx=0x7f2ad8005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:40.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.243+0000 7f2ae8bb8740 1 -- start start 2024-04-20T21:36:40.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.243+0000 7f2ae8bb8740 1 Processor -- start 2024-04-20T21:36:40.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.244+0000 7f2ae8bb8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 0x55d1011398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.244+0000 7f2ae8bb8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101139e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.244+0000 7f2ae8bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 0x55d10113a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.244+0000 7f2ae8bb8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d10113f670 con 0x55d101133ca0 2024-04-20T21:36:40.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.244+0000 7f2ae8bb8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d101052740 con 0x55d101133380 2024-04-20T21:36:40.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.244+0000 7f2ae8bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d10113f7f0 con 0x55d10113c880 2024-04-20T21:36:40.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.244+0000 7f2ae8bb8740 10 client.0 _ll_register_callbacks cb 0x55d10104db20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:40.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.244+0000 7f2adea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 0x55d1011398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.244+0000 7f2adea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 0x55d1011398d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41372/0 (socket says 172.21.15.22:41372) 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.244+0000 7f2adea00640 1 -- 172.21.15.22:0/1111582995 learned_addr learned my addr 172.21.15.22:0/1111582995 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.244+0000 7f2ade000640 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101139e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.244+0000 7f2adf400640 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 0x55d10113a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.245+0000 7f2adea00640 1 -- 172.21.15.22:0/1111582995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 msgr2=0x55d10113a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.245+0000 7f2adea00640 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 0x55d10113a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.245+0000 7f2adea00640 1 -- 172.21.15.22:0/1111582995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 msgr2=0x55d101139e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.245+0000 7f2adea00640 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101139e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.245+0000 7f2adea00640 1 -- 172.21.15.22:0/1111582995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d100dab3f0 con 0x55d101133380 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.245+0000 7f2adea00640 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 0x55d1011398d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009a70 tx=0x7f2ad0004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96d42ad4ab41c6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.245+0000 7f2accc00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.245+0000 7f2accc00640 1 -- 172.21.15.22:0/1111582995 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0004e60 con 0x55d101133380 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.245+0000 7f2accc00640 1 -- 172.21.15.22:0/1111582995 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f2ad0019830 con 0x55d101133380 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.245+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1111582995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad0011dc0 con 0x55d101133380 2024-04-20T21:36:40.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.246+0000 7f2accc00640 1 -- 172.21.15.22:0/1111582995 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2ad0019a30 con 0x55d101133380 2024-04-20T21:36:40.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.246+0000 7f2ae8bb8740 10 client.25730 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:40.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.246+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1111582995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d100d6b0f0 con 0x55d101133380 2024-04-20T21:36:40.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.246+0000 7f2accc00640 1 -- 172.21.15.22:0/1111582995 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ad0011dc0 con 0x55d101133380 2024-04-20T21:36:40.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.246+0000 7f2accc00640 1 -- 172.21.15.22:0/1111582995 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f2ad00184d0 con 0x55d101133380 2024-04-20T21:36:40.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.246+0000 7f2ae8bb8740 10 client.25730 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:40.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.246+0000 7f2ae8bb8740 10 client.25730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:40.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.247+0000 7f2ae8bb8740 20 client.25730 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:40.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.247+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d10114cc50 0x55d10114f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.247+0000 7f2ae8bb8740 4 client.25730 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:40.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.247+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1111582995 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1010a6b70 con 0x55d10114cc50 2024-04-20T21:36:40.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.247+0000 7f2ade000640 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d10114cc50 0x55d10114f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.247+0000 7f2ade000640 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d10114cc50 0x55d10114f040 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e21a8678ee95da1 server_cookie=513865335617f54c in_seq=0 out_seq=0 2024-04-20T21:36:40.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.247+0000 7f2accc00640 10 client.25730 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:40.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.248+0000 7f2accc00640 1 -- 172.21.15.22:0/1111582995 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1010a6b70 con 0x55d10114cc50 2024-04-20T21:36:40.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.248+0000 7f2accc00640 10 client.25730 handle_command_reply: tid=0 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.248+0000 7f2ae8bb8740 1 client.25730 shutdown 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.248+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1111582995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 msgr2=0x55d1011398d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.248+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 0x55d1011398d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009a70 tx=0x7f2ad0004c70 comp rx=0 tx=0).stop 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.248+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1111582995 shutdown_connections 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.249+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 0x55d10113a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.249+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101139e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.249+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d10114cc50 0x55d10114f040 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.249+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/1111582995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 0x55d1011398d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.249+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1111582995 >> 172.21.15.22:0/1111582995 conn(0x55d10112dab0 msgr2=0x55d10112f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.249+0000 7f2accc00640 0 client.25730 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.249+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1111582995 shutdown_connections 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.249+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/1111582995 wait complete. 2024-04-20T21:36:40.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.249+0000 7f2ae8bb8740 20 client.25730 trim_cache size 0 max 16384 2024-04-20T21:36:40.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.271+0000 7f2ae8bb8740 1 Processor -- start 2024-04-20T21:36:40.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.271+0000 7f2ae8bb8740 1 -- start start 2024-04-20T21:36:40.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.271+0000 7f2ae8bb8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 0x55d10113c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.271+0000 7f2ae8bb8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101148560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.271+0000 7f2ae8bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 0x55d101138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.271+0000 7f2ae8bb8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae01add10 con 0x55d101133ca0 2024-04-20T21:36:40.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.271+0000 7f2ae8bb8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae01b0590 con 0x55d101133380 2024-04-20T21:36:40.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.271+0000 7f2ae8bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae01ae380 con 0x55d10113c880 2024-04-20T21:36:40.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.272+0000 7f2adea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 0x55d10113c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.272+0000 7f2adea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 0x55d10113c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41386/0 (socket says 172.21.15.22:41386) 2024-04-20T21:36:40.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.272+0000 7f2adea00640 1 -- 172.21.15.22:0/655988392 learned_addr learned my addr 172.21.15.22:0/655988392 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:40.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.272+0000 7f2ade000640 1 --2- 172.21.15.22:0/655988392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.272+0000 7f2adf400640 1 --2- 172.21.15.22:0/655988392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 0x55d101138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.272+0000 7f2adea00640 1 -- 172.21.15.22:0/655988392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 msgr2=0x55d101138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:40.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.272+0000 7f2adea00640 1 --2- 172.21.15.22:0/655988392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 0x55d101138810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.272+0000 7f2adea00640 1 -- 172.21.15.22:0/655988392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 msgr2=0x55d101148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:40.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.272+0000 7f2adea00640 1 --2- 172.21.15.22:0/655988392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.272+0000 7f2adea00640 1 -- 172.21.15.22:0/655988392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d101137e20 con 0x55d101133380 2024-04-20T21:36:40.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.272+0000 7f2adea00640 1 --2- 172.21.15.22:0/655988392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 0x55d10113c000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00188a0 tx=0x7f2ad0019de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91130f6fb06ae515 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:40.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.273+0000 7f2acf400640 1 -- 172.21.15.22:0/655988392 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0037070 con 0x55d101133380 2024-04-20T21:36:40.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.273+0000 7f2acf400640 1 -- 172.21.15.22:0/655988392 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ad0003e50 con 0x55d101133380 2024-04-20T21:36:40.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.273+0000 7f2acf400640 1 -- 172.21.15.22:0/655988392 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad00183d0 con 0x55d101133380 2024-04-20T21:36:40.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.273+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/655988392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 msgr2=0x55d10113c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:40.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.273+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/655988392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 0x55d10113c000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00188a0 tx=0x7f2ad0019de0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.274+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/655988392 shutdown_connections 2024-04-20T21:36:40.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.274+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/655988392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10113c880 0x55d101138810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.274+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/655988392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.274+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/655988392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d101133380 0x55d10113c000 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.274+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/655988392 >> 172.21.15.22:0/655988392 conn(0x55d10112dab0 msgr2=0x55d10112cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:40.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.274+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/655988392 shutdown_connections 2024-04-20T21:36:40.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.274+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/655988392 wait complete. 2024-04-20T21:36:40.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.274+0000 7f2ae8bb8740 1 -- start start 2024-04-20T21:36:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.274+0000 7f2ae8bb8740 1 Processor -- start 2024-04-20T21:36:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2ae8bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d101133380 0x55d1011368d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2ae8bb8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101136e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2ae8bb8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d10113c880 0x55d1011b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2ae8bb8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae01aea50 con 0x55d101133ca0 2024-04-20T21:36:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2ae8bb8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae01afec0 con 0x55d10113c880 2024-04-20T21:36:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2ae8bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae01af120 con 0x55d101133380 2024-04-20T21:36:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2ae8bb8740 10 client.0 _ll_register_callbacks cb 0x7f2ac80f5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2ade000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101136e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2adf400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d10113c880 0x55d1011b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2adf400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d10113c880 0x55d1011b2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:41394/0 (socket says 172.21.15.22:41394) 2024-04-20T21:36:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2adf400640 1 -- 172.21.15.22:0/57006653 learned_addr learned my addr 172.21.15.22:0/57006653 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2adea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d101133380 0x55d1011368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.275+0000 7f2ade000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101136e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40654/0 (socket says 172.21.15.22:40654) 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.276+0000 7f2adf400640 1 -- 172.21.15.22:0/57006653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d101133380 msgr2=0x55d1011368d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.276+0000 7f2adf400640 1 --2- 172.21.15.22:0/57006653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d101133380 0x55d1011368d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.276+0000 7f2adf400640 1 -- 172.21.15.22:0/57006653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 msgr2=0x55d101136e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.276+0000 7f2adf400640 1 --2- 172.21.15.22:0/57006653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101136e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.276+0000 7f2adf400640 1 -- 172.21.15.22:0/57006653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1011b2560 con 0x55d10113c880 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.276+0000 7f2adf400640 1 --2- 172.21.15.22:0/57006653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d10113c880 0x55d1011b2020 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8003830 tx=0x7f2ad80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6edb535e47112ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.276+0000 7f2accc00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.276+0000 7f2accc00640 1 -- 172.21.15.22:0/57006653 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad802f060 con 0x55d10113c880 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.276+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/57006653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad0003e50 con 0x55d10113c880 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.276+0000 7f2accc00640 1 -- 172.21.15.22:0/57006653 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f2ad800fc10 con 0x55d10113c880 2024-04-20T21:36:40.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.277+0000 7f2accc00640 1 -- 172.21.15.22:0/57006653 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2ad800fe10 con 0x55d10113c880 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.277+0000 7f2ae8bb8740 10 client.25736 fetch_fsmap learned FSMap version 20 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.277+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/57006653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d1011b3720 con 0x55d10113c880 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.277+0000 7f2accc00640 1 -- 172.21.15.22:0/57006653 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ad0003e50 con 0x55d10113c880 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.277+0000 7f2accc00640 1 -- 172.21.15.22:0/57006653 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f2ad801e420 con 0x55d10113c880 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.277+0000 7f2ae8bb8740 10 client.25736 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.277+0000 7f2ae8bb8740 10 client.25736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.277+0000 7f2ae8bb8740 20 client.25736 populate_metadata read hostname 'smithi022' 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.277+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/57006653 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d1011b3df0 0x55d1011b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.277+0000 7f2ae8bb8740 4 client.25736 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.277+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/57006653 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1010a09e0 con 0x55d1011b3df0 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.277+0000 7f2adea00640 1 --2- 172.21.15.22:0/57006653 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d1011b3df0 0x55d1011b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.278+0000 7f2adea00640 1 --2- 172.21.15.22:0/57006653 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d1011b3df0 0x55d1011b6210 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a281f00ff76a4882 server_cookie=f7fa576cc920af3 in_seq=0 out_seq=0 2024-04-20T21:36:40.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.278+0000 7f2accc00640 10 client.25736 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.278+0000 7f2accc00640 1 -- 172.21.15.22:0/57006653 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d1010a09e0 con 0x55d1011b3df0 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.278+0000 7f2accc00640 10 client.25736 handle_command_reply: tid=0 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.278+0000 7f2ae8bb8740 1 client.25736 shutdown 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.279+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/57006653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d10113c880 msgr2=0x55d1011b2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.279+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/57006653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d10113c880 0x55d1011b2020 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8003830 tx=0x7f2ad80046c0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.279+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/57006653 shutdown_connections 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.279+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/57006653 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d1011b3df0 0x55d1011b6210 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.279+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/57006653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d10113c880 0x55d1011b2020 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.279+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/57006653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d101133ca0 0x55d101136e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.279+0000 7f2ae8bb8740 1 --2- 172.21.15.22:0/57006653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d101133380 0x55d1011368d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.279+0000 7f2accc00640 0 client.25736 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.279+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/57006653 >> 172.21.15.22:0/57006653 conn(0x55d10112dab0 msgr2=0x55d101139870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:40.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.279+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/57006653 shutdown_connections 2024-04-20T21:36:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.279+0000 7f2ae8bb8740 1 -- 172.21.15.22:0/57006653 wait complete. 2024-04-20T21:36:40.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.279+0000 7f2ae8bb8740 20 client.25736 trim_cache size 0 max 16384 2024-04-20T21:36:40.282 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:36:40.282 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:36:40.282 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:36:40.282 INFO:teuthology.orchestra.run.smithi022.stdout: "906c0cef-e3a7-4aff-b41a-24db6226ac7d": { 2024-04-20T21:36:40.282 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:36:40.282 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "906c0cef-e3a7-4aff-b41a-24db6226ac7d", 2024-04-20T21:36:40.282 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:36:40.282 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:36:40.282 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:36:40.283 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:36:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.280+0000 7f2adfe00640 1 -- 172.21.15.22:0/1808804797 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2ac40a3da0 msgr2=0x7f2ac40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.280+0000 7f2adfe00640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2ac40a3da0 0x7f2ac40a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8012560 tx=0x7f2ad8005f90 comp rx=0 tx=0).stop 2024-04-20T21:36:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.280+0000 7f2adfe00640 1 -- 172.21.15.22:0/1808804797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ae01038e0 msgr2=0x7f2ae0101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:40.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.280+0000 7f2adfe00640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ae01038e0 0x7f2ae0101150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2ac802c210 tx=0x7f2ac80042f0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.280+0000 7f2adfe00640 1 -- 172.21.15.22:0/1808804797 shutdown_connections 2024-04-20T21:36:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.280+0000 7f2adfe00640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2ac40a3da0 0x7f2ac40a6260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.280+0000 7f2adfe00640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ae010c3e0 0x7f2ae0101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.280+0000 7f2adfe00640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ae01038e0 0x7f2ae0101150 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.280+0000 7f2adfe00640 1 --2- 172.21.15.22:0/1808804797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ae0102fc0 0x7f2ae0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.280+0000 7f2adfe00640 1 -- 172.21.15.22:0/1808804797 >> 172.21.15.22:0/1808804797 conn(0x7f2ae00fc7e0 msgr2=0x7f2ae00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.280+0000 7f2adfe00640 1 -- 172.21.15.22:0/1808804797 shutdown_connections 2024-04-20T21:36:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:36:40.280+0000 7f2adfe00640 1 -- 172.21.15.22:0/1808804797 wait complete. 2024-04-20T21:36:40.300 INFO:tasks.cephfs.filesystem:scrub status for tag:906c0cef-e3a7-4aff-b41a-24db6226ac7d - {'path': '/', 'tag': '906c0cef-e3a7-4aff-b41a-24db6226ac7d', 'options': 'recursive,force'} 2024-04-20T21:36:40.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:40 smithi022 ceph-mon[16843]: pgmap v88: 97 pgs: 97 active+clean; 78 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-04-20T21:36:40.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:40 smithi122 ceph-mon[22147]: pgmap v88: 97 pgs: 97 active+clean; 78 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-04-20T21:36:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:40 smithi033 ceph-mon[21626]: pgmap v88: 97 pgs: 97 active+clean; 78 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-04-20T21:36:41.169 INFO:tasks.workunit.client.0.smithi022.stdout: 1 136210 10.91 MB/sec warmup 69 sec latency 16.573 ms 2024-04-20T21:36:41.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:41 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:36:41.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:41 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:41.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:41 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:36:41.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:41 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:41.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:41 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:36:41.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:41 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:41.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:41 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:36:41.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:41 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:41.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:41 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:36:41.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:41 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:41.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:41 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:36:41.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:41 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:42.169 INFO:tasks.workunit.client.0.smithi022.stdout: 1 138373 10.93 MB/sec warmup 70 sec latency 14.541 ms 2024-04-20T21:36:42.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:42 smithi022 ceph-mon[16843]: pgmap v89: 97 pgs: 97 active+clean; 83 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-04-20T21:36:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:42 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:36:42.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:42 smithi122 ceph-mon[22147]: pgmap v89: 97 pgs: 97 active+clean; 83 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-04-20T21:36:42.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:42 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:36:42.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:42 smithi033 ceph-mon[21626]: pgmap v89: 97 pgs: 97 active+clean; 83 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-04-20T21:36:42.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:42 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:36:43.169 INFO:tasks.workunit.client.0.smithi022.stdout: 1 139589 10.83 MB/sec warmup 71 sec latency 17.164 ms 2024-04-20T21:36:44.169 INFO:tasks.workunit.client.0.smithi022.stdout: 1 141742 10.85 MB/sec warmup 72 sec latency 39.174 ms 2024-04-20T21:36:44.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:44 smithi022 ceph-mon[16843]: pgmap v90: 97 pgs: 97 active+clean; 83 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 162 op/s 2024-04-20T21:36:44.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:44 smithi122 ceph-mon[22147]: pgmap v90: 97 pgs: 97 active+clean; 83 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 162 op/s 2024-04-20T21:36:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:44 smithi033 ceph-mon[21626]: pgmap v90: 97 pgs: 97 active+clean; 83 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 162 op/s 2024-04-20T21:36:45.170 INFO:tasks.workunit.client.0.smithi022.stdout: 1 143367 10.83 MB/sec warmup 73 sec latency 12.614 ms 2024-04-20T21:36:46.170 INFO:tasks.workunit.client.0.smithi022.stdout: 1 145837 10.88 MB/sec warmup 74 sec latency 7.612 ms 2024-04-20T21:36:46.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:46 smithi022 ceph-mon[16843]: pgmap v91: 97 pgs: 97 active+clean; 86 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-04-20T21:36:46.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:46 smithi122 ceph-mon[22147]: pgmap v91: 97 pgs: 97 active+clean; 86 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-04-20T21:36:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:46 smithi033 ceph-mon[21626]: pgmap v91: 97 pgs: 97 active+clean; 86 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-04-20T21:36:47.170 INFO:tasks.workunit.client.0.smithi022.stdout: 1 148019 10.84 MB/sec warmup 75 sec latency 18.914 ms 2024-04-20T21:36:48.170 INFO:tasks.workunit.client.0.smithi022.stdout: 1 149927 10.85 MB/sec warmup 76 sec latency 12.851 ms 2024-04-20T21:36:48.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:48 smithi022 ceph-mon[16843]: pgmap v92: 97 pgs: 97 active+clean; 86 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-04-20T21:36:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:48 smithi122 ceph-mon[22147]: pgmap v92: 97 pgs: 97 active+clean; 86 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-04-20T21:36:48.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:48 smithi033 ceph-mon[21626]: pgmap v92: 97 pgs: 97 active+clean; 86 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-04-20T21:36:49.170 INFO:tasks.workunit.client.0.smithi022.stdout: 1 152211 10.88 MB/sec warmup 77 sec latency 51.855 ms 2024-04-20T21:36:50.170 INFO:tasks.workunit.client.0.smithi022.stdout: 1 153664 10.84 MB/sec warmup 78 sec latency 50.077 ms 2024-04-20T21:36:50.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:50 smithi022 ceph-mon[16843]: pgmap v93: 97 pgs: 97 active+clean; 86 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-04-20T21:36:50.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:50 smithi122 ceph-mon[22147]: pgmap v93: 97 pgs: 97 active+clean; 86 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-04-20T21:36:50.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:50 smithi033 ceph-mon[21626]: pgmap v93: 97 pgs: 97 active+clean; 86 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 163 op/s 2024-04-20T21:36:51.170 INFO:tasks.workunit.client.0.smithi022.stdout: 1 156181 10.90 MB/sec warmup 79 sec latency 9.516 ms 2024-04-20T21:36:52.171 INFO:tasks.workunit.client.0.smithi022.stdout: 1 157581 10.84 MB/sec warmup 80 sec latency 13.808 ms 2024-04-20T21:36:52.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:52 smithi022 ceph-mon[16843]: pgmap v94: 97 pgs: 97 active+clean; 100 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 236 op/s 2024-04-20T21:36:52.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:52 smithi122 ceph-mon[22147]: pgmap v94: 97 pgs: 97 active+clean; 100 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 236 op/s 2024-04-20T21:36:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:52 smithi033 ceph-mon[21626]: pgmap v94: 97 pgs: 97 active+clean; 100 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 236 op/s 2024-04-20T21:36:53.171 INFO:tasks.workunit.client.0.smithi022.stdout: 1 159877 10.88 MB/sec warmup 81 sec latency 9.348 ms 2024-04-20T21:36:54.171 INFO:tasks.workunit.client.0.smithi022.stdout: 1 161049 10.81 MB/sec warmup 82 sec latency 73.698 ms 2024-04-20T21:36:54.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:54 smithi022 ceph-mon[16843]: pgmap v95: 97 pgs: 97 active+clean; 100 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 2.5 MiB/s wr, 154 op/s 2024-04-20T21:36:54.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:54 smithi122 ceph-mon[22147]: pgmap v95: 97 pgs: 97 active+clean; 100 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 2.5 MiB/s wr, 154 op/s 2024-04-20T21:36:54.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:54 smithi033 ceph-mon[21626]: pgmap v95: 97 pgs: 97 active+clean; 100 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 2.5 MiB/s wr, 154 op/s 2024-04-20T21:36:55.171 INFO:tasks.workunit.client.0.smithi022.stdout: 1 163321 10.83 MB/sec warmup 83 sec latency 39.911 ms 2024-04-20T21:36:56.171 INFO:tasks.workunit.client.0.smithi022.stdout: 1 164635 10.78 MB/sec warmup 84 sec latency 15.698 ms 2024-04-20T21:36:56.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:56 smithi022 ceph-mon[16843]: pgmap v96: 97 pgs: 97 active+clean; 104 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 238 op/s 2024-04-20T21:36:56.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:56 smithi122 ceph-mon[22147]: pgmap v96: 97 pgs: 97 active+clean; 104 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 238 op/s 2024-04-20T21:36:56.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:56 smithi033 ceph-mon[21626]: pgmap v96: 97 pgs: 97 active+clean; 104 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 238 op/s 2024-04-20T21:36:57.171 INFO:tasks.workunit.client.0.smithi022.stdout: 1 166873 10.80 MB/sec warmup 85 sec latency 10.720 ms 2024-04-20T21:36:58.171 INFO:tasks.workunit.client.0.smithi022.stdout: 1 168239 10.75 MB/sec warmup 86 sec latency 17.543 ms 2024-04-20T21:36:58.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:36:58 smithi122 ceph-mon[22147]: pgmap v97: 97 pgs: 97 active+clean; 104 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-04-20T21:36:58.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:36:58 smithi033 ceph-mon[21626]: pgmap v97: 97 pgs: 97 active+clean; 104 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-04-20T21:36:58.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:36:58 smithi022 ceph-mon[16843]: pgmap v97: 97 pgs: 97 active+clean; 104 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-04-20T21:36:59.172 INFO:tasks.workunit.client.0.smithi022.stdout: 1 170433 10.78 MB/sec warmup 87 sec latency 12.048 ms 2024-04-20T21:37:00.171 INFO:tasks.workunit.client.0.smithi022.stdout: 1 171427 10.69 MB/sec warmup 88 sec latency 39.891 ms 2024-04-20T21:37:00.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:00 smithi122 ceph-mon[22147]: pgmap v98: 97 pgs: 97 active+clean; 104 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-04-20T21:37:00.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:00 smithi033 ceph-mon[21626]: pgmap v98: 97 pgs: 97 active+clean; 104 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-04-20T21:37:00.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:00 smithi022 ceph-mon[16843]: pgmap v98: 97 pgs: 97 active+clean; 104 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-04-20T21:37:01.171 INFO:tasks.workunit.client.0.smithi022.stdout: 1 173514 10.70 MB/sec warmup 89 sec latency 55.803 ms 2024-04-20T21:37:02.171 INFO:tasks.workunit.client.0.smithi022.stdout: 1 174106 10.64 MB/sec warmup 90 sec latency 58.820 ms 2024-04-20T21:37:02.459 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:02 smithi022 ceph-mon[16843]: pgmap v99: 97 pgs: 97 active+clean; 107 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2024-04-20T21:37:02.459 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:02 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:37:02.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:02 smithi122 ceph-mon[22147]: pgmap v99: 97 pgs: 97 active+clean; 107 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2024-04-20T21:37:02.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:02 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:37:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:02 smithi033 ceph-mon[21626]: pgmap v99: 97 pgs: 97 active+clean; 107 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2024-04-20T21:37:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:02 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:37:03.172 INFO:tasks.workunit.client.0.smithi022.stdout: 1 176022 10.59 MB/sec warmup 91 sec latency 16.337 ms 2024-04-20T21:37:04.172 INFO:tasks.workunit.client.0.smithi022.stdout: 1 177789 10.62 MB/sec warmup 92 sec latency 11.397 ms 2024-04-20T21:37:04.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:04 smithi122 ceph-mon[22147]: pgmap v100: 97 pgs: 97 active+clean; 107 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-04-20T21:37:04.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:04 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:37:04.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:04 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:37:04.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:04 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:37:04.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:04 smithi033 ceph-mon[21626]: pgmap v100: 97 pgs: 97 active+clean; 107 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-04-20T21:37:04.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:04 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:37:04.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:04 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:37:04.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:04 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:37:04.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:04 smithi022 ceph-mon[16843]: pgmap v100: 97 pgs: 97 active+clean; 107 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-04-20T21:37:04.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:04 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:37:04.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:04 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:37:04.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:04 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:37:05.172 INFO:tasks.workunit.client.0.smithi022.stdout: 1 179360 10.56 MB/sec warmup 93 sec latency 44.805 ms 2024-04-20T21:37:06.172 INFO:tasks.workunit.client.0.smithi022.stdout: 1 181510 10.60 MB/sec warmup 94 sec latency 8.208 ms 2024-04-20T21:37:06.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:06 smithi122 ceph-mon[22147]: pgmap v101: 97 pgs: 97 active+clean; 110 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 266 op/s 2024-04-20T21:37:06.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:06 smithi033 ceph-mon[21626]: pgmap v101: 97 pgs: 97 active+clean; 110 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 266 op/s 2024-04-20T21:37:06.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:06 smithi022 ceph-mon[16843]: pgmap v101: 97 pgs: 97 active+clean; 110 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 266 op/s 2024-04-20T21:37:07.172 INFO:tasks.workunit.client.0.smithi022.stdout: 1 183923 10.62 MB/sec warmup 95 sec latency 13.600 ms 2024-04-20T21:37:08.172 INFO:tasks.workunit.client.0.smithi022.stdout: 1 185626 10.60 MB/sec warmup 96 sec latency 12.657 ms 2024-04-20T21:37:08.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:08 smithi122 ceph-mon[22147]: pgmap v102: 97 pgs: 97 active+clean; 110 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-04-20T21:37:08.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:08 smithi033 ceph-mon[21626]: pgmap v102: 97 pgs: 97 active+clean; 110 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-04-20T21:37:08.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:08 smithi022 ceph-mon[16843]: pgmap v102: 97 pgs: 97 active+clean; 110 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-04-20T21:37:09.172 INFO:tasks.workunit.client.0.smithi022.stdout: 1 187969 10.65 MB/sec warmup 97 sec latency 11.908 ms 2024-04-20T21:37:10.173 INFO:tasks.workunit.client.0.smithi022.stdout: 1 188877 10.56 MB/sec warmup 98 sec latency 70.885 ms 2024-04-20T21:37:10.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:37:10.301 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:37:10.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:10 smithi122 ceph-mon[22147]: pgmap v103: 97 pgs: 97 active+clean; 110 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-04-20T21:37:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.538+0000 7fbd73600640 1 -- 172.21.15.22:0/2061967300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c074b50 msgr2=0x7fbd6c10e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.538+0000 7fbd73600640 1 --2- 172.21.15.22:0/2061967300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c074b50 0x7fbd6c10e300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbd58009900 tx=0x7fbd5802bcd0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.538+0000 7fbd73600640 1 -- 172.21.15.22:0/2061967300 shutdown_connections 2024-04-20T21:37:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.538+0000 7fbd73600640 1 --2- 172.21.15.22:0/2061967300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd6c10e840 0x7fbd6c110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.538+0000 7fbd73600640 1 --2- 172.21.15.22:0/2061967300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c074b50 0x7fbd6c10e300 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.538+0000 7fbd73600640 1 --2- 172.21.15.22:0/2061967300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd6c074230 0x7fbd6c074610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.538+0000 7fbd73600640 1 -- 172.21.15.22:0/2061967300 >> 172.21.15.22:0/2061967300 conn(0x7fbd6c0793b0 msgr2=0x7fbd6c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.538+0000 7fbd73600640 1 -- 172.21.15.22:0/2061967300 shutdown_connections 2024-04-20T21:37:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.538+0000 7fbd73600640 1 -- 172.21.15.22:0/2061967300 wait complete. 2024-04-20T21:37:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.539+0000 7fbd73600640 1 Processor -- start 2024-04-20T21:37:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.539+0000 7fbd73600640 1 -- start start 2024-04-20T21:37:10.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.540+0000 7fbd73600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c074230 0x7fbd6c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.540+0000 7fbd73600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd6c074b50 0x7fbd6c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.540+0000 7fbd73600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd6c10e840 0x7fbd6c19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.540+0000 7fbd73600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c114580 con 0x7fbd6c074b50 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.540+0000 7fbd73600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c114400 con 0x7fbd6c074230 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.540+0000 7fbd73600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c114700 con 0x7fbd6c10e840 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.540+0000 7fbd70c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c074230 0x7fbd6c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.540+0000 7fbd70c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c074230 0x7fbd6c1a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:52292/0 (socket says 172.21.15.22:52292) 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.540+0000 7fbd70c00640 1 -- 172.21.15.22:0/1060101499 learned_addr learned my addr 172.21.15.22:0/1060101499 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.540+0000 7fbd71600640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd6c10e840 0x7fbd6c19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.540+0000 7fbd6be00640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd6c074b50 0x7fbd6c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.541+0000 7fbd70c00640 1 -- 172.21.15.22:0/1060101499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd6c10e840 msgr2=0x7fbd6c19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.541+0000 7fbd70c00640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd6c10e840 0x7fbd6c19a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.541+0000 7fbd70c00640 1 -- 172.21.15.22:0/1060101499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd6c074b50 msgr2=0x7fbd6c1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.541+0000 7fbd70c00640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd6c074b50 0x7fbd6c1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.541+0000 7fbd70c00640 1 -- 172.21.15.22:0/1060101499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6c19b040 con 0x7fbd6c074230 2024-04-20T21:37:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.541+0000 7fbd70c00640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c074230 0x7fbd6c1a0710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000d950 tx=0x7fbd6000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.541+0000 7fbd69600640 1 -- 172.21.15.22:0/1060101499 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60017030 con 0x7fbd6c074230 2024-04-20T21:37:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.541+0000 7fbd69600640 1 -- 172.21.15.22:0/1060101499 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd60004020 con 0x7fbd6c074230 2024-04-20T21:37:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.541+0000 7fbd73600640 1 -- 172.21.15.22:0/1060101499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6c19b330 con 0x7fbd6c074230 2024-04-20T21:37:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.542+0000 7fbd69600640 1 -- 172.21.15.22:0/1060101499 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd600047a0 con 0x7fbd6c074230 2024-04-20T21:37:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.542+0000 7fbd73600640 1 -- 172.21.15.22:0/1060101499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd6c1a75d0 con 0x7fbd6c074230 2024-04-20T21:37:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.546+0000 7fbd76368740 1 Processor -- start 2024-04-20T21:37:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.546+0000 7fbd76368740 1 -- start start 2024-04-20T21:37:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.547+0000 7fbd76368740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 0x55cd0b11e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.547+0000 7fbd76368740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11eac0 0x55cd0b127160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.547+0000 7fbd69600640 1 -- 172.21.15.22:0/1060101499 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbd600041d0 con 0x7fbd6c074230 2024-04-20T21:37:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.547+0000 7fbd76368740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b1276a0 0x55cd0b129a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.547+0000 7fbd76368740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cd0b084150 con 0x55cd0b11eac0 2024-04-20T21:37:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.547+0000 7fbd76368740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cd0b02a370 con 0x55cd0b11e1a0 2024-04-20T21:37:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.547+0000 7fbd76368740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd0b080110 con 0x55cd0b1276a0 2024-04-20T21:37:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.547+0000 7fbd71600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 0x55cd0b11e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.547+0000 7fbd71600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 0x55cd0b11e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:52304/0 (socket says 172.21.15.22:52304) 2024-04-20T21:37:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.547+0000 7fbd71600640 1 -- 172.21.15.22:0/3822985018 learned_addr learned my addr 172.21.15.22:0/3822985018 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.547+0000 7fbd70c00640 1 --2- 172.21.15.22:0/3822985018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11eac0 0x55cd0b127160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.548+0000 7fbd71600640 1 -- 172.21.15.22:0/3822985018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b1276a0 msgr2=0x55cd0b129a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.548+0000 7fbd71600640 1 --2- 172.21.15.22:0/3822985018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b1276a0 0x55cd0b129a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.548+0000 7fbd71600640 1 -- 172.21.15.22:0/3822985018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11eac0 msgr2=0x55cd0b127160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.548+0000 7fbd71600640 1 --2- 172.21.15.22:0/3822985018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11eac0 0x55cd0b127160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.548+0000 7fbd71600640 1 -- 172.21.15.22:0/3822985018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd0ae607d0 con 0x55cd0b11e1a0 2024-04-20T21:37:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.548+0000 7fbd70c00640 1 --2- 172.21.15.22:0/3822985018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11eac0 0x55cd0b127160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:37:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.548+0000 7fbd71600640 1 --2- 172.21.15.22:0/3822985018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 0x55cd0b11e580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009f40 tx=0x7fbd64010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5fe24da48559098 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.548+0000 7fbd5dc00640 1 -- 172.21.15.22:0/3822985018 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64016070 con 0x55cd0b11e1a0 2024-04-20T21:37:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.549+0000 7fbd5dc00640 1 -- 172.21.15.22:0/3822985018 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd64004d20 con 0x55cd0b11e1a0 2024-04-20T21:37:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.549+0000 7fbd5dc00640 1 -- 172.21.15.22:0/3822985018 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd640128b0 con 0x55cd0b11e1a0 2024-04-20T21:37:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.549+0000 7fbd76368740 1 -- 172.21.15.22:0/3822985018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 msgr2=0x55cd0b11e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.549+0000 7fbd76368740 1 --2- 172.21.15.22:0/3822985018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 0x55cd0b11e580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009f40 tx=0x7fbd64010800 comp rx=0 tx=0).stop 2024-04-20T21:37:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.550+0000 7fbd76368740 1 -- 172.21.15.22:0/3822985018 shutdown_connections 2024-04-20T21:37:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.550+0000 7fbd76368740 1 --2- 172.21.15.22:0/3822985018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b1276a0 0x55cd0b129a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.550+0000 7fbd76368740 1 --2- 172.21.15.22:0/3822985018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11eac0 0x55cd0b127160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.550+0000 7fbd76368740 1 --2- 172.21.15.22:0/3822985018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 0x55cd0b11e580 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.550+0000 7fbd76368740 1 -- 172.21.15.22:0/3822985018 >> 172.21.15.22:0/3822985018 conn(0x55cd0b1188d0 msgr2=0x55cd0b118cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.550+0000 7fbd76368740 1 -- 172.21.15.22:0/3822985018 shutdown_connections 2024-04-20T21:37:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.550+0000 7fbd76368740 1 -- 172.21.15.22:0/3822985018 wait complete. 2024-04-20T21:37:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.550+0000 7fbd69600640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbd540a3cb0 0x7fbd540a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.550+0000 7fbd76368740 1 -- start start 2024-04-20T21:37:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.550+0000 7fbd69600640 1 -- 172.21.15.22:0/1060101499 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fbd600d0120 con 0x7fbd6c074230 2024-04-20T21:37:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.550+0000 7fbd6be00640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbd540a3cb0 0x7fbd540a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.550+0000 7fbd76368740 1 Processor -- start 2024-04-20T21:37:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.551+0000 7fbd76368740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 0x55cd0b124800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.551+0000 7fbd76368740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b124d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.551+0000 7fbd76368740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b1276a0 0x55cd0b125280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.551+0000 7fbd76368740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cd0b12a490 con 0x55cd0b1276a0 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.551+0000 7fbd76368740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cd0b03d740 con 0x55cd0b11e1a0 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.551+0000 7fbd76368740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd0b12a610 con 0x55cd0b11eac0 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.551+0000 7fbd76368740 10 client.0 _ll_register_callbacks cb 0x55cd0b038b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.551+0000 7fbd70c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 0x55cd0b124800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.551+0000 7fbd70c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 0x55cd0b124800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:52320/0 (socket says 172.21.15.22:52320) 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.551+0000 7fbd70c00640 1 -- 172.21.15.22:0/2684899192 learned_addr learned my addr 172.21.15.22:0/2684899192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.551+0000 7fbd6be00640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbd540a3cb0 0x7fbd540a6170 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbd58005ab0 tx=0x7fbd58005fd0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.551+0000 7fbd71600640 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b1276a0 0x55cd0b125280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.552+0000 7fbd70c00640 1 -- 172.21.15.22:0/2684899192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 msgr2=0x55cd0b124d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.552+0000 7fbd70c00640 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b124d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.552+0000 7fbd70c00640 1 -- 172.21.15.22:0/2684899192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b1276a0 msgr2=0x55cd0b125280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.552+0000 7fbd70c00640 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b1276a0 0x55cd0b125280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.552+0000 7fbd70c00640 1 -- 172.21.15.22:0/2684899192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd0ad963f0 con 0x55cd0b11e1a0 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.552+0000 7fbd71600640 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b1276a0 0x55cd0b125280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.552+0000 7fbd70c00640 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 0x55cd0b124800 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009fd0 tx=0x7fbd60098340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=330596da9bd2713 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.552+0000 7fbd52000640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.552+0000 7fbd52000640 1 -- 172.21.15.22:0/2684899192 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6009ba00 con 0x55cd0b11e1a0 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.552+0000 7fbd52000640 1 -- 172.21.15.22:0/2684899192 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fbd600a00b0 con 0x55cd0b11e1a0 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.552+0000 7fbd76368740 1 -- 172.21.15.22:0/2684899192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd64004d20 con 0x55cd0b11e1a0 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.553+0000 7fbd52000640 1 -- 172.21.15.22:0/2684899192 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbd6009c7a0 con 0x55cd0b11e1a0 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.553+0000 7fbd76368740 10 client.25748 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.553+0000 7fbd76368740 1 -- 172.21.15.22:0/2684899192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55cd0ad560f0 con 0x55cd0b11e1a0 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.553+0000 7fbd52000640 1 -- 172.21.15.22:0/2684899192 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd64004d20 con 0x55cd0b11e1a0 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.553+0000 7fbd52000640 1 -- 172.21.15.22:0/2684899192 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fbd600d3b40 con 0x55cd0b11e1a0 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.553+0000 7fbd76368740 10 client.25748 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.553+0000 7fbd76368740 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.553+0000 7fbd76368740 20 client.25748 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.553+0000 7fbd76368740 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cd0b1379c0 0x55cd0b139e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.553+0000 7fbd76368740 4 client.25748 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:37:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.553+0000 7fbd76368740 1 -- 172.21.15.22:0/2684899192 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd0b091b70 con 0x55cd0b1379c0 2024-04-20T21:37:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.554+0000 7fbd6be00640 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cd0b1379c0 0x55cd0b139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.554+0000 7fbd6be00640 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cd0b1379c0 0x55cd0b139e50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb8e892627a0630 server_cookie=4a8205cf128feddc in_seq=0 out_seq=0 2024-04-20T21:37:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.554+0000 7fbd52000640 10 client.25748 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:10.556 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:10 smithi033 ceph-mon[21626]: pgmap v103: 97 pgs: 97 active+clean; 110 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-04-20T21:37:10.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd52000640 1 -- 172.21.15.22:0/2684899192 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd0b091b70 con 0x55cd0b1379c0 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd52000640 10 client.25748 handle_command_reply: tid=0 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd76368740 1 client.25748 shutdown 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd76368740 1 -- 172.21.15.22:0/2684899192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 msgr2=0x55cd0b124800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd76368740 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 0x55cd0b124800 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009fd0 tx=0x7fbd60098340 comp rx=0 tx=0).stop 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd76368740 1 -- 172.21.15.22:0/2684899192 shutdown_connections 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd76368740 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b1276a0 0x55cd0b125280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd76368740 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b124d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd76368740 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cd0b1379c0 0x55cd0b139e50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd76368740 1 --2- 172.21.15.22:0/2684899192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b11e1a0 0x55cd0b124800 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd76368740 1 -- 172.21.15.22:0/2684899192 >> 172.21.15.22:0/2684899192 conn(0x55cd0b1188d0 msgr2=0x55cd0b11a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd52000640 0 client.25748 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd76368740 1 -- 172.21.15.22:0/2684899192 shutdown_connections 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd76368740 1 -- 172.21.15.22:0/2684899192 wait complete. 2024-04-20T21:37:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.557+0000 7fbd76368740 20 client.25748 trim_cache size 0 max 16384 2024-04-20T21:37:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.578+0000 7fbd76368740 1 Processor -- start 2024-04-20T21:37:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.578+0000 7fbd76368740 1 -- start start 2024-04-20T21:37:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.579+0000 7fbd76368740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11e1a0 0x55cd0b126f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.579+0000 7fbd76368740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b133350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.579+0000 7fbd76368740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b1276a0 0x55cd0b123760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.579+0000 7fbd76368740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c1ae050 con 0x55cd0b11e1a0 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.579+0000 7fbd76368740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c1b08d0 con 0x55cd0b1276a0 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.579+0000 7fbd76368740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c1ae6c0 con 0x55cd0b11eac0 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.579+0000 7fbd6be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b133350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.579+0000 7fbd6be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b133350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33014/0 (socket says 172.21.15.22:33014) 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.579+0000 7fbd6be00640 1 -- 172.21.15.22:0/21309674 learned_addr learned my addr 172.21.15.22:0/21309674 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.579+0000 7fbd71600640 1 --2- 172.21.15.22:0/21309674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b1276a0 0x55cd0b123760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.579+0000 7fbd70c00640 1 --2- 172.21.15.22:0/21309674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11e1a0 0x55cd0b126f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.580+0000 7fbd6be00640 1 -- 172.21.15.22:0/21309674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b1276a0 msgr2=0x55cd0b123760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.580+0000 7fbd6be00640 1 --2- 172.21.15.22:0/21309674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b1276a0 0x55cd0b123760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.580+0000 7fbd6be00640 1 -- 172.21.15.22:0/21309674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11e1a0 msgr2=0x55cd0b126f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.580+0000 7fbd6be00640 1 --2- 172.21.15.22:0/21309674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11e1a0 0x55cd0b126f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.580+0000 7fbd6be00640 1 -- 172.21.15.22:0/21309674 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd0b119210 con 0x55cd0b11eac0 2024-04-20T21:37:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.580+0000 7fbd6be00640 1 --2- 172.21.15.22:0/21309674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b133350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbd58009900 tx=0x7fbd58005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e935148592dc0070 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.580+0000 7fbd5d200640 1 -- 172.21.15.22:0/21309674 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd58055070 con 0x55cd0b11eac0 2024-04-20T21:37:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.580+0000 7fbd5d200640 1 -- 172.21.15.22:0/21309674 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd5802ce00 con 0x55cd0b11eac0 2024-04-20T21:37:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.580+0000 7fbd5d200640 1 -- 172.21.15.22:0/21309674 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd58004ee0 con 0x55cd0b11eac0 2024-04-20T21:37:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.581+0000 7fbd76368740 1 -- 172.21.15.22:0/21309674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 msgr2=0x55cd0b133350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.581+0000 7fbd76368740 1 --2- 172.21.15.22:0/21309674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b133350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbd58009900 tx=0x7fbd58005a20 comp rx=0 tx=0).stop 2024-04-20T21:37:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.581+0000 7fbd76368740 1 -- 172.21.15.22:0/21309674 shutdown_connections 2024-04-20T21:37:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.581+0000 7fbd76368740 1 --2- 172.21.15.22:0/21309674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b1276a0 0x55cd0b123760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.581+0000 7fbd76368740 1 --2- 172.21.15.22:0/21309674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b133350 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.581+0000 7fbd76368740 1 --2- 172.21.15.22:0/21309674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11e1a0 0x55cd0b126f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.581+0000 7fbd76368740 1 -- 172.21.15.22:0/21309674 >> 172.21.15.22:0/21309674 conn(0x55cd0b1188d0 msgr2=0x55cd0b11a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.581+0000 7fbd76368740 1 -- 172.21.15.22:0/21309674 shutdown_connections 2024-04-20T21:37:10.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.581+0000 7fbd76368740 1 -- 172.21.15.22:0/21309674 wait complete. 2024-04-20T21:37:10.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.582+0000 7fbd76368740 1 -- start start 2024-04-20T21:37:10.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.582+0000 7fbd76368740 1 Processor -- start 2024-04-20T21:37:10.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.582+0000 7fbd76368740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11e1a0 0x55cd0b121590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.582+0000 7fbd76368740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b121ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.582+0000 7fbd76368740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b1276a0 0x55cd0b122010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.582+0000 7fbd76368740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c1aed90 con 0x55cd0b11e1a0 2024-04-20T21:37:10.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.582+0000 7fbd76368740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c1b0200 con 0x55cd0b1276a0 2024-04-20T21:37:10.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.582+0000 7fbd76368740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c1af460 con 0x55cd0b11eac0 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.582+0000 7fbd76368740 10 client.0 _ll_register_callbacks cb 0x7fbd60005130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd71600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b1276a0 0x55cd0b122010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd70c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11e1a0 0x55cd0b121590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd71600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b1276a0 0x55cd0b122010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:52334/0 (socket says 172.21.15.22:52334) 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd71600640 1 -- 172.21.15.22:0/3387506159 learned_addr learned my addr 172.21.15.22:0/3387506159 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd70c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11e1a0 0x55cd0b121590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52832/0 (socket says 172.21.15.22:52832) 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd6be00640 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b121ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd71600640 1 -- 172.21.15.22:0/3387506159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 msgr2=0x55cd0b121ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd71600640 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b121ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd71600640 1 -- 172.21.15.22:0/3387506159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11e1a0 msgr2=0x55cd0b121590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd71600640 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11e1a0 0x55cd0b121590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd71600640 1 -- 172.21.15.22:0/3387506159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd0b19d040 con 0x55cd0b1276a0 2024-04-20T21:37:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd70c00640 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11e1a0 0x55cd0b121590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd71600640 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b1276a0 0x55cd0b122010 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbd64019000 tx=0x7fbd64012680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7481b5640f1468 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.583+0000 7fbd52000640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.584+0000 7fbd52000640 1 -- 172.21.15.22:0/3387506159 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64016070 con 0x55cd0b1276a0 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.584+0000 7fbd52000640 1 -- 172.21.15.22:0/3387506159 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fbd64003d80 con 0x55cd0b1276a0 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.584+0000 7fbd76368740 1 -- 172.21.15.22:0/3387506159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd5802ce00 con 0x55cd0b1276a0 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.584+0000 7fbd52000640 1 -- 172.21.15.22:0/3387506159 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbd64032050 con 0x55cd0b1276a0 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.584+0000 7fbd76368740 10 client.25754 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.584+0000 7fbd76368740 1 -- 172.21.15.22:0/3387506159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55cd0b19e330 con 0x55cd0b1276a0 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.584+0000 7fbd52000640 1 -- 172.21.15.22:0/3387506159 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd5802ce00 con 0x55cd0b1276a0 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.585+0000 7fbd52000640 1 -- 172.21.15.22:0/3387506159 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fbd6400f040 con 0x55cd0b1276a0 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.585+0000 7fbd76368740 10 client.25754 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.585+0000 7fbd76368740 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.585+0000 7fbd76368740 20 client.25754 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.585+0000 7fbd76368740 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cd0b19e7f0 0x55cd0b1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.585+0000 7fbd76368740 4 client.25754 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.585+0000 7fbd76368740 1 -- 172.21.15.22:0/3387506159 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd0b08b9e0 con 0x55cd0b19e7f0 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.585+0000 7fbd70c00640 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cd0b19e7f0 0x55cd0b1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.585+0000 7fbd70c00640 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cd0b19e7f0 0x55cd0b1a0c10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b344936fe949c0b server_cookie=44d2c90541ccf7a3 in_seq=0 out_seq=0 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.585+0000 7fbd52000640 10 client.25754 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd52000640 1 -- 172.21.15.22:0/3387506159 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cd0b08b9e0 con 0x55cd0b19e7f0 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd52000640 10 client.25754 handle_command_reply: tid=0 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd76368740 1 client.25754 shutdown 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd76368740 1 -- 172.21.15.22:0/3387506159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b1276a0 msgr2=0x55cd0b122010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd76368740 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b1276a0 0x55cd0b122010 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbd64019000 tx=0x7fbd64012680 comp rx=0 tx=0).stop 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd76368740 1 -- 172.21.15.22:0/3387506159 shutdown_connections 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd76368740 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cd0b19e7f0 0x55cd0b1a0c10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd76368740 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd0b1276a0 0x55cd0b122010 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd76368740 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd0b11eac0 0x55cd0b121ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd76368740 1 --2- 172.21.15.22:0/3387506159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd0b11e1a0 0x55cd0b121590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd76368740 1 -- 172.21.15.22:0/3387506159 >> 172.21.15.22:0/3387506159 conn(0x55cd0b1188d0 msgr2=0x55cd0b117030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:10.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd52000640 0 client.25754 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:10.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd76368740 1 -- 172.21.15.22:0/3387506159 shutdown_connections 2024-04-20T21:37:10.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd76368740 1 -- 172.21.15.22:0/3387506159 wait complete. 2024-04-20T21:37:10.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.586+0000 7fbd76368740 20 client.25754 trim_cache size 0 max 16384 2024-04-20T21:37:10.589 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:37:10.589 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:37:10.590 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:37:10.590 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:37:10.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.587+0000 7fbd73600640 1 -- 172.21.15.22:0/1060101499 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbd540a3cb0 msgr2=0x7fbd540a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:10.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.587+0000 7fbd73600640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbd540a3cb0 0x7fbd540a6170 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbd58005ab0 tx=0x7fbd58005fd0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.587+0000 7fbd73600640 1 -- 172.21.15.22:0/1060101499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c074230 msgr2=0x7fbd6c1a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:10.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.587+0000 7fbd73600640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c074230 0x7fbd6c1a0710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000d950 tx=0x7fbd6000de10 comp rx=0 tx=0).stop 2024-04-20T21:37:10.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.587+0000 7fbd73600640 1 -- 172.21.15.22:0/1060101499 shutdown_connections 2024-04-20T21:37:10.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.587+0000 7fbd73600640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbd540a3cb0 0x7fbd540a6170 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.587+0000 7fbd73600640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd6c10e840 0x7fbd6c19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.587+0000 7fbd73600640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd6c074b50 0x7fbd6c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.587+0000 7fbd73600640 1 --2- 172.21.15.22:0/1060101499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c074230 0x7fbd6c1a0710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.587+0000 7fbd73600640 1 -- 172.21.15.22:0/1060101499 >> 172.21.15.22:0/1060101499 conn(0x7fbd6c0793b0 msgr2=0x7fbd6c0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:10.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.587+0000 7fbd73600640 1 -- 172.21.15.22:0/1060101499 shutdown_connections 2024-04-20T21:37:10.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.587+0000 7fbd73600640 1 -- 172.21.15.22:0/1060101499 wait complete. 2024-04-20T21:37:10.604 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:10 smithi022 ceph-mon[16843]: pgmap v103: 97 pgs: 97 active+clean; 110 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-04-20T21:37:10.606 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:37:10.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:37:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.820+0000 7f0269400640 1 -- 172.21.15.22:0/2577757818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0264102b50 msgr2=0x7f0264106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.820+0000 7f0269400640 1 --2- 172.21.15.22:0/2577757818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0264102b50 0x7f0264106b90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0250009860 tx=0x7f025002bb70 comp rx=0 tx=0).stop 2024-04-20T21:37:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.820+0000 7f0269400640 1 -- 172.21.15.22:0/2577757818 shutdown_connections 2024-04-20T21:37:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.820+0000 7f0269400640 1 --2- 172.21.15.22:0/2577757818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0264107270 0x7f026410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.820+0000 7f0269400640 1 --2- 172.21.15.22:0/2577757818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0264102b50 0x7f0264106b90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.820+0000 7f0269400640 1 --2- 172.21.15.22:0/2577757818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02641021a0 0x7f0264102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.820+0000 7f0269400640 1 -- 172.21.15.22:0/2577757818 >> 172.21.15.22:0/2577757818 conn(0x7f0264079320 msgr2=0x7f0264079730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:10.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.820+0000 7f0269400640 1 -- 172.21.15.22:0/2577757818 shutdown_connections 2024-04-20T21:37:10.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.820+0000 7f0269400640 1 -- 172.21.15.22:0/2577757818 wait complete. 2024-04-20T21:37:10.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.821+0000 7f0269400640 1 Processor -- start 2024-04-20T21:37:10.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.821+0000 7f0269400640 1 -- start start 2024-04-20T21:37:10.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0269400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02641021a0 0x7f02641a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0269400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0264107270 0x7f02641a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0269400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f026419a930 0x7f026419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0269400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0264112300 con 0x7f0264107270 2024-04-20T21:37:10.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0269400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0264112180 con 0x7f026419a930 2024-04-20T21:37:10.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0269400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0264112480 con 0x7f02641021a0 2024-04-20T21:37:10.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0261e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0264107270 0x7f02641a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0262800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02641021a0 0x7f02641a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0263200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f026419a930 0x7f026419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0261e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0264107270 0x7f02641a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52854/0 (socket says 172.21.15.22:52854) 2024-04-20T21:37:10.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0262800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02641021a0 0x7f02641a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33046/0 (socket says 172.21.15.22:33046) 2024-04-20T21:37:10.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0261e00640 1 -- 172.21.15.22:0/738390472 learned_addr learned my addr 172.21.15.22:0/738390472 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:10.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0263200640 1 -- 172.21.15.22:0/738390472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02641021a0 msgr2=0x7f02641a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:10.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0263200640 1 --2- 172.21.15.22:0/738390472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02641021a0 0x7f02641a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0263200640 1 -- 172.21.15.22:0/738390472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0264107270 msgr2=0x7f02641a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:10.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0263200640 1 --2- 172.21.15.22:0/738390472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0264107270 0x7f02641a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:10.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.822+0000 7f0263200640 1 -- 172.21.15.22:0/738390472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026419b450 con 0x7f026419a930 2024-04-20T21:37:10.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.823+0000 7f0263200640 1 --2- 172.21.15.22:0/738390472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f026419a930 0x7f026419ade0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f025c009fd0 tx=0x7f025c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:10.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.823+0000 7f025b400640 1 -- 172.21.15.22:0/738390472 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c00cd60 con 0x7f026419a930 2024-04-20T21:37:10.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.823+0000 7f025b400640 1 -- 172.21.15.22:0/738390472 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f025c005020 con 0x7f026419a930 2024-04-20T21:37:10.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.823+0000 7f0269400640 1 -- 172.21.15.22:0/738390472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026419b740 con 0x7f026419a930 2024-04-20T21:37:10.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.823+0000 7f025b400640 1 -- 172.21.15.22:0/738390472 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c010690 con 0x7f026419a930 2024-04-20T21:37:10.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.823+0000 7f0269400640 1 -- 172.21.15.22:0/738390472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02641a7940 con 0x7f026419a930 2024-04-20T21:37:10.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.825+0000 7f0269400640 1 -- 172.21.15.22:0/738390472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02640691d0 con 0x7f026419a930 2024-04-20T21:37:10.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.826+0000 7f025b400640 1 -- 172.21.15.22:0/738390472 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f025c005540 con 0x7f026419a930 2024-04-20T21:37:10.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.828+0000 7f025b400640 1 --2- 172.21.15.22:0/738390472 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f02480a3da0 0x7f02480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:10.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.828+0000 7f025b400640 1 -- 172.21.15.22:0/738390472 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f025c0d0d30 con 0x7f026419a930 2024-04-20T21:37:10.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.828+0000 7f0262800640 1 --2- 172.21.15.22:0/738390472 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f02480a3da0 0x7f02480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:10.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.829+0000 7f0262800640 1 --2- 172.21.15.22:0/738390472 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f02480a3da0 0x7f02480a6260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f026419ba70 tx=0x7f0254009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:10.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:10.832+0000 7f025b400640 1 -- 172.21.15.22:0/738390472 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f025c09d3b0 con 0x7f026419a930 2024-04-20T21:37:11.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.124+0000 7f0269400640 1 -- 172.21.15.22:0/738390472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0264102cf0 con 0x7f026419a930 2024-04-20T21:37:11.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.125+0000 7f025b400640 1 -- 172.21.15.22:0/738390472 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f025c09ccb0 con 0x7f026419a930 2024-04-20T21:37:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:37:11.125 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:37:11.125 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:37:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.128+0000 7f0269400640 1 -- 172.21.15.22:0/738390472 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f02480a3da0 msgr2=0x7f02480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.128+0000 7f0269400640 1 --2- 172.21.15.22:0/738390472 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f02480a3da0 0x7f02480a6260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f026419ba70 tx=0x7f0254009260 comp rx=0 tx=0).stop 2024-04-20T21:37:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.128+0000 7f0269400640 1 -- 172.21.15.22:0/738390472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f026419a930 msgr2=0x7f026419ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.128+0000 7f0269400640 1 --2- 172.21.15.22:0/738390472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f026419a930 0x7f026419ade0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f025c009fd0 tx=0x7f025c00ee50 comp rx=0 tx=0).stop 2024-04-20T21:37:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.128+0000 7f0269400640 1 -- 172.21.15.22:0/738390472 shutdown_connections 2024-04-20T21:37:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.128+0000 7f0269400640 1 --2- 172.21.15.22:0/738390472 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f02480a3da0 0x7f02480a6260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.128+0000 7f0269400640 1 --2- 172.21.15.22:0/738390472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f026419a930 0x7f026419ade0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.128+0000 7f0269400640 1 --2- 172.21.15.22:0/738390472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0264107270 0x7f02641a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.128+0000 7f0269400640 1 --2- 172.21.15.22:0/738390472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02641021a0 0x7f02641a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.128+0000 7f0269400640 1 -- 172.21.15.22:0/738390472 >> 172.21.15.22:0/738390472 conn(0x7f0264079320 msgr2=0x7f02640fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.128+0000 7f0269400640 1 -- 172.21.15.22:0/738390472 shutdown_connections 2024-04-20T21:37:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.128+0000 7f0269400640 1 -- 172.21.15.22:0/738390472 wait complete. 2024-04-20T21:37:11.151 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:37:11.173 INFO:tasks.workunit.client.0.smithi022.stdout: 1 191367 10.59 MB/sec warmup 99 sec latency 15.854 ms 2024-04-20T21:37:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.368+0000 7f735ec00640 1 -- 172.21.15.22:0/3344909341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7358108f50 msgr2=0x7f73581153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.368+0000 7f735ec00640 1 --2- 172.21.15.22:0/3344909341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7358108f50 0x7f73581153c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7350009cd0 tx=0x7f735002c0f0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.368+0000 7f735ec00640 1 -- 172.21.15.22:0/3344909341 shutdown_connections 2024-04-20T21:37:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.368+0000 7f735ec00640 1 --2- 172.21.15.22:0/3344909341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7358108f50 0x7f73581153c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.368+0000 7f735ec00640 1 --2- 172.21.15.22:0/3344909341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7358108590 0x7f7358108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.368+0000 7f735ec00640 1 --2- 172.21.15.22:0/3344909341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f735810e600 0x7f735810e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.368+0000 7f735ec00640 1 -- 172.21.15.22:0/3344909341 >> 172.21.15.22:0/3344909341 conn(0x7f73580764a0 msgr2=0x7f73580788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.369+0000 7f735ec00640 1 -- 172.21.15.22:0/3344909341 shutdown_connections 2024-04-20T21:37:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.369+0000 7f735ec00640 1 -- 172.21.15.22:0/3344909341 wait complete. 2024-04-20T21:37:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.369+0000 7f735ec00640 1 Processor -- start 2024-04-20T21:37:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.370+0000 7f735ec00640 1 -- start start 2024-04-20T21:37:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.370+0000 7f735ec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7358108590 0x7f735819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.370+0000 7f735ec00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7358108f50 0x7f735819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.370+0000 7f735ec00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f735810e600 0x7f73581a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.370+0000 7f735ec00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f73581188d0 con 0x7f735810e600 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.370+0000 7f735ec00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7358118750 con 0x7f7358108f50 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.370+0000 7f735ec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7358118a50 con 0x7f7358108590 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.370+0000 7f7357400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7358108f50 0x7f735819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.370+0000 7f735cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f735810e600 0x7f73581a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.370+0000 7f7357400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7358108f50 0x7f735819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:52384/0 (socket says 172.21.15.22:52384) 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.370+0000 7f7357400640 1 -- 172.21.15.22:0/16799171 learned_addr learned my addr 172.21.15.22:0/16799171 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.370+0000 7f7357e00640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7358108590 0x7f735819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.371+0000 7f7357400640 1 -- 172.21.15.22:0/16799171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7358108590 msgr2=0x7f735819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.371+0000 7f7357400640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7358108590 0x7f735819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.371+0000 7f7357400640 1 -- 172.21.15.22:0/16799171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f735810e600 msgr2=0x7f73581a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.371+0000 7f7357400640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f735810e600 0x7f73581a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.371+0000 7f7357400640 1 -- 172.21.15.22:0/16799171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73581a3dc0 con 0x7f7358108f50 2024-04-20T21:37:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.371+0000 7f7357400640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7358108f50 0x7f735819f310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f734400e9f0 tx=0x7f734400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.371+0000 7f7354c00640 1 -- 172.21.15.22:0/16799171 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734400cd80 con 0x7f7358108f50 2024-04-20T21:37:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.371+0000 7f7354c00640 1 -- 172.21.15.22:0/16799171 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7344010910 con 0x7f7358108f50 2024-04-20T21:37:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.371+0000 7f735ec00640 1 -- 172.21.15.22:0/16799171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73581a40b0 con 0x7f7358108f50 2024-04-20T21:37:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.371+0000 7f7354c00640 1 -- 172.21.15.22:0/16799171 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344018990 con 0x7f7358108f50 2024-04-20T21:37:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.372+0000 7f735ec00640 1 -- 172.21.15.22:0/16799171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73581ab940 con 0x7f7358108f50 2024-04-20T21:37:11.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.375+0000 7f73619b6740 1 Processor -- start 2024-04-20T21:37:11.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.375+0000 7f73619b6740 1 -- start start 2024-04-20T21:37:11.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.376+0000 7f73619b6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc60fb0 0x563e6fc61390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.376+0000 7f73619b6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc618d0 0x563e6fc69f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.376+0000 7f73619b6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc6a4b0 0x563e6fc6c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.376+0000 7f73619b6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563e6fbc6050 con 0x563e6fc6a4b0 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.376+0000 7f73619b6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563e6fb6c490 con 0x563e6fc618d0 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.376+0000 7f73619b6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563e6fbc2010 con 0x563e6fc60fb0 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.376+0000 7f7354c00640 1 -- 172.21.15.22:0/16799171 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7344010ac0 con 0x7f7358108f50 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.376+0000 7f735cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc60fb0 0x563e6fc61390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.376+0000 7f735cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc60fb0 0x563e6fc61390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:33078/0 (socket says 172.21.15.22:33078) 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.376+0000 7f735cc00640 1 -- 172.21.15.22:0/1040134945 learned_addr learned my addr 172.21.15.22:0/1040134945 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.376+0000 7f7357e00640 1 --2- 172.21.15.22:0/1040134945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc618d0 0x563e6fc69f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.376+0000 7f7357400640 1 --2- 172.21.15.22:0/1040134945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc6a4b0 0x563e6fc6c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.377+0000 7f735cc00640 1 -- 172.21.15.22:0/1040134945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc618d0 msgr2=0x563e6fc69f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:11.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.377+0000 7f735cc00640 1 --2- 172.21.15.22:0/1040134945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc618d0 0x563e6fc69f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.377+0000 7f735cc00640 1 -- 172.21.15.22:0/1040134945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc6a4b0 msgr2=0x563e6fc6c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:11.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.377+0000 7f735cc00640 1 --2- 172.21.15.22:0/1040134945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc6a4b0 0x563e6fc6c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.377+0000 7f735cc00640 1 -- 172.21.15.22:0/1040134945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6fb74070 con 0x563e6fc60fb0 2024-04-20T21:37:11.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.377+0000 7f735cc00640 1 --2- 172.21.15.22:0/1040134945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc60fb0 0x563e6fc61390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7350004af0 tx=0x7f7350005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82818b6b03cc5f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:11.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.377+0000 7f734d200640 1 -- 172.21.15.22:0/1040134945 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350043070 con 0x563e6fc60fb0 2024-04-20T21:37:11.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.378+0000 7f734d200640 1 -- 172.21.15.22:0/1040134945 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f735002cca0 con 0x563e6fc60fb0 2024-04-20T21:37:11.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.378+0000 7f734d200640 1 -- 172.21.15.22:0/1040134945 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350039040 con 0x563e6fc60fb0 2024-04-20T21:37:11.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.378+0000 7f7354c00640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f733c0a3da0 0x7f733c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.378+0000 7f7354c00640 1 -- 172.21.15.22:0/16799171 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f73440d0ac0 con 0x7f7358108f50 2024-04-20T21:37:11.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.378+0000 7f7357e00640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f733c0a3da0 0x7f733c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.378+0000 7f73619b6740 1 -- 172.21.15.22:0/1040134945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc60fb0 msgr2=0x563e6fc61390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.378+0000 7f73619b6740 1 --2- 172.21.15.22:0/1040134945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc60fb0 0x563e6fc61390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7350004af0 tx=0x7f7350005f80 comp rx=0 tx=0).stop 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.378+0000 7f73619b6740 1 -- 172.21.15.22:0/1040134945 shutdown_connections 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.378+0000 7f73619b6740 1 --2- 172.21.15.22:0/1040134945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc6a4b0 0x563e6fc6c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.378+0000 7f73619b6740 1 --2- 172.21.15.22:0/1040134945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc618d0 0x563e6fc69f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.378+0000 7f73619b6740 1 --2- 172.21.15.22:0/1040134945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc60fb0 0x563e6fc61390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.379+0000 7f73619b6740 1 -- 172.21.15.22:0/1040134945 >> 172.21.15.22:0/1040134945 conn(0x563e6fc5b6e0 msgr2=0x563e6fc5bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.379+0000 7f73619b6740 1 -- 172.21.15.22:0/1040134945 shutdown_connections 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.379+0000 7f73619b6740 1 -- 172.21.15.22:0/1040134945 wait complete. 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.379+0000 7f7357e00640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f733c0a3da0 0x7f733c0a6260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x563e6fc60a80 tx=0x7f7348005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.379+0000 7f73619b6740 1 -- start start 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.379+0000 7f73619b6740 1 Processor -- start 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.380+0000 7f73619b6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc60fb0 0x563e6fc67520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.380+0000 7f73619b6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc618d0 0x563e6fc67a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.380+0000 7f73619b6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc6a4b0 0x563e6fc67fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.380+0000 7f73619b6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563e6fc6d240 con 0x563e6fc60fb0 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.380+0000 7f73619b6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563e6fb7f860 con 0x563e6fc6a4b0 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.380+0000 7f73619b6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563e6fc6d3c0 con 0x563e6fc618d0 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.380+0000 7f73619b6740 10 client.0 _ll_register_callbacks cb 0x563e6fb4dfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.380+0000 7f7357e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc60fb0 0x563e6fc67520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.380+0000 7f7357e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc60fb0 0x563e6fc67520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52904/0 (socket says 172.21.15.22:52904) 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.380+0000 7f7357e00640 1 -- 172.21.15.22:0/3622772584 learned_addr learned my addr 172.21.15.22:0/3622772584 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.380+0000 7f735cc00640 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc6a4b0 0x563e6fc67fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.380+0000 7f7357400640 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc618d0 0x563e6fc67a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.381+0000 7f735cc00640 1 -- 172.21.15.22:0/3622772584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc618d0 msgr2=0x563e6fc67a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.381+0000 7f735cc00640 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc618d0 0x563e6fc67a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.381+0000 7f735cc00640 1 -- 172.21.15.22:0/3622772584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc60fb0 msgr2=0x563e6fc67520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.381+0000 7f735cc00640 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc60fb0 0x563e6fc67520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.381+0000 7f735cc00640 1 -- 172.21.15.22:0/3622772584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6fb83950 con 0x563e6fc6a4b0 2024-04-20T21:37:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.381+0000 7f7357400640 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc618d0 0x563e6fc67a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:37:11.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.381+0000 7f735cc00640 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc6a4b0 0x563e6fc67fa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7350037040 tx=0x7f7350003b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7be049c194955f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:11.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.381+0000 7f7341600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:11.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.381+0000 7f7341600640 1 -- 172.21.15.22:0/3622772584 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735004c030 con 0x563e6fc6a4b0 2024-04-20T21:37:11.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.381+0000 7f7341600640 1 -- 172.21.15.22:0/3622772584 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f73500327a0 con 0x563e6fc6a4b0 2024-04-20T21:37:11.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.382+0000 7f73619b6740 1 -- 172.21.15.22:0/3622772584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f735002cca0 con 0x563e6fc6a4b0 2024-04-20T21:37:11.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.382+0000 7f7341600640 1 -- 172.21.15.22:0/3622772584 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7350039070 con 0x563e6fc6a4b0 2024-04-20T21:37:11.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.382+0000 7f73619b6740 10 client.25772 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:11.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.382+0000 7f73619b6740 1 -- 172.21.15.22:0/3622772584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563e6fb51820 con 0x563e6fc6a4b0 2024-04-20T21:37:11.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.382+0000 7f7341600640 1 -- 172.21.15.22:0/3622772584 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f735002cca0 con 0x563e6fc6a4b0 2024-04-20T21:37:11.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.382+0000 7f7341600640 1 -- 172.21.15.22:0/3622772584 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f735003f440 con 0x563e6fc6a4b0 2024-04-20T21:37:11.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.382+0000 7f73619b6740 10 client.25772 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:11.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.382+0000 7f73619b6740 10 client.25772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.383+0000 7f73619b6740 20 client.25772 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.383+0000 7f73619b6740 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563e6fc7a380 0x563e6fc7c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.383+0000 7f73619b6740 4 client.25772 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.383+0000 7f73619b6740 1 -- 172.21.15.22:0/3622772584 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e6fbd3a70 con 0x563e6fc7a380 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.383+0000 7f7357e00640 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563e6fc7a380 0x563e6fc7c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.384+0000 7f7341600640 10 client.25772 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.384+0000 7f7357e00640 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563e6fc7a380 0x563e6fc7c770 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8318d7091b3a159e server_cookie=7dbef3fc100e85b8 in_seq=0 out_seq=0 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f7341600640 1 -- 172.21.15.22:0/3622772584 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e6fbd3a70 con 0x563e6fc7a380 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f7341600640 10 client.25772 handle_command_reply: tid=0 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f73619b6740 1 client.25772 shutdown 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f73619b6740 1 -- 172.21.15.22:0/3622772584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc6a4b0 msgr2=0x563e6fc67fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f73619b6740 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc6a4b0 0x563e6fc67fa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7350037040 tx=0x7f7350003b40 comp rx=0 tx=0).stop 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f73619b6740 1 -- 172.21.15.22:0/3622772584 shutdown_connections 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f73619b6740 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563e6fc7a380 0x563e6fc7c770 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f73619b6740 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc6a4b0 0x563e6fc67fa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f73619b6740 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc618d0 0x563e6fc67a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f7341600640 0 client.25772 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:11.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f73619b6740 1 --2- 172.21.15.22:0/3622772584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc60fb0 0x563e6fc67520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f73619b6740 1 -- 172.21.15.22:0/3622772584 >> 172.21.15.22:0/3622772584 conn(0x563e6fc5b6e0 msgr2=0x563e6fc5d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:11.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f73619b6740 1 -- 172.21.15.22:0/3622772584 shutdown_connections 2024-04-20T21:37:11.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.385+0000 7f73619b6740 1 -- 172.21.15.22:0/3622772584 wait complete. 2024-04-20T21:37:11.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.386+0000 7f73619b6740 20 client.25772 trim_cache size 0 max 16384 2024-04-20T21:37:11.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.401+0000 7f73619b6740 1 Processor -- start 2024-04-20T21:37:11.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.401+0000 7f73619b6740 1 -- start start 2024-04-20T21:37:11.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.401+0000 7f73619b6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 0x563e6fc69c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f73619b6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc618d0 0x563e6fc66790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f73619b6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc6a4b0 0x563e6fc7f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f73619b6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f73581b2400 con 0x563e6fc618d0 2024-04-20T21:37:11.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f73619b6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73581b4c80 con 0x563e6fc60fb0 2024-04-20T21:37:11.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f73619b6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73581b2a70 con 0x563e6fc6a4b0 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f7357e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 0x563e6fc69c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f7357e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 0x563e6fc69c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:52404/0 (socket says 172.21.15.22:52404) 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f7357e00640 1 -- 172.21.15.22:0/83797270 learned_addr learned my addr 172.21.15.22:0/83797270 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f7357400640 1 --2- 172.21.15.22:0/83797270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc618d0 0x563e6fc66790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f735cc00640 1 --2- 172.21.15.22:0/83797270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc6a4b0 0x563e6fc7f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f7357e00640 1 -- 172.21.15.22:0/83797270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc6a4b0 msgr2=0x563e6fc7f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f7357e00640 1 --2- 172.21.15.22:0/83797270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc6a4b0 0x563e6fc7f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f7357e00640 1 -- 172.21.15.22:0/83797270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc618d0 msgr2=0x563e6fc66790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f7357e00640 1 --2- 172.21.15.22:0/83797270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc618d0 0x563e6fc66790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f7357e00640 1 -- 172.21.15.22:0/83797270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6f8d83f0 con 0x563e6fc60fb0 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.402+0000 7f735cc00640 1 --2- 172.21.15.22:0/83797270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc6a4b0 0x563e6fc7f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.403+0000 7f7357e00640 1 --2- 172.21.15.22:0/83797270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 0x563e6fc69c50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x563e6fc6d430 tx=0x7f734801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57881b09732f0da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.403+0000 7f7343e00640 1 -- 172.21.15.22:0/83797270 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348004380 con 0x563e6fc60fb0 2024-04-20T21:37:11.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.403+0000 7f7343e00640 1 -- 172.21.15.22:0/83797270 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7348005020 con 0x563e6fc60fb0 2024-04-20T21:37:11.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.403+0000 7f7343e00640 1 -- 172.21.15.22:0/83797270 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348004910 con 0x563e6fc60fb0 2024-04-20T21:37:11.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.404+0000 7f73619b6740 1 -- 172.21.15.22:0/83797270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 msgr2=0x563e6fc69c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.404+0000 7f73619b6740 1 --2- 172.21.15.22:0/83797270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 0x563e6fc69c50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x563e6fc6d430 tx=0x7f734801b000 comp rx=0 tx=0).stop 2024-04-20T21:37:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.404+0000 7f73619b6740 1 -- 172.21.15.22:0/83797270 shutdown_connections 2024-04-20T21:37:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.404+0000 7f73619b6740 1 --2- 172.21.15.22:0/83797270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc6a4b0 0x563e6fc7f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.404+0000 7f73619b6740 1 --2- 172.21.15.22:0/83797270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc618d0 0x563e6fc66790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.404+0000 7f73619b6740 1 --2- 172.21.15.22:0/83797270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 0x563e6fc69c50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.404+0000 7f73619b6740 1 -- 172.21.15.22:0/83797270 >> 172.21.15.22:0/83797270 conn(0x563e6fc5b6e0 msgr2=0x563e6fc75f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.404+0000 7f73619b6740 1 -- 172.21.15.22:0/83797270 shutdown_connections 2024-04-20T21:37:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.404+0000 7f73619b6740 1 -- 172.21.15.22:0/83797270 wait complete. 2024-04-20T21:37:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.404+0000 7f73619b6740 1 -- start start 2024-04-20T21:37:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.405+0000 7f73619b6740 1 Processor -- start 2024-04-20T21:37:11.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.405+0000 7f73619b6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 0x563e6fc7d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.405+0000 7f73619b6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc618d0 0x563e6fc7d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.405+0000 7f73619b6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc6a4b0 0x563e6fc7dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.405+0000 7f73619b6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f73581b3140 con 0x563e6fc618d0 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.405+0000 7f73619b6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73581b45b0 con 0x563e6fc60fb0 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.405+0000 7f73619b6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73581b3810 con 0x563e6fc6a4b0 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.405+0000 7f73619b6740 10 client.0 _ll_register_callbacks cb 0x7f7350009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.405+0000 7f7357e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 0x563e6fc7d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.405+0000 7f7357400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc618d0 0x563e6fc7d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.405+0000 7f7357e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 0x563e6fc7d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:52410/0 (socket says 172.21.15.22:52410) 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.405+0000 7f7357e00640 1 -- 172.21.15.22:0/3097272587 learned_addr learned my addr 172.21.15.22:0/3097272587 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.406+0000 7f735cc00640 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc6a4b0 0x563e6fc7dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.406+0000 7f7357e00640 1 -- 172.21.15.22:0/3097272587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc6a4b0 msgr2=0x563e6fc7dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:11.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.406+0000 7f7357e00640 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc6a4b0 0x563e6fc7dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.406+0000 7f7357e00640 1 -- 172.21.15.22:0/3097272587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc618d0 msgr2=0x563e6fc7d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.406+0000 7f7357e00640 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc618d0 0x563e6fc7d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.406+0000 7f7357e00640 1 -- 172.21.15.22:0/3097272587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6f9a27d0 con 0x563e6fc60fb0 2024-04-20T21:37:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.406+0000 7f7357400640 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc618d0 0x563e6fc7d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.406+0000 7f7357e00640 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 0x563e6fc7d230 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7348009fa0 tx=0x7f7348004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0cc61de512c3a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.406+0000 7f7341600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.406+0000 7f7341600640 1 -- 172.21.15.22:0/3097272587 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348038030 con 0x563e6fc60fb0 2024-04-20T21:37:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.406+0000 7f7341600640 1 -- 172.21.15.22:0/3097272587 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7348030050 con 0x563e6fc60fb0 2024-04-20T21:37:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.406+0000 7f73619b6740 1 -- 172.21.15.22:0/3097272587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7348005020 con 0x563e6fc60fb0 2024-04-20T21:37:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.407+0000 7f7341600640 1 -- 172.21.15.22:0/3097272587 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f734802b470 con 0x563e6fc60fb0 2024-04-20T21:37:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.407+0000 7f73619b6740 10 client.25778 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.407+0000 7f73619b6740 1 -- 172.21.15.22:0/3097272587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563e6fce06e0 con 0x563e6fc60fb0 2024-04-20T21:37:11.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:11 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/738390472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:37:11.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.407+0000 7f7341600640 1 -- 172.21.15.22:0/3097272587 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7348005020 con 0x563e6fc60fb0 2024-04-20T21:37:11.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.407+0000 7f7341600640 1 -- 172.21.15.22:0/3097272587 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f7348004c60 con 0x563e6fc60fb0 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.407+0000 7f73619b6740 10 client.25778 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.407+0000 7f73619b6740 10 client.25778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.407+0000 7f73619b6740 20 client.25778 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.407+0000 7f73619b6740 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563e6fce0d90 0x563e6fce31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.407+0000 7f73619b6740 4 client.25778 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.407+0000 7f73619b6740 1 -- 172.21.15.22:0/3097272587 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e6fbcd8e0 con 0x563e6fce0d90 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.408+0000 7f7357400640 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563e6fce0d90 0x563e6fce31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.408+0000 7f7357400640 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563e6fce0d90 0x563e6fce31b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=210d67e1ea0910e2 server_cookie=8d18cef0fb1380f0 in_seq=0 out_seq=0 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.408+0000 7f7341600640 10 client.25778 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f7341600640 1 -- 172.21.15.22:0/3097272587 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e6fbcd8e0 con 0x563e6fce0d90 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f7341600640 10 client.25778 handle_command_reply: tid=0 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f73619b6740 1 client.25778 shutdown 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f73619b6740 1 -- 172.21.15.22:0/3097272587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 msgr2=0x563e6fc7d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f73619b6740 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 0x563e6fc7d230 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7348009fa0 tx=0x7f7348004700 comp rx=0 tx=0).stop 2024-04-20T21:37:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f73619b6740 1 -- 172.21.15.22:0/3097272587 shutdown_connections 2024-04-20T21:37:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f73619b6740 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e6fc6a4b0 0x563e6fc7dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f73619b6740 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e6fc618d0 0x563e6fc7d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f73619b6740 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563e6fce0d90 0x563e6fce31b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f73619b6740 1 --2- 172.21.15.22:0/3097272587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e6fc60fb0 0x563e6fc7d230 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f7341600640 0 client.25778 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f73619b6740 1 -- 172.21.15.22:0/3097272587 >> 172.21.15.22:0/3097272587 conn(0x563e6fc5b6e0 msgr2=0x563e6fc674c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f73619b6740 1 -- 172.21.15.22:0/3097272587 shutdown_connections 2024-04-20T21:37:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.409+0000 7f73619b6740 1 -- 172.21.15.22:0/3097272587 wait complete. 2024-04-20T21:37:11.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.410+0000 7f73619b6740 20 client.25778 trim_cache size 0 max 16384 2024-04-20T21:37:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:37:11.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.410+0000 7f735ec00640 1 -- 172.21.15.22:0/16799171 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f733c0a3da0 msgr2=0x7f733c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:11.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.410+0000 7f735ec00640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f733c0a3da0 0x7f733c0a6260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x563e6fc60a80 tx=0x7f7348005f90 comp rx=0 tx=0).stop 2024-04-20T21:37:11.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.410+0000 7f735ec00640 1 -- 172.21.15.22:0/16799171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7358108f50 msgr2=0x7f735819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:11.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.410+0000 7f735ec00640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7358108f50 0x7f735819f310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f734400e9f0 tx=0x7f734400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.411+0000 7f735ec00640 1 -- 172.21.15.22:0/16799171 shutdown_connections 2024-04-20T21:37:11.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.411+0000 7f735ec00640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f733c0a3da0 0x7f733c0a6260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.411+0000 7f735ec00640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f735810e600 0x7f73581a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.411+0000 7f735ec00640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7358108f50 0x7f735819f310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.411+0000 7f735ec00640 1 --2- 172.21.15.22:0/16799171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7358108590 0x7f735819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:11.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.411+0000 7f735ec00640 1 -- 172.21.15.22:0/16799171 >> 172.21.15.22:0/16799171 conn(0x7f73580764a0 msgr2=0x7f7358077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:11.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.411+0000 7f735ec00640 1 -- 172.21.15.22:0/16799171 shutdown_connections 2024-04-20T21:37:11.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:11.411+0000 7f735ec00640 1 -- 172.21.15.22:0/16799171 wait complete. 2024-04-20T21:37:11.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:11 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/738390472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:37:11.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:11 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/738390472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:37:12.173 INFO:tasks.workunit.client.0.smithi022.stdout: 1 192587 10.55 MB/sec warmup 100 sec latency 20.346 ms 2024-04-20T21:37:12.428 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:37:12.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:37:12.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:37:12.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:12 smithi022 ceph-mon[16843]: pgmap v104: 97 pgs: 97 active+clean; 117 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 263 op/s 2024-04-20T21:37:12.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:12 smithi122 ceph-mon[22147]: pgmap v104: 97 pgs: 97 active+clean; 117 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 263 op/s 2024-04-20T21:37:12.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:12 smithi033 ceph-mon[21626]: pgmap v104: 97 pgs: 97 active+clean; 117 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 263 op/s 2024-04-20T21:37:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.667+0000 7f9ccd800640 1 -- 172.21.15.22:0/2400373687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cc8112e10 msgr2=0x7f9cc8115200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.667+0000 7f9ccd800640 1 --2- 172.21.15.22:0/2400373687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cc8112e10 0x7f9cc8115200 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009be0 tx=0x7f9cc002bf80 comp rx=0 tx=0).stop 2024-04-20T21:37:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.667+0000 7f9ccd800640 1 -- 172.21.15.22:0/2400373687 shutdown_connections 2024-04-20T21:37:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.667+0000 7f9ccd800640 1 --2- 172.21.15.22:0/2400373687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cc8112e10 0x7f9cc8115200 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.667+0000 7f9ccd800640 1 --2- 172.21.15.22:0/2400373687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc8074c20 0x7f9cc81128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.667+0000 7f9ccd800640 1 --2- 172.21.15.22:0/2400373687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cc8074300 0x7f9cc80746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.667+0000 7f9ccd800640 1 -- 172.21.15.22:0/2400373687 >> 172.21.15.22:0/2400373687 conn(0x7f9cc80fd930 msgr2=0x7f9cc80ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.667+0000 7f9ccd800640 1 -- 172.21.15.22:0/2400373687 shutdown_connections 2024-04-20T21:37:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.667+0000 7f9ccd800640 1 -- 172.21.15.22:0/2400373687 wait complete. 2024-04-20T21:37:12.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.668+0000 7f9ccd800640 1 Processor -- start 2024-04-20T21:37:12.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.668+0000 7f9ccd800640 1 -- start start 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9ccd800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc8074300 0x7f9cc81a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9ccd800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cc8074c20 0x7f9cc81a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9ccd800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cc8112e10 0x7f9cc819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9ccd800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8118800 con 0x7f9cc8074c20 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9ccd800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8118680 con 0x7f9cc8112e10 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9ccd800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8118980 con 0x7f9cc8074300 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9cc7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc8074300 0x7f9cc81a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9cc7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc8074300 0x7f9cc81a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:53004/0 (socket says 172.21.15.22:53004) 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9cc7e00640 1 -- 172.21.15.22:0/1809161122 learned_addr learned my addr 172.21.15.22:0/1809161122 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9ccce00640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cc8112e10 0x7f9cc819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9cc7400640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cc8074c20 0x7f9cc81a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9cc7e00640 1 -- 172.21.15.22:0/1809161122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cc8112e10 msgr2=0x7f9cc819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9cc7e00640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cc8112e10 0x7f9cc819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9cc7e00640 1 -- 172.21.15.22:0/1809161122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cc8074c20 msgr2=0x7f9cc81a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9cc7e00640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cc8074c20 0x7f9cc81a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.669+0000 7f9cc7e00640 1 -- 172.21.15.22:0/1809161122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc819f240 con 0x7f9cc8074300 2024-04-20T21:37:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.670+0000 7f9cc7e00640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc8074300 0x7f9cc81a4910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800e9c0 tx=0x7f9cb800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.670+0000 7f9cc4c00640 1 -- 172.21.15.22:0/1809161122 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800cde0 con 0x7f9cc8074300 2024-04-20T21:37:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.670+0000 7f9cc4c00640 1 -- 172.21.15.22:0/1809161122 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb80035f0 con 0x7f9cc8074300 2024-04-20T21:37:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.670+0000 7f9ccd800640 1 -- 172.21.15.22:0/1809161122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc819f530 con 0x7f9cc8074300 2024-04-20T21:37:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.670+0000 7f9cc4c00640 1 -- 172.21.15.22:0/1809161122 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8010690 con 0x7f9cc8074300 2024-04-20T21:37:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.670+0000 7f9ccd800640 1 -- 172.21.15.22:0/1809161122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc81ab7d0 con 0x7f9cc8074300 2024-04-20T21:37:12.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.674+0000 7f9cd1be1740 1 Processor -- start 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.674+0000 7f9cd1be1740 1 -- start start 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.674+0000 7f9cc4c00640 1 -- 172.21.15.22:0/1809161122 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9cb80037a0 con 0x7f9cc8074300 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.674+0000 7f9cd1be1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820db90f0 0x556820db94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.674+0000 7f9cd1be1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db9a10 0x556820dc20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.674+0000 7f9cd1be1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820dc25f0 0x556820dc49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.674+0000 7f9cd1be1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556820d1dd20 con 0x556820db90f0 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.674+0000 7f9cd1be1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556820cc43e0 con 0x556820db9a10 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.674+0000 7f9cd1be1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556820d19ce0 con 0x556820dc25f0 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.675+0000 7f9cc7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db9a10 0x556820dc20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.675+0000 7f9ccce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820db90f0 0x556820db94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.675+0000 7f9cc7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820dc25f0 0x556820dc49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.675+0000 7f9ccce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820db90f0 0x556820db94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60358/0 (socket says 172.21.15.22:60358) 2024-04-20T21:37:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.675+0000 7f9ccce00640 1 -- 172.21.15.22:0/4016275633 learned_addr learned my addr 172.21.15.22:0/4016275633 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:12.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.675+0000 7f9cc7400640 1 -- 172.21.15.22:0/4016275633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db9a10 msgr2=0x556820dc20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.675+0000 7f9cc7400640 1 --2- 172.21.15.22:0/4016275633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db9a10 0x556820dc20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.675+0000 7f9cc7400640 1 -- 172.21.15.22:0/4016275633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820db90f0 msgr2=0x556820db94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.675+0000 7f9cc7400640 1 --2- 172.21.15.22:0/4016275633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820db90f0 0x556820db94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.675+0000 7f9cc7400640 1 -- 172.21.15.22:0/4016275633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556820ce3980 con 0x556820dc25f0 2024-04-20T21:37:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.676+0000 7f9cc7400640 1 --2- 172.21.15.22:0/4016275633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820dc25f0 0x556820dc49e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9cc81a02b0 tx=0x7f9cb400f360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d158fad48c16a627 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.676+0000 7f9cbd600640 1 -- 172.21.15.22:0/4016275633 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4015070 con 0x556820dc25f0 2024-04-20T21:37:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.676+0000 7f9cbd600640 1 -- 172.21.15.22:0/4016275633 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb400fd90 con 0x556820dc25f0 2024-04-20T21:37:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.676+0000 7f9cbd600640 1 -- 172.21.15.22:0/4016275633 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40116a0 con 0x556820dc25f0 2024-04-20T21:37:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.677+0000 7f9cc4c00640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9cac0a3d20 0x7f9cac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.677+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4016275633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820dc25f0 msgr2=0x556820dc49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.677+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/4016275633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820dc25f0 0x556820dc49e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9cc81a02b0 tx=0x7f9cb400f360 comp rx=0 tx=0).stop 2024-04-20T21:37:12.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.677+0000 7f9cc4c00640 1 -- 172.21.15.22:0/1809161122 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f9cb80d0c50 con 0x7f9cc8074300 2024-04-20T21:37:12.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.677+0000 7f9ccce00640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9cac0a3d20 0x7f9cac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.677+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4016275633 shutdown_connections 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.677+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/4016275633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820dc25f0 0x556820dc49e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.677+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/4016275633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db9a10 0x556820dc20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.677+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/4016275633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820db90f0 0x556820db94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.677+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4016275633 >> 172.21.15.22:0/4016275633 conn(0x556820db3820 msgr2=0x556820db3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.677+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4016275633 shutdown_connections 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.677+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4016275633 wait complete. 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.678+0000 7f9ccce00640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9cac0a3d20 0x7f9cac0a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00099a0 tx=0x7f9cc0005fd0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.678+0000 7f9cd1be1740 1 -- start start 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.678+0000 7f9cd1be1740 1 Processor -- start 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cd1be1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db90f0 0x556820dbf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cd1be1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db9a10 0x556820dbfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cd1be1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820dc25f0 0x556820dc00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cd1be1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556820dc53e0 con 0x556820dc25f0 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cd1be1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556820cd77a0 con 0x556820db9a10 2024-04-20T21:37:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cd1be1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556820dc5560 con 0x556820db90f0 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cd1be1740 10 client.0 _ll_register_callbacks cb 0x556820cd9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cc7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db9a10 0x556820dbfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cc7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db9a10 0x556820dbfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:36072/0 (socket says 172.21.15.22:36072) 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cc7400640 1 -- 172.21.15.22:0/827570370 learned_addr learned my addr 172.21.15.22:0/827570370 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9ccce00640 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820dc25f0 0x556820dc00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cc7e00640 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db90f0 0x556820dbf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cc7400640 1 -- 172.21.15.22:0/827570370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db90f0 msgr2=0x556820dbf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.679+0000 7f9cc7400640 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db90f0 0x556820dbf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.680+0000 7f9cc7400640 1 -- 172.21.15.22:0/827570370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820dc25f0 msgr2=0x556820dc00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.680+0000 7f9cc7400640 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820dc25f0 0x556820dc00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.680+0000 7f9cc7400640 1 -- 172.21.15.22:0/827570370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556820af87e0 con 0x556820db9a10 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.680+0000 7f9cc7e00640 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db90f0 0x556820dbf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.680+0000 7f9cc7400640 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db9a10 0x556820dbfba0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4003820 tx=0x7f9cb4003850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac10ddc4de5ea7a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.680+0000 7f9cb2000640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.680+0000 7f9cb2000640 1 -- 172.21.15.22:0/827570370 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4028060 con 0x556820db9a10 2024-04-20T21:37:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.680+0000 7f9cd1be1740 1 -- 172.21.15.22:0/827570370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb400fd90 con 0x556820db9a10 2024-04-20T21:37:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.680+0000 7f9cb2000640 1 -- 172.21.15.22:0/827570370 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f9cb4004000 con 0x556820db9a10 2024-04-20T21:37:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.680+0000 7f9cb2000640 1 -- 172.21.15.22:0/827570370 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9cb4015070 con 0x556820db9a10 2024-04-20T21:37:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.680+0000 7f9cd1be1740 10 client.25790 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.680+0000 7f9cd1be1740 1 -- 172.21.15.22:0/827570370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556820b58f70 con 0x556820db9a10 2024-04-20T21:37:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.681+0000 7f9cb2000640 1 -- 172.21.15.22:0/827570370 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb400fd90 con 0x556820db9a10 2024-04-20T21:37:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.681+0000 7f9cb2000640 1 -- 172.21.15.22:0/827570370 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f9cb4023b00 con 0x556820db9a10 2024-04-20T21:37:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.681+0000 7f9cd1be1740 10 client.25790 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.681+0000 7f9cd1be1740 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.681+0000 7f9cd1be1740 20 client.25790 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.681+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556820dd2930 0x556820dd4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.681+0000 7f9cd1be1740 4 client.25790 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:37:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.681+0000 7f9cd1be1740 1 -- 172.21.15.22:0/827570370 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556820d2b740 con 0x556820dd2930 2024-04-20T21:37:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.682+0000 7f9cc7e00640 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556820dd2930 0x556820dd4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.682+0000 7f9cc7e00640 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556820dd2930 0x556820dd4d20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=835f4937bc07736 server_cookie=507846764ed45ae5 in_seq=0 out_seq=0 2024-04-20T21:37:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.682+0000 7f9cb2000640 10 client.25790 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cb2000640 1 -- 172.21.15.22:0/827570370 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556820d2b740 con 0x556820dd2930 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cb2000640 10 client.25790 handle_command_reply: tid=0 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cd1be1740 1 client.25790 shutdown 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cd1be1740 1 -- 172.21.15.22:0/827570370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db9a10 msgr2=0x556820dbfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db9a10 0x556820dbfba0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4003820 tx=0x7f9cb4003850 comp rx=0 tx=0).stop 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cd1be1740 1 -- 172.21.15.22:0/827570370 shutdown_connections 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820dc25f0 0x556820dc00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556820dd2930 0x556820dd4d20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db9a10 0x556820dbfba0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/827570370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db90f0 0x556820dbf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cb2000640 0 client.25790 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cd1be1740 1 -- 172.21.15.22:0/827570370 >> 172.21.15.22:0/827570370 conn(0x556820db3820 msgr2=0x556820db5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cd1be1740 1 -- 172.21.15.22:0/827570370 shutdown_connections 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.683+0000 7f9cd1be1740 1 -- 172.21.15.22:0/827570370 wait complete. 2024-04-20T21:37:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.684+0000 7f9cd1be1740 20 client.25790 trim_cache size 0 max 16384 2024-04-20T21:37:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.706+0000 7f9cd1be1740 1 Processor -- start 2024-04-20T21:37:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.706+0000 7f9cd1be1740 1 -- start start 2024-04-20T21:37:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.706+0000 7f9cd1be1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db90f0 0x556820dc1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.706+0000 7f9cd1be1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820dc25f0 0x556820dce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.706+0000 7f9cd1be1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820db2880 0x556820dda3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.706+0000 7f9cd1be1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81b2230 con 0x556820db2880 2024-04-20T21:37:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.706+0000 7f9cd1be1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81b4ab0 con 0x556820dc25f0 2024-04-20T21:37:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.706+0000 7f9cd1be1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81b28a0 con 0x556820db90f0 2024-04-20T21:37:12.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.707+0000 7f9cc7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820dc25f0 0x556820dce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.707+0000 7f9cc7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820dc25f0 0x556820dce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:36078/0 (socket says 172.21.15.22:36078) 2024-04-20T21:37:12.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.707+0000 7f9cc7400640 1 -- 172.21.15.22:0/563639223 learned_addr learned my addr 172.21.15.22:0/563639223 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:12.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.707+0000 7f9ccce00640 1 --2- 172.21.15.22:0/563639223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820db2880 0x556820dda3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.707+0000 7f9cc7e00640 1 --2- 172.21.15.22:0/563639223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db90f0 0x556820dc1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.707+0000 7f9cc7400640 1 -- 172.21.15.22:0/563639223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db90f0 msgr2=0x556820dc1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.707+0000 7f9cc7400640 1 --2- 172.21.15.22:0/563639223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db90f0 0x556820dc1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.707+0000 7f9cc7400640 1 -- 172.21.15.22:0/563639223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820db2880 msgr2=0x556820dda3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.707+0000 7f9cc7400640 1 --2- 172.21.15.22:0/563639223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820db2880 0x556820dda3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.707+0000 7f9cc7400640 1 -- 172.21.15.22:0/563639223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568209f0170 con 0x556820dc25f0 2024-04-20T21:37:12.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.707+0000 7f9cc7400640 1 --2- 172.21.15.22:0/563639223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820dc25f0 0x556820dce2d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4023f20 tx=0x7f9cb4011710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e6d566b3ed66ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:12.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.708+0000 7f9cbcc00640 1 -- 172.21.15.22:0/563639223 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb403a070 con 0x556820dc25f0 2024-04-20T21:37:12.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.708+0000 7f9cbcc00640 1 -- 172.21.15.22:0/563639223 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb400b730 con 0x556820dc25f0 2024-04-20T21:37:12.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.708+0000 7f9cbcc00640 1 -- 172.21.15.22:0/563639223 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4028070 con 0x556820dc25f0 2024-04-20T21:37:12.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.708+0000 7f9cd1be1740 1 -- 172.21.15.22:0/563639223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820dc25f0 msgr2=0x556820dce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.708+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/563639223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820dc25f0 0x556820dce2d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4023f20 tx=0x7f9cb4011710 comp rx=0 tx=0).stop 2024-04-20T21:37:12.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.709+0000 7f9cd1be1740 1 -- 172.21.15.22:0/563639223 shutdown_connections 2024-04-20T21:37:12.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.709+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/563639223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820db2880 0x556820dda3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.709+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/563639223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820dc25f0 0x556820dce2d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.709+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/563639223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db90f0 0x556820dc1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.709+0000 7f9cd1be1740 1 -- 172.21.15.22:0/563639223 >> 172.21.15.22:0/563639223 conn(0x556820db3820 msgr2=0x556820db2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.709+0000 7f9cd1be1740 1 -- 172.21.15.22:0/563639223 shutdown_connections 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.709+0000 7f9cd1be1740 1 -- 172.21.15.22:0/563639223 wait complete. 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.709+0000 7f9cd1be1740 1 -- start start 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9cd1be1740 1 Processor -- start 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9cd1be1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db2880 0x556820dc6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9cd1be1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db90f0 0x556820dc6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9cd1be1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820dc25f0 0x556820e37c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9cd1be1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81b2f70 con 0x556820dc25f0 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9cd1be1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81b43e0 con 0x556820db90f0 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9cd1be1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81b3640 con 0x556820db2880 2024-04-20T21:37:12.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9cd1be1740 10 client.0 _ll_register_callbacks cb 0x7f9cc0009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9cc7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db2880 0x556820dc6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9cc7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db2880 0x556820dc6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:53040/0 (socket says 172.21.15.22:53040) 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9cc7e00640 1 -- 172.21.15.22:0/4023963442 learned_addr learned my addr 172.21.15.22:0/4023963442 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9cc7400640 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db90f0 0x556820dc6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.710+0000 7f9ccce00640 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820dc25f0 0x556820e37c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cc7e00640 1 -- 172.21.15.22:0/4023963442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db90f0 msgr2=0x556820dc6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cc7e00640 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db90f0 0x556820dc6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cc7e00640 1 -- 172.21.15.22:0/4023963442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820dc25f0 msgr2=0x556820e37c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cc7e00640 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820dc25f0 0x556820e37c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cc7e00640 1 -- 172.21.15.22:0/4023963442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556820e38160 con 0x556820db2880 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cc7e00640 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db2880 0x556820dc6880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8099540 tx=0x7f9cb80d4700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89b159ac0e294880 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cb2000640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cb2000640 1 -- 172.21.15.22:0/4023963442 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb809c6b0 con 0x556820db2880 2024-04-20T21:37:12.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cb2000640 1 -- 172.21.15.22:0/4023963442 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f9cb80bd0b0 con 0x556820db2880 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4023963442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb400b730 con 0x556820db2880 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cb2000640 1 -- 172.21.15.22:0/4023963442 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9cb809c4d0 con 0x556820db2880 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cd1be1740 10 client.25825 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.711+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4023963442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556820e39400 con 0x556820db2880 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.712+0000 7f9cb2000640 1 -- 172.21.15.22:0/4023963442 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb400b730 con 0x556820db2880 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.712+0000 7f9cb2000640 1 -- 172.21.15.22:0/4023963442 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f9cb80b7070 con 0x556820db2880 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.712+0000 7f9cd1be1740 10 client.25825 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.712+0000 7f9cd1be1740 10 client.25825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.712+0000 7f9cd1be1740 20 client.25825 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.712+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556820e39b20 0x556820e3bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.712+0000 7f9cd1be1740 4 client.25825 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.712+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4023963442 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556820d255b0 con 0x556820e39b20 2024-04-20T21:37:12.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.712+0000 7f9cc7400640 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556820e39b20 0x556820e3bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.713+0000 7f9cc7400640 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556820e39b20 0x556820e3bf10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f5d9b2b57787bad server_cookie=81639385db48455a in_seq=0 out_seq=0 2024-04-20T21:37:12.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.713+0000 7f9cb2000640 10 client.25825 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cb2000640 1 -- 172.21.15.22:0/4023963442 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556820d255b0 con 0x556820e39b20 2024-04-20T21:37:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cb2000640 10 client.25825 handle_command_reply: tid=0 2024-04-20T21:37:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cd1be1740 1 client.25825 shutdown 2024-04-20T21:37:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4023963442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db2880 msgr2=0x556820dc6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db2880 0x556820dc6880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8099540 tx=0x7f9cb80d4700 comp rx=0 tx=0).stop 2024-04-20T21:37:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4023963442 shutdown_connections 2024-04-20T21:37:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556820dc25f0 0x556820e37c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556820e39b20 0x556820e3bf10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556820db90f0 0x556820dc6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cd1be1740 1 --2- 172.21.15.22:0/4023963442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556820db2880 0x556820dc6880 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4023963442 >> 172.21.15.22:0/4023963442 conn(0x556820db3820 msgr2=0x556820dbf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cb2000640 0 client.25825 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:12.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4023963442 shutdown_connections 2024-04-20T21:37:12.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cd1be1740 1 -- 172.21.15.22:0/4023963442 wait complete. 2024-04-20T21:37:12.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.714+0000 7f9cd1be1740 20 client.25825 trim_cache size 0 max 16384 2024-04-20T21:37:12.717 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:37:12.717 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:37:12.717 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "00841d9e-4bc8-42a7-a9cd-01f6ef46de48", 2024-04-20T21:37:12.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:37:12.717 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:37:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.715+0000 7f9ccd800640 1 -- 172.21.15.22:0/1809161122 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9cac0a3d20 msgr2=0x7f9cac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.715+0000 7f9ccd800640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9cac0a3d20 0x7f9cac0a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00099a0 tx=0x7f9cc0005fd0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.715+0000 7f9ccd800640 1 -- 172.21.15.22:0/1809161122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc8074300 msgr2=0x7f9cc81a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.715+0000 7f9ccd800640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc8074300 0x7f9cc81a4910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800e9c0 tx=0x7f9cb800ee80 comp rx=0 tx=0).stop 2024-04-20T21:37:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.715+0000 7f9ccd800640 1 -- 172.21.15.22:0/1809161122 shutdown_connections 2024-04-20T21:37:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.715+0000 7f9ccd800640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9cac0a3d20 0x7f9cac0a61e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.715+0000 7f9ccd800640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cc8112e10 0x7f9cc819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.715+0000 7f9ccd800640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cc8074c20 0x7f9cc81a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.715+0000 7f9ccd800640 1 --2- 172.21.15.22:0/1809161122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc8074300 0x7f9cc81a4910 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.715+0000 7f9ccd800640 1 -- 172.21.15.22:0/1809161122 >> 172.21.15.22:0/1809161122 conn(0x7f9cc80fd930 msgr2=0x7f9cc80ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.716+0000 7f9ccd800640 1 -- 172.21.15.22:0/1809161122 shutdown_connections 2024-04-20T21:37:12.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.716+0000 7f9ccd800640 1 -- 172.21.15.22:0/1809161122 wait complete. 2024-04-20T21:37:12.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:37:12.737 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:37:12.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.946+0000 7f574f200640 1 -- 172.21.15.22:0/2367867639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f575010e650 msgr2=0x7f5750110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.946+0000 7f574f200640 1 --2- 172.21.15.22:0/2367867639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f575010e650 0x7f5750110aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5748009ca0 tx=0x7f574802c190 comp rx=0 tx=0).stop 2024-04-20T21:37:12.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.946+0000 7f574f200640 1 -- 172.21.15.22:0/2367867639 shutdown_connections 2024-04-20T21:37:12.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.946+0000 7f574f200640 1 --2- 172.21.15.22:0/2367867639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f575010e650 0x7f5750110aa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.946+0000 7f574f200640 1 --2- 172.21.15.22:0/2367867639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5750074cb0 0x7f575010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.946+0000 7f574f200640 1 --2- 172.21.15.22:0/2367867639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5750074390 0x7f5750074770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.946+0000 7f574f200640 1 -- 172.21.15.22:0/2367867639 >> 172.21.15.22:0/2367867639 conn(0x7f57500fd900 msgr2=0x7f57500ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:12.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.946+0000 7f574f200640 1 -- 172.21.15.22:0/2367867639 shutdown_connections 2024-04-20T21:37:12.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.946+0000 7f574f200640 1 -- 172.21.15.22:0/2367867639 wait complete. 2024-04-20T21:37:12.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.947+0000 7f574f200640 1 Processor -- start 2024-04-20T21:37:12.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.947+0000 7f574f200640 1 -- start start 2024-04-20T21:37:12.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.948+0000 7f574f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5750074390 0x7f57501a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.948+0000 7f574f200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5750074cb0 0x7f57501a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.948+0000 7f574f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f575010e650 0x7f575019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.948+0000 7f574f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57501143d0 con 0x7f575010e650 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.948+0000 7f574f200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5750114250 con 0x7f5750074cb0 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.948+0000 7f574f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5750114550 con 0x7f5750074390 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.948+0000 7f574de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5750074390 0x7f57501a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.948+0000 7f574de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5750074390 0x7f57501a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:53058/0 (socket says 172.21.15.22:53058) 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.948+0000 7f574de00640 1 -- 172.21.15.22:0/1654801470 learned_addr learned my addr 172.21.15.22:0/1654801470 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.948+0000 7f5747e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5750074cb0 0x7f57501a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.948+0000 7f574e800640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f575010e650 0x7f575019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f574de00640 1 -- 172.21.15.22:0/1654801470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5750074cb0 msgr2=0x7f57501a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f574de00640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5750074cb0 0x7f57501a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f574de00640 1 -- 172.21.15.22:0/1654801470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f575010e650 msgr2=0x7f575019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f574de00640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f575010e650 0x7f575019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f574de00640 1 -- 172.21.15.22:0/1654801470 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575019aff0 con 0x7f5750074390 2024-04-20T21:37:12.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f574e800640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f575010e650 0x7f575019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:37:12.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f5747e00640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5750074cb0 0x7f57501a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:12.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f574de00640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5750074390 0x7f57501a0620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f574000ea60 tx=0x7f574000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:12.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f5746a00640 1 -- 172.21.15.22:0/1654801470 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574000cde0 con 0x7f5750074390 2024-04-20T21:37:12.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f574f200640 1 -- 172.21.15.22:0/1654801470 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575019b2e0 con 0x7f5750074390 2024-04-20T21:37:12.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f5746a00640 1 -- 172.21.15.22:0/1654801470 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57400035f0 con 0x7f5750074390 2024-04-20T21:37:12.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f5746a00640 1 -- 172.21.15.22:0/1654801470 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740010670 con 0x7f5750074390 2024-04-20T21:37:12.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.949+0000 7f574f200640 1 -- 172.21.15.22:0/1654801470 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57501a75a0 con 0x7f5750074390 2024-04-20T21:37:12.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.952+0000 7f5746a00640 1 -- 172.21.15.22:0/1654801470 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f57400037a0 con 0x7f5750074390 2024-04-20T21:37:12.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.953+0000 7f5757c65740 1 Processor -- start 2024-04-20T21:37:12.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.953+0000 7f5757c65740 1 -- start start 2024-04-20T21:37:12.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.953+0000 7f5757c65740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7380 0x556ab08f7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.953+0000 7f5757c65740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7ca0 0x556ab0900340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.953+0000 7f5757c65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab0902c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.953+0000 7f5757c65740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556ab085d150 con 0x556ab08f7ca0 2024-04-20T21:37:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.953+0000 7f5757c65740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556ab0803370 con 0x556ab08f7380 2024-04-20T21:37:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.953+0000 7f5757c65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556ab0859110 con 0x556ab0900880 2024-04-20T21:37:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.953+0000 7f574e800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7380 0x556ab08f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.953+0000 7f574e800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7380 0x556ab08f7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:36110/0 (socket says 172.21.15.22:36110) 2024-04-20T21:37:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.953+0000 7f574e800640 1 -- 172.21.15.22:0/1989400318 learned_addr learned my addr 172.21.15.22:0/1989400318 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.953+0000 7f5747e00640 1 --2- 172.21.15.22:0/1989400318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab0902c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.954+0000 7f574de00640 1 --2- 172.21.15.22:0/1989400318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7ca0 0x556ab0900340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.954+0000 7f574e800640 1 -- 172.21.15.22:0/1989400318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 msgr2=0x556ab0902c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.954+0000 7f574e800640 1 --2- 172.21.15.22:0/1989400318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab0902c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.954+0000 7f574e800640 1 -- 172.21.15.22:0/1989400318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7ca0 msgr2=0x556ab0900340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.954+0000 7f574e800640 1 --2- 172.21.15.22:0/1989400318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7ca0 0x556ab0900340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.954+0000 7f574e800640 1 -- 172.21.15.22:0/1989400318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab06397d0 con 0x556ab08f7380 2024-04-20T21:37:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.954+0000 7f574e800640 1 --2- 172.21.15.22:0/1989400318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7380 0x556ab08f7760 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f574802c7a0 tx=0x7f5748009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce0b21f0c19944db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.954+0000 7f5746a00640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57340a3d20 0x7f57340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.955+0000 7f573f400640 1 -- 172.21.15.22:0/1989400318 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748043020 con 0x556ab08f7380 2024-04-20T21:37:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.955+0000 7f573f400640 1 -- 172.21.15.22:0/1989400318 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5748003e50 con 0x556ab08f7380 2024-04-20T21:37:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.955+0000 7f573f400640 1 -- 172.21.15.22:0/1989400318 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748039040 con 0x556ab08f7380 2024-04-20T21:37:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.955+0000 7f5746a00640 1 -- 172.21.15.22:0/1654801470 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f57400d0bf0 con 0x7f5750074390 2024-04-20T21:37:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.955+0000 7f574de00640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57340a3d20 0x7f57340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.955+0000 7f5757c65740 1 -- 172.21.15.22:0/1989400318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7380 msgr2=0x556ab08f7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.955+0000 7f5757c65740 1 --2- 172.21.15.22:0/1989400318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7380 0x556ab08f7760 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f574802c7a0 tx=0x7f5748009890 comp rx=0 tx=0).stop 2024-04-20T21:37:12.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.956+0000 7f5757c65740 1 -- 172.21.15.22:0/1989400318 shutdown_connections 2024-04-20T21:37:12.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.956+0000 7f5757c65740 1 --2- 172.21.15.22:0/1989400318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab0902c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.956+0000 7f5757c65740 1 --2- 172.21.15.22:0/1989400318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7ca0 0x556ab0900340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.956+0000 7f5757c65740 1 --2- 172.21.15.22:0/1989400318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7380 0x556ab08f7760 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.956+0000 7f5757c65740 1 -- 172.21.15.22:0/1989400318 >> 172.21.15.22:0/1989400318 conn(0x556ab08f1ab0 msgr2=0x556ab08f1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.956+0000 7f5757c65740 1 -- 172.21.15.22:0/1989400318 shutdown_connections 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.956+0000 7f574de00640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57340a3d20 0x7f57340a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x556ab08f6e50 tx=0x7f574000ef90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.956+0000 7f5757c65740 1 -- 172.21.15.22:0/1989400318 wait complete. 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.956+0000 7f5757c65740 1 -- start start 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.956+0000 7f5757c65740 1 Processor -- start 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.957+0000 7f5757c65740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7380 0x556ab08fd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.957+0000 7f5757c65740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 0x556ab08fde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.957+0000 7f5757c65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.957+0000 7f5757c65740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556ab0903610 con 0x556ab08f7380 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.957+0000 7f5757c65740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556ab0816740 con 0x556ab08f7ca0 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.957+0000 7f5757c65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556ab0903790 con 0x556ab0900880 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.957+0000 7f5757c65740 10 client.0 _ll_register_callbacks cb 0x556ab0811b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:12.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.957+0000 7f5747e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 0x556ab08fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.957+0000 7f5747e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 0x556ab08fde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:36114/0 (socket says 172.21.15.22:36114) 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.957+0000 7f5747e00640 1 -- 172.21.15.22:0/4274513795 learned_addr learned my addr 172.21.15.22:0/4274513795 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.957+0000 7f574e800640 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7380 0x556ab08fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.958+0000 7f574e800640 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.958+0000 7f5747e00640 1 -- 172.21.15.22:0/4274513795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 msgr2=0x556ab08fe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.958+0000 7f5747e00640 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.958+0000 7f5747e00640 1 -- 172.21.15.22:0/4274513795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7380 msgr2=0x556ab08fd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.958+0000 7f5747e00640 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7380 0x556ab08fd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.958+0000 7f5747e00640 1 -- 172.21.15.22:0/4274513795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab056f3f0 con 0x556ab08f7ca0 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.958+0000 7f574e800640 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.958+0000 7f5747e00640 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 0x556ab08fde80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5738012d20 tx=0x7f573800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b29577279b306c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.958+0000 7f5733e00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.958+0000 7f5733e00640 1 -- 172.21.15.22:0/4274513795 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57380120d0 con 0x556ab08f7ca0 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.958+0000 7f5757c65740 1 -- 172.21.15.22:0/4274513795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5748003e50 con 0x556ab08f7ca0 2024-04-20T21:37:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.958+0000 7f5733e00640 1 -- 172.21.15.22:0/4274513795 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5738010430 con 0x556ab08f7ca0 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.959+0000 7f5733e00640 1 -- 172.21.15.22:0/4274513795 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5738012270 con 0x556ab08f7ca0 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.959+0000 7f5757c65740 10 client.25808 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.959+0000 7f5757c65740 1 -- 172.21.15.22:0/4274513795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556ab052f0f0 con 0x556ab08f7ca0 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.959+0000 7f5733e00640 1 -- 172.21.15.22:0/4274513795 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5748003e50 con 0x556ab08f7ca0 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.959+0000 7f5733e00640 1 -- 172.21.15.22:0/4274513795 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f5738024780 con 0x556ab08f7ca0 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.959+0000 7f5757c65740 10 client.25808 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.959+0000 7f5757c65740 10 client.25808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.959+0000 7f5757c65740 20 client.25808 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.959+0000 7f5757c65740 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556ab0910c30 0x556ab09130b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.959+0000 7f5757c65740 4 client.25808 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.959+0000 7f5757c65740 1 -- 172.21.15.22:0/4274513795 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ab086ab70 con 0x556ab0910c30 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.960+0000 7f574e800640 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556ab0910c30 0x556ab09130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.960+0000 7f574e800640 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556ab0910c30 0x556ab09130b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=546f3eb6d541231b server_cookie=4d3b74dc3ffdb169 in_seq=0 out_seq=0 2024-04-20T21:37:12.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.960+0000 7f5733e00640 10 client.25808 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:12.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5733e00640 1 -- 172.21.15.22:0/4274513795 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ab086ab70 con 0x556ab0910c30 2024-04-20T21:37:12.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5733e00640 10 client.25808 handle_command_reply: tid=0 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5757c65740 1 client.25808 shutdown 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5757c65740 1 -- 172.21.15.22:0/4274513795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 msgr2=0x556ab08fde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5757c65740 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 0x556ab08fde80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5738012d20 tx=0x7f573800f790 comp rx=0 tx=0).stop 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5757c65740 1 -- 172.21.15.22:0/4274513795 shutdown_connections 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5757c65740 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5757c65740 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556ab0910c30 0x556ab09130b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5757c65740 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 0x556ab08fde80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5757c65740 1 --2- 172.21.15.22:0/4274513795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7380 0x556ab08fd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5733e00640 0 client.25808 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5757c65740 1 -- 172.21.15.22:0/4274513795 >> 172.21.15.22:0/4274513795 conn(0x556ab08f1ab0 msgr2=0x556ab08f3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.961+0000 7f5757c65740 1 -- 172.21.15.22:0/4274513795 shutdown_connections 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.962+0000 7f5757c65740 1 -- 172.21.15.22:0/4274513795 wait complete. 2024-04-20T21:37:12.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.962+0000 7f5757c65740 20 client.25808 trim_cache size 0 max 16384 2024-04-20T21:37:12.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.982+0000 7f5757c65740 1 Processor -- start 2024-04-20T21:37:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.982+0000 7f5757c65740 1 -- start start 2024-04-20T21:37:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.983+0000 7f5757c65740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7380 0x556ab0900070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.983+0000 7f5757c65740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7ca0 0x556ab090c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.983+0000 7f5757c65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.983+0000 7f5757c65740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57501adde0 con 0x556ab08f7ca0 2024-04-20T21:37:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.983+0000 7f5757c65740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57501b0660 con 0x556ab08f7380 2024-04-20T21:37:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.983+0000 7f5757c65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57501ae450 con 0x556ab0900880 2024-04-20T21:37:12.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.983+0000 7f5747e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7ca0 0x556ab090c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.983+0000 7f5747e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7ca0 0x556ab090c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60446/0 (socket says 172.21.15.22:60446) 2024-04-20T21:37:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.983+0000 7f5747e00640 1 -- 172.21.15.22:0/543843011 learned_addr learned my addr 172.21.15.22:0/543843011 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.983+0000 7f574e800640 1 --2- 172.21.15.22:0/543843011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7380 0x556ab0900070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.984+0000 7f574e800640 1 --2- 172.21.15.22:0/543843011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.984+0000 7f5747e00640 1 -- 172.21.15.22:0/543843011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 msgr2=0x556ab08fc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.984+0000 7f5747e00640 1 --2- 172.21.15.22:0/543843011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.984+0000 7f5747e00640 1 -- 172.21.15.22:0/543843011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7380 msgr2=0x556ab0900070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.984+0000 7f5747e00640 1 --2- 172.21.15.22:0/543843011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7380 0x556ab0900070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.984+0000 7f5747e00640 1 -- 172.21.15.22:0/543843011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab08f7ad0 con 0x556ab08f7ca0 2024-04-20T21:37:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.984+0000 7f574e800640 1 --2- 172.21.15.22:0/543843011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:37:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.984+0000 7f5747e00640 1 --2- 172.21.15.22:0/543843011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7ca0 0x556ab090c4d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5738024de0 tx=0x7f5738005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f674ced7043a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.984+0000 7f573ea00640 1 -- 172.21.15.22:0/543843011 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57380120d0 con 0x556ab08f7ca0 2024-04-20T21:37:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.984+0000 7f573ea00640 1 -- 172.21.15.22:0/543843011 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57380055b0 con 0x556ab08f7ca0 2024-04-20T21:37:12.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.984+0000 7f573ea00640 1 -- 172.21.15.22:0/543843011 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738003830 con 0x556ab08f7ca0 2024-04-20T21:37:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.985+0000 7f5757c65740 1 -- 172.21.15.22:0/543843011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7ca0 msgr2=0x556ab090c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.985+0000 7f5757c65740 1 --2- 172.21.15.22:0/543843011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7ca0 0x556ab090c4d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5738024de0 tx=0x7f5738005870 comp rx=0 tx=0).stop 2024-04-20T21:37:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.985+0000 7f5757c65740 1 -- 172.21.15.22:0/543843011 shutdown_connections 2024-04-20T21:37:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.985+0000 7f5757c65740 1 --2- 172.21.15.22:0/543843011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.985+0000 7f5757c65740 1 --2- 172.21.15.22:0/543843011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7ca0 0x556ab090c4d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.985+0000 7f5757c65740 1 --2- 172.21.15.22:0/543843011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7380 0x556ab0900070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.985+0000 7f5757c65740 1 -- 172.21.15.22:0/543843011 >> 172.21.15.22:0/543843011 conn(0x556ab08f1ab0 msgr2=0x556ab08fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.985+0000 7f5757c65740 1 -- 172.21.15.22:0/543843011 shutdown_connections 2024-04-20T21:37:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.985+0000 7f5757c65740 1 -- 172.21.15.22:0/543843011 wait complete. 2024-04-20T21:37:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.986+0000 7f5757c65740 1 -- start start 2024-04-20T21:37:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.986+0000 7f5757c65740 1 Processor -- start 2024-04-20T21:37:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.986+0000 7f5757c65740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7380 0x556ab08fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.986+0000 7f5757c65740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 0x556ab08fabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.986+0000 7f5757c65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.986+0000 7f5757c65740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57501aeb20 con 0x556ab08f7380 2024-04-20T21:37:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.986+0000 7f5757c65740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57501aff90 con 0x556ab08f7ca0 2024-04-20T21:37:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.986+0000 7f5757c65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57501af1f0 con 0x556ab0900880 2024-04-20T21:37:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f5757c65740 10 client.0 _ll_register_callbacks cb 0x7f5738009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f5747e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 0x556ab08fabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f574e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7380 0x556ab08fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f5747e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 0x556ab08fabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:36146/0 (socket says 172.21.15.22:36146) 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f5747e00640 1 -- 172.21.15.22:0/1610020250 learned_addr learned my addr 172.21.15.22:0/1610020250 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f574e800640 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f5747e00640 1 -- 172.21.15.22:0/1610020250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 msgr2=0x556ab08fb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f5747e00640 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f5747e00640 1 -- 172.21.15.22:0/1610020250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7380 msgr2=0x556ab08fa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f5747e00640 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7380 0x556ab08fa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f5747e00640 1 -- 172.21.15.22:0/1610020250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab0976270 con 0x556ab08f7ca0 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f574e800640 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7380 0x556ab08fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.987+0000 7f5747e00640 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 0x556ab08fabb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5738005330 tx=0x7f5738005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a06329d79b161080 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.988+0000 7f5733e00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.988+0000 7f5733e00640 1 -- 172.21.15.22:0/1610020250 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738024e60 con 0x556ab08f7ca0 2024-04-20T21:37:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.988+0000 7f5757c65740 1 -- 172.21.15.22:0/1610020250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57380055b0 con 0x556ab08f7ca0 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.988+0000 7f5733e00640 1 -- 172.21.15.22:0/1610020250 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5738003800 con 0x556ab08f7ca0 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.988+0000 7f5733e00640 1 -- 172.21.15.22:0/1610020250 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f573803b050 con 0x556ab08f7ca0 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.988+0000 7f5757c65740 10 client.25814 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.988+0000 7f5757c65740 1 -- 172.21.15.22:0/1610020250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556ab09774f0 con 0x556ab08f7ca0 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.988+0000 7f5733e00640 1 -- 172.21.15.22:0/1610020250 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57380055b0 con 0x556ab08f7ca0 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.989+0000 7f5733e00640 1 -- 172.21.15.22:0/1610020250 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f573802b070 con 0x556ab08f7ca0 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.989+0000 7f5757c65740 10 client.25814 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.989+0000 7f5757c65740 10 client.25814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.989+0000 7f5757c65740 20 client.25814 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.989+0000 7f5757c65740 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556ab0977c00 0x556ab097a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.989+0000 7f5757c65740 4 client.25814 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.989+0000 7f5757c65740 1 -- 172.21.15.22:0/1610020250 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556ab08649e0 con 0x556ab0977c00 2024-04-20T21:37:12.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.989+0000 7f574e800640 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556ab0977c00 0x556ab097a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:12.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f574e800640 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556ab0977c00 0x556ab097a020 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b1a7f50e4d1ab0e server_cookie=2ef8a31f4f58f916 in_seq=0 out_seq=0 2024-04-20T21:37:12.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5733e00640 10 client.25814 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5733e00640 1 -- 172.21.15.22:0/1610020250 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556ab08649e0 con 0x556ab0977c00 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5733e00640 10 client.25814 handle_command_reply: tid=0 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5757c65740 1 client.25814 shutdown 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5757c65740 1 -- 172.21.15.22:0/1610020250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 msgr2=0x556ab08fabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5757c65740 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 0x556ab08fabb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5738005330 tx=0x7f5738005fd0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5757c65740 1 -- 172.21.15.22:0/1610020250 shutdown_connections 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5757c65740 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ab0900880 0x556ab08fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5757c65740 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556ab0977c00 0x556ab097a020 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5757c65740 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab08f7ca0 0x556ab08fabb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5757c65740 1 --2- 172.21.15.22:0/1610020250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ab08f7380 0x556ab08fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5757c65740 1 -- 172.21.15.22:0/1610020250 >> 172.21.15.22:0/1610020250 conn(0x556ab08f1ab0 msgr2=0x556ab08fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.990+0000 7f5733e00640 0 client.25814 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.991+0000 7f5757c65740 1 -- 172.21.15.22:0/1610020250 shutdown_connections 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.991+0000 7f5757c65740 1 -- 172.21.15.22:0/1610020250 wait complete. 2024-04-20T21:37:12.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.991+0000 7f5757c65740 20 client.25814 trim_cache size 0 max 16384 2024-04-20T21:37:12.994 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:37:12.994 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:37:12.994 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:37:12.994 INFO:teuthology.orchestra.run.smithi022.stdout: "00841d9e-4bc8-42a7-a9cd-01f6ef46de48": { 2024-04-20T21:37:12.994 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:37:12.994 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "00841d9e-4bc8-42a7-a9cd-01f6ef46de48", 2024-04-20T21:37:12.994 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:37:12.994 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:37:12.994 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:37:12.994 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:37:12.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.992+0000 7f574f200640 1 -- 172.21.15.22:0/1654801470 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57340a3d20 msgr2=0x7f57340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.992+0000 7f574f200640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57340a3d20 0x7f57340a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x556ab08f6e50 tx=0x7f574000ef90 comp rx=0 tx=0).stop 2024-04-20T21:37:12.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.992+0000 7f574f200640 1 -- 172.21.15.22:0/1654801470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5750074390 msgr2=0x7f57501a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:12.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.992+0000 7f574f200640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5750074390 0x7f57501a0620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f574000ea60 tx=0x7f574000ef20 comp rx=0 tx=0).stop 2024-04-20T21:37:12.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.992+0000 7f574f200640 1 -- 172.21.15.22:0/1654801470 shutdown_connections 2024-04-20T21:37:12.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.992+0000 7f574f200640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57340a3d20 0x7f57340a61e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.992+0000 7f574f200640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f575010e650 0x7f575019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.992+0000 7f574f200640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5750074cb0 0x7f57501a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.992+0000 7f574f200640 1 --2- 172.21.15.22:0/1654801470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5750074390 0x7f57501a0620 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:12.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.992+0000 7f574f200640 1 -- 172.21.15.22:0/1654801470 >> 172.21.15.22:0/1654801470 conn(0x7f57500fd900 msgr2=0x7f57500ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:12.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.992+0000 7f574f200640 1 -- 172.21.15.22:0/1654801470 shutdown_connections 2024-04-20T21:37:12.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:12.992+0000 7f574f200640 1 -- 172.21.15.22:0/1654801470 wait complete. 2024-04-20T21:37:13.010 INFO:tasks.cephfs.filesystem:scrub status for tag:00841d9e-4bc8-42a7-a9cd-01f6ef46de48 - {'path': '/', 'tag': '00841d9e-4bc8-42a7-a9cd-01f6ef46de48', 'options': 'recursive,force'} 2024-04-20T21:37:13.173 INFO:tasks.workunit.client.0.smithi022.stdout: 1 195305 10.62 MB/sec warmup 101 sec latency 10.938 ms 2024-04-20T21:37:14.173 INFO:tasks.workunit.client.0.smithi022.stdout: 1 197396 10.58 MB/sec warmup 102 sec latency 12.750 ms 2024-04-20T21:37:14.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:14 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:37:14.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:14 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:14.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:14 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:37:14.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:14 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:14.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:14 smithi122 ceph-mon[22147]: pgmap v105: 97 pgs: 97 active+clean; 117 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-04-20T21:37:14.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:14 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:37:14.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:14 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:14.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:14 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:37:14.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:14 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:14.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:14 smithi033 ceph-mon[21626]: pgmap v105: 97 pgs: 97 active+clean; 117 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-04-20T21:37:14.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:14 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:37:14.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:14 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:14.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:14 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:37:14.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:14 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:14.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:14 smithi022 ceph-mon[16843]: pgmap v105: 97 pgs: 97 active+clean; 117 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-04-20T21:37:15.173 INFO:tasks.workunit.client.0.smithi022.stdout: 1 199064 10.60 MB/sec warmup 103 sec latency 48.134 ms 2024-04-20T21:37:16.173 INFO:tasks.workunit.client.0.smithi022.stdout: 1 201210 10.57 MB/sec warmup 104 sec latency 27.727 ms 2024-04-20T21:37:16.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:16 smithi122 ceph-mon[22147]: pgmap v106: 97 pgs: 97 active+clean; 122 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-04-20T21:37:16.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:16 smithi033 ceph-mon[21626]: pgmap v106: 97 pgs: 97 active+clean; 122 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-04-20T21:37:16.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:16 smithi022 ceph-mon[16843]: pgmap v106: 97 pgs: 97 active+clean; 122 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-04-20T21:37:17.173 INFO:tasks.workunit.client.0.smithi022.stdout: 1 202993 10.58 MB/sec warmup 105 sec latency 18.225 ms 2024-04-20T21:37:17.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:17 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:37:17.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:17 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:37:17.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:17 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:37:18.173 INFO:tasks.workunit.client.0.smithi022.stdout: 1 205562 10.62 MB/sec warmup 106 sec latency 9.384 ms 2024-04-20T21:37:18.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:18 smithi122 ceph-mon[22147]: pgmap v107: 97 pgs: 97 active+clean; 122 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.6 MiB/s wr, 167 op/s 2024-04-20T21:37:18.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:18 smithi033 ceph-mon[21626]: pgmap v107: 97 pgs: 97 active+clean; 122 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.6 MiB/s wr, 167 op/s 2024-04-20T21:37:18.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:18 smithi022 ceph-mon[16843]: pgmap v107: 97 pgs: 97 active+clean; 122 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.6 MiB/s wr, 167 op/s 2024-04-20T21:37:19.174 INFO:tasks.workunit.client.0.smithi022.stdout: 1 207028 10.58 MB/sec warmup 107 sec latency 24.568 ms 2024-04-20T21:37:20.174 INFO:tasks.workunit.client.0.smithi022.stdout: 1 209281 10.62 MB/sec warmup 108 sec latency 72.871 ms 2024-04-20T21:37:20.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:20 smithi122 ceph-mon[22147]: pgmap v108: 97 pgs: 97 active+clean; 122 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.6 MiB/s wr, 167 op/s 2024-04-20T21:37:20.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:20 smithi033 ceph-mon[21626]: pgmap v108: 97 pgs: 97 active+clean; 122 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.6 MiB/s wr, 167 op/s 2024-04-20T21:37:20.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:20 smithi022 ceph-mon[16843]: pgmap v108: 97 pgs: 97 active+clean; 122 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.6 MiB/s wr, 167 op/s 2024-04-20T21:37:21.174 INFO:tasks.workunit.client.0.smithi022.stdout: 1 211104 10.59 MB/sec warmup 109 sec latency 11.259 ms 2024-04-20T21:37:22.174 INFO:tasks.workunit.client.0.smithi022.stdout: 1 213366 10.62 MB/sec warmup 110 sec latency 13.541 ms 2024-04-20T21:37:22.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:22 smithi122 ceph-mon[22147]: pgmap v109: 97 pgs: 97 active+clean; 128 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 252 op/s 2024-04-20T21:37:22.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:22 smithi033 ceph-mon[21626]: pgmap v109: 97 pgs: 97 active+clean; 128 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 252 op/s 2024-04-20T21:37:22.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:22 smithi022 ceph-mon[16843]: pgmap v109: 97 pgs: 97 active+clean; 128 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 252 op/s 2024-04-20T21:37:23.174 INFO:tasks.workunit.client.0.smithi022.stdout: 1 215866 10.64 MB/sec warmup 111 sec latency 12.383 ms 2024-04-20T21:37:24.175 INFO:tasks.workunit.client.0.smithi022.stdout: 1 217482 10.62 MB/sec warmup 112 sec latency 24.443 ms 2024-04-20T21:37:24.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:24 smithi122 ceph-mon[22147]: pgmap v110: 97 pgs: 97 active+clean; 128 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.8 MiB/s wr, 171 op/s 2024-04-20T21:37:24.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:24 smithi033 ceph-mon[21626]: pgmap v110: 97 pgs: 97 active+clean; 128 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.8 MiB/s wr, 171 op/s 2024-04-20T21:37:24.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:24 smithi022 ceph-mon[16843]: pgmap v110: 97 pgs: 97 active+clean; 128 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.8 MiB/s wr, 171 op/s 2024-04-20T21:37:25.174 INFO:tasks.workunit.client.0.smithi022.stdout: 1 219778 10.64 MB/sec warmup 113 sec latency 49.877 ms 2024-04-20T21:37:26.174 INFO:tasks.workunit.client.0.smithi022.stdout: 1 221217 10.60 MB/sec warmup 114 sec latency 16.195 ms 2024-04-20T21:37:26.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:26 smithi122 ceph-mon[22147]: pgmap v111: 97 pgs: 97 active+clean; 133 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 259 op/s 2024-04-20T21:37:26.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:26 smithi033 ceph-mon[21626]: pgmap v111: 97 pgs: 97 active+clean; 133 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 259 op/s 2024-04-20T21:37:26.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:26 smithi022 ceph-mon[16843]: pgmap v111: 97 pgs: 97 active+clean; 133 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 259 op/s 2024-04-20T21:37:27.175 INFO:tasks.workunit.client.0.smithi022.stdout: 1 223632 10.65 MB/sec warmup 115 sec latency 12.130 ms 2024-04-20T21:37:28.175 INFO:tasks.workunit.client.0.smithi022.stdout: 1 224966 10.61 MB/sec warmup 116 sec latency 33.151 ms 2024-04-20T21:37:28.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:28 smithi122 ceph-mon[22147]: pgmap v112: 97 pgs: 97 active+clean; 133 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.7 MiB/s wr, 173 op/s 2024-04-20T21:37:28.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:28 smithi033 ceph-mon[21626]: pgmap v112: 97 pgs: 97 active+clean; 133 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.7 MiB/s wr, 173 op/s 2024-04-20T21:37:28.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:28 smithi022 ceph-mon[16843]: pgmap v112: 97 pgs: 97 active+clean; 133 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.7 MiB/s wr, 173 op/s 2024-04-20T21:37:29.175 INFO:tasks.workunit.client.0.smithi022.stdout: 1 227207 10.63 MB/sec warmup 117 sec latency 10.165 ms 2024-04-20T21:37:30.175 INFO:tasks.workunit.client.0.smithi022.stdout: 1 228266 10.57 MB/sec warmup 118 sec latency 56.393 ms 2024-04-20T21:37:30.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:30 smithi122 ceph-mon[22147]: pgmap v113: 97 pgs: 97 active+clean; 133 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.7 MiB/s wr, 173 op/s 2024-04-20T21:37:30.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:30 smithi033 ceph-mon[21626]: pgmap v113: 97 pgs: 97 active+clean; 133 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.7 MiB/s wr, 173 op/s 2024-04-20T21:37:30.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:30 smithi022 ceph-mon[16843]: pgmap v113: 97 pgs: 97 active+clean; 133 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 2.7 MiB/s wr, 173 op/s 2024-04-20T21:37:31.175 INFO:tasks.workunit.client.0.smithi022.stdout: 1 230751 10.61 MB/sec warmup 119 sec latency 9.630 ms 2024-04-20T21:37:32.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:32 smithi122 ceph-mon[22147]: pgmap v114: 97 pgs: 97 active+clean; 130 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 243 op/s 2024-04-20T21:37:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:32 smithi033 ceph-mon[21626]: pgmap v114: 97 pgs: 97 active+clean; 130 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 243 op/s 2024-04-20T21:37:32.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:32 smithi022 ceph-mon[16843]: pgmap v114: 97 pgs: 97 active+clean; 130 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 243 op/s 2024-04-20T21:37:33.175 INFO:tasks.workunit.client.0.smithi022.stdout: 1 234359 13.75 MB/sec execute 1 sec latency 18.462 ms 2024-04-20T21:37:34.175 INFO:tasks.workunit.client.0.smithi022.stdout: 1 235839 9.66 MB/sec execute 2 sec latency 14.625 ms 2024-04-20T21:37:34.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:34 smithi122 ceph-mon[22147]: pgmap v115: 97 pgs: 97 active+clean; 130 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-04-20T21:37:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:34 smithi033 ceph-mon[21626]: pgmap v115: 97 pgs: 97 active+clean; 130 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-04-20T21:37:34.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:34 smithi022 ceph-mon[16843]: pgmap v115: 97 pgs: 97 active+clean; 130 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-04-20T21:37:35.175 INFO:tasks.workunit.client.0.smithi022.stdout: 1 237949 10.96 MB/sec execute 3 sec latency 36.166 ms 2024-04-20T21:37:36.175 INFO:tasks.workunit.client.0.smithi022.stdout: 1 240022 10.01 MB/sec execute 4 sec latency 16.356 ms 2024-04-20T21:37:36.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:36 smithi122 ceph-mon[22147]: pgmap v116: 97 pgs: 97 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 231 op/s 2024-04-20T21:37:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:36 smithi033 ceph-mon[21626]: pgmap v116: 97 pgs: 97 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 231 op/s 2024-04-20T21:37:36.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:36 smithi022 ceph-mon[16843]: pgmap v116: 97 pgs: 97 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 231 op/s 2024-04-20T21:37:37.176 INFO:tasks.workunit.client.0.smithi022.stdout: 1 241712 10.46 MB/sec execute 5 sec latency 10.334 ms 2024-04-20T21:37:38.176 INFO:tasks.workunit.client.0.smithi022.stdout: 1 244001 10.55 MB/sec execute 6 sec latency 21.921 ms 2024-04-20T21:37:38.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:38 smithi122 ceph-mon[22147]: pgmap v117: 97 pgs: 97 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-04-20T21:37:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:38 smithi033 ceph-mon[21626]: pgmap v117: 97 pgs: 97 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-04-20T21:37:38.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:38 smithi022 ceph-mon[16843]: pgmap v117: 97 pgs: 97 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-04-20T21:37:39.176 INFO:tasks.workunit.client.0.smithi022.stdout: 1 245596 10.28 MB/sec execute 7 sec latency 13.098 ms 2024-04-20T21:37:40.176 INFO:tasks.workunit.client.0.smithi022.stdout: 1 248125 10.73 MB/sec execute 8 sec latency 17.415 ms 2024-04-20T21:37:40.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:40 smithi122 ceph-mon[22147]: pgmap v118: 97 pgs: 97 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-04-20T21:37:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:40 smithi033 ceph-mon[21626]: pgmap v118: 97 pgs: 97 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-04-20T21:37:40.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:40 smithi022 ceph-mon[16843]: pgmap v118: 97 pgs: 97 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-04-20T21:37:41.176 INFO:tasks.workunit.client.0.smithi022.stdout: 1 249397 10.27 MB/sec execute 9 sec latency 39.491 ms 2024-04-20T21:37:42.176 INFO:tasks.workunit.client.0.smithi022.stdout: 1 251985 10.83 MB/sec execute 10 sec latency 10.564 ms 2024-04-20T21:37:42.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:42 smithi033 ceph-mon[21626]: pgmap v119: 97 pgs: 97 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-04-20T21:37:42.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:42 smithi022 ceph-mon[16843]: pgmap v119: 97 pgs: 97 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-04-20T21:37:42.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:42 smithi122 ceph-mon[22147]: pgmap v119: 97 pgs: 97 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-04-20T21:37:43.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:37:43.012 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:37:43.176 INFO:tasks.workunit.client.0.smithi022.stdout: 1 253049 10.16 MB/sec execute 11 sec latency 21.645 ms 2024-04-20T21:37:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.241+0000 7fabfca00640 1 -- 172.21.15.22:0/2036977721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabf81038f0 msgr2=0x7fabf810bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.241+0000 7fabfca00640 1 --2- 172.21.15.22:0/2036977721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabf81038f0 0x7fabf810bdd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009900 tx=0x7fabe002bcc0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.242+0000 7fabfca00640 1 -- 172.21.15.22:0/2036977721 shutdown_connections 2024-04-20T21:37:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.242+0000 7fabfca00640 1 --2- 172.21.15.22:0/2036977721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabf810c3f0 0x7fabf810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.242+0000 7fabfca00640 1 --2- 172.21.15.22:0/2036977721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabf81038f0 0x7fabf810bdd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.242+0000 7fabfca00640 1 --2- 172.21.15.22:0/2036977721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf8102fd0 0x7fabf81033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.242+0000 7fabfca00640 1 -- 172.21.15.22:0/2036977721 >> 172.21.15.22:0/2036977721 conn(0x7fabf80fc7d0 msgr2=0x7fabf80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.242+0000 7fabfca00640 1 -- 172.21.15.22:0/2036977721 shutdown_connections 2024-04-20T21:37:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.242+0000 7fabfca00640 1 -- 172.21.15.22:0/2036977721 wait complete. 2024-04-20T21:37:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.243+0000 7fabfca00640 1 Processor -- start 2024-04-20T21:37:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.243+0000 7fabfca00640 1 -- start start 2024-04-20T21:37:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.244+0000 7fabfca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf8102fd0 0x7fabf81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.244+0000 7fabfca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabf81038f0 0x7fabf81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.244+0000 7fabfca00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabf810c3f0 0x7fabf819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.244+0000 7fabfca00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fabf8112200 con 0x7fabf8102fd0 2024-04-20T21:37:43.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.244+0000 7fabfca00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabf8112080 con 0x7fabf810c3f0 2024-04-20T21:37:43.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.244+0000 7fabfca00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fabf8112380 con 0x7fabf81038f0 2024-04-20T21:37:43.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.244+0000 7fabf6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf8102fd0 0x7fabf81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.244+0000 7fabf7200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabf810c3f0 0x7fabf819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.244+0000 7fabf7200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabf810c3f0 0x7fabf819a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33648/0 (socket says 172.21.15.22:33648) 2024-04-20T21:37:43.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.244+0000 7fabf7200640 1 -- 172.21.15.22:0/4012897385 learned_addr learned my addr 172.21.15.22:0/4012897385 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:43.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.244+0000 7fabede00640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabf81038f0 0x7fabf81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.245+0000 7fabf7200640 1 -- 172.21.15.22:0/4012897385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabf81038f0 msgr2=0x7fabf81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.245+0000 7fabf7200640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabf81038f0 0x7fabf81a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.245+0000 7fabf7200640 1 -- 172.21.15.22:0/4012897385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf8102fd0 msgr2=0x7fabf81a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.245+0000 7fabf7200640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf8102fd0 0x7fabf81a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.245+0000 7fabf7200640 1 -- 172.21.15.22:0/4012897385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf819b0c0 con 0x7fabf810c3f0 2024-04-20T21:37:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.245+0000 7fabede00640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabf81038f0 0x7fabf81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.245+0000 7fabf7200640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabf810c3f0 0x7fabf819a800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fabf000eb10 tx=0x7fabf000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.245+0000 7fabefe00640 1 -- 172.21.15.22:0/4012897385 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0004b20 con 0x7fabf810c3f0 2024-04-20T21:37:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.245+0000 7fabefe00640 1 -- 172.21.15.22:0/4012897385 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf0004cc0 con 0x7fabf810c3f0 2024-04-20T21:37:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.245+0000 7fabfca00640 1 -- 172.21.15.22:0/4012897385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf819b3b0 con 0x7fabf810c3f0 2024-04-20T21:37:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.245+0000 7fabefe00640 1 -- 172.21.15.22:0/4012897385 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0010760 con 0x7fabf810c3f0 2024-04-20T21:37:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.246+0000 7fabfca00640 1 -- 172.21.15.22:0/4012897385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf81a76a0 con 0x7fabf810c3f0 2024-04-20T21:37:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.249+0000 7fac001bb740 1 Processor -- start 2024-04-20T21:37:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.249+0000 7fabefe00640 1 -- 172.21.15.22:0/4012897385 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fabf0005020 con 0x7fabf810c3f0 2024-04-20T21:37:43.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.250+0000 7fac001bb740 1 -- start start 2024-04-20T21:37:43.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.250+0000 7fac001bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6380 0x5627444f6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.250+0000 7fac001bb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444f6ca0 0x5627444ff340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.250+0000 7fac001bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 0x562744501c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.250+0000 7fac001bb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56274445c150 con 0x5627444f6ca0 2024-04-20T21:37:43.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.250+0000 7fac001bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562744402370 con 0x5627444ff880 2024-04-20T21:37:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.250+0000 7fac001bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562744458110 con 0x5627444f6380 2024-04-20T21:37:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.251+0000 7fabede00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 0x562744501c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.251+0000 7fabede00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 0x562744501c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33662/0 (socket says 172.21.15.22:33662) 2024-04-20T21:37:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.251+0000 7fabede00640 1 -- 172.21.15.22:0/163138402 learned_addr learned my addr 172.21.15.22:0/163138402 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.251+0000 7fabede00640 1 --2- 172.21.15.22:0/163138402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6380 0x5627444f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.251+0000 7fabf6800640 1 --2- 172.21.15.22:0/163138402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444f6ca0 0x5627444ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.251+0000 7fabede00640 1 -- 172.21.15.22:0/163138402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6380 msgr2=0x5627444f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.251+0000 7fabede00640 1 --2- 172.21.15.22:0/163138402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6380 0x5627444f6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.251+0000 7fabede00640 1 -- 172.21.15.22:0/163138402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444f6ca0 msgr2=0x5627444ff340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.251+0000 7fabede00640 1 --2- 172.21.15.22:0/163138402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444f6ca0 0x5627444ff340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.251+0000 7fabede00640 1 -- 172.21.15.22:0/163138402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627442387d0 con 0x5627444ff880 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.252+0000 7fabede00640 1 --2- 172.21.15.22:0/163138402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 0x562744501c70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fabe0005c40 tx=0x7fabe0005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f42ffa1f22e7e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.252+0000 7fabe7400640 1 -- 172.21.15.22:0/163138402 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe00040d0 con 0x5627444ff880 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.252+0000 7fabe7400640 1 -- 172.21.15.22:0/163138402 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe002cdf0 con 0x5627444ff880 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.252+0000 7fabe7400640 1 -- 172.21.15.22:0/163138402 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe00054b0 con 0x5627444ff880 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.252+0000 7fabefe00640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fabdc0a3d20 0x7fabdc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.252+0000 7fabefe00640 1 -- 172.21.15.22:0/4012897385 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fabf00d0d20 con 0x7fabf810c3f0 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.252+0000 7fabf6800640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fabdc0a3d20 0x7fabdc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.253+0000 7fac001bb740 1 -- 172.21.15.22:0/163138402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 msgr2=0x562744501c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.253+0000 7fac001bb740 1 --2- 172.21.15.22:0/163138402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 0x562744501c70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fabe0005c40 tx=0x7fabe0005b70 comp rx=0 tx=0).stop 2024-04-20T21:37:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.253+0000 7fac001bb740 1 -- 172.21.15.22:0/163138402 shutdown_connections 2024-04-20T21:37:43.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.253+0000 7fac001bb740 1 --2- 172.21.15.22:0/163138402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 0x562744501c70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.253+0000 7fac001bb740 1 --2- 172.21.15.22:0/163138402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444f6ca0 0x5627444ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.253+0000 7fac001bb740 1 --2- 172.21.15.22:0/163138402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6380 0x5627444f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.253+0000 7fac001bb740 1 -- 172.21.15.22:0/163138402 >> 172.21.15.22:0/163138402 conn(0x5627444f0ab0 msgr2=0x5627444f0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:43.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.253+0000 7fac001bb740 1 -- 172.21.15.22:0/163138402 shutdown_connections 2024-04-20T21:37:43.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.253+0000 7fabf6800640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fabdc0a3d20 0x7fabdc0a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5627444f5e50 tx=0x7fabe8005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:43.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.253+0000 7fac001bb740 1 -- 172.21.15.22:0/163138402 wait complete. 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.253+0000 7fac001bb740 1 -- start start 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.254+0000 7fac001bb740 1 Processor -- start 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.254+0000 7fac001bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444f6380 0x5627444fc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.254+0000 7fac001bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6ca0 0x5627444fce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.254+0000 7fac001bb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444ff880 0x5627444fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.254+0000 7fac001bb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562744502610 con 0x5627444ff880 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.254+0000 7fac001bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562744415740 con 0x5627444f6380 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.254+0000 7fac001bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562744502790 con 0x5627444f6ca0 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fac001bb740 10 client.0 _ll_register_callbacks cb 0x562744410b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabf6800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444f6380 0x5627444fc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabf6800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444f6380 0x5627444fc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33676/0 (socket says 172.21.15.22:33676) 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabf6800640 1 -- 172.21.15.22:0/3053085084 learned_addr learned my addr 172.21.15.22:0/3053085084 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabede00640 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6ca0 0x5627444fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabf7200640 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444ff880 0x5627444fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabf6800640 1 -- 172.21.15.22:0/3053085084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6ca0 msgr2=0x5627444fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabf6800640 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6ca0 0x5627444fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabf6800640 1 -- 172.21.15.22:0/3053085084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444ff880 msgr2=0x5627444fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabf6800640 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444ff880 0x5627444fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabf6800640 1 -- 172.21.15.22:0/3053085084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56274416e3f0 con 0x5627444f6380 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabf7200640 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444ff880 0x5627444fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabf6800640 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444f6380 0x5627444fc930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fabe80045e0 tx=0x7fabe8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e41a59a0eb7ec601 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.255+0000 7fabdbe00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.256+0000 7fabdbe00640 1 -- 172.21.15.22:0/3053085084 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe801e680 con 0x5627444f6380 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.256+0000 7fac001bb740 1 -- 172.21.15.22:0/3053085084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabe002cdf0 con 0x5627444f6380 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.256+0000 7fabdbe00640 1 -- 172.21.15.22:0/3053085084 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fabe801e820 con 0x5627444f6380 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.256+0000 7fabdbe00640 1 -- 172.21.15.22:0/3053085084 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fabe801ea20 con 0x5627444f6380 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.256+0000 7fac001bb740 10 client.25826 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.256+0000 7fac001bb740 1 -- 172.21.15.22:0/3053085084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56274412e0f0 con 0x5627444f6380 2024-04-20T21:37:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.256+0000 7fabdbe00640 1 -- 172.21.15.22:0/3053085084 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe002cdf0 con 0x5627444f6380 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.256+0000 7fabdbe00640 1 -- 172.21.15.22:0/3053085084 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fabe8027070 con 0x5627444f6380 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.257+0000 7fac001bb740 10 client.25826 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.257+0000 7fac001bb740 10 client.25826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.257+0000 7fac001bb740 20 client.25826 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.257+0000 7fac001bb740 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56274450fc90 0x562744512110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.257+0000 7fac001bb740 4 client.25826 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.257+0000 7fac001bb740 1 -- 172.21.15.22:0/3053085084 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562744469b70 con 0x56274450fc90 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.257+0000 7fabede00640 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56274450fc90 0x562744512110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.257+0000 7fabede00640 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56274450fc90 0x562744512110 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e5af405312b7602 server_cookie=8b1de72b83f26f51 in_seq=0 out_seq=0 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.257+0000 7fabdbe00640 10 client.25826 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.258+0000 7fabdbe00640 1 -- 172.21.15.22:0/3053085084 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562744469b70 con 0x56274450fc90 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.258+0000 7fabdbe00640 10 client.25826 handle_command_reply: tid=0 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.258+0000 7fac001bb740 1 client.25826 shutdown 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.258+0000 7fac001bb740 1 -- 172.21.15.22:0/3053085084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444f6380 msgr2=0x5627444fc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.258+0000 7fac001bb740 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444f6380 0x5627444fc930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fabe80045e0 tx=0x7fabe8004730 comp rx=0 tx=0).stop 2024-04-20T21:37:43.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.259+0000 7fac001bb740 1 -- 172.21.15.22:0/3053085084 shutdown_connections 2024-04-20T21:37:43.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.259+0000 7fac001bb740 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444ff880 0x5627444fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.259+0000 7fac001bb740 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6ca0 0x5627444fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.259+0000 7fac001bb740 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56274450fc90 0x562744512110 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.259+0000 7fac001bb740 1 --2- 172.21.15.22:0/3053085084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444f6380 0x5627444fc930 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.259+0000 7fac001bb740 1 -- 172.21.15.22:0/3053085084 >> 172.21.15.22:0/3053085084 conn(0x5627444f0ab0 msgr2=0x5627444f2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:43.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.259+0000 7fabdbe00640 0 client.25826 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:43.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.259+0000 7fac001bb740 1 -- 172.21.15.22:0/3053085084 shutdown_connections 2024-04-20T21:37:43.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.259+0000 7fac001bb740 1 -- 172.21.15.22:0/3053085084 wait complete. 2024-04-20T21:37:43.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.259+0000 7fac001bb740 20 client.25826 trim_cache size 0 max 16384 2024-04-20T21:37:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.280+0000 7fac001bb740 1 Processor -- start 2024-04-20T21:37:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.280+0000 7fac001bb740 1 -- start start 2024-04-20T21:37:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.280+0000 7fac001bb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444f6380 0x5627444ff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.280+0000 7fac001bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6ca0 0x5627444fbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.280+0000 7fac001bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 0x56274450b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.280+0000 7fac001bb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fabf81add50 con 0x5627444f6380 2024-04-20T21:37:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.280+0000 7fac001bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabf81b05d0 con 0x5627444ff880 2024-04-20T21:37:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.280+0000 7fac001bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fabf81ae3c0 con 0x5627444f6ca0 2024-04-20T21:37:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.281+0000 7fabf7200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 0x56274450b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.281+0000 7fabede00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6ca0 0x5627444fbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.281+0000 7fabf6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444f6380 0x5627444ff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.281+0000 7fabede00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6ca0 0x5627444fbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:57808/0 (socket says 172.21.15.22:57808) 2024-04-20T21:37:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.281+0000 7fabede00640 1 -- 172.21.15.22:0/634437181 learned_addr learned my addr 172.21.15.22:0/634437181 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.281+0000 7fabede00640 1 -- 172.21.15.22:0/634437181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 msgr2=0x56274450b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.281+0000 7fabede00640 1 --2- 172.21.15.22:0/634437181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 0x56274450b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.281+0000 7fabede00640 1 -- 172.21.15.22:0/634437181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444f6380 msgr2=0x5627444ff060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.281+0000 7fabede00640 1 --2- 172.21.15.22:0/634437181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444f6380 0x5627444ff060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.281+0000 7fabede00640 1 -- 172.21.15.22:0/634437181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56274450ba10 con 0x5627444f6ca0 2024-04-20T21:37:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.281+0000 7fabf7200640 1 --2- 172.21.15.22:0/634437181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 0x56274450b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.281+0000 7fabf6800640 1 --2- 172.21.15.22:0/634437181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444f6380 0x5627444ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.282+0000 7fabede00640 1 --2- 172.21.15.22:0/634437181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6ca0 0x5627444fbad0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fabe0000c00 tx=0x7fabe002bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7810c2b7487df2f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:43.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.283+0000 7fabe6a00640 1 -- 172.21.15.22:0/634437181 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe00042d0 con 0x5627444f6ca0 2024-04-20T21:37:43.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.283+0000 7fabe6a00640 1 -- 172.21.15.22:0/634437181 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe0004470 con 0x5627444f6ca0 2024-04-20T21:37:43.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.283+0000 7fabe6a00640 1 -- 172.21.15.22:0/634437181 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0053170 con 0x5627444f6ca0 2024-04-20T21:37:43.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.284+0000 7fac001bb740 1 -- 172.21.15.22:0/634437181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6ca0 msgr2=0x5627444fbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.284+0000 7fac001bb740 1 --2- 172.21.15.22:0/634437181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6ca0 0x5627444fbad0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fabe0000c00 tx=0x7fabe002bcc0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.284+0000 7fac001bb740 1 -- 172.21.15.22:0/634437181 shutdown_connections 2024-04-20T21:37:43.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.284+0000 7fac001bb740 1 --2- 172.21.15.22:0/634437181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444ff880 0x56274450b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.284+0000 7fac001bb740 1 --2- 172.21.15.22:0/634437181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6ca0 0x5627444fbad0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.284+0000 7fac001bb740 1 --2- 172.21.15.22:0/634437181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444f6380 0x5627444ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.284+0000 7fac001bb740 1 -- 172.21.15.22:0/634437181 >> 172.21.15.22:0/634437181 conn(0x5627444f0ab0 msgr2=0x5627444f1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.284+0000 7fac001bb740 1 -- 172.21.15.22:0/634437181 shutdown_connections 2024-04-20T21:37:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.284+0000 7fac001bb740 1 -- 172.21.15.22:0/634437181 wait complete. 2024-04-20T21:37:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.285+0000 7fac001bb740 1 -- start start 2024-04-20T21:37:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.285+0000 7fac001bb740 1 Processor -- start 2024-04-20T21:37:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.285+0000 7fac001bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6380 0x562744507d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.285+0000 7fac001bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444f6ca0 0x5627444f9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.285+0000 7fac001bb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444ff880 0x5627444f9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.285+0000 7fac001bb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fabf81aea90 con 0x5627444ff880 2024-04-20T21:37:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.285+0000 7fac001bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabf81aff00 con 0x5627444f6ca0 2024-04-20T21:37:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.285+0000 7fac001bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fabf81af160 con 0x5627444f6380 2024-04-20T21:37:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.285+0000 7fac001bb740 10 client.0 _ll_register_callbacks cb 0x7fabe8009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.286+0000 7fabf6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6380 0x562744507d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.286+0000 7fabf6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6380 0x562744507d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:57824/0 (socket says 172.21.15.22:57824) 2024-04-20T21:37:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.286+0000 7fabf6800640 1 -- 172.21.15.22:0/2685117075 learned_addr learned my addr 172.21.15.22:0/2685117075 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.286+0000 7fabf7200640 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444ff880 0x5627444f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.286+0000 7fabede00640 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444f6ca0 0x5627444f9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.286+0000 7fabf6800640 1 -- 172.21.15.22:0/2685117075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444f6ca0 msgr2=0x5627444f9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.286+0000 7fabf6800640 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444f6ca0 0x5627444f9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.286+0000 7fabf6800640 1 -- 172.21.15.22:0/2685117075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444ff880 msgr2=0x5627444f9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.286+0000 7fabf6800640 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444ff880 0x5627444f9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.286+0000 7fabf6800640 1 -- 172.21.15.22:0/2685117075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627444fa050 con 0x5627444f6380 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.286+0000 7fabf6800640 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6380 0x562744507d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fabe8029f70 tx=0x7fabe8031890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e002ebe240a3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.286+0000 7fabdbe00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.287+0000 7fabdbe00640 1 -- 172.21.15.22:0/2685117075 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe80043d0 con 0x5627444f6380 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.287+0000 7fabdbe00640 1 -- 172.21.15.22:0/2685117075 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fabe801edd0 con 0x5627444f6380 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.287+0000 7fac001bb740 1 -- 172.21.15.22:0/2685117075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabe0004470 con 0x5627444f6380 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.287+0000 7fabdbe00640 1 -- 172.21.15.22:0/2685117075 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fabe8031900 con 0x5627444f6380 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.287+0000 7fac001bb740 10 client.25855 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.287+0000 7fac001bb740 1 -- 172.21.15.22:0/2685117075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56274457ea90 con 0x5627444f6380 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.288+0000 7fabdbe00640 1 -- 172.21.15.22:0/2685117075 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe0004470 con 0x5627444f6380 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.288+0000 7fabdbe00640 1 -- 172.21.15.22:0/2685117075 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fabe8038020 con 0x5627444f6380 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.288+0000 7fac001bb740 10 client.25855 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.288+0000 7fac001bb740 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.288+0000 7fac001bb740 20 client.25855 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.288+0000 7fac001bb740 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56274457f150 0x562744581570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.288+0000 7fac001bb740 4 client.25855 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.288+0000 7fac001bb740 1 -- 172.21.15.22:0/2685117075 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5627444ec170 con 0x56274457f150 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.288+0000 7fabede00640 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56274457f150 0x562744581570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.289+0000 7fabede00640 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56274457f150 0x562744581570 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95909de53c9c79cd server_cookie=f6398d6ea008496 in_seq=0 out_seq=0 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.289+0000 7fabdbe00640 10 client.25855 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.289+0000 7fabdbe00640 1 -- 172.21.15.22:0/2685117075 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5627444ec170 con 0x56274457f150 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.289+0000 7fabdbe00640 10 client.25855 handle_command_reply: tid=0 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.289+0000 7fac001bb740 1 client.25855 shutdown 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.289+0000 7fac001bb740 1 -- 172.21.15.22:0/2685117075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6380 msgr2=0x562744507d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.289+0000 7fac001bb740 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6380 0x562744507d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fabe8029f70 tx=0x7fabe8031890 comp rx=0 tx=0).stop 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.290+0000 7fac001bb740 1 -- 172.21.15.22:0/2685117075 shutdown_connections 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.290+0000 7fac001bb740 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627444ff880 0x5627444f9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.290+0000 7fac001bb740 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56274457f150 0x562744581570 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.290+0000 7fac001bb740 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627444f6ca0 0x5627444f9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.290+0000 7fac001bb740 1 --2- 172.21.15.22:0/2685117075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627444f6380 0x562744507d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.290+0000 7fac001bb740 1 -- 172.21.15.22:0/2685117075 >> 172.21.15.22:0/2685117075 conn(0x5627444f0ab0 msgr2=0x5627444f2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:43.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.290+0000 7fabdbe00640 0 client.25855 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:43.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.290+0000 7fac001bb740 1 -- 172.21.15.22:0/2685117075 shutdown_connections 2024-04-20T21:37:43.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.290+0000 7fac001bb740 1 -- 172.21.15.22:0/2685117075 wait complete. 2024-04-20T21:37:43.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.290+0000 7fac001bb740 20 client.25855 trim_cache size 0 max 16384 2024-04-20T21:37:43.293 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:37:43.293 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:37:43.294 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:37:43.294 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:37:43.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.291+0000 7fabfca00640 1 -- 172.21.15.22:0/4012897385 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fabdc0a3d20 msgr2=0x7fabdc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.291+0000 7fabfca00640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fabdc0a3d20 0x7fabdc0a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5627444f5e50 tx=0x7fabe8005f90 comp rx=0 tx=0).stop 2024-04-20T21:37:43.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.291+0000 7fabfca00640 1 -- 172.21.15.22:0/4012897385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabf810c3f0 msgr2=0x7fabf819a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.291+0000 7fabfca00640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabf810c3f0 0x7fabf819a800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fabf000eb10 tx=0x7fabf000c2d0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.291+0000 7fabfca00640 1 -- 172.21.15.22:0/4012897385 shutdown_connections 2024-04-20T21:37:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.291+0000 7fabfca00640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fabdc0a3d20 0x7fabdc0a61e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.291+0000 7fabfca00640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabf810c3f0 0x7fabf819a800 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.291+0000 7fabfca00640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabf81038f0 0x7fabf81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.291+0000 7fabfca00640 1 --2- 172.21.15.22:0/4012897385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabf8102fd0 0x7fabf81a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.291+0000 7fabfca00640 1 -- 172.21.15.22:0/4012897385 >> 172.21.15.22:0/4012897385 conn(0x7fabf80fc7d0 msgr2=0x7fabf8100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.291+0000 7fabfca00640 1 -- 172.21.15.22:0/4012897385 shutdown_connections 2024-04-20T21:37:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.291+0000 7fabfca00640 1 -- 172.21.15.22:0/4012897385 wait complete. 2024-04-20T21:37:43.309 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:37:43.309 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:37:43.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.517+0000 7f89dfe00640 1 -- 172.21.15.22:0/3460216961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e010e470 msgr2=0x7f89e010e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.517+0000 7f89dfe00640 1 --2- 172.21.15.22:0/3460216961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e010e470 0x7f89e010e850 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009900 tx=0x7f89d002bcc0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.517+0000 7f89dfe00640 1 -- 172.21.15.22:0/3460216961 shutdown_connections 2024-04-20T21:37:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.517+0000 7f89dfe00640 1 --2- 172.21.15.22:0/3460216961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e0108da0 0x7f89e0115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.517+0000 7f89dfe00640 1 --2- 172.21.15.22:0/3460216961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e0108400 0x7f89e0108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.517+0000 7f89dfe00640 1 --2- 172.21.15.22:0/3460216961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e010e470 0x7f89e010e850 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.517+0000 7f89dfe00640 1 -- 172.21.15.22:0/3460216961 >> 172.21.15.22:0/3460216961 conn(0x7f89e0077ae0 msgr2=0x7f89e0079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.517+0000 7f89dfe00640 1 -- 172.21.15.22:0/3460216961 shutdown_connections 2024-04-20T21:37:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.517+0000 7f89dfe00640 1 -- 172.21.15.22:0/3460216961 wait complete. 2024-04-20T21:37:43.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.518+0000 7f89dfe00640 1 Processor -- start 2024-04-20T21:37:43.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.519+0000 7f89dfe00640 1 -- start start 2024-04-20T21:37:43.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.519+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e0108400 0x7f89e01a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.519+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e0108da0 0x7f89e01a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.519+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e010e470 0x7f89e019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.519+0000 7f89dfe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89e0118780 con 0x7f89e0108400 2024-04-20T21:37:43.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.519+0000 7f89dfe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f89e0118600 con 0x7f89e010e470 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.519+0000 7f89dfe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89e0118900 con 0x7f89e0108da0 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.519+0000 7f89dea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e0108400 0x7f89e01a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.519+0000 7f89df400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e010e470 0x7f89e019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.519+0000 7f89dea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e0108400 0x7f89e01a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53718/0 (socket says 172.21.15.22:53718) 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.519+0000 7f89dea00640 1 -- 172.21.15.22:0/59340857 learned_addr learned my addr 172.21.15.22:0/59340857 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.520+0000 7f89de000640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e0108da0 0x7f89e01a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.520+0000 7f89df400640 1 -- 172.21.15.22:0/59340857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e0108da0 msgr2=0x7f89e01a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.520+0000 7f89df400640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e0108da0 0x7f89e01a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.520+0000 7f89df400640 1 -- 172.21.15.22:0/59340857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e0108400 msgr2=0x7f89e01a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.520+0000 7f89df400640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e0108400 0x7f89e01a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.520+0000 7f89df400640 1 -- 172.21.15.22:0/59340857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e019f160 con 0x7f89e010e470 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.520+0000 7f89dea00640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e0108400 0x7f89e01a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.520+0000 7f89df400640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e010e470 0x7f89e019e9e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f89d800ec50 tx=0x7f89d800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:43.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.521+0000 7f89d7400640 1 -- 172.21.15.22:0/59340857 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d8003b70 con 0x7f89e010e470 2024-04-20T21:37:43.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.521+0000 7f89d7400640 1 -- 172.21.15.22:0/59340857 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89d8003d10 con 0x7f89e010e470 2024-04-20T21:37:43.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.521+0000 7f89dfe00640 1 -- 172.21.15.22:0/59340857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e019f450 con 0x7f89e010e470 2024-04-20T21:37:43.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.521+0000 7f89d7400640 1 -- 172.21.15.22:0/59340857 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d8010890 con 0x7f89e010e470 2024-04-20T21:37:43.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.521+0000 7f89dfe00640 1 -- 172.21.15.22:0/59340857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e01ab6f0 con 0x7f89e010e470 2024-04-20T21:37:43.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.522+0000 7f89dfe00640 1 -- 172.21.15.22:0/59340857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e00690f0 con 0x7f89e010e470 2024-04-20T21:37:43.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.525+0000 7f89d7400640 1 -- 172.21.15.22:0/59340857 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f89d8004b40 con 0x7f89e010e470 2024-04-20T21:37:43.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.527+0000 7f89d7400640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f89c40a3da0 0x7f89c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:43.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.527+0000 7f89d7400640 1 -- 172.21.15.22:0/59340857 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f89d80d0f50 con 0x7f89e010e470 2024-04-20T21:37:43.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.527+0000 7f89dea00640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f89c40a3da0 0x7f89c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:43.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.528+0000 7f89dea00640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f89c40a3da0 0x7f89c40a6260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f89d00096c0 tx=0x7f89d0005a20 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:43.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.531+0000 7f89d7400640 1 -- 172.21.15.22:0/59340857 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f89d809d570 con 0x7f89e010e470 2024-04-20T21:37:43.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.826+0000 7f89dfe00640 1 -- 172.21.15.22:0/59340857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89e019ff10 con 0x7f89e010e470 2024-04-20T21:37:43.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.827+0000 7f89d7400640 1 -- 172.21.15.22:0/59340857 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f89d809ce70 con 0x7f89e010e470 2024-04-20T21:37:43.825 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:37:43.826 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:37:43.826 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:37:43.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.830+0000 7f89dfe00640 1 -- 172.21.15.22:0/59340857 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f89c40a3da0 msgr2=0x7f89c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.830+0000 7f89dfe00640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f89c40a3da0 0x7f89c40a6260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f89d00096c0 tx=0x7f89d0005a20 comp rx=0 tx=0).stop 2024-04-20T21:37:43.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.830+0000 7f89dfe00640 1 -- 172.21.15.22:0/59340857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e010e470 msgr2=0x7f89e019e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:43.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.830+0000 7f89dfe00640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e010e470 0x7f89e019e9e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f89d800ec50 tx=0x7f89d800c6a0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.831+0000 7f89dfe00640 1 -- 172.21.15.22:0/59340857 shutdown_connections 2024-04-20T21:37:43.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.831+0000 7f89dfe00640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f89c40a3da0 0x7f89c40a6260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.831+0000 7f89dfe00640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e010e470 0x7f89e019e9e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.831+0000 7f89dfe00640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e0108da0 0x7f89e01a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.831+0000 7f89dfe00640 1 --2- 172.21.15.22:0/59340857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e0108400 0x7f89e01a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:43.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.831+0000 7f89dfe00640 1 -- 172.21.15.22:0/59340857 >> 172.21.15.22:0/59340857 conn(0x7f89e0077ae0 msgr2=0x7f89e00792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:43.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.831+0000 7f89dfe00640 1 -- 172.21.15.22:0/59340857 shutdown_connections 2024-04-20T21:37:43.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:43.831+0000 7f89dfe00640 1 -- 172.21.15.22:0/59340857 wait complete. 2024-04-20T21:37:43.850 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:37:44.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.074+0000 7f0bdcc00640 1 -- 172.21.15.22:0/922823788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bd81038e0 msgr2=0x7f0bd810bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.074+0000 7f0bdcc00640 1 --2- 172.21.15.22:0/922823788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bd81038e0 0x7f0bd810bdc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009900 tx=0x7f0bc402bce0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.074+0000 7f0bdcc00640 1 -- 172.21.15.22:0/922823788 shutdown_connections 2024-04-20T21:37:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.074+0000 7f0bdcc00640 1 --2- 172.21.15.22:0/922823788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd810c3e0 0x7f0bd810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.074+0000 7f0bdcc00640 1 --2- 172.21.15.22:0/922823788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bd81038e0 0x7f0bd810bdc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.074+0000 7f0bdcc00640 1 --2- 172.21.15.22:0/922823788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bd8102fc0 0x7f0bd81033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.074+0000 7f0bdcc00640 1 -- 172.21.15.22:0/922823788 >> 172.21.15.22:0/922823788 conn(0x7f0bd80fc7e0 msgr2=0x7f0bd80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.074+0000 7f0bdcc00640 1 -- 172.21.15.22:0/922823788 shutdown_connections 2024-04-20T21:37:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.074+0000 7f0bdcc00640 1 -- 172.21.15.22:0/922823788 wait complete. 2024-04-20T21:37:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.075+0000 7f0bdcc00640 1 Processor -- start 2024-04-20T21:37:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.075+0000 7f0bdcc00640 1 -- start start 2024-04-20T21:37:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.076+0000 7f0bdcc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bd8102fc0 0x7f0bd8116560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.076+0000 7f0bdcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd81038e0 0x7f0bd8116aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.076+0000 7f0bdcc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bd810c3e0 0x7f0bd81116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.076+0000 7f0bdcc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8110020 con 0x7f0bd81038e0 2024-04-20T21:37:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.076+0000 7f0bdcc00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd810fea0 con 0x7f0bd8102fc0 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.076+0000 7f0bdcc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81101a0 con 0x7f0bd810c3e0 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.076+0000 7f0bd7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bd8102fc0 0x7f0bd8116560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.076+0000 7f0bd7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bd8102fc0 0x7f0bd8116560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33728/0 (socket says 172.21.15.22:33728) 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.076+0000 7f0bd7400640 1 -- 172.21.15.22:0/2489974563 learned_addr learned my addr 172.21.15.22:0/2489974563 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.076+0000 7f0bd6a00640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd81038e0 0x7f0bd8116aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.076+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bd810c3e0 0x7f0bd81116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.077+0000 7f0bd7400640 1 -- 172.21.15.22:0/2489974563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bd810c3e0 msgr2=0x7f0bd81116c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.077+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bd810c3e0 0x7f0bd81116c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.077+0000 7f0bd7400640 1 -- 172.21.15.22:0/2489974563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd81038e0 msgr2=0x7f0bd8116aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.077+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd81038e0 0x7f0bd8116aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.077+0000 7f0bd7400640 1 -- 172.21.15.22:0/2489974563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd8111df0 con 0x7f0bd8102fc0 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.077+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bd8102fc0 0x7f0bd8116560 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800d950 tx=0x7f0bc800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.077+0000 7f0bcfe00640 1 -- 172.21.15.22:0/2489974563 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8017070 con 0x7f0bd8102fc0 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.077+0000 7f0bcfe00640 1 -- 172.21.15.22:0/2489974563 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc8009c30 con 0x7f0bd8102fc0 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.077+0000 7f0bcfe00640 1 -- 172.21.15.22:0/2489974563 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc800f6b0 con 0x7f0bd8102fc0 2024-04-20T21:37:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.077+0000 7f0bdcc00640 1 -- 172.21.15.22:0/2489974563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd81120e0 con 0x7f0bd8102fc0 2024-04-20T21:37:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.077+0000 7f0bdcc00640 1 -- 172.21.15.22:0/2489974563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd81b41a0 con 0x7f0bd8102fc0 2024-04-20T21:37:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.081+0000 7f0bcfe00640 1 -- 172.21.15.22:0/2489974563 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0bc80109d0 con 0x7f0bd8102fc0 2024-04-20T21:37:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.081+0000 7f0be0e06740 1 Processor -- start 2024-04-20T21:37:44.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.082+0000 7f0be0e06740 1 -- start start 2024-04-20T21:37:44.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.082+0000 7f0be0e06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178f170 0x556e0178f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.082+0000 7f0be0e06740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e0178fa90 0x556e01798130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.082+0000 7f0be0e06740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e01798670 0x556e0179aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.082+0000 7f0be0e06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556e016f4050 con 0x556e0178f170 2024-04-20T21:37:44.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.082+0000 7f0be0e06740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556e0169a490 con 0x556e0178fa90 2024-04-20T21:37:44.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.082+0000 7f0be0e06740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556e016f0010 con 0x556e01798670 2024-04-20T21:37:44.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.082+0000 7f0bd7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e0178fa90 0x556e01798130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.082+0000 7f0bd7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e0178fa90 0x556e01798130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33734/0 (socket says 172.21.15.22:33734) 2024-04-20T21:37:44.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.082+0000 7f0bd7400640 1 -- 172.21.15.22:0/2192556521 learned_addr learned my addr 172.21.15.22:0/2192556521 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:44.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.082+0000 7f0bd6a00640 1 --2- 172.21.15.22:0/2192556521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e01798670 0x556e0179aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.082+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2192556521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178f170 0x556e0178f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.083+0000 7f0bd7400640 1 -- 172.21.15.22:0/2192556521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e01798670 msgr2=0x556e0179aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:44.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.083+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2192556521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e01798670 0x556e0179aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.083+0000 7f0bd7400640 1 -- 172.21.15.22:0/2192556521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178f170 msgr2=0x556e0178f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:44.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.083+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2192556521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178f170 0x556e0178f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.083+0000 7f0bd7400640 1 -- 172.21.15.22:0/2192556521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e016a2070 con 0x556e0178fa90 2024-04-20T21:37:44.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.083+0000 7f0bd6a00640 1 --2- 172.21.15.22:0/2192556521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e01798670 0x556e0179aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:44.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.083+0000 7f0bcfe00640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0bbc0a3da0 0x7f0bbc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.083+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2192556521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e0178fa90 0x556e01798130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80cd9c0 tx=0x7f0bc80d89b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a63b9311897d2f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.084+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0bbc0a3da0 0x7f0bbc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.084+0000 7f0bc3e00640 1 -- 172.21.15.22:0/2192556521 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80985f0 con 0x556e0178fa90 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.084+0000 7f0bc3e00640 1 -- 172.21.15.22:0/2192556521 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc8018430 con 0x556e0178fa90 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.084+0000 7f0bc3e00640 1 -- 172.21.15.22:0/2192556521 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80ab8a0 con 0x556e0178fa90 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.084+0000 7f0bcfe00640 1 -- 172.21.15.22:0/2489974563 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f0bc80d0320 con 0x7f0bd8102fc0 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.084+0000 7f0be0e06740 1 -- 172.21.15.22:0/2192556521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e0178fa90 msgr2=0x556e01798130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.084+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2192556521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e0178fa90 0x556e01798130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80cd9c0 tx=0x7f0bc80d89b0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.084+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0bbc0a3da0 0x7f0bbc0a6260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80feeb0 tx=0x7f0bd0009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.085+0000 7f0be0e06740 1 -- 172.21.15.22:0/2192556521 shutdown_connections 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.085+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2192556521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e01798670 0x556e0179aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.085+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2192556521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e0178fa90 0x556e01798130 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.085+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2192556521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178f170 0x556e0178f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.085+0000 7f0be0e06740 1 -- 172.21.15.22:0/2192556521 >> 172.21.15.22:0/2192556521 conn(0x556e017898a0 msgr2=0x556e01789c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:44.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.085+0000 7f0be0e06740 1 -- 172.21.15.22:0/2192556521 shutdown_connections 2024-04-20T21:37:44.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.085+0000 7f0be0e06740 1 -- 172.21.15.22:0/2192556521 wait complete. 2024-04-20T21:37:44.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.085+0000 7f0be0e06740 1 -- start start 2024-04-20T21:37:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.085+0000 7f0be0e06740 1 Processor -- start 2024-04-20T21:37:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.086+0000 7f0be0e06740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178f170 0x556e01795610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.086+0000 7f0be0e06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178fa90 0x556e01795b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.086+0000 7f0be0e06740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e01796090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.086+0000 7f0be0e06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556e0179b460 con 0x556e0178fa90 2024-04-20T21:37:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.086+0000 7f0be0e06740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556e016ad860 con 0x556e01798670 2024-04-20T21:37:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.086+0000 7f0be0e06740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556e0179b5e0 con 0x556e0178f170 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.086+0000 7f0be0e06740 10 client.0 _ll_register_callbacks cb 0x556e0167bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.086+0000 7f0bd7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e01796090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.086+0000 7f0bd6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178fa90 0x556e01795b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.086+0000 7f0bd7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e01796090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33740/0 (socket says 172.21.15.22:33740) 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.086+0000 7f0bd7e00640 1 -- 172.21.15.22:0/2559347147 learned_addr learned my addr 172.21.15.22:0/2559347147 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.086+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2559347147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178f170 0x556e01795610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.087+0000 7f0bd7e00640 1 -- 172.21.15.22:0/2559347147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178f170 msgr2=0x556e01795610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.087+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2559347147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178f170 0x556e01795610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.087+0000 7f0bd7e00640 1 -- 172.21.15.22:0/2559347147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178fa90 msgr2=0x556e01795b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.087+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2559347147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178fa90 0x556e01795b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.087+0000 7f0bd7e00640 1 -- 172.21.15.22:0/2559347147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e016b1950 con 0x556e01798670 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.087+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2559347147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e01796090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009c30 tx=0x7f0bd0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d7260f90ac51bb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.087+0000 7f0bc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.087+0000 7f0bc0c00640 1 -- 172.21.15.22:0/2559347147 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0004b20 con 0x556e01798670 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.087+0000 7f0bc0c00640 1 -- 172.21.15.22:0/2559347147 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f0bd0004cc0 con 0x556e01798670 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.087+0000 7f0be0e06740 1 -- 172.21.15.22:0/2559347147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc8018430 con 0x556e01798670 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.088+0000 7f0bc0c00640 1 -- 172.21.15.22:0/2559347147 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0bd000f7d0 con 0x556e01798670 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.088+0000 7f0be0e06740 10 client.25850 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.088+0000 7f0be0e06740 1 -- 172.21.15.22:0/2559347147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556e0167f820 con 0x556e01798670 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.088+0000 7f0bc0c00640 1 -- 172.21.15.22:0/2559347147 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc8018430 con 0x556e01798670 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.088+0000 7f0bc0c00640 1 -- 172.21.15.22:0/2559347147 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f0bd0026070 con 0x556e01798670 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.089+0000 7f0be0e06740 10 client.25850 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.089+0000 7f0be0e06740 10 client.25850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.089+0000 7f0be0e06740 20 client.25850 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.089+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2559347147 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e017a86f0 0x556e017aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.089+0000 7f0be0e06740 4 client.25850 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.089+0000 7f0be0e06740 1 -- 172.21.15.22:0/2559347147 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e01701a70 con 0x556e017a86f0 2024-04-20T21:37:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.089+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2559347147 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e017a86f0 0x556e017aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.090+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2559347147 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e017a86f0 0x556e017aaae0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4802f009d1af8fb5 server_cookie=13caa078e8b0a3ee in_seq=0 out_seq=0 2024-04-20T21:37:44.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.090+0000 7f0bc0c00640 10 client.25850 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:44.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.091+0000 7f0bc0c00640 1 -- 172.21.15.22:0/2559347147 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e01701a70 con 0x556e017a86f0 2024-04-20T21:37:44.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.091+0000 7f0bc0c00640 10 client.25850 handle_command_reply: tid=0 2024-04-20T21:37:44.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.091+0000 7f0be0e06740 1 client.25850 shutdown 2024-04-20T21:37:44.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.091+0000 7f0be0e06740 1 -- 172.21.15.22:0/2559347147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 msgr2=0x556e01796090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:44.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.091+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2559347147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e01796090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009c30 tx=0x7f0bd0004ab0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.092+0000 7f0be0e06740 1 -- 172.21.15.22:0/2559347147 shutdown_connections 2024-04-20T21:37:44.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.092+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2559347147 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e017a86f0 0x556e017aaae0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.092+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2559347147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e01796090 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.092+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2559347147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178fa90 0x556e01795b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.092+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2559347147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178f170 0x556e01795610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.092+0000 7f0bc0c00640 0 client.25850 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:44.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.092+0000 7f0be0e06740 1 -- 172.21.15.22:0/2559347147 >> 172.21.15.22:0/2559347147 conn(0x556e017898a0 msgr2=0x556e0178b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:44.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.092+0000 7f0be0e06740 1 -- 172.21.15.22:0/2559347147 shutdown_connections 2024-04-20T21:37:44.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.092+0000 7f0be0e06740 1 -- 172.21.15.22:0/2559347147 wait complete. 2024-04-20T21:37:44.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.092+0000 7f0be0e06740 20 client.25850 trim_cache size 0 max 16384 2024-04-20T21:37:44.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.110+0000 7f0be0e06740 1 Processor -- start 2024-04-20T21:37:44.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.110+0000 7f0be0e06740 1 -- start start 2024-04-20T21:37:44.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.110+0000 7f0be0e06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178f170 0x556e01797d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.110+0000 7f0be0e06740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178fa90 0x556e017a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.110+0000 7f0be0e06740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e017887e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.110+0000 7f0be0e06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81baa00 con 0x556e0178f170 2024-04-20T21:37:44.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.110+0000 7f0be0e06740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81bd280 con 0x556e01798670 2024-04-20T21:37:44.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.110+0000 7f0be0e06740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81bb070 con 0x556e0178fa90 2024-04-20T21:37:44.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.111+0000 7f0bd7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e017887e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.111+0000 7f0bd7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e017887e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33756/0 (socket says 172.21.15.22:33756) 2024-04-20T21:37:44.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.111+0000 7f0bd7e00640 1 -- 172.21.15.22:0/2350202666 learned_addr learned my addr 172.21.15.22:0/2350202666 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:44.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.111+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2350202666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178f170 0x556e01797d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.111+0000 7f0bd6a00640 1 --2- 172.21.15.22:0/2350202666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178fa90 0x556e017a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.111+0000 7f0bd7e00640 1 -- 172.21.15.22:0/2350202666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178fa90 msgr2=0x556e017a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:44.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.111+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2350202666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178fa90 0x556e017a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.111+0000 7f0bd7e00640 1 -- 172.21.15.22:0/2350202666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178f170 msgr2=0x556e01797d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:44.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.111+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2350202666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178f170 0x556e01797d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.111+0000 7f0bd7e00640 1 -- 172.21.15.22:0/2350202666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e014063f0 con 0x556e01798670 2024-04-20T21:37:44.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.111+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2350202666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178f170 0x556e01797d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:37:44.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.111+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2350202666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e017887e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005a20 tx=0x7f0bd001e640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89584a3fe188ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:44.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.112+0000 7f0bc3400640 1 -- 172.21.15.22:0/2350202666 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0004d20 con 0x556e01798670 2024-04-20T21:37:44.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.112+0000 7f0bc3400640 1 -- 172.21.15.22:0/2350202666 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bd000fe50 con 0x556e01798670 2024-04-20T21:37:44.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.112+0000 7f0bc3400640 1 -- 172.21.15.22:0/2350202666 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd001e6e0 con 0x556e01798670 2024-04-20T21:37:44.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.113+0000 7f0be0e06740 1 -- 172.21.15.22:0/2350202666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 msgr2=0x556e017887e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:44.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.113+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2350202666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e017887e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005a20 tx=0x7f0bd001e640 comp rx=0 tx=0).stop 2024-04-20T21:37:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.113+0000 7f0be0e06740 1 -- 172.21.15.22:0/2350202666 shutdown_connections 2024-04-20T21:37:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.113+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2350202666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e017887e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.113+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2350202666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178fa90 0x556e017a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.113+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2350202666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178f170 0x556e01797d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.113+0000 7f0be0e06740 1 -- 172.21.15.22:0/2350202666 >> 172.21.15.22:0/2350202666 conn(0x556e017898a0 msgr2=0x556e017880d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.113+0000 7f0be0e06740 1 -- 172.21.15.22:0/2350202666 shutdown_connections 2024-04-20T21:37:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.113+0000 7f0be0e06740 1 -- 172.21.15.22:0/2350202666 wait complete. 2024-04-20T21:37:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.114+0000 7f0be0e06740 1 -- start start 2024-04-20T21:37:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.114+0000 7f0be0e06740 1 Processor -- start 2024-04-20T21:37:44.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.115+0000 7f0be0e06740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178f170 0x556e017a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.115+0000 7f0be0e06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178fa90 0x556e0179c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.115+0000 7f0be0e06740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e0179c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.115+0000 7f0be0e06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81bb740 con 0x556e0178fa90 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.115+0000 7f0be0e06740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81bcbb0 con 0x556e01798670 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.115+0000 7f0be0e06740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81bbe10 con 0x556e0178f170 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.115+0000 7f0be0e06740 10 client.0 _ll_register_callbacks cb 0x7f0bd0003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.115+0000 7f0bd7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e0179c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.115+0000 7f0bd7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e0179c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33770/0 (socket says 172.21.15.22:33770) 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.115+0000 7f0bd6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178fa90 0x556e0179c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.115+0000 7f0bd7e00640 1 -- 172.21.15.22:0/2432936051 learned_addr learned my addr 172.21.15.22:0/2432936051 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.115+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2432936051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178f170 0x556e017a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.116+0000 7f0bd7e00640 1 -- 172.21.15.22:0/2432936051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178f170 msgr2=0x556e017a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:44.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.116+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2432936051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178f170 0x556e017a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.116+0000 7f0bd7e00640 1 -- 172.21.15.22:0/2432936051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178fa90 msgr2=0x556e0179c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:44.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.116+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2432936051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178fa90 0x556e0179c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.116+0000 7f0bd7e00640 1 -- 172.21.15.22:0/2432936051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e014d07d0 con 0x556e01798670 2024-04-20T21:37:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.116+0000 7f0bd7e00640 1 --2- 172.21.15.22:0/2432936051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e0179c850 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009c30 tx=0x7f0bd0003dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=637fe03332b2960c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.116+0000 7f0bc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.116+0000 7f0bc0c00640 1 -- 172.21.15.22:0/2432936051 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000f970 con 0x556e01798670 2024-04-20T21:37:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.116+0000 7f0bc0c00640 1 -- 172.21.15.22:0/2432936051 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f0bd000fb10 con 0x556e01798670 2024-04-20T21:37:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.116+0000 7f0be0e06740 1 -- 172.21.15.22:0/2432936051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd000fe50 con 0x556e01798670 2024-04-20T21:37:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.117+0000 7f0bc0c00640 1 -- 172.21.15.22:0/2432936051 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0bd00048a0 con 0x556e01798670 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.117+0000 7f0be0e06740 10 client.25856 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.117+0000 7f0be0e06740 1 -- 172.21.15.22:0/2432936051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556e0180ebf0 con 0x556e01798670 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.117+0000 7f0bc0c00640 1 -- 172.21.15.22:0/2432936051 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bd000fe50 con 0x556e01798670 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.117+0000 7f0bc0c00640 1 -- 172.21.15.22:0/2432936051 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f0bd0026050 con 0x556e01798670 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.117+0000 7f0be0e06740 10 client.25856 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.117+0000 7f0be0e06740 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.117+0000 7f0be0e06740 20 client.25856 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.117+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2432936051 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e0180f1a0 0x556e01811590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.117+0000 7f0be0e06740 4 client.25856 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.117+0000 7f0be0e06740 1 -- 172.21.15.22:0/2432936051 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e016fb8e0 con 0x556e0180f1a0 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.118+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2432936051 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e0180f1a0 0x556e01811590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.118+0000 7f0bd7400640 1 --2- 172.21.15.22:0/2432936051 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e0180f1a0 0x556e01811590 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74bed3b3e5800e07 server_cookie=6287c387f48b0f in_seq=0 out_seq=0 2024-04-20T21:37:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.118+0000 7f0bc0c00640 10 client.25856 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.119+0000 7f0bc0c00640 1 -- 172.21.15.22:0/2432936051 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e016fb8e0 con 0x556e0180f1a0 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.119+0000 7f0bc0c00640 10 client.25856 handle_command_reply: tid=0 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.119+0000 7f0be0e06740 1 client.25856 shutdown 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.119+0000 7f0be0e06740 1 -- 172.21.15.22:0/2432936051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 msgr2=0x556e0179c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.119+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2432936051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e0179c850 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009c30 tx=0x7f0bd0003dd0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.119+0000 7f0be0e06740 1 -- 172.21.15.22:0/2432936051 shutdown_connections 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.119+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2432936051 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e0180f1a0 0x556e01811590 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.119+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2432936051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e01798670 0x556e0179c850 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.119+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2432936051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e0178fa90 0x556e0179c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.119+0000 7f0be0e06740 1 --2- 172.21.15.22:0/2432936051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e0178f170 0x556e017a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.119+0000 7f0bc0c00640 0 client.25856 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.119+0000 7f0be0e06740 1 -- 172.21.15.22:0/2432936051 >> 172.21.15.22:0/2432936051 conn(0x556e017898a0 msgr2=0x556e01795e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.120+0000 7f0be0e06740 1 -- 172.21.15.22:0/2432936051 shutdown_connections 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.120+0000 7f0be0e06740 1 -- 172.21.15.22:0/2432936051 wait complete. 2024-04-20T21:37:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.120+0000 7f0be0e06740 20 client.25856 trim_cache size 0 max 16384 2024-04-20T21:37:44.120 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:37:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.121+0000 7f0bdcc00640 1 -- 172.21.15.22:0/2489974563 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0bbc0a3da0 msgr2=0x7f0bbc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.121+0000 7f0bdcc00640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0bbc0a3da0 0x7f0bbc0a6260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80feeb0 tx=0x7f0bd0009500 comp rx=0 tx=0).stop 2024-04-20T21:37:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.121+0000 7f0bdcc00640 1 -- 172.21.15.22:0/2489974563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bd8102fc0 msgr2=0x7f0bd8116560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.121+0000 7f0bdcc00640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bd8102fc0 0x7f0bd8116560 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800d950 tx=0x7f0bc800de10 comp rx=0 tx=0).stop 2024-04-20T21:37:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.121+0000 7f0bdcc00640 1 -- 172.21.15.22:0/2489974563 shutdown_connections 2024-04-20T21:37:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.121+0000 7f0bdcc00640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0bbc0a3da0 0x7f0bbc0a6260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.121+0000 7f0bdcc00640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bd810c3e0 0x7f0bd81116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.121+0000 7f0bdcc00640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd81038e0 0x7f0bd8116aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.121+0000 7f0bdcc00640 1 --2- 172.21.15.22:0/2489974563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bd8102fc0 0x7f0bd8116560 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.121+0000 7f0bdcc00640 1 -- 172.21.15.22:0/2489974563 >> 172.21.15.22:0/2489974563 conn(0x7f0bd80fc7e0 msgr2=0x7f0bd80fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.121+0000 7f0bdcc00640 1 -- 172.21.15.22:0/2489974563 shutdown_connections 2024-04-20T21:37:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:44.121+0000 7f0bdcc00640 1 -- 172.21.15.22:0/2489974563 wait complete. 2024-04-20T21:37:44.176 INFO:tasks.workunit.client.0.smithi022.stdout: 1 255637 10.71 MB/sec execute 12 sec latency 34.688 ms 2024-04-20T21:37:44.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:44 smithi022 ceph-mon[16843]: pgmap v120: 97 pgs: 97 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-04-20T21:37:44.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:44 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/59340857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:37:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:44 smithi033 ceph-mon[21626]: pgmap v120: 97 pgs: 97 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-04-20T21:37:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:44 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/59340857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:37:44.577 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:44 smithi122 ceph-mon[22147]: pgmap v120: 97 pgs: 97 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-04-20T21:37:44.577 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:44 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/59340857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:37:45.148 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:37:45.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:37:45.149 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:37:45.176 INFO:tasks.workunit.client.0.smithi022.stdout: 1 256652 10.07 MB/sec execute 13 sec latency 25.271 ms 2024-04-20T21:37:45.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.411+0000 7f98c4600640 1 -- 172.21.15.22:0/3711646105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98bc101a60 msgr2=0x7f98bc105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.411+0000 7f98c4600640 1 --2- 172.21.15.22:0/3711646105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98bc101a60 0x7f98bc105aa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009860 tx=0x7f98a802bb70 comp rx=0 tx=0).stop 2024-04-20T21:37:45.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.411+0000 7f98c4600640 1 -- 172.21.15.22:0/3711646105 shutdown_connections 2024-04-20T21:37:45.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.411+0000 7f98c4600640 1 --2- 172.21.15.22:0/3711646105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98bc106180 0x7f98bc10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.411+0000 7f98c4600640 1 --2- 172.21.15.22:0/3711646105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98bc101a60 0x7f98bc105aa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.411+0000 7f98c4600640 1 --2- 172.21.15.22:0/3711646105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98bc1010b0 0x7f98bc101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.411+0000 7f98c4600640 1 -- 172.21.15.22:0/3711646105 >> 172.21.15.22:0/3711646105 conn(0x7f98bc077ec0 msgr2=0x7f98bc0782d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:45.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.411+0000 7f98c4600640 1 -- 172.21.15.22:0/3711646105 shutdown_connections 2024-04-20T21:37:45.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.411+0000 7f98c4600640 1 -- 172.21.15.22:0/3711646105 wait complete. 2024-04-20T21:37:45.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.413+0000 7f98c4600640 1 Processor -- start 2024-04-20T21:37:45.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.413+0000 7f98c4600640 1 -- start start 2024-04-20T21:37:45.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.413+0000 7f98c4600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98bc1010b0 0x7f98bc19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.413+0000 7f98c4600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98bc106180 0x7f98bc19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.413+0000 7f98c4600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98bc198630 0x7f98bc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.413+0000 7f98c4600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc111210 con 0x7f98bc198630 2024-04-20T21:37:45.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.413+0000 7f98c4600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc111090 con 0x7f98bc106180 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.413+0000 7f98c4600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc111390 con 0x7f98bc1010b0 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.414+0000 7f98c1200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98bc106180 0x7f98bc19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.414+0000 7f98c1200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98bc106180 0x7f98bc19eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33790/0 (socket says 172.21.15.22:33790) 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.414+0000 7f98c2600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98bc198630 0x7f98bc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.414+0000 7f98c1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98bc1010b0 0x7f98bc19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.414+0000 7f98c1200640 1 -- 172.21.15.22:0/3813177544 learned_addr learned my addr 172.21.15.22:0/3813177544 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.414+0000 7f98c1200640 1 -- 172.21.15.22:0/3813177544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98bc1010b0 msgr2=0x7f98bc19e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.414+0000 7f98c1200640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98bc1010b0 0x7f98bc19e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.414+0000 7f98c1200640 1 -- 172.21.15.22:0/3813177544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98bc198630 msgr2=0x7f98bc198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.414+0000 7f98c1200640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98bc198630 0x7f98bc198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.414+0000 7f98c1200640 1 -- 172.21.15.22:0/3813177544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc199230 con 0x7f98bc106180 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.414+0000 7f98c1c00640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98bc1010b0 0x7f98bc19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:45.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.414+0000 7f98c1200640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98bc106180 0x7f98bc19eab0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f98a8037040 tx=0x7f98a80045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:45.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.415+0000 7f98ba000640 1 -- 172.21.15.22:0/3813177544 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8041020 con 0x7f98bc106180 2024-04-20T21:37:45.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.415+0000 7f98ba000640 1 -- 172.21.15.22:0/3813177544 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a802c9f0 con 0x7f98bc106180 2024-04-20T21:37:45.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.415+0000 7f98c4600640 1 -- 172.21.15.22:0/3813177544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc1994c0 con 0x7f98bc106180 2024-04-20T21:37:45.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.415+0000 7f98ba000640 1 -- 172.21.15.22:0/3813177544 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8039040 con 0x7f98bc106180 2024-04-20T21:37:45.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.415+0000 7f98c4600640 1 -- 172.21.15.22:0/3813177544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc1a5650 con 0x7f98bc106180 2024-04-20T21:37:45.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.418+0000 7f98c739a740 1 Processor -- start 2024-04-20T21:37:45.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.419+0000 7f98c739a740 1 -- start start 2024-04-20T21:37:45.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.419+0000 7f98ba000640 1 -- 172.21.15.22:0/3813177544 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f98a8044020 con 0x7f98bc106180 2024-04-20T21:37:45.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.419+0000 7f98c739a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x5646831152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.419+0000 7f98c739a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564683115830 0x56468311ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.419+0000 7f98c739a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56468311e410 0x564683120800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.419+0000 7f98c739a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564683079d20 con 0x564683115830 2024-04-20T21:37:45.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.419+0000 7f98c739a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5646830203e0 con 0x564683114f10 2024-04-20T21:37:45.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.419+0000 7f98c739a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564683075ce0 con 0x56468311e410 2024-04-20T21:37:45.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.419+0000 7f98c2600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x5646831152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.419+0000 7f98c2600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x5646831152f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33800/0 (socket says 172.21.15.22:33800) 2024-04-20T21:37:45.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.419+0000 7f98c2600640 1 -- 172.21.15.22:0/1583383247 learned_addr learned my addr 172.21.15.22:0/1583383247 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:45.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.420+0000 7f98c1c00640 1 --2- 172.21.15.22:0/1583383247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564683115830 0x56468311ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.420+0000 7f98c1200640 1 --2- 172.21.15.22:0/1583383247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56468311e410 0x564683120800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.420+0000 7f98c2600640 1 -- 172.21.15.22:0/1583383247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56468311e410 msgr2=0x564683120800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.420+0000 7f98c2600640 1 --2- 172.21.15.22:0/1583383247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56468311e410 0x564683120800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.420+0000 7f98c2600640 1 -- 172.21.15.22:0/1583383247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564683115830 msgr2=0x56468311ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.420+0000 7f98c2600640 1 --2- 172.21.15.22:0/1583383247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564683115830 0x56468311ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.420+0000 7f98c2600640 1 -- 172.21.15.22:0/1583383247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468303f980 con 0x564683114f10 2024-04-20T21:37:45.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.420+0000 7f98c1c00640 1 --2- 172.21.15.22:0/1583383247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564683115830 0x56468311ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:37:45.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.420+0000 7f98c2600640 1 --2- 172.21.15.22:0/1583383247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x5646831152f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009f50 tx=0x7f98b40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae8134228f0979a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.420+0000 7f98aea00640 1 -- 172.21.15.22:0/1583383247 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4004d20 con 0x564683114f10 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.420+0000 7f98aea00640 1 -- 172.21.15.22:0/1583383247 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98b4004050 con 0x564683114f10 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.421+0000 7f98aea00640 1 -- 172.21.15.22:0/1583383247 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4005280 con 0x564683114f10 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.421+0000 7f98ba000640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f98a40a3da0 0x7f98a40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.421+0000 7f98ba000640 1 -- 172.21.15.22:0/3813177544 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f98a80f1ef0 con 0x7f98bc106180 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.421+0000 7f98c739a740 1 -- 172.21.15.22:0/1583383247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 msgr2=0x5646831152f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.421+0000 7f98c739a740 1 --2- 172.21.15.22:0/1583383247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x5646831152f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009f50 tx=0x7f98b40118d0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.421+0000 7f98c1c00640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f98a40a3da0 0x7f98a40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.422+0000 7f98c739a740 1 -- 172.21.15.22:0/1583383247 shutdown_connections 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.422+0000 7f98c739a740 1 --2- 172.21.15.22:0/1583383247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56468311e410 0x564683120800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.422+0000 7f98c739a740 1 --2- 172.21.15.22:0/1583383247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564683115830 0x56468311ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.422+0000 7f98c739a740 1 --2- 172.21.15.22:0/1583383247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x5646831152f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.422+0000 7f98c739a740 1 -- 172.21.15.22:0/1583383247 >> 172.21.15.22:0/1583383247 conn(0x56468310f640 msgr2=0x56468310fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:45.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.422+0000 7f98c739a740 1 -- 172.21.15.22:0/1583383247 shutdown_connections 2024-04-20T21:37:45.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.422+0000 7f98c739a740 1 -- 172.21.15.22:0/1583383247 wait complete. 2024-04-20T21:37:45.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.422+0000 7f98c739a740 1 -- start start 2024-04-20T21:37:45.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.422+0000 7f98c1c00640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f98a40a3da0 0x7f98a40a6260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5646831149e0 tx=0x7f98b0014040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:45.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.423+0000 7f98c739a740 1 Processor -- start 2024-04-20T21:37:45.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.423+0000 7f98c739a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683114f10 0x56468311b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.423+0000 7f98c739a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683115830 0x56468311b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.423+0000 7f98c739a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468311be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.423+0000 7f98c739a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564683121200 con 0x56468311e410 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.423+0000 7f98c739a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5646830337a0 con 0x564683115830 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.423+0000 7f98c739a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564683121380 con 0x564683114f10 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.424+0000 7f98c739a740 10 client.0 _ll_register_callbacks cb 0x564683035b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.424+0000 7f98c1200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683115830 0x56468311b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.424+0000 7f98c2600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468311be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.424+0000 7f98c1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683114f10 0x56468311b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.424+0000 7f98c1200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683115830 0x56468311b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33814/0 (socket says 172.21.15.22:33814) 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.424+0000 7f98c1200640 1 -- 172.21.15.22:0/2616527636 learned_addr learned my addr 172.21.15.22:0/2616527636 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.424+0000 7f98c2600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468311be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53774/0 (socket says 172.21.15.22:53774) 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.424+0000 7f98c1200640 1 -- 172.21.15.22:0/2616527636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683114f10 msgr2=0x56468311b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.424+0000 7f98c1200640 1 --2- 172.21.15.22:0/2616527636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683114f10 0x56468311b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.424+0000 7f98c1200640 1 -- 172.21.15.22:0/2616527636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 msgr2=0x56468311be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.424+0000 7f98c1200640 1 --2- 172.21.15.22:0/2616527636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468311be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.424+0000 7f98c1200640 1 -- 172.21.15.22:0/2616527636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564682e547e0 con 0x564683115830 2024-04-20T21:37:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.425+0000 7f98c1200640 1 --2- 172.21.15.22:0/2616527636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683115830 0x56468311b950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f98a8000c00 tx=0x7f98a8032820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e614e386327b14e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.425+0000 7f98a3400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.425+0000 7f98a3400640 1 -- 172.21.15.22:0/2616527636 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8032640 con 0x564683115830 2024-04-20T21:37:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.425+0000 7f98a3400640 1 -- 172.21.15.22:0/2616527636 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f98a80bc220 con 0x564683115830 2024-04-20T21:37:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.425+0000 7f98c739a740 1 -- 172.21.15.22:0/2616527636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98b4004050 con 0x564683115830 2024-04-20T21:37:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.425+0000 7f98a3400640 1 -- 172.21.15.22:0/2616527636 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f98a80be620 con 0x564683115830 2024-04-20T21:37:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.425+0000 7f98c739a740 10 client.25868 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.425+0000 7f98c739a740 1 -- 172.21.15.22:0/2616527636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564682eb4f70 con 0x564683115830 2024-04-20T21:37:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.426+0000 7f98a3400640 1 -- 172.21.15.22:0/2616527636 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98b4004050 con 0x564683115830 2024-04-20T21:37:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.426+0000 7f98a3400640 1 -- 172.21.15.22:0/2616527636 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f98a80c1020 con 0x564683115830 2024-04-20T21:37:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.426+0000 7f98c739a740 10 client.25868 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:45.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.426+0000 7f98c739a740 10 client.25868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:45.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.426+0000 7f98c739a740 20 client.25868 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:45.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.426+0000 7f98c739a740 1 --2- 172.21.15.22:0/2616527636 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x564683136fd0 0x564683139410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.426+0000 7f98c739a740 4 client.25868 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:37:45.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.426+0000 7f98c739a740 1 -- 172.21.15.22:0/2616527636 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564683087740 con 0x564683136fd0 2024-04-20T21:37:45.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.426+0000 7f98c1c00640 1 --2- 172.21.15.22:0/2616527636 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x564683136fd0 0x564683139410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.427+0000 7f98c1c00640 1 --2- 172.21.15.22:0/2616527636 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x564683136fd0 0x564683139410 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d10b0b821418407e server_cookie=d423385358046cba in_seq=0 out_seq=0 2024-04-20T21:37:45.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.427+0000 7f98a3400640 10 client.25868 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:45.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.428+0000 7f98a3400640 1 -- 172.21.15.22:0/2616527636 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564683087740 con 0x564683136fd0 2024-04-20T21:37:45.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.428+0000 7f98a3400640 10 client.25868 handle_command_reply: tid=0 2024-04-20T21:37:45.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.428+0000 7f98c739a740 1 client.25868 shutdown 2024-04-20T21:37:45.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.429+0000 7f98c739a740 1 -- 172.21.15.22:0/2616527636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683115830 msgr2=0x56468311b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.429+0000 7f98c739a740 1 --2- 172.21.15.22:0/2616527636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683115830 0x56468311b950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f98a8000c00 tx=0x7f98a8032820 comp rx=0 tx=0).stop 2024-04-20T21:37:45.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.429+0000 7f98c739a740 1 -- 172.21.15.22:0/2616527636 shutdown_connections 2024-04-20T21:37:45.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.429+0000 7f98c739a740 1 --2- 172.21.15.22:0/2616527636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468311be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.429+0000 7f98c739a740 1 --2- 172.21.15.22:0/2616527636 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x564683136fd0 0x564683139410 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.429+0000 7f98c739a740 1 --2- 172.21.15.22:0/2616527636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683115830 0x56468311b950 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.429+0000 7f98c739a740 1 --2- 172.21.15.22:0/2616527636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683114f10 0x56468311b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.429+0000 7f98c739a740 1 -- 172.21.15.22:0/2616527636 >> 172.21.15.22:0/2616527636 conn(0x56468310f640 msgr2=0x564683111390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:45.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.429+0000 7f98a3400640 0 client.25868 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:45.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.429+0000 7f98c739a740 1 -- 172.21.15.22:0/2616527636 shutdown_connections 2024-04-20T21:37:45.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.430+0000 7f98c739a740 1 -- 172.21.15.22:0/2616527636 wait complete. 2024-04-20T21:37:45.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.430+0000 7f98c739a740 20 client.25868 trim_cache size 0 max 16384 2024-04-20T21:37:45.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.450+0000 7f98c739a740 1 Processor -- start 2024-04-20T21:37:45.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.450+0000 7f98c739a740 1 -- start start 2024-04-20T21:37:45.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.450+0000 7f98c739a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x56468311db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.450+0000 7f98c739a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683115830 0x564683132980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.450+0000 7f98c739a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468310e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.450+0000 7f98c739a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1a9b90 con 0x56468311e410 2024-04-20T21:37:45.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.450+0000 7f98c739a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1ac410 con 0x564683114f10 2024-04-20T21:37:45.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.450+0000 7f98c739a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1aa200 con 0x564683115830 2024-04-20T21:37:45.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.451+0000 7f98c1c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x56468311db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.451+0000 7f98c1c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x56468311db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33816/0 (socket says 172.21.15.22:33816) 2024-04-20T21:37:45.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.451+0000 7f98c1c00640 1 -- 172.21.15.22:0/1881056960 learned_addr learned my addr 172.21.15.22:0/1881056960 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:45.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.451+0000 7f98c2600640 1 --2- 172.21.15.22:0/1881056960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468310e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.451+0000 7f98c1200640 1 --2- 172.21.15.22:0/1881056960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683115830 0x564683132980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.451+0000 7f98c1c00640 1 -- 172.21.15.22:0/1881056960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683115830 msgr2=0x564683132980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.451+0000 7f98c1c00640 1 --2- 172.21.15.22:0/1881056960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683115830 0x564683132980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.451+0000 7f98c1c00640 1 -- 172.21.15.22:0/1881056960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 msgr2=0x56468310e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.451+0000 7f98c1c00640 1 --2- 172.21.15.22:0/1881056960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468310e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.451+0000 7f98c1c00640 1 -- 172.21.15.22:0/1881056960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468311a420 con 0x564683114f10 2024-04-20T21:37:45.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.452+0000 7f98c1c00640 1 --2- 172.21.15.22:0/1881056960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x56468311db40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009530 tx=0x7f98b0016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c409a8bf9c281f6e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:45.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.452+0000 7f98ae000640 1 -- 172.21.15.22:0/1881056960 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b001e960 con 0x564683114f10 2024-04-20T21:37:45.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.452+0000 7f98ae000640 1 -- 172.21.15.22:0/1881056960 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98b00293b0 con 0x564683114f10 2024-04-20T21:37:45.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.452+0000 7f98ae000640 1 -- 172.21.15.22:0/1881056960 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b00051a0 con 0x564683114f10 2024-04-20T21:37:45.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.453+0000 7f98c739a740 1 -- 172.21.15.22:0/1881056960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 msgr2=0x56468311db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.453+0000 7f98c739a740 1 --2- 172.21.15.22:0/1881056960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x56468311db40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009530 tx=0x7f98b0016000 comp rx=0 tx=0).stop 2024-04-20T21:37:45.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.453+0000 7f98c739a740 1 -- 172.21.15.22:0/1881056960 shutdown_connections 2024-04-20T21:37:45.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.453+0000 7f98c739a740 1 --2- 172.21.15.22:0/1881056960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468310e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.453+0000 7f98c739a740 1 --2- 172.21.15.22:0/1881056960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683115830 0x564683132980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.453+0000 7f98c739a740 1 --2- 172.21.15.22:0/1881056960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x56468311db40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.453+0000 7f98c739a740 1 -- 172.21.15.22:0/1881056960 >> 172.21.15.22:0/1881056960 conn(0x56468310f640 msgr2=0x56468310ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:45.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.453+0000 7f98c739a740 1 -- 172.21.15.22:0/1881056960 shutdown_connections 2024-04-20T21:37:45.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.453+0000 7f98c739a740 1 -- 172.21.15.22:0/1881056960 wait complete. 2024-04-20T21:37:45.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.453+0000 7f98c739a740 1 -- start start 2024-04-20T21:37:45.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.454+0000 7f98c739a740 1 Processor -- start 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.454+0000 7f98c739a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x56468312ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.454+0000 7f98c739a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683115830 0x56468312f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.454+0000 7f98c739a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468312be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.454+0000 7f98c739a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1aa8d0 con 0x56468311e410 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.454+0000 7f98c739a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1abd40 con 0x564683114f10 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.454+0000 7f98c739a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1aafa0 con 0x564683115830 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.454+0000 7f98c739a740 10 client.0 _ll_register_callbacks cb 0x7f98a80094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.455+0000 7f98c1c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x56468312ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.455+0000 7f98c1c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x56468312ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33820/0 (socket says 172.21.15.22:33820) 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.455+0000 7f98c1c00640 1 -- 172.21.15.22:0/4281931850 learned_addr learned my addr 172.21.15.22:0/4281931850 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.455+0000 7f98c1200640 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683115830 0x56468312f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.455+0000 7f98c2600640 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468312be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.455+0000 7f98c1c00640 1 -- 172.21.15.22:0/4281931850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683115830 msgr2=0x56468312f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.455+0000 7f98c1c00640 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683115830 0x56468312f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.455+0000 7f98c1c00640 1 -- 172.21.15.22:0/4281931850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 msgr2=0x56468312be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.455+0000 7f98c1c00640 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468312be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.455+0000 7f98c1c00640 1 -- 172.21.15.22:0/4281931850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564682d4c170 con 0x564683114f10 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.455+0000 7f98c1c00640 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x56468312ee90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f98b0012f30 tx=0x7f98b0029810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a2902f66c60516c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.455+0000 7f98a3400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.456+0000 7f98a3400640 1 -- 172.21.15.22:0/4281931850 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0038020 con 0x564683114f10 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.456+0000 7f98c739a740 1 -- 172.21.15.22:0/4281931850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98b00293b0 con 0x564683114f10 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.456+0000 7f98a3400640 1 -- 172.21.15.22:0/4281931850 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f98b001eb70 con 0x564683114f10 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.456+0000 7f98a3400640 1 -- 172.21.15.22:0/4281931850 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f98b0043050 con 0x564683114f10 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.456+0000 7f98c739a740 10 client.25874 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.456+0000 7f98c739a740 1 -- 172.21.15.22:0/4281931850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56468319de20 con 0x564683114f10 2024-04-20T21:37:45.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.456+0000 7f98a3400640 1 -- 172.21.15.22:0/4281931850 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98b00293b0 con 0x564683114f10 2024-04-20T21:37:45.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.457+0000 7f98a3400640 1 -- 172.21.15.22:0/4281931850 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f98b0032430 con 0x564683114f10 2024-04-20T21:37:45.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.457+0000 7f98c739a740 10 client.25874 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:45.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.457+0000 7f98c739a740 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:45.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.457+0000 7f98c739a740 20 client.25874 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:45.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.457+0000 7f98c739a740 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56468319e550 0x5646831a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.457+0000 7f98c739a740 4 client.25874 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:37:45.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.457+0000 7f98c739a740 1 -- 172.21.15.22:0/4281931850 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5646830815b0 con 0x56468319e550 2024-04-20T21:37:45.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.457+0000 7f98c1200640 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56468319e550 0x5646831a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.457+0000 7f98c1200640 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56468319e550 0x5646831a0970 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f3d38169b66bfe server_cookie=d3448c1325819fb0 in_seq=0 out_seq=0 2024-04-20T21:37:45.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.457+0000 7f98a3400640 10 client.25874 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98a3400640 1 -- 172.21.15.22:0/4281931850 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5646830815b0 con 0x56468319e550 2024-04-20T21:37:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98a3400640 10 client.25874 handle_command_reply: tid=0 2024-04-20T21:37:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98c739a740 1 client.25874 shutdown 2024-04-20T21:37:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98c739a740 1 -- 172.21.15.22:0/4281931850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 msgr2=0x56468312ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98c739a740 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x56468312ee90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f98b0012f30 tx=0x7f98b0029810 comp rx=0 tx=0).stop 2024-04-20T21:37:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98c739a740 1 -- 172.21.15.22:0/4281931850 shutdown_connections 2024-04-20T21:37:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98c739a740 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56468311e410 0x56468312be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98c739a740 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564683115830 0x56468312f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98c739a740 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56468319e550 0x5646831a0970 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98c739a740 1 --2- 172.21.15.22:0/4281931850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564683114f10 0x56468312ee90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98c739a740 1 -- 172.21.15.22:0/4281931850 >> 172.21.15.22:0/4281931850 conn(0x56468310f640 msgr2=0x56468310e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:45.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98a3400640 0 client.25874 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:45.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98c739a740 1 -- 172.21.15.22:0/4281931850 shutdown_connections 2024-04-20T21:37:45.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98c739a740 1 -- 172.21.15.22:0/4281931850 wait complete. 2024-04-20T21:37:45.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.459+0000 7f98c739a740 20 client.25874 trim_cache size 0 max 16384 2024-04-20T21:37:45.461 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:37:45.461 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:37:45.461 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "d710ccb6-60d7-4672-929e-96c027aa252e", 2024-04-20T21:37:45.461 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:37:45.461 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:37:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.460+0000 7f98c4600640 1 -- 172.21.15.22:0/3813177544 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f98a40a3da0 msgr2=0x7f98a40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.460+0000 7f98c4600640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f98a40a3da0 0x7f98a40a6260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5646831149e0 tx=0x7f98b0014040 comp rx=0 tx=0).stop 2024-04-20T21:37:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.460+0000 7f98c4600640 1 -- 172.21.15.22:0/3813177544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98bc106180 msgr2=0x7f98bc19eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.460+0000 7f98c4600640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98bc106180 0x7f98bc19eab0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f98a8037040 tx=0x7f98a80045b0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.461+0000 7f98c4600640 1 -- 172.21.15.22:0/3813177544 shutdown_connections 2024-04-20T21:37:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.461+0000 7f98c4600640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f98a40a3da0 0x7f98a40a6260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.461+0000 7f98c4600640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98bc198630 0x7f98bc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.461+0000 7f98c4600640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98bc106180 0x7f98bc19eab0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.461+0000 7f98c4600640 1 --2- 172.21.15.22:0/3813177544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98bc1010b0 0x7f98bc19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.461+0000 7f98c4600640 1 -- 172.21.15.22:0/3813177544 >> 172.21.15.22:0/3813177544 conn(0x7f98bc077ec0 msgr2=0x7f98bc06a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.461+0000 7f98c4600640 1 -- 172.21.15.22:0/3813177544 shutdown_connections 2024-04-20T21:37:45.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.461+0000 7f98c4600640 1 -- 172.21.15.22:0/3813177544 wait complete. 2024-04-20T21:37:45.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:37:45.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:37:45.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.750+0000 7fdf20a00640 1 -- 172.21.15.22:0/3609444317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf1c104270 msgr2=0x7fdf1c104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.750+0000 7fdf20a00640 1 --2- 172.21.15.22:0/3609444317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf1c104270 0x7fdf1c104650 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009900 tx=0x7fdf0c02bc40 comp rx=0 tx=0).stop 2024-04-20T21:37:45.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.750+0000 7fdf20a00640 1 -- 172.21.15.22:0/3609444317 shutdown_connections 2024-04-20T21:37:45.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.750+0000 7fdf20a00640 1 --2- 172.21.15.22:0/3609444317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c109340 0x7fdf1c110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.750+0000 7fdf20a00640 1 --2- 172.21.15.22:0/3609444317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c104c20 0x7fdf1c108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.750+0000 7fdf20a00640 1 --2- 172.21.15.22:0/3609444317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf1c104270 0x7fdf1c104650 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.750+0000 7fdf20a00640 1 -- 172.21.15.22:0/3609444317 >> 172.21.15.22:0/3609444317 conn(0x7fdf1c0ffa70 msgr2=0x7fdf1c101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:45.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.750+0000 7fdf20a00640 1 -- 172.21.15.22:0/3609444317 shutdown_connections 2024-04-20T21:37:45.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.750+0000 7fdf20a00640 1 -- 172.21.15.22:0/3609444317 wait complete. 2024-04-20T21:37:45.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.751+0000 7fdf20a00640 1 Processor -- start 2024-04-20T21:37:45.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.751+0000 7fdf20a00640 1 -- start start 2024-04-20T21:37:45.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.752+0000 7fdf20a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c104270 0x7fdf1c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.752+0000 7fdf20a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf1c109340 0x7fdf1c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.752+0000 7fdf20a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c19a720 0x7fdf1c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.752+0000 7fdf20a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c114420 con 0x7fdf1c104270 2024-04-20T21:37:45.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.752+0000 7fdf20a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1142a0 con 0x7fdf1c109340 2024-04-20T21:37:45.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.752+0000 7fdf20a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1145a0 con 0x7fdf1c19a720 2024-04-20T21:37:45.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.752+0000 7fdf1a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c104270 0x7fdf1c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.752+0000 7fdf19e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf1c109340 0x7fdf1c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.752+0000 7fdf1a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c104270 0x7fdf1c1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53814/0 (socket says 172.21.15.22:53814) 2024-04-20T21:37:45.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.752+0000 7fdf1a800640 1 -- 172.21.15.22:0/1723526134 learned_addr learned my addr 172.21.15.22:0/1723526134 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:45.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.752+0000 7fdf1b200640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c19a720 0x7fdf1c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.752+0000 7fdf1a800640 1 -- 172.21.15.22:0/1723526134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c19a720 msgr2=0x7fdf1c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.753+0000 7fdf1a800640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c19a720 0x7fdf1c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.753+0000 7fdf1a800640 1 -- 172.21.15.22:0/1723526134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf1c109340 msgr2=0x7fdf1c1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.753+0000 7fdf1a800640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf1c109340 0x7fdf1c1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.753+0000 7fdf1a800640 1 -- 172.21.15.22:0/1723526134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c19b320 con 0x7fdf1c104270 2024-04-20T21:37:45.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.753+0000 7fdf19e00640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf1c109340 0x7fdf1c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:45.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.753+0000 7fdf1a800640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c104270 0x7fdf1c1a0660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c005ab0 tx=0x7fdf0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:45.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.753+0000 7fdf13400640 1 -- 172.21.15.22:0/1723526134 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c041070 con 0x7fdf1c104270 2024-04-20T21:37:45.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.753+0000 7fdf20a00640 1 -- 172.21.15.22:0/1723526134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c19b5b0 con 0x7fdf1c104270 2024-04-20T21:37:45.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.753+0000 7fdf13400640 1 -- 172.21.15.22:0/1723526134 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c02cca0 con 0x7fdf1c104270 2024-04-20T21:37:45.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.754+0000 7fdf13400640 1 -- 172.21.15.22:0/1723526134 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c003d20 con 0x7fdf1c104270 2024-04-20T21:37:45.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.754+0000 7fdf20a00640 1 -- 172.21.15.22:0/1723526134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c1a76f0 con 0x7fdf1c104270 2024-04-20T21:37:45.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.757+0000 7fdf24051740 1 Processor -- start 2024-04-20T21:37:45.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.757+0000 7fdf13400640 1 -- 172.21.15.22:0/1723526134 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fdf0c044070 con 0x7fdf1c104270 2024-04-20T21:37:45.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.757+0000 7fdf24051740 1 -- start start 2024-04-20T21:37:45.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.757+0000 7fdf24051740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bac8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf24051740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bad1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf24051740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bad1880 0x56519bad3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf24051740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56519ba2e150 con 0x56519bad1880 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf24051740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56519b9d4370 con 0x56519bac8ca0 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf24051740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56519ba2a110 con 0x56519bac8380 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf1a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bad1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf19e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bad1880 0x56519bad3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf1a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bad1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33842/0 (socket says 172.21.15.22:33842) 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf1a800640 1 -- 172.21.15.22:0/1848215233 learned_addr learned my addr 172.21.15.22:0/1848215233 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf1b200640 1 --2- 172.21.15.22:0/1848215233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bac8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf19e00640 1 -- 172.21.15.22:0/1848215233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 msgr2=0x56519bac8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf19e00640 1 --2- 172.21.15.22:0/1848215233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bac8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf19e00640 1 -- 172.21.15.22:0/1848215233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 msgr2=0x56519bad1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf19e00640 1 --2- 172.21.15.22:0/1848215233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bad1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.758+0000 7fdf19e00640 1 -- 172.21.15.22:0/1848215233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56519b80a7d0 con 0x56519bad1880 2024-04-20T21:37:45.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.759+0000 7fdf19e00640 1 --2- 172.21.15.22:0/1848215233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bad1880 0x56519bad3c70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c06b630 tx=0x7fdf080118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c716f5945015cf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:45.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.759+0000 7fdf13400640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdf000a3d20 0x7fdf000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.759+0000 7fdf07400640 1 -- 172.21.15.22:0/1848215233 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08004030 con 0x56519bad1880 2024-04-20T21:37:45.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.759+0000 7fdf07400640 1 -- 172.21.15.22:0/1848215233 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf08005500 con 0x56519bad1880 2024-04-20T21:37:45.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.759+0000 7fdf07400640 1 -- 172.21.15.22:0/1848215233 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08004d80 con 0x56519bad1880 2024-04-20T21:37:45.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.759+0000 7fdf13400640 1 -- 172.21.15.22:0/1723526134 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fdf0c0f20a0 con 0x7fdf1c104270 2024-04-20T21:37:45.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.760+0000 7fdf1b200640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdf000a3d20 0x7fdf000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.760+0000 7fdf24051740 1 -- 172.21.15.22:0/1848215233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bad1880 msgr2=0x56519bad3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.760+0000 7fdf24051740 1 --2- 172.21.15.22:0/1848215233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bad1880 0x56519bad3c70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c06b630 tx=0x7fdf080118d0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.760+0000 7fdf24051740 1 -- 172.21.15.22:0/1848215233 shutdown_connections 2024-04-20T21:37:45.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.760+0000 7fdf24051740 1 --2- 172.21.15.22:0/1848215233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bad1880 0x56519bad3c70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.760+0000 7fdf24051740 1 --2- 172.21.15.22:0/1848215233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bad1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.760+0000 7fdf24051740 1 --2- 172.21.15.22:0/1848215233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bac8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.760+0000 7fdf24051740 1 -- 172.21.15.22:0/1848215233 >> 172.21.15.22:0/1848215233 conn(0x56519bac2ab0 msgr2=0x56519bac2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:45.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.761+0000 7fdf24051740 1 -- 172.21.15.22:0/1848215233 shutdown_connections 2024-04-20T21:37:45.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.761+0000 7fdf1b200640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdf000a3d20 0x7fdf000a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdf14005fd0 tx=0x7fdf14009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:45.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.761+0000 7fdf24051740 1 -- 172.21.15.22:0/1848215233 wait complete. 2024-04-20T21:37:45.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.761+0000 7fdf24051740 1 -- start start 2024-04-20T21:37:45.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.761+0000 7fdf24051740 1 Processor -- start 2024-04-20T21:37:45.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.762+0000 7fdf24051740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bace8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.762+0000 7fdf24051740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bacee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.762+0000 7fdf24051740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bad1880 0x56519bacf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.762+0000 7fdf24051740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56519bad4670 con 0x56519bad1880 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.762+0000 7fdf24051740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56519b9e7740 con 0x56519bac8ca0 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.762+0000 7fdf24051740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56519bad47f0 con 0x56519bac8380 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.762+0000 7fdf24051740 10 client.0 _ll_register_callbacks cb 0x56519b9e2b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.762+0000 7fdf19e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bacee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.762+0000 7fdf19e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bacee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33846/0 (socket says 172.21.15.22:33846) 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.762+0000 7fdf19e00640 1 -- 172.21.15.22:0/2360359397 learned_addr learned my addr 172.21.15.22:0/2360359397 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.762+0000 7fdf1b200640 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bad1880 0x56519bacf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.762+0000 7fdf1a800640 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bace8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.763+0000 7fdf19e00640 1 -- 172.21.15.22:0/2360359397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 msgr2=0x56519bace8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.763+0000 7fdf19e00640 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bace8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.763+0000 7fdf19e00640 1 -- 172.21.15.22:0/2360359397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bad1880 msgr2=0x56519bacf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.763+0000 7fdf19e00640 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bad1880 0x56519bacf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.763+0000 7fdf19e00640 1 -- 172.21.15.22:0/2360359397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519b7403f0 con 0x56519bac8ca0 2024-04-20T21:37:45.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.763+0000 7fdf1a800640 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bace8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:45.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.763+0000 7fdf19e00640 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bacee30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf0801b040 tx=0x7fdf08005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d6f5527fac37c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:45.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.763+0000 7fdefbe00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:45.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.763+0000 7fdefbe00640 1 -- 172.21.15.22:0/2360359397 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08028020 con 0x56519bac8ca0 2024-04-20T21:37:45.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdefbe00640 1 -- 172.21.15.22:0/2360359397 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fdf08016040 con 0x56519bac8ca0 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdf24051740 1 -- 172.21.15.22:0/2360359397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf08005500 con 0x56519bac8ca0 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdefbe00640 1 -- 172.21.15.22:0/2360359397 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdf08003800 con 0x56519bac8ca0 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdf24051740 10 client.25886 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdf24051740 1 -- 172.21.15.22:0/2360359397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56519b7000f0 con 0x56519bac8ca0 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdefbe00640 1 -- 172.21.15.22:0/2360359397 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf08005500 con 0x56519bac8ca0 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdefbe00640 1 -- 172.21.15.22:0/2360359397 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fdf08005030 con 0x56519bac8ca0 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdf24051740 10 client.25886 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdf24051740 10 client.25886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdf24051740 20 client.25886 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdf24051740 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56519bae1cb0 0x56519bae40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdf24051740 4 client.25886 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.764+0000 7fdf24051740 1 -- 172.21.15.22:0/2360359397 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56519ba3bb70 con 0x56519bae1cb0 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.765+0000 7fdf1a800640 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56519bae1cb0 0x56519bae40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.765+0000 7fdf1a800640 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56519bae1cb0 0x56519bae40a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c51ce05ec297f2 server_cookie=a80572b0db46320e in_seq=0 out_seq=0 2024-04-20T21:37:45.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.765+0000 7fdefbe00640 10 client.25886 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.766+0000 7fdefbe00640 1 -- 172.21.15.22:0/2360359397 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56519ba3bb70 con 0x56519bae1cb0 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.766+0000 7fdefbe00640 10 client.25886 handle_command_reply: tid=0 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.766+0000 7fdf24051740 1 client.25886 shutdown 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.766+0000 7fdf24051740 1 -- 172.21.15.22:0/2360359397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 msgr2=0x56519bacee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.766+0000 7fdf24051740 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bacee30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf0801b040 tx=0x7fdf08005d70 comp rx=0 tx=0).stop 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.766+0000 7fdf24051740 1 -- 172.21.15.22:0/2360359397 shutdown_connections 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.766+0000 7fdf24051740 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bad1880 0x56519bacf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.766+0000 7fdf24051740 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56519bae1cb0 0x56519bae40a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.767+0000 7fdf24051740 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bacee30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.767+0000 7fdf24051740 1 --2- 172.21.15.22:0/2360359397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bace8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.767+0000 7fdefbe00640 0 client.25886 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.767+0000 7fdf24051740 1 -- 172.21.15.22:0/2360359397 >> 172.21.15.22:0/2360359397 conn(0x56519bac2ab0 msgr2=0x56519bac4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:45.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.767+0000 7fdf24051740 1 -- 172.21.15.22:0/2360359397 shutdown_connections 2024-04-20T21:37:45.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.767+0000 7fdf24051740 1 -- 172.21.15.22:0/2360359397 wait complete. 2024-04-20T21:37:45.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.767+0000 7fdf24051740 20 client.25886 trim_cache size 0 max 16384 2024-04-20T21:37:45.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.792+0000 7fdf24051740 1 Processor -- start 2024-04-20T21:37:45.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.792+0000 7fdf24051740 1 -- start start 2024-04-20T21:37:45.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.792+0000 7fdf24051740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bad1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.792+0000 7fdf24051740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bac8ca0 0x56519badd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.792+0000 7fdf24051740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bad1880 0x56519bae9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.792+0000 7fdf24051740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1add60 con 0x56519bac8ca0 2024-04-20T21:37:45.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.792+0000 7fdf24051740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1b05e0 con 0x56519bad1880 2024-04-20T21:37:45.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.792+0000 7fdf24051740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1ae3d0 con 0x56519bac8380 2024-04-20T21:37:45.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.793+0000 7fdf1b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bad1880 0x56519bae9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.793+0000 7fdf19e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bac8ca0 0x56519badd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.793+0000 7fdf1a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bad1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.793+0000 7fdf1b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bad1880 0x56519bae9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33850/0 (socket says 172.21.15.22:33850) 2024-04-20T21:37:45.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.793+0000 7fdf1b200640 1 -- 172.21.15.22:0/236665859 learned_addr learned my addr 172.21.15.22:0/236665859 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:45.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.793+0000 7fdf1a800640 1 -- 172.21.15.22:0/236665859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bad1880 msgr2=0x56519bae9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.793+0000 7fdf1a800640 1 --2- 172.21.15.22:0/236665859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bad1880 0x56519bae9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.793+0000 7fdf1a800640 1 -- 172.21.15.22:0/236665859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bac8ca0 msgr2=0x56519badd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.793+0000 7fdf1a800640 1 --2- 172.21.15.22:0/236665859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bac8ca0 0x56519badd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.793+0000 7fdf1a800640 1 -- 172.21.15.22:0/236665859 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56519bac8ad0 con 0x56519bac8380 2024-04-20T21:37:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.793+0000 7fdf1a800640 1 --2- 172.21.15.22:0/236665859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bad1020 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0bddd0 tx=0x7fdf0c0f44f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a761a6495a671f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.794+0000 7fdf06a00640 1 -- 172.21.15.22:0/236665859 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c04b040 con 0x56519bac8380 2024-04-20T21:37:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.794+0000 7fdf06a00640 1 -- 172.21.15.22:0/236665859 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c0be7d0 con 0x56519bac8380 2024-04-20T21:37:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.794+0000 7fdf06a00640 1 -- 172.21.15.22:0/236665859 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0be0d0 con 0x56519bac8380 2024-04-20T21:37:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.795+0000 7fdf24051740 1 -- 172.21.15.22:0/236665859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 msgr2=0x56519bad1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.795+0000 7fdf24051740 1 --2- 172.21.15.22:0/236665859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bad1020 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0bddd0 tx=0x7fdf0c0f44f0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.795+0000 7fdf24051740 1 -- 172.21.15.22:0/236665859 shutdown_connections 2024-04-20T21:37:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.795+0000 7fdf24051740 1 --2- 172.21.15.22:0/236665859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bad1880 0x56519bae9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.795+0000 7fdf24051740 1 --2- 172.21.15.22:0/236665859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bac8ca0 0x56519badd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.795+0000 7fdf24051740 1 --2- 172.21.15.22:0/236665859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bac8380 0x56519bad1020 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.795+0000 7fdf24051740 1 -- 172.21.15.22:0/236665859 >> 172.21.15.22:0/236665859 conn(0x56519bac2ab0 msgr2=0x56519bac2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.795+0000 7fdf24051740 1 -- 172.21.15.22:0/236665859 shutdown_connections 2024-04-20T21:37:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.795+0000 7fdf24051740 1 -- 172.21.15.22:0/236665859 wait complete. 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.795+0000 7fdf24051740 1 -- start start 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.796+0000 7fdf24051740 1 Processor -- start 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.796+0000 7fdf24051740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bac8380 0x56519bad9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.796+0000 7fdf24051740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bad5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.796+0000 7fdf24051740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bad1880 0x56519bad5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.796+0000 7fdf24051740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1aeaa0 con 0x56519bac8380 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.796+0000 7fdf24051740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1aff10 con 0x56519bac8ca0 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.796+0000 7fdf24051740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1af170 con 0x56519bad1880 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.796+0000 7fdf24051740 10 client.0 _ll_register_callbacks cb 0x7fdf0c0f5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.796+0000 7fdf19e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bad5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.796+0000 7fdf1a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bac8380 0x56519bad9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.796+0000 7fdf19e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bad5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33854/0 (socket says 172.21.15.22:33854) 2024-04-20T21:37:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdf19e00640 1 -- 172.21.15.22:0/2068529007 learned_addr learned my addr 172.21.15.22:0/2068529007 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdf1b200640 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bad1880 0x56519bad5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdf19e00640 1 -- 172.21.15.22:0/2068529007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bad1880 msgr2=0x56519bad5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdf19e00640 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bad1880 0x56519bad5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdf19e00640 1 -- 172.21.15.22:0/2068529007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bac8380 msgr2=0x56519bad9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdf19e00640 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bac8380 0x56519bad9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdf19e00640 1 -- 172.21.15.22:0/2068529007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519bad6010 con 0x56519bac8ca0 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdf1a800640 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bac8380 0x56519bad9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdf19e00640 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bad5230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdf08004810 tx=0x7fdf08005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71c510b2babbcf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdefbe00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdefbe00640 1 -- 172.21.15.22:0/2068529007 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08028070 con 0x56519bac8ca0 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdf24051740 1 -- 172.21.15.22:0/2068529007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf0c0be7d0 con 0x56519bac8ca0 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.797+0000 7fdefbe00640 1 -- 172.21.15.22:0/2068529007 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fdf0800fd30 con 0x56519bac8ca0 2024-04-20T21:37:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.798+0000 7fdefbe00640 1 -- 172.21.15.22:0/2068529007 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdf08005ab0 con 0x56519bac8ca0 2024-04-20T21:37:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.798+0000 7fdf24051740 10 client.25892 fetch_fsmap learned FSMap version 20 2024-04-20T21:37:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.798+0000 7fdf24051740 1 -- 172.21.15.22:0/2068529007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56519bb50830 con 0x56519bac8ca0 2024-04-20T21:37:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.798+0000 7fdefbe00640 1 -- 172.21.15.22:0/2068529007 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c0be7d0 con 0x56519bac8ca0 2024-04-20T21:37:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.798+0000 7fdefbe00640 1 -- 172.21.15.22:0/2068529007 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fdf08004df0 con 0x56519bac8ca0 2024-04-20T21:37:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.798+0000 7fdf24051740 10 client.25892 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:37:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.798+0000 7fdf24051740 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:45.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.798+0000 7fdf24051740 20 client.25892 populate_metadata read hostname 'smithi022' 2024-04-20T21:37:45.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.798+0000 7fdf24051740 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56519bb50f40 0x56519bb53330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:45.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.798+0000 7fdf24051740 4 client.25892 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:37:45.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.798+0000 7fdf24051740 1 -- 172.21.15.22:0/2068529007 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56519ba359e0 con 0x56519bb50f40 2024-04-20T21:37:45.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.798+0000 7fdf1a800640 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56519bb50f40 0x56519bb53330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:45.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.799+0000 7fdf1a800640 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56519bb50f40 0x56519bb53330 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f596feef43d491 server_cookie=c3ac31b0ad3707b8 in_seq=0 out_seq=0 2024-04-20T21:37:45.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.799+0000 7fdefbe00640 10 client.25892 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:45.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.799+0000 7fdefbe00640 1 -- 172.21.15.22:0/2068529007 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56519ba359e0 con 0x56519bb50f40 2024-04-20T21:37:45.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.799+0000 7fdefbe00640 10 client.25892 handle_command_reply: tid=0 2024-04-20T21:37:45.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.799+0000 7fdf24051740 1 client.25892 shutdown 2024-04-20T21:37:45.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.800+0000 7fdf24051740 1 -- 172.21.15.22:0/2068529007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 msgr2=0x56519bad5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.800+0000 7fdf24051740 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bad5230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdf08004810 tx=0x7fdf08005fa0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.800+0000 7fdf24051740 1 -- 172.21.15.22:0/2068529007 shutdown_connections 2024-04-20T21:37:45.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.800+0000 7fdf24051740 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56519bad1880 0x56519bad5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.800+0000 7fdf24051740 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56519bb50f40 0x56519bb53330 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.800+0000 7fdf24051740 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56519bac8ca0 0x56519bad5230 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.800+0000 7fdf24051740 1 --2- 172.21.15.22:0/2068529007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56519bac8380 0x56519bad9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.800+0000 7fdf24051740 1 -- 172.21.15.22:0/2068529007 >> 172.21.15.22:0/2068529007 conn(0x56519bac2ab0 msgr2=0x56519bace890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:45.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.800+0000 7fdefbe00640 0 client.25892 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:37:45.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.800+0000 7fdf24051740 1 -- 172.21.15.22:0/2068529007 shutdown_connections 2024-04-20T21:37:45.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.800+0000 7fdf24051740 1 -- 172.21.15.22:0/2068529007 wait complete. 2024-04-20T21:37:45.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.800+0000 7fdf24051740 20 client.25892 trim_cache size 0 max 16384 2024-04-20T21:37:45.806 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:37:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:37:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:37:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "d710ccb6-60d7-4672-929e-96c027aa252e": { 2024-04-20T21:37:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:37:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "d710ccb6-60d7-4672-929e-96c027aa252e", 2024-04-20T21:37:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:37:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:37:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:37:45.807 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:37:45.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.801+0000 7fdf20a00640 1 -- 172.21.15.22:0/1723526134 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdf000a3d20 msgr2=0x7fdf000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.801+0000 7fdf20a00640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdf000a3d20 0x7fdf000a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdf14005fd0 tx=0x7fdf14009500 comp rx=0 tx=0).stop 2024-04-20T21:37:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.801+0000 7fdf20a00640 1 -- 172.21.15.22:0/1723526134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c104270 msgr2=0x7fdf1c1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.801+0000 7fdf20a00640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c104270 0x7fdf1c1a0660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c005ab0 tx=0x7fdf0c0042f0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.802+0000 7fdf20a00640 1 -- 172.21.15.22:0/1723526134 shutdown_connections 2024-04-20T21:37:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.802+0000 7fdf20a00640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdf000a3d20 0x7fdf000a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.802+0000 7fdf20a00640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c19a720 0x7fdf1c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.802+0000 7fdf20a00640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf1c109340 0x7fdf1c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.802+0000 7fdf20a00640 1 --2- 172.21.15.22:0/1723526134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c104270 0x7fdf1c1a0660 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.802+0000 7fdf20a00640 1 -- 172.21.15.22:0/1723526134 >> 172.21.15.22:0/1723526134 conn(0x7fdf1c0ffa70 msgr2=0x7fdf1c101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.802+0000 7fdf20a00640 1 -- 172.21.15.22:0/1723526134 shutdown_connections 2024-04-20T21:37:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:37:45.802+0000 7fdf20a00640 1 -- 172.21.15.22:0/1723526134 wait complete. 2024-04-20T21:37:45.818 INFO:tasks.cephfs.filesystem:scrub status for tag:d710ccb6-60d7-4672-929e-96c027aa252e - {'path': '/', 'tag': 'd710ccb6-60d7-4672-929e-96c027aa252e', 'options': 'recursive,force'} 2024-04-20T21:37:46.176 INFO:tasks.workunit.client.0.smithi022.stdout: 1 258758 10.23 MB/sec execute 14 sec latency 40.305 ms 2024-04-20T21:37:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:46 smithi033 ceph-mon[21626]: pgmap v121: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 231 op/s 2024-04-20T21:37:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:46 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:37:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:46 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:46 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:37:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:46 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:46 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:37:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:46 smithi022 ceph-mon[16843]: pgmap v121: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 231 op/s 2024-04-20T21:37:46.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:46 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:37:46.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:46 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:46.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:46 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:37:46.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:46 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:46.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:46 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:37:46.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:46 smithi122 ceph-mon[22147]: pgmap v121: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 231 op/s 2024-04-20T21:37:46.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:46 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:37:46.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:46 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:46.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:46 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:37:46.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:46 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:46.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:46 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:37:47.177 INFO:tasks.workunit.client.0.smithi022.stdout: 1 260176 10.00 MB/sec execute 15 sec latency 13.228 ms 2024-04-20T21:37:48.177 INFO:tasks.workunit.client.0.smithi022.stdout: 1 262829 10.42 MB/sec execute 16 sec latency 8.174 ms 2024-04-20T21:37:48.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:48 smithi033 ceph-mon[21626]: pgmap v122: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-04-20T21:37:48.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:48 smithi022 ceph-mon[16843]: pgmap v122: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-04-20T21:37:48.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:48 smithi122 ceph-mon[22147]: pgmap v122: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-04-20T21:37:49.177 INFO:tasks.workunit.client.0.smithi022.stdout: 1 264915 10.24 MB/sec execute 17 sec latency 12.223 ms 2024-04-20T21:37:50.178 INFO:tasks.workunit.client.0.smithi022.stdout: 1 266783 10.35 MB/sec execute 18 sec latency 19.748 ms 2024-04-20T21:37:50.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:50 smithi033 ceph-mon[21626]: pgmap v123: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-04-20T21:37:50.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:50 smithi022 ceph-mon[16843]: pgmap v123: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-04-20T21:37:50.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:50 smithi122 ceph-mon[22147]: pgmap v123: 97 pgs: 97 active+clean; 143 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-04-20T21:37:51.177 INFO:tasks.workunit.client.0.smithi022.stdout: 1 268668 10.22 MB/sec execute 19 sec latency 65.790 ms 2024-04-20T21:37:52.177 INFO:tasks.workunit.client.0.smithi022.stdout: 1 270269 10.27 MB/sec execute 20 sec latency 40.640 ms 2024-04-20T21:37:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:52 smithi033 ceph-mon[21626]: pgmap v124: 97 pgs: 97 active+clean; 149 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 241 op/s 2024-04-20T21:37:52.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:52 smithi022 ceph-mon[16843]: pgmap v124: 97 pgs: 97 active+clean; 149 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 241 op/s 2024-04-20T21:37:52.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:52 smithi122 ceph-mon[22147]: pgmap v124: 97 pgs: 97 active+clean; 149 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 241 op/s 2024-04-20T21:37:53.177 INFO:tasks.workunit.client.0.smithi022.stdout: 1 272198 10.16 MB/sec execute 21 sec latency 11.751 ms 2024-04-20T21:37:54.178 INFO:tasks.workunit.client.0.smithi022.stdout: 1 273702 10.20 MB/sec execute 22 sec latency 12.069 ms 2024-04-20T21:37:54.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:54 smithi022 ceph-mon[16843]: pgmap v125: 97 pgs: 97 active+clean; 149 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 2.6 MiB/s wr, 162 op/s 2024-04-20T21:37:54.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:54 smithi122 ceph-mon[22147]: pgmap v125: 97 pgs: 97 active+clean; 149 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 2.6 MiB/s wr, 162 op/s 2024-04-20T21:37:54.805 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:54 smithi033 ceph-mon[21626]: pgmap v125: 97 pgs: 97 active+clean; 149 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 2.6 MiB/s wr, 162 op/s 2024-04-20T21:37:55.178 INFO:tasks.workunit.client.0.smithi022.stdout: 1 275569 10.07 MB/sec execute 23 sec latency 23.726 ms 2024-04-20T21:37:56.179 INFO:tasks.workunit.client.0.smithi022.stdout: 1 277015 10.13 MB/sec execute 24 sec latency 49.118 ms 2024-04-20T21:37:56.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:56 smithi022 ceph-mon[16843]: pgmap v126: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 245 op/s 2024-04-20T21:37:56.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:56 smithi122 ceph-mon[22147]: pgmap v126: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 245 op/s 2024-04-20T21:37:56.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:56 smithi033 ceph-mon[21626]: pgmap v126: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 245 op/s 2024-04-20T21:37:57.180 INFO:tasks.workunit.client.0.smithi022.stdout: 1 278755 9.97 MB/sec execute 25 sec latency 30.952 ms 2024-04-20T21:37:58.181 INFO:tasks.workunit.client.0.smithi022.stdout: 1 280560 10.09 MB/sec execute 26 sec latency 16.547 ms 2024-04-20T21:37:58.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:37:58 smithi022 ceph-mon[16843]: pgmap v127: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 165 op/s 2024-04-20T21:37:58.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:37:58 smithi122 ceph-mon[22147]: pgmap v127: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 165 op/s 2024-04-20T21:37:58.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:37:58 smithi033 ceph-mon[21626]: pgmap v127: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 165 op/s 2024-04-20T21:37:59.181 INFO:tasks.workunit.client.0.smithi022.stdout: 1 282017 9.92 MB/sec execute 27 sec latency 24.485 ms 2024-04-20T21:38:00.181 INFO:tasks.workunit.client.0.smithi022.stdout: 1 284206 10.06 MB/sec execute 28 sec latency 11.021 ms 2024-04-20T21:38:00.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:00 smithi022 ceph-mon[16843]: pgmap v128: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 165 op/s 2024-04-20T21:38:00.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:00 smithi122 ceph-mon[22147]: pgmap v128: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 165 op/s 2024-04-20T21:38:00.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:00 smithi033 ceph-mon[21626]: pgmap v128: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 165 op/s 2024-04-20T21:38:01.181 INFO:tasks.workunit.client.0.smithi022.stdout: 1 286050 9.91 MB/sec execute 29 sec latency 33.324 ms 2024-04-20T21:38:02.181 INFO:tasks.workunit.client.0.smithi022.stdout: 1 286801 9.81 MB/sec execute 30 sec latency 79.964 ms 2024-04-20T21:38:02.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:02 smithi022 ceph-mon[16843]: pgmap v129: 97 pgs: 97 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 243 op/s 2024-04-20T21:38:02.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:02 smithi122 ceph-mon[22147]: pgmap v129: 97 pgs: 97 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 243 op/s 2024-04-20T21:38:02.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:02 smithi033 ceph-mon[21626]: pgmap v129: 97 pgs: 97 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 243 op/s 2024-04-20T21:38:03.181 INFO:tasks.workunit.client.0.smithi022.stdout: 1 288247 9.73 MB/sec execute 31 sec latency 13.751 ms 2024-04-20T21:38:04.181 INFO:tasks.workunit.client.0.smithi022.stdout: 1 290843 9.94 MB/sec execute 32 sec latency 8.728 ms 2024-04-20T21:38:04.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:04 smithi022 ceph-mon[16843]: pgmap v130: 97 pgs: 97 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 160 op/s 2024-04-20T21:38:04.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:04 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:38:04.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:04 smithi122 ceph-mon[22147]: pgmap v130: 97 pgs: 97 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 160 op/s 2024-04-20T21:38:04.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:04 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:38:04.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:04 smithi033 ceph-mon[21626]: pgmap v130: 97 pgs: 97 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 160 op/s 2024-04-20T21:38:04.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:04 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:38:05.182 INFO:tasks.workunit.client.0.smithi022.stdout: 1 292300 9.85 MB/sec execute 33 sec latency 15.341 ms 2024-04-20T21:38:05.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:05 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:38:05.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:05 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:38:05.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:05 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:38:05.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:05 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:38:05.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:05 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:38:05.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:05 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:38:05.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:05 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:38:05.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:05 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:38:05.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:05 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:38:06.182 INFO:tasks.workunit.client.0.smithi022.stdout: 1 294491 9.93 MB/sec execute 34 sec latency 37.683 ms 2024-04-20T21:38:06.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:06 smithi022 ceph-mon[16843]: pgmap v131: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 268 op/s 2024-04-20T21:38:06.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:06 smithi122 ceph-mon[22147]: pgmap v131: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 268 op/s 2024-04-20T21:38:06.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:06 smithi033 ceph-mon[21626]: pgmap v131: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 268 op/s 2024-04-20T21:38:07.183 INFO:tasks.workunit.client.0.smithi022.stdout: 1 295635 9.75 MB/sec execute 35 sec latency 24.604 ms 2024-04-20T21:38:08.182 INFO:tasks.workunit.client.0.smithi022.stdout: 1 297982 9.90 MB/sec execute 36 sec latency 17.094 ms 2024-04-20T21:38:08.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:08 smithi022 ceph-mon[16843]: pgmap v132: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-04-20T21:38:08.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:08 smithi122 ceph-mon[22147]: pgmap v132: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-04-20T21:38:08.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:08 smithi033 ceph-mon[21626]: pgmap v132: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-04-20T21:38:09.182 INFO:tasks.workunit.client.0.smithi022.stdout: 1 299211 9.74 MB/sec execute 37 sec latency 15.418 ms 2024-04-20T21:38:10.183 INFO:tasks.workunit.client.0.smithi022.stdout: 1 301637 9.89 MB/sec execute 38 sec latency 11.463 ms 2024-04-20T21:38:10.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:10 smithi022 ceph-mon[16843]: pgmap v133: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-04-20T21:38:10.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:10 smithi122 ceph-mon[22147]: pgmap v133: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-04-20T21:38:10.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:10 smithi033 ceph-mon[21626]: pgmap v133: 97 pgs: 97 active+clean; 161 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-04-20T21:38:11.183 INFO:tasks.workunit.client.0.smithi022.stdout: 1 302560 9.72 MB/sec execute 39 sec latency 101.355 ms 2024-04-20T21:38:12.183 INFO:tasks.workunit.client.0.smithi022.stdout: 1 304858 9.79 MB/sec execute 40 sec latency 40.907 ms 2024-04-20T21:38:12.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:12 smithi022 ceph-mon[16843]: pgmap v134: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 263 op/s 2024-04-20T21:38:12.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:12 smithi122 ceph-mon[22147]: pgmap v134: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 263 op/s 2024-04-20T21:38:12.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:12 smithi033 ceph-mon[21626]: pgmap v134: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 263 op/s 2024-04-20T21:38:13.183 INFO:tasks.workunit.client.0.smithi022.stdout: 1 306138 9.72 MB/sec execute 41 sec latency 14.737 ms 2024-04-20T21:38:14.183 INFO:tasks.workunit.client.0.smithi022.stdout: 1 308439 9.78 MB/sec execute 42 sec latency 10.175 ms 2024-04-20T21:38:14.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:14 smithi022 ceph-mon[16843]: pgmap v135: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-04-20T21:38:14.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:14 smithi122 ceph-mon[22147]: pgmap v135: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-04-20T21:38:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:14 smithi033 ceph-mon[21626]: pgmap v135: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-04-20T21:38:15.183 INFO:tasks.workunit.client.0.smithi022.stdout: 1 309593 9.68 MB/sec execute 43 sec latency 16.330 ms 2024-04-20T21:38:15.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:38:15.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:38:16.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.072+0000 7f53ebe00640 1 -- 172.21.15.22:0/759176754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e41043b0 msgr2=0x7f53e4104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.072+0000 7f53ebe00640 1 --2- 172.21.15.22:0/759176754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e41043b0 0x7f53e4104790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009900 tx=0x7f53d802bc30 comp rx=0 tx=0).stop 2024-04-20T21:38:16.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.073+0000 7f53ebe00640 1 -- 172.21.15.22:0/759176754 shutdown_connections 2024-04-20T21:38:16.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.073+0000 7f53ebe00640 1 --2- 172.21.15.22:0/759176754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e4109480 0x7f53e4111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.073+0000 7f53ebe00640 1 --2- 172.21.15.22:0/759176754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e4104d60 0x7f53e4108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.073+0000 7f53ebe00640 1 --2- 172.21.15.22:0/759176754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e41043b0 0x7f53e4104790 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.073+0000 7f53ebe00640 1 -- 172.21.15.22:0/759176754 >> 172.21.15.22:0/759176754 conn(0x7f53e40ffbd0 msgr2=0x7f53e4101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.073+0000 7f53ebe00640 1 -- 172.21.15.22:0/759176754 shutdown_connections 2024-04-20T21:38:16.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.073+0000 7f53ebe00640 1 -- 172.21.15.22:0/759176754 wait complete. 2024-04-20T21:38:16.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.074+0000 7f53ebe00640 1 Processor -- start 2024-04-20T21:38:16.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.074+0000 7f53ebe00640 1 -- start start 2024-04-20T21:38:16.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.074+0000 7f53ebe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41043b0 0x7f53e41a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.074+0000 7f53ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4109480 0x7f53e41a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.074+0000 7f53ebe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e419a940 0x7f53e419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.074+0000 7f53ebe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53e4114560 con 0x7f53e4109480 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.074+0000 7f53ebe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f53e41143e0 con 0x7f53e419a940 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.074+0000 7f53ebe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53e41146e0 con 0x7f53e41043b0 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e9e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e419a940 0x7f53e419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41043b0 0x7f53e41a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e8a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4109480 0x7f53e41a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e9e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e419a940 0x7f53e419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45782/0 (socket says 172.21.15.22:45782) 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e9e00640 1 -- 172.21.15.22:0/3013672805 learned_addr learned my addr 172.21.15.22:0/3013672805 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e9400640 1 -- 172.21.15.22:0/3013672805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e419a940 msgr2=0x7f53e419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e9400640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e419a940 0x7f53e419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e9400640 1 -- 172.21.15.22:0/3013672805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4109480 msgr2=0x7f53e41a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e9400640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4109480 0x7f53e41a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e9400640 1 -- 172.21.15.22:0/3013672805 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e419b460 con 0x7f53e41043b0 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e9e00640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e419a940 0x7f53e419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:16.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e8a00640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4109480 0x7f53e41a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:16.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.075+0000 7f53e9400640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41043b0 0x7f53e41a0740 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f53d802c170 tx=0x7f53d8004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.076+0000 7f53e2000640 1 -- 172.21.15.22:0/3013672805 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8043020 con 0x7f53e41043b0 2024-04-20T21:38:16.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.076+0000 7f53ebe00640 1 -- 172.21.15.22:0/3013672805 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e419b6f0 con 0x7f53e41043b0 2024-04-20T21:38:16.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.076+0000 7f53e2000640 1 -- 172.21.15.22:0/3013672805 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d80054e0 con 0x7f53e41043b0 2024-04-20T21:38:16.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.076+0000 7f53e2000640 1 -- 172.21.15.22:0/3013672805 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8003b50 con 0x7f53e41043b0 2024-04-20T21:38:16.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.076+0000 7f53ebe00640 1 -- 172.21.15.22:0/3013672805 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e41a78f0 con 0x7f53e41043b0 2024-04-20T21:38:16.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.079+0000 7f53eebce740 1 Processor -- start 2024-04-20T21:38:16.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.079+0000 7f53eebce740 1 -- start start 2024-04-20T21:38:16.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.079+0000 7f53e2000640 1 -- 172.21.15.22:0/3013672805 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f53d8046020 con 0x7f53e41043b0 2024-04-20T21:38:16.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.079+0000 7f53eebce740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6216831a0 0x55c621683580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.079+0000 7f53eebce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c621683ac0 0x55c62168c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.079+0000 7f53eebce740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c62168c6a0 0x55c62168ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.079+0000 7f53eebce740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c6215e9150 con 0x55c6216831a0 2024-04-20T21:38:16.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.079+0000 7f53eebce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c62158f370 con 0x55c621683ac0 2024-04-20T21:38:16.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.079+0000 7f53eebce740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c6215e5110 con 0x55c62168c6a0 2024-04-20T21:38:16.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.080+0000 7f53e9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6216831a0 0x55c621683580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.080+0000 7f53e9400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c621683ac0 0x55c62168c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.080+0000 7f53e8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c62168c6a0 0x55c62168ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.080+0000 7f53e9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6216831a0 0x55c621683580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38198/0 (socket says 172.21.15.22:38198) 2024-04-20T21:38:16.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.080+0000 7f53e9e00640 1 -- 172.21.15.22:0/955407961 learned_addr learned my addr 172.21.15.22:0/955407961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:16.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.080+0000 7f53e8a00640 1 -- 172.21.15.22:0/955407961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c621683ac0 msgr2=0x55c62168c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.080+0000 7f53e8a00640 1 --2- 172.21.15.22:0/955407961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c621683ac0 0x55c62168c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.080+0000 7f53e8a00640 1 -- 172.21.15.22:0/955407961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6216831a0 msgr2=0x55c621683580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.080+0000 7f53e8a00640 1 --2- 172.21.15.22:0/955407961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6216831a0 0x55c621683580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.080+0000 7f53e8a00640 1 -- 172.21.15.22:0/955407961 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6213c57d0 con 0x55c62168c6a0 2024-04-20T21:38:16.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.080+0000 7f53e9400640 1 --2- 172.21.15.22:0/955407961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c621683ac0 0x55c62168c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:16.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.081+0000 7f53e8a00640 1 --2- 172.21.15.22:0/955407961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c62168c6a0 0x55c62168ea90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f53e419c0e0 tx=0x7f53d00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76e7aaaceb4f9f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.081+0000 7f53d6600640 1 -- 172.21.15.22:0/955407961 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0004030 con 0x55c62168c6a0 2024-04-20T21:38:16.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.081+0000 7f53d6600640 1 -- 172.21.15.22:0/955407961 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d00041d0 con 0x55c62168c6a0 2024-04-20T21:38:16.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.081+0000 7f53d6600640 1 -- 172.21.15.22:0/955407961 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d000f7e0 con 0x55c62168c6a0 2024-04-20T21:38:16.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.081+0000 7f53e2000640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f53cc0a3d20 0x7f53cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.081+0000 7f53e2000640 1 -- 172.21.15.22:0/3013672805 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f53d80f2740 con 0x7f53e41043b0 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.081+0000 7f53e9e00640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f53cc0a3d20 0x7f53cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.082+0000 7f53eebce740 1 -- 172.21.15.22:0/955407961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c62168c6a0 msgr2=0x55c62168ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.082+0000 7f53eebce740 1 --2- 172.21.15.22:0/955407961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c62168c6a0 0x55c62168ea90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f53e419c0e0 tx=0x7f53d00118d0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.082+0000 7f53eebce740 1 -- 172.21.15.22:0/955407961 shutdown_connections 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.082+0000 7f53eebce740 1 --2- 172.21.15.22:0/955407961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c62168c6a0 0x55c62168ea90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.082+0000 7f53eebce740 1 --2- 172.21.15.22:0/955407961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c621683ac0 0x55c62168c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.082+0000 7f53eebce740 1 --2- 172.21.15.22:0/955407961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6216831a0 0x55c621683580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.082+0000 7f53eebce740 1 -- 172.21.15.22:0/955407961 >> 172.21.15.22:0/955407961 conn(0x55c62167d8d0 msgr2=0x55c62167dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.082+0000 7f53eebce740 1 -- 172.21.15.22:0/955407961 shutdown_connections 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.082+0000 7f53eebce740 1 -- 172.21.15.22:0/955407961 wait complete. 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.082+0000 7f53e9e00640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f53cc0a3d20 0x7f53cc0a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009a50 tx=0x7f53dc009760 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.083+0000 7f53eebce740 1 -- start start 2024-04-20T21:38:16.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.083+0000 7f53eebce740 1 Processor -- start 2024-04-20T21:38:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.083+0000 7f53eebce740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6216831a0 0x55c621689690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.083+0000 7f53eebce740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c621683ac0 0x55c621689bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.083+0000 7f53eebce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 0x55c62168a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.083+0000 7f53eebce740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c62168f490 con 0x55c6216831a0 2024-04-20T21:38:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.083+0000 7f53eebce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6215a2740 con 0x55c62168c6a0 2024-04-20T21:38:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.083+0000 7f53eebce740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c62168f610 con 0x55c621683ac0 2024-04-20T21:38:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.083+0000 7f53eebce740 10 client.0 _ll_register_callbacks cb 0x55c62159db20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.084+0000 7f53e9e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 0x55c62168a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.084+0000 7f53e9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6216831a0 0x55c621689690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.084+0000 7f53e9e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 0x55c62168a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45800/0 (socket says 172.21.15.22:45800) 2024-04-20T21:38:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.084+0000 7f53e9e00640 1 -- 172.21.15.22:0/2955443349 learned_addr learned my addr 172.21.15.22:0/2955443349 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.084+0000 7f53e8a00640 1 --2- 172.21.15.22:0/2955443349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c621683ac0 0x55c621689bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.084+0000 7f53e9e00640 1 -- 172.21.15.22:0/2955443349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c621683ac0 msgr2=0x55c621689bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.084+0000 7f53e9e00640 1 --2- 172.21.15.22:0/2955443349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c621683ac0 0x55c621689bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.084+0000 7f53e9e00640 1 -- 172.21.15.22:0/2955443349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6216831a0 msgr2=0x55c621689690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.084+0000 7f53e9e00640 1 --2- 172.21.15.22:0/2955443349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6216831a0 0x55c621689690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.084+0000 7f53e9e00640 1 -- 172.21.15.22:0/2955443349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6212fb3f0 con 0x55c62168c6a0 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.084+0000 7f53e9e00640 1 --2- 172.21.15.22:0/2955443349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 0x55c62168a110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f53dc010d10 tx=0x7f53dc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbece01889292521 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.084+0000 7f53caa00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.085+0000 7f53caa00640 1 -- 172.21.15.22:0/2955443349 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc004b20 con 0x55c62168c6a0 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.085+0000 7f53eebce740 1 -- 172.21.15.22:0/2955443349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d00041d0 con 0x55c62168c6a0 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.085+0000 7f53caa00640 1 -- 172.21.15.22:0/2955443349 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f53dc017030 con 0x55c62168c6a0 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.085+0000 7f53caa00640 1 -- 172.21.15.22:0/2955443349 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f53dc004cc0 con 0x55c62168c6a0 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.085+0000 7f53eebce740 10 client.25904 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:16.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.085+0000 7f53eebce740 1 -- 172.21.15.22:0/2955443349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c6212bb0f0 con 0x55c62168c6a0 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.085+0000 7f53caa00640 1 -- 172.21.15.22:0/2955443349 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d00041d0 con 0x55c62168c6a0 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.086+0000 7f53caa00640 1 -- 172.21.15.22:0/2955443349 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f53dc027410 con 0x55c62168c6a0 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.086+0000 7f53eebce740 10 client.25904 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.086+0000 7f53eebce740 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.086+0000 7f53eebce740 20 client.25904 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.086+0000 7f53eebce740 1 --2- 172.21.15.22:0/2955443349 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c62169ca80 0x55c62169eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.086+0000 7f53eebce740 4 client.25904 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.086+0000 7f53eebce740 1 -- 172.21.15.22:0/2955443349 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6215f6b70 con 0x55c62169ca80 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.086+0000 7f53e9400640 1 --2- 172.21.15.22:0/2955443349 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c62169ca80 0x55c62169eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.087+0000 7f53caa00640 10 client.25904 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.087+0000 7f53e9400640 1 --2- 172.21.15.22:0/2955443349 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c62169ca80 0x55c62169eed0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fd572a16d5e36ad server_cookie=16f6f785840d364c in_seq=0 out_seq=0 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53caa00640 1 -- 172.21.15.22:0/2955443349 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6215f6b70 con 0x55c62169ca80 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53caa00640 10 client.25904 handle_command_reply: tid=0 2024-04-20T21:38:16.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53eebce740 1 client.25904 shutdown 2024-04-20T21:38:16.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53eebce740 1 -- 172.21.15.22:0/2955443349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 msgr2=0x55c62168a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53eebce740 1 --2- 172.21.15.22:0/2955443349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 0x55c62168a110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f53dc010d10 tx=0x7f53dc004ab0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53eebce740 1 -- 172.21.15.22:0/2955443349 shutdown_connections 2024-04-20T21:38:16.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53eebce740 1 --2- 172.21.15.22:0/2955443349 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c62169ca80 0x55c62169eed0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53eebce740 1 --2- 172.21.15.22:0/2955443349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 0x55c62168a110 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53eebce740 1 --2- 172.21.15.22:0/2955443349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c621683ac0 0x55c621689bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53eebce740 1 --2- 172.21.15.22:0/2955443349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6216831a0 0x55c621689690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53eebce740 1 -- 172.21.15.22:0/2955443349 >> 172.21.15.22:0/2955443349 conn(0x55c62167d8d0 msgr2=0x55c62167f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53caa00640 0 client.25904 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:16.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53eebce740 1 -- 172.21.15.22:0/2955443349 shutdown_connections 2024-04-20T21:38:16.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.088+0000 7f53eebce740 1 -- 172.21.15.22:0/2955443349 wait complete. 2024-04-20T21:38:16.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.089+0000 7f53eebce740 20 client.25904 trim_cache size 0 max 16384 2024-04-20T21:38:16.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.112+0000 7f53eebce740 1 Processor -- start 2024-04-20T21:38:16.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.112+0000 7f53eebce740 1 -- start start 2024-04-20T21:38:16.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.113+0000 7f53eebce740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6216831a0 0x55c62168bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.113+0000 7f53eebce740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c621683ac0 0x55c621698350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.113+0000 7f53eebce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 0x55c6216885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.113+0000 7f53eebce740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53e41adf80 con 0x55c621683ac0 2024-04-20T21:38:16.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.113+0000 7f53eebce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f53e41b0800 con 0x55c62168c6a0 2024-04-20T21:38:16.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.113+0000 7f53eebce740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53e41ae5f0 con 0x55c6216831a0 2024-04-20T21:38:16.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.113+0000 7f53e9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6216831a0 0x55c62168bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.113+0000 7f53e9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6216831a0 0x55c62168bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50696/0 (socket says 172.21.15.22:50696) 2024-04-20T21:38:16.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.113+0000 7f53e9400640 1 -- 172.21.15.22:0/1944483005 learned_addr learned my addr 172.21.15.22:0/1944483005 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:16.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.113+0000 7f53e8a00640 1 --2- 172.21.15.22:0/1944483005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c621683ac0 0x55c621698350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.114+0000 7f53e9e00640 1 --2- 172.21.15.22:0/1944483005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 0x55c6216885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.114+0000 7f53e9400640 1 -- 172.21.15.22:0/1944483005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 msgr2=0x55c6216885d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.114+0000 7f53e9400640 1 --2- 172.21.15.22:0/1944483005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 0x55c6216885d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.114+0000 7f53e9400640 1 -- 172.21.15.22:0/1944483005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c621683ac0 msgr2=0x55c621698350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.114+0000 7f53e9400640 1 --2- 172.21.15.22:0/1944483005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c621683ac0 0x55c621698350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.114+0000 7f53e9400640 1 -- 172.21.15.22:0/1944483005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c62167d400 con 0x55c6216831a0 2024-04-20T21:38:16.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.114+0000 7f53e9e00640 1 --2- 172.21.15.22:0/1944483005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 0x55c6216885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:16.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.114+0000 7f53e9400640 1 --2- 172.21.15.22:0/1944483005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6216831a0 0x55c62168bdc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f53d8006010 tx=0x7f53d80a9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=813f532ac3df19e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.115+0000 7f53d5c00640 1 -- 172.21.15.22:0/1944483005 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d80be310 con 0x55c6216831a0 2024-04-20T21:38:16.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.115+0000 7f53d5c00640 1 -- 172.21.15.22:0/1944483005 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d8004c80 con 0x55c6216831a0 2024-04-20T21:38:16.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.115+0000 7f53d5c00640 1 -- 172.21.15.22:0/1944483005 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d80be130 con 0x55c6216831a0 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.116+0000 7f53eebce740 1 -- 172.21.15.22:0/1944483005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6216831a0 msgr2=0x55c62168bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.116+0000 7f53eebce740 1 --2- 172.21.15.22:0/1944483005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6216831a0 0x55c62168bdc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f53d8006010 tx=0x7f53d80a9040 comp rx=0 tx=0).stop 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.116+0000 7f53eebce740 1 -- 172.21.15.22:0/1944483005 shutdown_connections 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.116+0000 7f53eebce740 1 --2- 172.21.15.22:0/1944483005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c62168c6a0 0x55c6216885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.116+0000 7f53eebce740 1 --2- 172.21.15.22:0/1944483005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c621683ac0 0x55c621698350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.116+0000 7f53eebce740 1 --2- 172.21.15.22:0/1944483005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6216831a0 0x55c62168bdc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.116+0000 7f53eebce740 1 -- 172.21.15.22:0/1944483005 >> 172.21.15.22:0/1944483005 conn(0x55c62167d8d0 msgr2=0x55c62167f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.116+0000 7f53eebce740 1 -- 172.21.15.22:0/1944483005 shutdown_connections 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.116+0000 7f53eebce740 1 -- 172.21.15.22:0/1944483005 wait complete. 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.116+0000 7f53eebce740 1 -- start start 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.117+0000 7f53eebce740 1 Processor -- start 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.117+0000 7f53eebce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6216831a0 0x55c621694d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.117+0000 7f53eebce740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c621683ac0 0x55c6216900b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.117+0000 7f53eebce740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c62168c6a0 0x55c6216906c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.117+0000 7f53eebce740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53e41aecc0 con 0x55c621683ac0 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.117+0000 7f53eebce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f53e41b0130 con 0x55c6216831a0 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.117+0000 7f53eebce740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53e41af390 con 0x55c62168c6a0 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.117+0000 7f53eebce740 10 client.0 _ll_register_callbacks cb 0x7f53dc0092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.117+0000 7f53e9400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6216831a0 0x55c621694d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.117+0000 7f53e9400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6216831a0 0x55c621694d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45818/0 (socket says 172.21.15.22:45818) 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.118+0000 7f53e9400640 1 -- 172.21.15.22:0/2542905971 learned_addr learned my addr 172.21.15.22:0/2542905971 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.118+0000 7f53e9e00640 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c62168c6a0 0x55c6216906c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.118+0000 7f53e9400640 1 -- 172.21.15.22:0/2542905971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c62168c6a0 msgr2=0x55c6216906c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.118+0000 7f53e9400640 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c62168c6a0 0x55c6216906c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.118+0000 7f53e8a00640 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c621683ac0 0x55c6216900b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.118+0000 7f53e9400640 1 -- 172.21.15.22:0/2542905971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c621683ac0 msgr2=0x55c6216900b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.118+0000 7f53e9400640 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c621683ac0 0x55c6216900b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.118+0000 7f53e9400640 1 -- 172.21.15.22:0/2542905971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c621690ef0 con 0x55c6216831a0 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.118+0000 7f53e8a00640 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c621683ac0 0x55c6216900b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.118+0000 7f53e9400640 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6216831a0 0x55c621694d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f53d802c210 tx=0x7f53d80035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaa4f6e23e08c626 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.118+0000 7f53caa00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.119+0000 7f53caa00640 1 -- 172.21.15.22:0/2542905971 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8005690 con 0x55c6216831a0 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.119+0000 7f53caa00640 1 -- 172.21.15.22:0/2542905971 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f53d8003cf0 con 0x55c6216831a0 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.119+0000 7f53eebce740 1 -- 172.21.15.22:0/2542905971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d8004c80 con 0x55c6216831a0 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.119+0000 7f53caa00640 1 -- 172.21.15.22:0/2542905971 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f53d802cb90 con 0x55c6216831a0 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.119+0000 7f53eebce740 10 client.25910 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.119+0000 7f53eebce740 1 -- 172.21.15.22:0/2542905971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c621703150 con 0x55c6216831a0 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.119+0000 7f53caa00640 1 -- 172.21.15.22:0/2542905971 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d8004c80 con 0x55c6216831a0 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.119+0000 7f53caa00640 1 -- 172.21.15.22:0/2542905971 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f53d80f00f0 con 0x55c6216831a0 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.119+0000 7f53eebce740 10 client.25910 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.119+0000 7f53eebce740 10 client.25910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.119+0000 7f53eebce740 20 client.25910 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:16.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.119+0000 7f53eebce740 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c6217037b0 0x55c621705bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.120+0000 7f53eebce740 4 client.25910 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:38:16.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.120+0000 7f53eebce740 1 -- 172.21.15.22:0/2542905971 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c6215f09e0 con 0x55c6217037b0 2024-04-20T21:38:16.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.120+0000 7f53e8a00640 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c6217037b0 0x55c621705bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53e8a00640 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c6217037b0 0x55c621705bd0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7153f2cf3785d953 server_cookie=a7ca064de1a275d9 in_seq=0 out_seq=0 2024-04-20T21:38:16.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53caa00640 10 client.25910 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:16.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53caa00640 1 -- 172.21.15.22:0/2542905971 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c6215f09e0 con 0x55c6217037b0 2024-04-20T21:38:16.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53caa00640 10 client.25910 handle_command_reply: tid=0 2024-04-20T21:38:16.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53eebce740 1 client.25910 shutdown 2024-04-20T21:38:16.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53eebce740 1 -- 172.21.15.22:0/2542905971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6216831a0 msgr2=0x55c621694d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53eebce740 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6216831a0 0x55c621694d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f53d802c210 tx=0x7f53d80035f0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53eebce740 1 -- 172.21.15.22:0/2542905971 shutdown_connections 2024-04-20T21:38:16.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53eebce740 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c62168c6a0 0x55c6216906c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53eebce740 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c621683ac0 0x55c6216900b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53eebce740 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c6217037b0 0x55c621705bd0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53eebce740 1 --2- 172.21.15.22:0/2542905971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6216831a0 0x55c621694d40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53eebce740 1 -- 172.21.15.22:0/2542905971 >> 172.21.15.22:0/2542905971 conn(0x55c62167d8d0 msgr2=0x55c621689650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53caa00640 0 client.25910 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:16.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53eebce740 1 -- 172.21.15.22:0/2542905971 shutdown_connections 2024-04-20T21:38:16.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53eebce740 1 -- 172.21.15.22:0/2542905971 wait complete. 2024-04-20T21:38:16.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.121+0000 7f53eebce740 20 client.25910 trim_cache size 0 max 16384 2024-04-20T21:38:16.124 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:38:16.124 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:38:16.124 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:38:16.125 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:38:16.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.122+0000 7f53ebe00640 1 -- 172.21.15.22:0/3013672805 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f53cc0a3d20 msgr2=0x7f53cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.122+0000 7f53ebe00640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f53cc0a3d20 0x7f53cc0a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009a50 tx=0x7f53dc009760 comp rx=0 tx=0).stop 2024-04-20T21:38:16.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.122+0000 7f53ebe00640 1 -- 172.21.15.22:0/3013672805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41043b0 msgr2=0x7f53e41a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.122+0000 7f53ebe00640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41043b0 0x7f53e41a0740 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f53d802c170 tx=0x7f53d8004360 comp rx=0 tx=0).stop 2024-04-20T21:38:16.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.122+0000 7f53ebe00640 1 -- 172.21.15.22:0/3013672805 shutdown_connections 2024-04-20T21:38:16.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.122+0000 7f53ebe00640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f53cc0a3d20 0x7f53cc0a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.122+0000 7f53ebe00640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53e419a940 0x7f53e419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.122+0000 7f53ebe00640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53e4109480 0x7f53e41a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.122+0000 7f53ebe00640 1 --2- 172.21.15.22:0/3013672805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53e41043b0 0x7f53e41a0740 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.122+0000 7f53ebe00640 1 -- 172.21.15.22:0/3013672805 >> 172.21.15.22:0/3013672805 conn(0x7f53e40ffbd0 msgr2=0x7f53e4101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.122+0000 7f53ebe00640 1 -- 172.21.15.22:0/3013672805 shutdown_connections 2024-04-20T21:38:16.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.122+0000 7f53ebe00640 1 -- 172.21.15.22:0/3013672805 wait complete. 2024-04-20T21:38:16.140 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:38:16.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:16.183 INFO:tasks.workunit.client.0.smithi022.stdout: 1 311800 9.75 MB/sec execute 44 sec latency 36.316 ms 2024-04-20T21:38:16.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.371+0000 7f7a81c00640 1 -- 172.21.15.22:0/3519984042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7c1043b0 msgr2=0x7f7a7c104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.371+0000 7f7a81c00640 1 --2- 172.21.15.22:0/3519984042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7c1043b0 0x7f7a7c104790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009900 tx=0x7f7a6c02bc40 comp rx=0 tx=0).stop 2024-04-20T21:38:16.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.372+0000 7f7a81c00640 1 -- 172.21.15.22:0/3519984042 shutdown_connections 2024-04-20T21:38:16.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.372+0000 7f7a81c00640 1 --2- 172.21.15.22:0/3519984042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a7c109480 0x7f7a7c111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.372+0000 7f7a81c00640 1 --2- 172.21.15.22:0/3519984042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a7c104d60 0x7f7a7c108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.372+0000 7f7a81c00640 1 --2- 172.21.15.22:0/3519984042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7c1043b0 0x7f7a7c104790 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.372+0000 7f7a81c00640 1 -- 172.21.15.22:0/3519984042 >> 172.21.15.22:0/3519984042 conn(0x7f7a7c0ffc50 msgr2=0x7f7a7c102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.372+0000 7f7a81c00640 1 -- 172.21.15.22:0/3519984042 shutdown_connections 2024-04-20T21:38:16.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.372+0000 7f7a81c00640 1 -- 172.21.15.22:0/3519984042 wait complete. 2024-04-20T21:38:16.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.373+0000 7f7a81c00640 1 Processor -- start 2024-04-20T21:38:16.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.373+0000 7f7a81c00640 1 -- start start 2024-04-20T21:38:16.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.373+0000 7f7a81c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a7c1043b0 0x7f7a7c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.373+0000 7f7a81c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7c109480 0x7f7a7c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.373+0000 7f7a81c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a7c19a890 0x7f7a7c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.373+0000 7f7a81c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c114590 con 0x7f7a7c1043b0 2024-04-20T21:38:16.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.373+0000 7f7a81c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c114410 con 0x7f7a7c19a890 2024-04-20T21:38:16.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.373+0000 7f7a81c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c114710 con 0x7f7a7c109480 2024-04-20T21:38:16.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.374+0000 7f7a7aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a7c1043b0 0x7f7a7c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.374+0000 7f7a7a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7c109480 0x7f7a7c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.374+0000 7f7a7a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7c109480 0x7f7a7c1a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50726/0 (socket says 172.21.15.22:50726) 2024-04-20T21:38:16.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.374+0000 7f7a7a000640 1 -- 172.21.15.22:0/3432346442 learned_addr learned my addr 172.21.15.22:0/3432346442 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:16.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.374+0000 7f7a7b400640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a7c19a890 0x7f7a7c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.375+0000 7f7a7a000640 1 -- 172.21.15.22:0/3432346442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a7c19a890 msgr2=0x7f7a7c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.375+0000 7f7a7a000640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a7c19a890 0x7f7a7c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.375+0000 7f7a7a000640 1 -- 172.21.15.22:0/3432346442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a7c1043b0 msgr2=0x7f7a7c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.375+0000 7f7a7a000640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a7c1043b0 0x7f7a7c1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.375+0000 7f7a7a000640 1 -- 172.21.15.22:0/3432346442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7c19b3b0 con 0x7f7a7c109480 2024-04-20T21:38:16.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.375+0000 7f7a7a000640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7c109480 0x7f7a7c1a0c20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7a68005fd0 tx=0x7f7a6800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.375+0000 7f7a73400640 1 -- 172.21.15.22:0/3432346442 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68017070 con 0x7f7a7c109480 2024-04-20T21:38:16.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.375+0000 7f7a81c00640 1 -- 172.21.15.22:0/3432346442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7c19b6a0 con 0x7f7a7c109480 2024-04-20T21:38:16.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.375+0000 7f7a73400640 1 -- 172.21.15.22:0/3432346442 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a68009c30 con 0x7f7a7c109480 2024-04-20T21:38:16.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.375+0000 7f7a73400640 1 -- 172.21.15.22:0/3432346442 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68010460 con 0x7f7a7c109480 2024-04-20T21:38:16.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.375+0000 7f7a81c00640 1 -- 172.21.15.22:0/3432346442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7c1a7940 con 0x7f7a7c109480 2024-04-20T21:38:16.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.377+0000 7f7a81c00640 1 -- 172.21.15.22:0/3432346442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a7c0691d0 con 0x7f7a7c109480 2024-04-20T21:38:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.380+0000 7f7a73400640 1 -- 172.21.15.22:0/3432346442 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7a68003e70 con 0x7f7a7c109480 2024-04-20T21:38:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.381+0000 7f7a73400640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7a600a3d20 0x7f7a600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.381+0000 7f7a73400640 1 -- 172.21.15.22:0/3432346442 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7a680d0490 con 0x7f7a7c109480 2024-04-20T21:38:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.381+0000 7f7a7aa00640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7a600a3d20 0x7f7a600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.382+0000 7f7a7aa00640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7a600a3d20 0x7f7a600a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c02efd0 tx=0x7f7a6c02cf90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.385+0000 7f7a73400640 1 -- 172.21.15.22:0/3432346442 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7a68020030 con 0x7f7a7c109480 2024-04-20T21:38:16.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:16 smithi022 ceph-mon[16843]: pgmap v136: 97 pgs: 97 active+clean; 172 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 263 op/s 2024-04-20T21:38:16.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.667+0000 7f7a81c00640 1 -- 172.21.15.22:0/3432346442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a7c19bdc0 con 0x7f7a7c109480 2024-04-20T21:38:16.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.669+0000 7f7a73400640 1 -- 172.21.15.22:0/3432346442 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f7a6809cab0 con 0x7f7a7c109480 2024-04-20T21:38:16.668 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:38:16.668 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:38:16.669 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:38:16.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.672+0000 7f7a81c00640 1 -- 172.21.15.22:0/3432346442 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7a600a3d20 msgr2=0x7f7a600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.672+0000 7f7a81c00640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7a600a3d20 0x7f7a600a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c02efd0 tx=0x7f7a6c02cf90 comp rx=0 tx=0).stop 2024-04-20T21:38:16.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.672+0000 7f7a81c00640 1 -- 172.21.15.22:0/3432346442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7c109480 msgr2=0x7f7a7c1a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.672+0000 7f7a81c00640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7c109480 0x7f7a7c1a0c20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7a68005fd0 tx=0x7f7a6800dde0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.672+0000 7f7a81c00640 1 -- 172.21.15.22:0/3432346442 shutdown_connections 2024-04-20T21:38:16.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.672+0000 7f7a81c00640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7a600a3d20 0x7f7a600a61e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.672+0000 7f7a81c00640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a7c19a890 0x7f7a7c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.672+0000 7f7a81c00640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7c109480 0x7f7a7c1a0c20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.672+0000 7f7a81c00640 1 --2- 172.21.15.22:0/3432346442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a7c1043b0 0x7f7a7c1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.672+0000 7f7a81c00640 1 -- 172.21.15.22:0/3432346442 >> 172.21.15.22:0/3432346442 conn(0x7f7a7c0ffc50 msgr2=0x7f7a7c101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.672+0000 7f7a81c00640 1 -- 172.21.15.22:0/3432346442 shutdown_connections 2024-04-20T21:38:16.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.672+0000 7f7a81c00640 1 -- 172.21.15.22:0/3432346442 wait complete. 2024-04-20T21:38:16.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:38:16.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:16 smithi122 ceph-mon[22147]: pgmap v136: 97 pgs: 97 active+clean; 172 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 263 op/s 2024-04-20T21:38:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:16 smithi033 ceph-mon[21626]: pgmap v136: 97 pgs: 97 active+clean; 172 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 263 op/s 2024-04-20T21:38:16.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.961+0000 7f9692e00640 1 -- 172.21.15.22:0/783116983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968c076490 msgr2=0x7f968c076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.961+0000 7f9692e00640 1 --2- 172.21.15.22:0/783116983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968c076490 0x7f968c076870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9680009890 tx=0x7f968002bb70 comp rx=0 tx=0).stop 2024-04-20T21:38:16.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.962+0000 7f9692e00640 1 -- 172.21.15.22:0/783116983 shutdown_connections 2024-04-20T21:38:16.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.962+0000 7f9692e00640 1 --2- 172.21.15.22:0/783116983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c112ca0 0x7f968c115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.962+0000 7f9692e00640 1 --2- 172.21.15.22:0/783116983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c076db0 0x7f968c112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.962+0000 7f9692e00640 1 --2- 172.21.15.22:0/783116983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968c076490 0x7f968c076870 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.962+0000 7f9692e00640 1 -- 172.21.15.22:0/783116983 >> 172.21.15.22:0/783116983 conn(0x7f968c0ffad0 msgr2=0x7f968c101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.962+0000 7f9692e00640 1 -- 172.21.15.22:0/783116983 shutdown_connections 2024-04-20T21:38:16.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.962+0000 7f9692e00640 1 -- 172.21.15.22:0/783116983 wait complete. 2024-04-20T21:38:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.963+0000 7f9692e00640 1 Processor -- start 2024-04-20T21:38:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.963+0000 7f9692e00640 1 -- start start 2024-04-20T21:38:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.963+0000 7f9692e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968c076490 0x7f968c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.963+0000 7f9692e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c076db0 0x7f968c19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.963+0000 7f9692e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c112ca0 0x7f968c1a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.963+0000 7f9692e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1187f0 con 0x7f968c076db0 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.963+0000 7f9692e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f968c118670 con 0x7f968c076490 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.963+0000 7f9692e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f968c118970 con 0x7f968c112ca0 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.963+0000 7f9691a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968c076490 0x7f968c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.964+0000 7f9691a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968c076490 0x7f968c19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45846/0 (socket says 172.21.15.22:45846) 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.964+0000 7f9691a00640 1 -- 172.21.15.22:0/3126026372 learned_addr learned my addr 172.21.15.22:0/3126026372 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.964+0000 7f9691000640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c076db0 0x7f968c19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.964+0000 7f9692400640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c112ca0 0x7f968c1a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.964+0000 7f9691a00640 1 -- 172.21.15.22:0/3126026372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c112ca0 msgr2=0x7f968c1a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.964+0000 7f9691a00640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c112ca0 0x7f968c1a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.964+0000 7f9691a00640 1 -- 172.21.15.22:0/3126026372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c076db0 msgr2=0x7f968c19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.964+0000 7f9691a00640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c076db0 0x7f968c19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.964+0000 7f9691a00640 1 -- 172.21.15.22:0/3126026372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c1a3d70 con 0x7f968c076490 2024-04-20T21:38:16.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.964+0000 7f9691000640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c076db0 0x7f968c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.965+0000 7f9691a00640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968c076490 0x7f968c19edd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f96800042f0 tx=0x7f9680004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.965+0000 7f968a000640 1 -- 172.21.15.22:0/3126026372 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680041020 con 0x7f968c076490 2024-04-20T21:38:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.965+0000 7f968a000640 1 -- 172.21.15.22:0/3126026372 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968002c9f0 con 0x7f968c076490 2024-04-20T21:38:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.965+0000 7f968a000640 1 -- 172.21.15.22:0/3126026372 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680039040 con 0x7f968c076490 2024-04-20T21:38:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.965+0000 7f9692e00640 1 -- 172.21.15.22:0/3126026372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968c1a4000 con 0x7f968c076490 2024-04-20T21:38:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.965+0000 7f9692e00640 1 -- 172.21.15.22:0/3126026372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f968c1ab8f0 con 0x7f968c076490 2024-04-20T21:38:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.969+0000 7f96971de740 1 Processor -- start 2024-04-20T21:38:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.969+0000 7f968a000640 1 -- 172.21.15.22:0/3126026372 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9680044020 con 0x7f968c076490 2024-04-20T21:38:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.970+0000 7f96971de740 1 -- start start 2024-04-20T21:38:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.970+0000 7f96971de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 0x556cb2a73550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.970+0000 7f96971de740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73a90 0x556cb2a7c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.970+0000 7f96971de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a7ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.970+0000 7f96971de740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556cb29d8050 con 0x556cb2a73a90 2024-04-20T21:38:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.970+0000 7f96971de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556cb297e490 con 0x556cb2a73170 2024-04-20T21:38:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.970+0000 7f96971de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556cb29d4010 con 0x556cb2a7c670 2024-04-20T21:38:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.970+0000 7f9692400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 0x556cb2a73550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.970+0000 7f9691000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a7ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.970+0000 7f9691a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73a90 0x556cb2a7c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.970+0000 7f9692400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 0x556cb2a73550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45860/0 (socket says 172.21.15.22:45860) 2024-04-20T21:38:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.970+0000 7f9692400640 1 -- 172.21.15.22:0/704976609 learned_addr learned my addr 172.21.15.22:0/704976609 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.971+0000 7f9692400640 1 -- 172.21.15.22:0/704976609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 msgr2=0x556cb2a7ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.971+0000 7f9692400640 1 --2- 172.21.15.22:0/704976609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a7ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.971+0000 7f9692400640 1 -- 172.21.15.22:0/704976609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73a90 msgr2=0x556cb2a7c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.971+0000 7f9692400640 1 --2- 172.21.15.22:0/704976609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73a90 0x556cb2a7c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.971+0000 7f9692400640 1 -- 172.21.15.22:0/704976609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cb2986070 con 0x556cb2a73170 2024-04-20T21:38:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.971+0000 7f9691000640 1 --2- 172.21.15.22:0/704976609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a7ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.971+0000 7f9692400640 1 --2- 172.21.15.22:0/704976609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 0x556cb2a73550 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9684009f30 tx=0x7f96840118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc7234b9aa01de61 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.971+0000 7f968a000640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f96740a3da0 0x7f96740a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.972+0000 7f967ea00640 1 -- 172.21.15.22:0/704976609 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684004d20 con 0x556cb2a73170 2024-04-20T21:38:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.972+0000 7f967ea00640 1 -- 172.21.15.22:0/704976609 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9684004050 con 0x556cb2a73170 2024-04-20T21:38:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.972+0000 7f967ea00640 1 -- 172.21.15.22:0/704976609 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684005260 con 0x556cb2a73170 2024-04-20T21:38:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.972+0000 7f968a000640 1 -- 172.21.15.22:0/3126026372 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f96800f1ff0 con 0x7f968c076490 2024-04-20T21:38:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.972+0000 7f9691a00640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f96740a3da0 0x7f96740a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.973+0000 7f96971de740 1 -- 172.21.15.22:0/704976609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 msgr2=0x556cb2a73550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.973+0000 7f96971de740 1 --2- 172.21.15.22:0/704976609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 0x556cb2a73550 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9684009f30 tx=0x7f96840118d0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.973+0000 7f96971de740 1 -- 172.21.15.22:0/704976609 shutdown_connections 2024-04-20T21:38:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.973+0000 7f96971de740 1 --2- 172.21.15.22:0/704976609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a7ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.973+0000 7f96971de740 1 --2- 172.21.15.22:0/704976609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73a90 0x556cb2a7c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.973+0000 7f96971de740 1 --2- 172.21.15.22:0/704976609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 0x556cb2a73550 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.973+0000 7f96971de740 1 -- 172.21.15.22:0/704976609 >> 172.21.15.22:0/704976609 conn(0x556cb2a6d8a0 msgr2=0x556cb2a6dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.973+0000 7f9691a00640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f96740a3da0 0x7f96740a6260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f96800ef8e0 tx=0x7f96800f5b40 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.973+0000 7f96971de740 1 -- 172.21.15.22:0/704976609 shutdown_connections 2024-04-20T21:38:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.973+0000 7f96971de740 1 -- 172.21.15.22:0/704976609 wait complete. 2024-04-20T21:38:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.974+0000 7f96971de740 1 -- start start 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.974+0000 7f96971de740 1 Processor -- start 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.975+0000 7f96971de740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 0x556cb2a796e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.975+0000 7f96971de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73a90 0x556cb2a79c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.975+0000 7f96971de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a7a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.975+0000 7f96971de740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556cb2a7f460 con 0x556cb2a73170 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.975+0000 7f96971de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556cb2991860 con 0x556cb2a73a90 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.975+0000 7f96971de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556cb2a7f5e0 con 0x556cb2a7c670 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.975+0000 7f96971de740 10 client.0 _ll_register_callbacks cb 0x556cb295ffe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.975+0000 7f9692400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 0x556cb2a796e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.975+0000 7f9692400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 0x556cb2a796e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38304/0 (socket says 172.21.15.22:38304) 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.975+0000 7f9692400640 1 -- 172.21.15.22:0/1758418197 learned_addr learned my addr 172.21.15.22:0/1758418197 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.975+0000 7f9691000640 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73a90 0x556cb2a79c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.975+0000 7f9692400640 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a7a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.976+0000 7f9691000640 1 -- 172.21.15.22:0/1758418197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 msgr2=0x556cb2a7a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.976+0000 7f9691000640 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a7a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.976+0000 7f9691000640 1 -- 172.21.15.22:0/1758418197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 msgr2=0x556cb2a796e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.976+0000 7f9691000640 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 0x556cb2a796e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.976+0000 7f9691000640 1 -- 172.21.15.22:0/1758418197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cb2995950 con 0x556cb2a73a90 2024-04-20T21:38:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.976+0000 7f9692400640 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 0x556cb2a796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.976+0000 7f9691000640 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73a90 0x556cb2a79c20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9678009530 tx=0x7f967800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97f2ae589b60167c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.976+0000 7f9673400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.976+0000 7f9673400640 1 -- 172.21.15.22:0/1758418197 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967801a3f0 con 0x556cb2a73a90 2024-04-20T21:38:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.977+0000 7f9673400640 1 -- 172.21.15.22:0/1758418197 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f967801a590 con 0x556cb2a73a90 2024-04-20T21:38:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.977+0000 7f96971de740 1 -- 172.21.15.22:0/1758418197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9684004050 con 0x556cb2a73a90 2024-04-20T21:38:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.977+0000 7f9673400640 1 -- 172.21.15.22:0/1758418197 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f967801a7d0 con 0x556cb2a73a90 2024-04-20T21:38:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.977+0000 7f96971de740 10 client.25928 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.977+0000 7f96971de740 1 -- 172.21.15.22:0/1758418197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556cb2963820 con 0x556cb2a73a90 2024-04-20T21:38:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.978+0000 7f9673400640 1 -- 172.21.15.22:0/1758418197 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9684004050 con 0x556cb2a73a90 2024-04-20T21:38:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.978+0000 7f9673400640 1 -- 172.21.15.22:0/1758418197 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f9678022030 con 0x556cb2a73a90 2024-04-20T21:38:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.978+0000 7f96971de740 10 client.25928 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.978+0000 7f96971de740 10 client.25928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.978+0000 7f96971de740 20 client.25928 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.978+0000 7f96971de740 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556cb2a8c740 0x556cb2a8eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.978+0000 7f96971de740 4 client.25928 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.978+0000 7f96971de740 1 -- 172.21.15.22:0/1758418197 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cb29e5a70 con 0x556cb2a8c740 2024-04-20T21:38:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.979+0000 7f9692400640 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556cb2a8c740 0x556cb2a8eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.979+0000 7f9692400640 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556cb2a8c740 0x556cb2a8eb80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa9f6c9c86ad4df3 server_cookie=763252e06af697e1 in_seq=0 out_seq=0 2024-04-20T21:38:16.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.979+0000 7f9673400640 10 client.25928 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:16.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.980+0000 7f9673400640 1 -- 172.21.15.22:0/1758418197 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cb29e5a70 con 0x556cb2a8c740 2024-04-20T21:38:16.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.980+0000 7f9673400640 10 client.25928 handle_command_reply: tid=0 2024-04-20T21:38:16.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.980+0000 7f96971de740 1 client.25928 shutdown 2024-04-20T21:38:16.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.980+0000 7f96971de740 1 -- 172.21.15.22:0/1758418197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73a90 msgr2=0x556cb2a79c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:16.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.980+0000 7f96971de740 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73a90 0x556cb2a79c20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9678009530 tx=0x7f967800f790 comp rx=0 tx=0).stop 2024-04-20T21:38:16.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.981+0000 7f96971de740 1 -- 172.21.15.22:0/1758418197 shutdown_connections 2024-04-20T21:38:16.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.981+0000 7f96971de740 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a7a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.981+0000 7f96971de740 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556cb2a8c740 0x556cb2a8eb80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.981+0000 7f96971de740 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73a90 0x556cb2a79c20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.981+0000 7f96971de740 1 --2- 172.21.15.22:0/1758418197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 0x556cb2a796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.981+0000 7f96971de740 1 -- 172.21.15.22:0/1758418197 >> 172.21.15.22:0/1758418197 conn(0x556cb2a6d8a0 msgr2=0x556cb2a6f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:16.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.981+0000 7f9673400640 0 client.25928 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:16.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.981+0000 7f96971de740 1 -- 172.21.15.22:0/1758418197 shutdown_connections 2024-04-20T21:38:16.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.981+0000 7f96971de740 1 -- 172.21.15.22:0/1758418197 wait complete. 2024-04-20T21:38:16.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.981+0000 7f96971de740 20 client.25928 trim_cache size 0 max 16384 2024-04-20T21:38:16.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.998+0000 7f96971de740 1 Processor -- start 2024-04-20T21:38:16.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.998+0000 7f96971de740 1 -- start start 2024-04-20T21:38:16.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.998+0000 7f96971de740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 0x556cb2a7be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.998+0000 7f96971de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a73a90 0x556cb2a88350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f96971de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a7c670 0x556cb2a79db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:16.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f96971de740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1b2200 con 0x556cb2a73170 2024-04-20T21:38:16.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f96971de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1b4a80 con 0x556cb2a7c670 2024-04-20T21:38:16.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f96971de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1b2870 con 0x556cb2a73a90 2024-04-20T21:38:16.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f9692400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 0x556cb2a7be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f9692400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 0x556cb2a7be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38318/0 (socket says 172.21.15.22:38318) 2024-04-20T21:38:16.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f9691000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a73a90 0x556cb2a88350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f9692400640 1 -- 172.21.15.22:0/66525710 learned_addr learned my addr 172.21.15.22:0/66525710 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:16.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f9692400640 1 --2- 172.21.15.22:0/66525710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a7c670 0x556cb2a79db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:16.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f9691000640 1 -- 172.21.15.22:0/66525710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a7c670 msgr2=0x556cb2a79db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f9691000640 1 --2- 172.21.15.22:0/66525710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a7c670 0x556cb2a79db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:16.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f9691000640 1 -- 172.21.15.22:0/66525710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 msgr2=0x556cb2a7be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:16.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f9691000640 1 --2- 172.21.15.22:0/66525710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 0x556cb2a7be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:16.999+0000 7f9691000640 1 -- 172.21.15.22:0/66525710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cb26ea3f0 con 0x556cb2a73a90 2024-04-20T21:38:17.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.000+0000 7f9691000640 1 --2- 172.21.15.22:0/66525710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a73a90 0x556cb2a88350 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9678000c00 tx=0x7f9678019f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=257fc91a3ad91947 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:17.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.000+0000 7f967e000640 1 -- 172.21.15.22:0/66525710 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967801a3f0 con 0x556cb2a73a90 2024-04-20T21:38:17.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.000+0000 7f967e000640 1 -- 172.21.15.22:0/66525710 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f967801a590 con 0x556cb2a73a90 2024-04-20T21:38:17.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.000+0000 7f967e000640 1 -- 172.21.15.22:0/66525710 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678005040 con 0x556cb2a73a90 2024-04-20T21:38:17.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.001+0000 7f96971de740 1 -- 172.21.15.22:0/66525710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a73a90 msgr2=0x556cb2a88350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:17.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.001+0000 7f96971de740 1 --2- 172.21.15.22:0/66525710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a73a90 0x556cb2a88350 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9678000c00 tx=0x7f9678019f70 comp rx=0 tx=0).stop 2024-04-20T21:38:17.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.001+0000 7f96971de740 1 -- 172.21.15.22:0/66525710 shutdown_connections 2024-04-20T21:38:17.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.001+0000 7f96971de740 1 --2- 172.21.15.22:0/66525710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a7c670 0x556cb2a79db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.001+0000 7f96971de740 1 --2- 172.21.15.22:0/66525710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a73a90 0x556cb2a88350 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.001+0000 7f96971de740 1 --2- 172.21.15.22:0/66525710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73170 0x556cb2a7be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.001+0000 7f96971de740 1 -- 172.21.15.22:0/66525710 >> 172.21.15.22:0/66525710 conn(0x556cb2a6d8a0 msgr2=0x556cb2a796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:17.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.001+0000 7f96971de740 1 -- 172.21.15.22:0/66525710 shutdown_connections 2024-04-20T21:38:17.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.001+0000 7f96971de740 1 -- 172.21.15.22:0/66525710 wait complete. 2024-04-20T21:38:17.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.002+0000 7f96971de740 1 -- start start 2024-04-20T21:38:17.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.002+0000 7f96971de740 1 Processor -- start 2024-04-20T21:38:17.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.002+0000 7f96971de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 0x556cb2a84bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:17.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f96971de740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73a90 0x556cb2a7ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:17.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f96971de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a804a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:17.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f96971de740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1b2f40 con 0x556cb2a73a90 2024-04-20T21:38:17.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f96971de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1b43b0 con 0x556cb2a73170 2024-04-20T21:38:17.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f96971de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1b3610 con 0x556cb2a7c670 2024-04-20T21:38:17.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f96971de740 10 client.0 _ll_register_callbacks cb 0x7f9678011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:17.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f9692400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 0x556cb2a84bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:17.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f9691000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73a90 0x556cb2a7ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f9692400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 0x556cb2a84bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45878/0 (socket says 172.21.15.22:45878) 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f9692400640 1 -- 172.21.15.22:0/2110439826 learned_addr learned my addr 172.21.15.22:0/2110439826 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f9692400640 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a804a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f9692400640 1 -- 172.21.15.22:0/2110439826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 msgr2=0x556cb2a804a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f9692400640 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a804a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f9692400640 1 -- 172.21.15.22:0/2110439826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73a90 msgr2=0x556cb2a7ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f9692400640 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73a90 0x556cb2a7ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f9692400640 1 -- 172.21.15.22:0/2110439826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cb27b47d0 con 0x556cb2a73170 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.003+0000 7f9691000640 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73a90 0x556cb2a7ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.004+0000 7f9692400640 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 0x556cb2a84bc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9684009730 tx=0x7f96840118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f90a7e595bb423f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.004+0000 7f9673400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.004+0000 7f9673400640 1 -- 172.21.15.22:0/2110439826 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684003df0 con 0x556cb2a73170 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.004+0000 7f9673400640 1 -- 172.21.15.22:0/2110439826 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f9684004960 con 0x556cb2a73170 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.004+0000 7f96971de740 1 -- 172.21.15.22:0/2110439826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f967801a590 con 0x556cb2a73170 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.004+0000 7f9673400640 1 -- 172.21.15.22:0/2110439826 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f968403c050 con 0x556cb2a73170 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.004+0000 7f96971de740 10 client.25934 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:17.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.004+0000 7f96971de740 1 -- 172.21.15.22:0/2110439826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556cb2af2a90 con 0x556cb2a73170 2024-04-20T21:38:17.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.004+0000 7f9673400640 1 -- 172.21.15.22:0/2110439826 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f967801a590 con 0x556cb2a73170 2024-04-20T21:38:17.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.005+0000 7f9673400640 1 -- 172.21.15.22:0/2110439826 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f9684025040 con 0x556cb2a73170 2024-04-20T21:38:17.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.005+0000 7f96971de740 10 client.25934 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:17.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.005+0000 7f96971de740 10 client.25934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:17.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.005+0000 7f96971de740 20 client.25934 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:17.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.005+0000 7f96971de740 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556cb2af3070 0x556cb2af5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.005+0000 7f96971de740 4 client.25934 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.005+0000 7f96971de740 1 -- 172.21.15.22:0/2110439826 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cb29df8e0 con 0x556cb2af3070 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.005+0000 7f9691000640 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556cb2af3070 0x556cb2af5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.005+0000 7f9691000640 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556cb2af3070 0x556cb2af5490 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fee1126ed9a4974 server_cookie=5a6e57461a8fb13d in_seq=0 out_seq=0 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.005+0000 7f9673400640 10 client.25934 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f9673400640 1 -- 172.21.15.22:0/2110439826 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cb29df8e0 con 0x556cb2af3070 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f9673400640 10 client.25934 handle_command_reply: tid=0 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f96971de740 1 client.25934 shutdown 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f96971de740 1 -- 172.21.15.22:0/2110439826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 msgr2=0x556cb2a84bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f96971de740 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 0x556cb2a84bc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9684009730 tx=0x7f96840118d0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f96971de740 1 -- 172.21.15.22:0/2110439826 shutdown_connections 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f96971de740 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556cb2a7c670 0x556cb2a804a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f96971de740 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556cb2a73a90 0x556cb2a7ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f96971de740 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556cb2af3070 0x556cb2af5490 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f96971de740 1 --2- 172.21.15.22:0/2110439826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb2a73170 0x556cb2a84bc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f96971de740 1 -- 172.21.15.22:0/2110439826 >> 172.21.15.22:0/2110439826 conn(0x556cb2a6d8a0 msgr2=0x556cb2a6c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f9673400640 0 client.25934 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:17.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f96971de740 1 -- 172.21.15.22:0/2110439826 shutdown_connections 2024-04-20T21:38:17.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.006+0000 7f96971de740 1 -- 172.21.15.22:0/2110439826 wait complete. 2024-04-20T21:38:17.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.007+0000 7f96971de740 20 client.25934 trim_cache size 0 max 16384 2024-04-20T21:38:17.008 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:38:17.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.007+0000 7f9692e00640 1 -- 172.21.15.22:0/3126026372 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f96740a3da0 msgr2=0x7f96740a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:17.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.007+0000 7f9692e00640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f96740a3da0 0x7f96740a6260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f96800ef8e0 tx=0x7f96800f5b40 comp rx=0 tx=0).stop 2024-04-20T21:38:17.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.007+0000 7f9692e00640 1 -- 172.21.15.22:0/3126026372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968c076490 msgr2=0x7f968c19edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:17.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.007+0000 7f9692e00640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968c076490 0x7f968c19edd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f96800042f0 tx=0x7f9680004320 comp rx=0 tx=0).stop 2024-04-20T21:38:17.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.007+0000 7f9692e00640 1 -- 172.21.15.22:0/3126026372 shutdown_connections 2024-04-20T21:38:17.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.007+0000 7f9692e00640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f96740a3da0 0x7f96740a6260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.007+0000 7f9692e00640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c112ca0 0x7f968c1a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.007+0000 7f9692e00640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c076db0 0x7f968c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.007+0000 7f9692e00640 1 --2- 172.21.15.22:0/3126026372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968c076490 0x7f968c19edd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:17.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.007+0000 7f9692e00640 1 -- 172.21.15.22:0/3126026372 >> 172.21.15.22:0/3126026372 conn(0x7f968c0ffad0 msgr2=0x7f968c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:17.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.008+0000 7f9692e00640 1 -- 172.21.15.22:0/3126026372 shutdown_connections 2024-04-20T21:38:17.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:17.008+0000 7f9692e00640 1 -- 172.21.15.22:0/3126026372 wait complete. 2024-04-20T21:38:17.183 INFO:tasks.workunit.client.0.smithi022.stdout: 1 313224 9.70 MB/sec execute 45 sec latency 31.319 ms 2024-04-20T21:38:17.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:17 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3432346442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:38:17.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:17 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3432346442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:38:17.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:17 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3432346442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:38:18.030 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:38:18.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:38:18.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:38:18.184 INFO:tasks.workunit.client.0.smithi022.stdout: 1 315428 9.76 MB/sec execute 46 sec latency 13.157 ms 2024-04-20T21:38:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.271+0000 7f4751000640 1 -- 172.21.15.22:0/4208818306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1016d0 msgr2=0x7f474c105b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.271+0000 7f4751000640 1 --2- 172.21.15.22:0/4208818306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1016d0 0x7f474c105b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4734009900 tx=0x7f473402bcc0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.272+0000 7f4751000640 1 -- 172.21.15.22:0/4208818306 shutdown_connections 2024-04-20T21:38:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.272+0000 7f4751000640 1 --2- 172.21.15.22:0/4208818306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f474c1061b0 0x7f474c106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.272+0000 7f4751000640 1 --2- 172.21.15.22:0/4208818306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1016d0 0x7f474c105b90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.272+0000 7f4751000640 1 --2- 172.21.15.22:0/4208818306 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474c100db0 0x7f474c101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.272+0000 7f4751000640 1 -- 172.21.15.22:0/4208818306 >> 172.21.15.22:0/4208818306 conn(0x7f474c0fc7e0 msgr2=0x7f474c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.272+0000 7f4751000640 1 -- 172.21.15.22:0/4208818306 shutdown_connections 2024-04-20T21:38:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.272+0000 7f4751000640 1 -- 172.21.15.22:0/4208818306 wait complete. 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.273+0000 7f4751000640 1 Processor -- start 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.273+0000 7f4751000640 1 -- start start 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.273+0000 7f4751000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f474c100db0 0x7f474c1a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.273+0000 7f4751000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1016d0 0x7f474c1a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.273+0000 7f4751000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474c1061b0 0x7f474c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.273+0000 7f4751000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f474c118760 con 0x7f474c1061b0 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.273+0000 7f4751000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1185e0 con 0x7f474c1016d0 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.273+0000 7f4751000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1188e0 con 0x7f474c100db0 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.274+0000 7f4742a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1016d0 0x7f474c1a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.274+0000 7f4742a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1016d0 0x7f474c1a4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45892/0 (socket says 172.21.15.22:45892) 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.274+0000 7f4742a00640 1 -- 172.21.15.22:0/259075382 learned_addr learned my addr 172.21.15.22:0/259075382 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.274+0000 7f474b400640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f474c100db0 0x7f474c1a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.274+0000 7f474be00640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474c1061b0 0x7f474c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.274+0000 7f4742a00640 1 -- 172.21.15.22:0/259075382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f474c100db0 msgr2=0x7f474c1a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.274+0000 7f4742a00640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f474c100db0 0x7f474c1a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.274+0000 7f4742a00640 1 -- 172.21.15.22:0/259075382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474c1061b0 msgr2=0x7f474c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.274+0000 7f4742a00640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474c1061b0 0x7f474c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.274+0000 7f4742a00640 1 -- 172.21.15.22:0/259075382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c19f1e0 con 0x7f474c1016d0 2024-04-20T21:38:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.274+0000 7f4742a00640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1016d0 0x7f474c1a4df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4734009230 tx=0x7f4734004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.275+0000 7f4748c00640 1 -- 172.21.15.22:0/259075382 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734043020 con 0x7f474c1016d0 2024-04-20T21:38:18.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.275+0000 7f4748c00640 1 -- 172.21.15.22:0/259075382 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47340054e0 con 0x7f474c1016d0 2024-04-20T21:38:18.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.275+0000 7f4751000640 1 -- 172.21.15.22:0/259075382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474c19f470 con 0x7f474c1016d0 2024-04-20T21:38:18.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.275+0000 7f4748c00640 1 -- 172.21.15.22:0/259075382 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734003b20 con 0x7f474c1016d0 2024-04-20T21:38:18.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.275+0000 7f4751000640 1 -- 172.21.15.22:0/259075382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474c1ab770 con 0x7f474c1016d0 2024-04-20T21:38:18.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.278+0000 7f475521e740 1 Processor -- start 2024-04-20T21:38:18.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.278+0000 7f4748c00640 1 -- 172.21.15.22:0/259075382 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4734046020 con 0x7f474c1016d0 2024-04-20T21:38:18.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.279+0000 7f475521e740 1 -- start start 2024-04-20T21:38:18.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.279+0000 7f475521e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.279+0000 7f475521e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.279+0000 7f475521e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc439e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.279+0000 7f475521e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a99bb9cd20 con 0x55a99bc38a10 2024-04-20T21:38:18.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.279+0000 7f475521e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a99bb433e0 con 0x55a99bc415f0 2024-04-20T21:38:18.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.279+0000 7f475521e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a99bb98ce0 con 0x55a99bc380f0 2024-04-20T21:38:18.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.279+0000 7f474be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f474b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f474be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc384d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50814/0 (socket says 172.21.15.22:50814) 2024-04-20T21:38:18.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f474be00640 1 -- 172.21.15.22:0/1689437272 learned_addr learned my addr 172.21.15.22:0/1689437272 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:18.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f4742a00640 1 --2- 172.21.15.22:0/1689437272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f474be00640 1 -- 172.21.15.22:0/1689437272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 msgr2=0x55a99bc439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f474be00640 1 --2- 172.21.15.22:0/1689437272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f474be00640 1 -- 172.21.15.22:0/1689437272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 msgr2=0x55a99bc410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f474be00640 1 --2- 172.21.15.22:0/1689437272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f474be00640 1 -- 172.21.15.22:0/1689437272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a99bb62980 con 0x55a99bc380f0 2024-04-20T21:38:18.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f4742a00640 1 --2- 172.21.15.22:0/1689437272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:18.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f4748c00640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f47300a3da0 0x7f47300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f474be00640 1 --2- 172.21.15.22:0/1689437272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc384d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4744009b90 tx=0x7f47440117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40c6dc9efc622a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.280+0000 7f4748c00640 1 -- 172.21.15.22:0/259075382 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f47340f2560 con 0x7f474c1016d0 2024-04-20T21:38:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.281+0000 7f4740c00640 1 -- 172.21.15.22:0/1689437272 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744012450 con 0x55a99bc380f0 2024-04-20T21:38:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.281+0000 7f474b400640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f47300a3da0 0x7f47300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.281+0000 7f4740c00640 1 -- 172.21.15.22:0/1689437272 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4744012ad0 con 0x55a99bc380f0 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.281+0000 7f4740c00640 1 -- 172.21.15.22:0/1689437272 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474400fac0 con 0x55a99bc380f0 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.281+0000 7f475521e740 1 -- 172.21.15.22:0/1689437272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 msgr2=0x55a99bc384d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.281+0000 7f474b400640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f47300a3da0 0x7f47300a6260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55a99bc37bc0 tx=0x7f473c009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.281+0000 7f475521e740 1 --2- 172.21.15.22:0/1689437272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc384d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4744009b90 tx=0x7f47440117b0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.282+0000 7f475521e740 1 -- 172.21.15.22:0/1689437272 shutdown_connections 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.282+0000 7f475521e740 1 --2- 172.21.15.22:0/1689437272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.282+0000 7f475521e740 1 --2- 172.21.15.22:0/1689437272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.282+0000 7f475521e740 1 --2- 172.21.15.22:0/1689437272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc384d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.282+0000 7f475521e740 1 -- 172.21.15.22:0/1689437272 >> 172.21.15.22:0/1689437272 conn(0x55a99bc32820 msgr2=0x55a99bc32c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.282+0000 7f475521e740 1 -- 172.21.15.22:0/1689437272 shutdown_connections 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.282+0000 7f475521e740 1 -- 172.21.15.22:0/1689437272 wait complete. 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.282+0000 7f475521e740 1 -- start start 2024-04-20T21:38:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.282+0000 7f475521e740 1 Processor -- start 2024-04-20T21:38:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.283+0000 7f475521e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc3e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.283+0000 7f475521e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc3ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.283+0000 7f475521e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc3f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.283+0000 7f475521e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a99bc443e0 con 0x55a99bc38a10 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.283+0000 7f475521e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a99bb567a0 con 0x55a99bc415f0 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.283+0000 7f475521e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a99bc44560 con 0x55a99bc380f0 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.283+0000 7f475521e740 10 client.0 _ll_register_callbacks cb 0x55a99bb58b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.283+0000 7f4742a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc3ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.283+0000 7f474be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc3f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.283+0000 7f4742a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc3ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38360/0 (socket says 172.21.15.22:38360) 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.283+0000 7f4742a00640 1 -- 172.21.15.22:0/1988998168 learned_addr learned my addr 172.21.15.22:0/1988998168 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.283+0000 7f474b400640 1 --2- 172.21.15.22:0/1988998168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc3e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.284+0000 7f474be00640 1 -- 172.21.15.22:0/1988998168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 msgr2=0x55a99bc3e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.284+0000 7f474be00640 1 --2- 172.21.15.22:0/1988998168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc3e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.284+0000 7f474be00640 1 -- 172.21.15.22:0/1988998168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 msgr2=0x55a99bc3ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.284+0000 7f474be00640 1 --2- 172.21.15.22:0/1988998168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc3ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.284+0000 7f474be00640 1 -- 172.21.15.22:0/1988998168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a99b9777e0 con 0x55a99bc415f0 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.284+0000 7f474be00640 1 --2- 172.21.15.22:0/1988998168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc3f100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4744009b60 tx=0x7f4744009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b2f02afa9f796e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.284+0000 7f4739600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.284+0000 7f4739600640 1 -- 172.21.15.22:0/1988998168 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744027070 con 0x55a99bc415f0 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.284+0000 7f475521e740 1 -- 172.21.15.22:0/1988998168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4744012ad0 con 0x55a99bc415f0 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.284+0000 7f4739600640 1 -- 172.21.15.22:0/1988998168 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f474400f800 con 0x55a99bc415f0 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.285+0000 7f4739600640 1 -- 172.21.15.22:0/1988998168 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4744031050 con 0x55a99bc415f0 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.285+0000 7f475521e740 10 client.25946 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.285+0000 7f475521e740 1 -- 172.21.15.22:0/1988998168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a99b9d7f70 con 0x55a99bc415f0 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.285+0000 7f4739600640 1 -- 172.21.15.22:0/1988998168 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4744012ad0 con 0x55a99bc415f0 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.286+0000 7f4739600640 1 -- 172.21.15.22:0/1988998168 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f47440055a0 con 0x55a99bc415f0 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.286+0000 7f475521e740 10 client.25946 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.286+0000 7f475521e740 10 client.25946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.286+0000 7f475521e740 20 client.25946 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.286+0000 7f475521e740 1 --2- 172.21.15.22:0/1988998168 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a99bc51970 0x55a99bc53d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.286+0000 7f475521e740 4 client.25946 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.286+0000 7f475521e740 1 -- 172.21.15.22:0/1988998168 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a99bbaa740 con 0x55a99bc51970 2024-04-20T21:38:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.286+0000 7f474b400640 1 --2- 172.21.15.22:0/1988998168 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a99bc51970 0x55a99bc53d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.287+0000 7f4739600640 10 client.25946 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.287+0000 7f474b400640 1 --2- 172.21.15.22:0/1988998168 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a99bc51970 0x55a99bc53d60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3b7c147fb087c59 server_cookie=e354b7c29089610b in_seq=0 out_seq=0 2024-04-20T21:38:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.288+0000 7f4739600640 1 -- 172.21.15.22:0/1988998168 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a99bbaa740 con 0x55a99bc51970 2024-04-20T21:38:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.288+0000 7f4739600640 10 client.25946 handle_command_reply: tid=0 2024-04-20T21:38:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.288+0000 7f475521e740 1 client.25946 shutdown 2024-04-20T21:38:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.288+0000 7f475521e740 1 -- 172.21.15.22:0/1988998168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 msgr2=0x55a99bc3f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.288+0000 7f475521e740 1 --2- 172.21.15.22:0/1988998168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc3f100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4744009b60 tx=0x7f4744009840 comp rx=0 tx=0).stop 2024-04-20T21:38:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.288+0000 7f475521e740 1 -- 172.21.15.22:0/1988998168 shutdown_connections 2024-04-20T21:38:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.288+0000 7f475521e740 1 --2- 172.21.15.22:0/1988998168 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a99bc51970 0x55a99bc53d60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.288+0000 7f475521e740 1 --2- 172.21.15.22:0/1988998168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc3f100 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.288+0000 7f475521e740 1 --2- 172.21.15.22:0/1988998168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc3ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.288+0000 7f475521e740 1 --2- 172.21.15.22:0/1988998168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc3e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.288+0000 7f4739600640 0 client.25946 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:18.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.288+0000 7f475521e740 1 -- 172.21.15.22:0/1988998168 >> 172.21.15.22:0/1988998168 conn(0x55a99bc32820 msgr2=0x55a99bc34570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.289+0000 7f475521e740 1 -- 172.21.15.22:0/1988998168 shutdown_connections 2024-04-20T21:38:18.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.289+0000 7f475521e740 1 -- 172.21.15.22:0/1988998168 wait complete. 2024-04-20T21:38:18.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.289+0000 7f475521e740 20 client.25946 trim_cache size 0 max 16384 2024-04-20T21:38:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.309+0000 7f475521e740 1 Processor -- start 2024-04-20T21:38:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.310+0000 7f475521e740 1 -- start start 2024-04-20T21:38:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.310+0000 7f475521e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc4d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.310+0000 7f475521e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc3ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.310+0000 7f475521e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc59330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.310+0000 7f475521e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b2170 con 0x55a99bc38a10 2024-04-20T21:38:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.310+0000 7f475521e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b49f0 con 0x55a99bc415f0 2024-04-20T21:38:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.310+0000 7f475521e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b27e0 con 0x55a99bc380f0 2024-04-20T21:38:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.310+0000 7f474be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc59330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.310+0000 7f4742a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc3ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.310+0000 7f474be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc59330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45924/0 (socket says 172.21.15.22:45924) 2024-04-20T21:38:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.310+0000 7f474be00640 1 -- 172.21.15.22:0/1837286652 learned_addr learned my addr 172.21.15.22:0/1837286652 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.310+0000 7f474b400640 1 --2- 172.21.15.22:0/1837286652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc4d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.311+0000 7f474be00640 1 -- 172.21.15.22:0/1837286652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 msgr2=0x55a99bc4d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.311+0000 7f474be00640 1 --2- 172.21.15.22:0/1837286652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc4d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.311+0000 7f474be00640 1 -- 172.21.15.22:0/1837286652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 msgr2=0x55a99bc3ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.311+0000 7f474be00640 1 --2- 172.21.15.22:0/1837286652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc3ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.311+0000 7f474be00640 1 -- 172.21.15.22:0/1837286652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a99b86f170 con 0x55a99bc415f0 2024-04-20T21:38:18.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.311+0000 7f474b400640 1 --2- 172.21.15.22:0/1837286652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc4d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:18.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.311+0000 7f474be00640 1 --2- 172.21.15.22:0/1837286652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc59330 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4744005e00 tx=0x7f474400f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e41df5fd45cdad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.311+0000 7f473be00640 1 -- 172.21.15.22:0/1837286652 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744027040 con 0x55a99bc415f0 2024-04-20T21:38:18.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.311+0000 7f473be00640 1 -- 172.21.15.22:0/1837286652 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4744011cb0 con 0x55a99bc415f0 2024-04-20T21:38:18.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.311+0000 7f473be00640 1 -- 172.21.15.22:0/1837286652 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474400fb50 con 0x55a99bc415f0 2024-04-20T21:38:18.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.312+0000 7f475521e740 1 -- 172.21.15.22:0/1837286652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 msgr2=0x55a99bc59330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.312+0000 7f475521e740 1 --2- 172.21.15.22:0/1837286652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc59330 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4744005e00 tx=0x7f474400f440 comp rx=0 tx=0).stop 2024-04-20T21:38:18.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.312+0000 7f475521e740 1 -- 172.21.15.22:0/1837286652 shutdown_connections 2024-04-20T21:38:18.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.312+0000 7f475521e740 1 --2- 172.21.15.22:0/1837286652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc415f0 0x55a99bc59330 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.312+0000 7f475521e740 1 --2- 172.21.15.22:0/1837286652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc38a10 0x55a99bc3ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.312+0000 7f475521e740 1 --2- 172.21.15.22:0/1837286652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc380f0 0x55a99bc4d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.312+0000 7f475521e740 1 -- 172.21.15.22:0/1837286652 >> 172.21.15.22:0/1837286652 conn(0x55a99bc32820 msgr2=0x55a99bc31a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.312+0000 7f475521e740 1 -- 172.21.15.22:0/1837286652 shutdown_connections 2024-04-20T21:38:18.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.313+0000 7f475521e740 1 -- 172.21.15.22:0/1837286652 wait complete. 2024-04-20T21:38:18.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.313+0000 7f475521e740 1 -- start start 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.313+0000 7f475521e740 1 Processor -- start 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.313+0000 7f475521e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc380f0 0x55a99bc45560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f475521e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc38a10 0x55a99bc45aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f475521e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc415f0 0x55a99bcb6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f475521e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b2eb0 con 0x55a99bc380f0 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f475521e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b4320 con 0x55a99bc38a10 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f475521e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b3580 con 0x55a99bc415f0 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f475521e740 10 client.0 _ll_register_callbacks cb 0x7f473c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f4742a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc38a10 0x55a99bc45aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f474b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc380f0 0x55a99bc45560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f4742a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc38a10 0x55a99bc45aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45938/0 (socket says 172.21.15.22:45938) 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f4742a00640 1 -- 172.21.15.22:0/2968419241 learned_addr learned my addr 172.21.15.22:0/2968419241 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f474be00640 1 --2- 172.21.15.22:0/2968419241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc415f0 0x55a99bcb6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f4742a00640 1 -- 172.21.15.22:0/2968419241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc415f0 msgr2=0x55a99bcb6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f4742a00640 1 --2- 172.21.15.22:0/2968419241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc415f0 0x55a99bcb6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f4742a00640 1 -- 172.21.15.22:0/2968419241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc380f0 msgr2=0x55a99bc45560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f4742a00640 1 --2- 172.21.15.22:0/2968419241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc380f0 0x55a99bc45560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f4742a00640 1 -- 172.21.15.22:0/2968419241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a99bcb70e0 con 0x55a99bc38a10 2024-04-20T21:38:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.314+0000 7f4742a00640 1 --2- 172.21.15.22:0/2968419241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc38a10 0x55a99bc45aa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f47340f0290 tx=0x7f473402bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea6e3b5a2c3ff89e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.315+0000 7f4739600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.315+0000 7f4739600640 1 -- 172.21.15.22:0/2968419241 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47340be130 con 0x55a99bc38a10 2024-04-20T21:38:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.315+0000 7f475521e740 1 -- 172.21.15.22:0/2968419241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4744011cb0 con 0x55a99bc38a10 2024-04-20T21:38:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.315+0000 7f4739600640 1 -- 172.21.15.22:0/2968419241 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f47340042e0 con 0x55a99bc38a10 2024-04-20T21:38:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.315+0000 7f4739600640 1 -- 172.21.15.22:0/2968419241 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f47340bdf50 con 0x55a99bc38a10 2024-04-20T21:38:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.315+0000 7f475521e740 10 client.25952 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.315+0000 7f475521e740 1 -- 172.21.15.22:0/2968419241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a99bcb82b0 con 0x55a99bc38a10 2024-04-20T21:38:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.315+0000 7f4739600640 1 -- 172.21.15.22:0/2968419241 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4744011cb0 con 0x55a99bc38a10 2024-04-20T21:38:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.315+0000 7f4739600640 1 -- 172.21.15.22:0/2968419241 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f47340c6dd0 con 0x55a99bc38a10 2024-04-20T21:38:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.315+0000 7f475521e740 10 client.25952 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.315+0000 7f475521e740 10 client.25952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.315+0000 7f475521e740 20 client.25952 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.316+0000 7f475521e740 1 --2- 172.21.15.22:0/2968419241 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a99bcb8a80 0x55a99bcbaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.316+0000 7f475521e740 4 client.25952 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:38:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.316+0000 7f475521e740 1 -- 172.21.15.22:0/2968419241 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a99bba45b0 con 0x55a99bcb8a80 2024-04-20T21:38:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.316+0000 7f474b400640 1 --2- 172.21.15.22:0/2968419241 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a99bcb8a80 0x55a99bcbaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.316+0000 7f474b400640 1 --2- 172.21.15.22:0/2968419241 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a99bcb8a80 0x55a99bcbaea0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f11a8007c130e3c3 server_cookie=c6ca99852322cbd8 in_seq=0 out_seq=0 2024-04-20T21:38:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.316+0000 7f4739600640 10 client.25952 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:18.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.318+0000 7f4739600640 1 -- 172.21.15.22:0/2968419241 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a99bba45b0 con 0x55a99bcb8a80 2024-04-20T21:38:18.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.318+0000 7f4739600640 10 client.25952 handle_command_reply: tid=0 2024-04-20T21:38:18.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.318+0000 7f475521e740 1 client.25952 shutdown 2024-04-20T21:38:18.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.318+0000 7f475521e740 1 -- 172.21.15.22:0/2968419241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc38a10 msgr2=0x55a99bc45aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.318+0000 7f475521e740 1 --2- 172.21.15.22:0/2968419241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc38a10 0x55a99bc45aa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f47340f0290 tx=0x7f473402bcc0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.318+0000 7f475521e740 1 -- 172.21.15.22:0/2968419241 shutdown_connections 2024-04-20T21:38:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.318+0000 7f475521e740 1 --2- 172.21.15.22:0/2968419241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99bc415f0 0x55a99bcb6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.318+0000 7f475521e740 1 --2- 172.21.15.22:0/2968419241 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a99bcb8a80 0x55a99bcbaea0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.318+0000 7f475521e740 1 --2- 172.21.15.22:0/2968419241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a99bc38a10 0x55a99bc45aa0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.318+0000 7f475521e740 1 --2- 172.21.15.22:0/2968419241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a99bc380f0 0x55a99bc45560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.318+0000 7f4739600640 0 client.25952 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.318+0000 7f475521e740 1 -- 172.21.15.22:0/2968419241 >> 172.21.15.22:0/2968419241 conn(0x55a99bc32820 msgr2=0x55a99bc33c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.319+0000 7f475521e740 1 -- 172.21.15.22:0/2968419241 shutdown_connections 2024-04-20T21:38:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.319+0000 7f475521e740 1 -- 172.21.15.22:0/2968419241 wait complete. 2024-04-20T21:38:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.319+0000 7f475521e740 20 client.25952 trim_cache size 0 max 16384 2024-04-20T21:38:18.319 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:38:18.320 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:38:18.320 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "65f20df3-7c17-4f72-82fe-30774f04c5bf", 2024-04-20T21:38:18.320 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:38:18.320 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:38:18.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.319+0000 7f4751000640 1 -- 172.21.15.22:0/259075382 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f47300a3da0 msgr2=0x7f47300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.319+0000 7f4751000640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f47300a3da0 0x7f47300a6260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55a99bc37bc0 tx=0x7f473c009500 comp rx=0 tx=0).stop 2024-04-20T21:38:18.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.319+0000 7f4751000640 1 -- 172.21.15.22:0/259075382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1016d0 msgr2=0x7f474c1a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.319+0000 7f4751000640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1016d0 0x7f474c1a4df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4734009230 tx=0x7f4734004970 comp rx=0 tx=0).stop 2024-04-20T21:38:18.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.320+0000 7f4751000640 1 -- 172.21.15.22:0/259075382 shutdown_connections 2024-04-20T21:38:18.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.320+0000 7f4751000640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f47300a3da0 0x7f47300a6260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.320+0000 7f4751000640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f474c1061b0 0x7f474c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.320+0000 7f4751000640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f474c1016d0 0x7f474c1a4df0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.320+0000 7f4751000640 1 --2- 172.21.15.22:0/259075382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f474c100db0 0x7f474c1a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.320+0000 7f4751000640 1 -- 172.21.15.22:0/259075382 >> 172.21.15.22:0/259075382 conn(0x7f474c0fc7e0 msgr2=0x7f474c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.320+0000 7f4751000640 1 -- 172.21.15.22:0/259075382 shutdown_connections 2024-04-20T21:38:18.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.321+0000 7f4751000640 1 -- 172.21.15.22:0/259075382 wait complete. 2024-04-20T21:38:18.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:38:18.340 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:38:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.547+0000 7f6a34400640 1 -- 172.21.15.22:0/361830582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2c102ba0 msgr2=0x7f6a2c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.547+0000 7f6a34400640 1 --2- 172.21.15.22:0/361830582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2c102ba0 0x7f6a2c106be0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009900 tx=0x7f6a1802bc30 comp rx=0 tx=0).stop 2024-04-20T21:38:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.547+0000 7f6a34400640 1 -- 172.21.15.22:0/361830582 shutdown_connections 2024-04-20T21:38:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.547+0000 7f6a34400640 1 --2- 172.21.15.22:0/361830582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2c1072c0 0x7f6a2c10ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.547+0000 7f6a34400640 1 --2- 172.21.15.22:0/361830582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2c102ba0 0x7f6a2c106be0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.547+0000 7f6a34400640 1 --2- 172.21.15.22:0/361830582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a2c1021f0 0x7f6a2c1025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.547+0000 7f6a34400640 1 -- 172.21.15.22:0/361830582 >> 172.21.15.22:0/361830582 conn(0x7f6a2c0fda70 msgr2=0x7f6a2c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.547+0000 7f6a34400640 1 -- 172.21.15.22:0/361830582 shutdown_connections 2024-04-20T21:38:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.547+0000 7f6a34400640 1 -- 172.21.15.22:0/361830582 wait complete. 2024-04-20T21:38:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.548+0000 7f6a34400640 1 Processor -- start 2024-04-20T21:38:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.548+0000 7f6a34400640 1 -- start start 2024-04-20T21:38:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a34400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2c1021f0 0x7f6a2c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a34400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2c1072c0 0x7f6a2c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a34400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a2c19a8c0 0x7f6a2c19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a34400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c112380 con 0x7f6a2c19a8c0 2024-04-20T21:38:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a34400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c112200 con 0x7f6a2c1072c0 2024-04-20T21:38:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a34400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c112500 con 0x7f6a2c1021f0 2024-04-20T21:38:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a31000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2c1072c0 0x7f6a2c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a31000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2c1072c0 0x7f6a2c1a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45952/0 (socket says 172.21.15.22:45952) 2024-04-20T21:38:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a31000640 1 -- 172.21.15.22:0/2058351178 learned_addr learned my addr 172.21.15.22:0/2058351178 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a31a00640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2c1021f0 0x7f6a2c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a32400640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a2c19a8c0 0x7f6a2c19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a31000640 1 -- 172.21.15.22:0/2058351178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2c1021f0 msgr2=0x7f6a2c1a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a31000640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2c1021f0 0x7f6a2c1a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a31000640 1 -- 172.21.15.22:0/2058351178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a2c19a8c0 msgr2=0x7f6a2c19ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a31000640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a2c19a8c0 0x7f6a2c19ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.549+0000 7f6a31000640 1 -- 172.21.15.22:0/2058351178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c19b3e0 con 0x7f6a2c1072c0 2024-04-20T21:38:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.550+0000 7f6a31000640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2c1072c0 0x7f6a2c1a0c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a1802c240 tx=0x7f6a18004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.550+0000 7f6a2a000640 1 -- 172.21.15.22:0/2058351178 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18043020 con 0x7f6a2c1072c0 2024-04-20T21:38:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.550+0000 7f6a2a000640 1 -- 172.21.15.22:0/2058351178 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a180054e0 con 0x7f6a2c1072c0 2024-04-20T21:38:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.550+0000 7f6a34400640 1 -- 172.21.15.22:0/2058351178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2c19b670 con 0x7f6a2c1072c0 2024-04-20T21:38:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.550+0000 7f6a2a000640 1 -- 172.21.15.22:0/2058351178 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18003b80 con 0x7f6a2c1072c0 2024-04-20T21:38:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.550+0000 7f6a34400640 1 -- 172.21.15.22:0/2058351178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2c1a7910 con 0x7f6a2c1072c0 2024-04-20T21:38:18.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.554+0000 7f6a3700e740 1 Processor -- start 2024-04-20T21:38:18.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.554+0000 7f6a2a000640 1 -- 172.21.15.22:0/2058351178 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f6a18046020 con 0x7f6a2c1072c0 2024-04-20T21:38:18.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.554+0000 7f6a3700e740 1 -- start start 2024-04-20T21:38:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a3700e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93b31a0 0x5590b93b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a3700e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b3ac0 0x5590b93bc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a3700e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93bc6a0 0x5590b93bea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a3700e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5590b9319150 con 0x5590b93bc6a0 2024-04-20T21:38:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a3700e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5590b92bf370 con 0x5590b93b3ac0 2024-04-20T21:38:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a3700e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5590b9315110 con 0x5590b93b31a0 2024-04-20T21:38:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a31a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b3ac0 0x5590b93bc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a31a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b3ac0 0x5590b93bc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45962/0 (socket says 172.21.15.22:45962) 2024-04-20T21:38:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a31a00640 1 -- 172.21.15.22:0/2747552793 learned_addr learned my addr 172.21.15.22:0/2747552793 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a31000640 1 --2- 172.21.15.22:0/2747552793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93bc6a0 0x5590b93bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a32400640 1 --2- 172.21.15.22:0/2747552793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93b31a0 0x5590b93b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a31a00640 1 -- 172.21.15.22:0/2747552793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93b31a0 msgr2=0x5590b93b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a31a00640 1 --2- 172.21.15.22:0/2747552793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93b31a0 0x5590b93b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a31a00640 1 -- 172.21.15.22:0/2747552793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93bc6a0 msgr2=0x5590b93bea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a31a00640 1 --2- 172.21.15.22:0/2747552793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93bc6a0 0x5590b93bea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a31a00640 1 -- 172.21.15.22:0/2747552793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b90f57d0 con 0x5590b93b3ac0 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.555+0000 7f6a32400640 1 --2- 172.21.15.22:0/2747552793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93b31a0 0x5590b93b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.556+0000 7f6a31a00640 1 --2- 172.21.15.22:0/2747552793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b3ac0 0x5590b93bc160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000bc90 tx=0x7f6a200107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f1505dd9e15623e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.556+0000 7f6a1ea00640 1 -- 172.21.15.22:0/2747552793 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20005220 con 0x5590b93b3ac0 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.556+0000 7f6a1ea00640 1 -- 172.21.15.22:0/2747552793 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2000be40 con 0x5590b93b3ac0 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.556+0000 7f6a1ea00640 1 -- 172.21.15.22:0/2747552793 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20011770 con 0x5590b93b3ac0 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.556+0000 7f6a2a000640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f6a140a3da0 0x7f6a140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.556+0000 7f6a2a000640 1 -- 172.21.15.22:0/2058351178 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f6a180f2770 con 0x7f6a2c1072c0 2024-04-20T21:38:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.556+0000 7f6a32400640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f6a140a3da0 0x7f6a140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.557+0000 7f6a3700e740 1 -- 172.21.15.22:0/2747552793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b3ac0 msgr2=0x5590b93bc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.557+0000 7f6a3700e740 1 --2- 172.21.15.22:0/2747552793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b3ac0 0x5590b93bc160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000bc90 tx=0x7f6a200107a0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.557+0000 7f6a3700e740 1 -- 172.21.15.22:0/2747552793 shutdown_connections 2024-04-20T21:38:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.557+0000 7f6a3700e740 1 --2- 172.21.15.22:0/2747552793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93bc6a0 0x5590b93bea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.557+0000 7f6a3700e740 1 --2- 172.21.15.22:0/2747552793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b3ac0 0x5590b93bc160 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.557+0000 7f6a3700e740 1 --2- 172.21.15.22:0/2747552793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93b31a0 0x5590b93b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.557+0000 7f6a3700e740 1 -- 172.21.15.22:0/2747552793 >> 172.21.15.22:0/2747552793 conn(0x5590b93ad8d0 msgr2=0x5590b93adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.557+0000 7f6a3700e740 1 -- 172.21.15.22:0/2747552793 shutdown_connections 2024-04-20T21:38:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.557+0000 7f6a32400640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f6a140a3da0 0x7f6a140a6260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a24005fd0 tx=0x7f6a24009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.557+0000 7f6a3700e740 1 -- 172.21.15.22:0/2747552793 wait complete. 2024-04-20T21:38:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.558+0000 7f6a3700e740 1 -- start start 2024-04-20T21:38:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.558+0000 7f6a3700e740 1 Processor -- start 2024-04-20T21:38:18.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.559+0000 7f6a3700e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 0x5590b93b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.559+0000 7f6a3700e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93b9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.559+0000 7f6a3700e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93bc6a0 0x5590b93ba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.559+0000 7f6a3700e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5590b93bf490 con 0x5590b93b3ac0 2024-04-20T21:38:18.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.559+0000 7f6a3700e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5590b92d2740 con 0x5590b93b31a0 2024-04-20T21:38:18.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.559+0000 7f6a3700e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5590b93bf610 con 0x5590b93bc6a0 2024-04-20T21:38:18.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.559+0000 7f6a3700e740 10 client.0 _ll_register_callbacks cb 0x5590b92cdb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:18.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.559+0000 7f6a31a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 0x5590b93b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.559+0000 7f6a31000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93b9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.559+0000 7f6a31a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 0x5590b93b96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45972/0 (socket says 172.21.15.22:45972) 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.559+0000 7f6a31a00640 1 -- 172.21.15.22:0/2342840001 learned_addr learned my addr 172.21.15.22:0/2342840001 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.559+0000 7f6a32400640 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93bc6a0 0x5590b93ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.560+0000 7f6a31a00640 1 -- 172.21.15.22:0/2342840001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93bc6a0 msgr2=0x5590b93ba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.560+0000 7f6a31a00640 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93bc6a0 0x5590b93ba160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.560+0000 7f6a31a00640 1 -- 172.21.15.22:0/2342840001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 msgr2=0x5590b93b9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.560+0000 7f6a31a00640 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93b9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.560+0000 7f6a31a00640 1 -- 172.21.15.22:0/2342840001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b902b3f0 con 0x5590b93b31a0 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.560+0000 7f6a31000640 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93b9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.560+0000 7f6a32400640 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93bc6a0 0x5590b93ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.560+0000 7f6a31a00640 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 0x5590b93b96e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6a20011010 tx=0x7f6a200052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55fabf09aa3eb5af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.560+0000 7f6a13400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:18.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.561+0000 7f6a13400640 1 -- 172.21.15.22:0/2342840001 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2002a060 con 0x5590b93b31a0 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.561+0000 7f6a13400640 1 -- 172.21.15.22:0/2342840001 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f6a20004ad0 con 0x5590b93b31a0 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.561+0000 7f6a3700e740 1 -- 172.21.15.22:0/2342840001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5590b9326b70 con 0x5590b93b31a0 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.561+0000 7f6a13400640 1 -- 172.21.15.22:0/2342840001 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6a20004cd0 con 0x5590b93b31a0 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.561+0000 7f6a3700e740 10 client.25964 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.561+0000 7f6a3700e740 1 -- 172.21.15.22:0/2342840001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5590b8feb0f0 con 0x5590b93b31a0 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.561+0000 7f6a13400640 1 -- 172.21.15.22:0/2342840001 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5590b9326b70 con 0x5590b93b31a0 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.562+0000 7f6a13400640 1 -- 172.21.15.22:0/2342840001 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f6a2001b430 con 0x5590b93b31a0 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.562+0000 7f6a3700e740 10 client.25964 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.562+0000 7f6a3700e740 10 client.25964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.562+0000 7f6a3700e740 20 client.25964 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.562+0000 7f6a3700e740 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5590b93cca10 0x5590b93cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.562+0000 7f6a3700e740 4 client.25964 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.562+0000 7f6a3700e740 1 -- 172.21.15.22:0/2342840001 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590b93209e0 con 0x5590b93cca10 2024-04-20T21:38:18.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.562+0000 7f6a31000640 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5590b93cca10 0x5590b93cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.563+0000 7f6a31000640 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5590b93cca10 0x5590b93cee00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=942a1b2c601603e3 server_cookie=b8c01335b15cfa4f in_seq=0 out_seq=0 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.563+0000 7f6a13400640 10 client.25964 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a13400640 1 -- 172.21.15.22:0/2342840001 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590b93209e0 con 0x5590b93cca10 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a13400640 10 client.25964 handle_command_reply: tid=0 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a3700e740 1 client.25964 shutdown 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a3700e740 1 -- 172.21.15.22:0/2342840001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 msgr2=0x5590b93b96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a3700e740 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 0x5590b93b96e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6a20011010 tx=0x7f6a200052a0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a3700e740 1 -- 172.21.15.22:0/2342840001 shutdown_connections 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a3700e740 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93bc6a0 0x5590b93ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a3700e740 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93b9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a3700e740 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5590b93cca10 0x5590b93cee00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a3700e740 1 --2- 172.21.15.22:0/2342840001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 0x5590b93b96e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a3700e740 1 -- 172.21.15.22:0/2342840001 >> 172.21.15.22:0/2342840001 conn(0x5590b93ad8d0 msgr2=0x5590b93af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a13400640 0 client.25964 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a3700e740 1 -- 172.21.15.22:0/2342840001 shutdown_connections 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.564+0000 7f6a3700e740 1 -- 172.21.15.22:0/2342840001 wait complete. 2024-04-20T21:38:18.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.565+0000 7f6a3700e740 20 client.25964 trim_cache size 0 max 16384 2024-04-20T21:38:18.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.587+0000 7f6a3700e740 1 Processor -- start 2024-04-20T21:38:18.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.587+0000 7f6a3700e740 1 -- start start 2024-04-20T21:38:18.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.587+0000 7f6a3700e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 0x5590b93c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.587+0000 7f6a3700e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.587+0000 7f6a3700e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93bc6a0 0x5590b93b9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.587+0000 7f6a3700e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1adf90 con 0x5590b93b3ac0 2024-04-20T21:38:18.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.587+0000 7f6a3700e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1b0810 con 0x5590b93b31a0 2024-04-20T21:38:18.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.587+0000 7f6a3700e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1ae600 con 0x5590b93bc6a0 2024-04-20T21:38:18.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.588+0000 7f6a31a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 0x5590b93c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.588+0000 7f6a31000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.588+0000 7f6a31a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 0x5590b93c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45978/0 (socket says 172.21.15.22:45978) 2024-04-20T21:38:18.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.588+0000 7f6a31a00640 1 -- 172.21.15.22:0/1160072827 learned_addr learned my addr 172.21.15.22:0/1160072827 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:18.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.588+0000 7f6a32400640 1 --2- 172.21.15.22:0/1160072827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93bc6a0 0x5590b93b9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.588+0000 7f6a31a00640 1 -- 172.21.15.22:0/1160072827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93bc6a0 msgr2=0x5590b93b9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.588+0000 7f6a31a00640 1 --2- 172.21.15.22:0/1160072827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93bc6a0 0x5590b93b9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.588+0000 7f6a31a00640 1 -- 172.21.15.22:0/1160072827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 msgr2=0x5590b93adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.588+0000 7f6a31a00640 1 --2- 172.21.15.22:0/1160072827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.588+0000 7f6a31a00640 1 -- 172.21.15.22:0/1160072827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b93bea90 con 0x5590b93b31a0 2024-04-20T21:38:18.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.589+0000 7f6a31a00640 1 --2- 172.21.15.22:0/1160072827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 0x5590b93c8380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6a200107e0 tx=0x7f6a20005000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79b4b8e2a90de48 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.589+0000 7f6a1e000640 1 -- 172.21.15.22:0/1160072827 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2002a070 con 0x5590b93b31a0 2024-04-20T21:38:18.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.589+0000 7f6a1e000640 1 -- 172.21.15.22:0/1160072827 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a20005310 con 0x5590b93b31a0 2024-04-20T21:38:18.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.589+0000 7f6a1e000640 1 -- 172.21.15.22:0/1160072827 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20004b30 con 0x5590b93b31a0 2024-04-20T21:38:18.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.590+0000 7f6a3700e740 1 -- 172.21.15.22:0/1160072827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 msgr2=0x5590b93c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.590+0000 7f6a3700e740 1 --2- 172.21.15.22:0/1160072827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 0x5590b93c8380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6a200107e0 tx=0x7f6a20005000 comp rx=0 tx=0).stop 2024-04-20T21:38:18.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.590+0000 7f6a3700e740 1 -- 172.21.15.22:0/1160072827 shutdown_connections 2024-04-20T21:38:18.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.590+0000 7f6a3700e740 1 --2- 172.21.15.22:0/1160072827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93bc6a0 0x5590b93b9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.590+0000 7f6a3700e740 1 --2- 172.21.15.22:0/1160072827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.590+0000 7f6a3700e740 1 --2- 172.21.15.22:0/1160072827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93b31a0 0x5590b93c8380 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.590+0000 7f6a3700e740 1 -- 172.21.15.22:0/1160072827 >> 172.21.15.22:0/1160072827 conn(0x5590b93ad8d0 msgr2=0x5590b93ac030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.590+0000 7f6a3700e740 1 -- 172.21.15.22:0/1160072827 shutdown_connections 2024-04-20T21:38:18.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.590+0000 7f6a3700e740 1 -- 172.21.15.22:0/1160072827 wait complete. 2024-04-20T21:38:18.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.590+0000 7f6a3700e740 1 -- start start 2024-04-20T21:38:18.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.591+0000 7f6a3700e740 1 Processor -- start 2024-04-20T21:38:18.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.591+0000 7f6a3700e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93b31a0 0x5590b93bf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.591+0000 7f6a3700e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93bfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.591+0000 7f6a3700e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93bc6a0 0x5590b9430c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.591+0000 7f6a3700e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1aecd0 con 0x5590b93b3ac0 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.591+0000 7f6a3700e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1b0140 con 0x5590b93bc6a0 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.591+0000 7f6a3700e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1af3a0 con 0x5590b93b31a0 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.591+0000 7f6a3700e740 10 client.0 _ll_register_callbacks cb 0x7f6a2000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.592+0000 7f6a32400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93bc6a0 0x5590b9430c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.592+0000 7f6a32400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93bc6a0 0x5590b9430c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:45982/0 (socket says 172.21.15.22:45982) 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.592+0000 7f6a32400640 1 -- 172.21.15.22:0/1415159133 learned_addr learned my addr 172.21.15.22:0/1415159133 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.592+0000 7f6a31000640 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93bfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.592+0000 7f6a31a00640 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93b31a0 0x5590b93bf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.592+0000 7f6a32400640 1 -- 172.21.15.22:0/1415159133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93b31a0 msgr2=0x5590b93bf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.592+0000 7f6a32400640 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93b31a0 0x5590b93bf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.592+0000 7f6a32400640 1 -- 172.21.15.22:0/1415159133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 msgr2=0x5590b93bfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.592+0000 7f6a32400640 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93bfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.592+0000 7f6a32400640 1 -- 172.21.15.22:0/1415159133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b9431170 con 0x5590b93bc6a0 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.592+0000 7f6a32400640 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93bc6a0 0x5590b9430c30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a240041b0 tx=0x7f6a24004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36bb85c747aa7d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.592+0000 7f6a13400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a13400640 1 -- 172.21.15.22:0/1415159133 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24004b20 con 0x5590b93bc6a0 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a13400640 1 -- 172.21.15.22:0/1415159133 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f6a24018030 con 0x5590b93bc6a0 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a3700e740 1 -- 172.21.15.22:0/1415159133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a20005310 con 0x5590b93bc6a0 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a13400640 1 -- 172.21.15.22:0/1415159133 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6a24037050 con 0x5590b93bc6a0 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a3700e740 10 client.25970 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a3700e740 1 -- 172.21.15.22:0/1415159133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5590b94323e0 con 0x5590b93bc6a0 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a13400640 1 -- 172.21.15.22:0/1415159133 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a20005310 con 0x5590b93bc6a0 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a13400640 1 -- 172.21.15.22:0/1415159133 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f6a24005160 con 0x5590b93bc6a0 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a3700e740 10 client.25970 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a3700e740 10 client.25970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a3700e740 20 client.25970 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a3700e740 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5590b9432b50 0x5590b9434f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a3700e740 4 client.25970 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.593+0000 7f6a3700e740 1 -- 172.21.15.22:0/1415159133 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5590b929c450 con 0x5590b9432b50 2024-04-20T21:38:18.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.594+0000 7f6a31a00640 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5590b9432b50 0x5590b9434f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:18.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.594+0000 7f6a31a00640 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5590b9432b50 0x5590b9434f70 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d4e1d0b518f980 server_cookie=97d0060c553c8c61 in_seq=0 out_seq=0 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.594+0000 7f6a13400640 10 client.25970 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.594+0000 7f6a13400640 1 -- 172.21.15.22:0/1415159133 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5590b929c450 con 0x5590b9432b50 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.594+0000 7f6a13400640 10 client.25970 handle_command_reply: tid=0 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.594+0000 7f6a3700e740 1 client.25970 shutdown 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.595+0000 7f6a3700e740 1 -- 172.21.15.22:0/1415159133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93bc6a0 msgr2=0x5590b9430c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.595+0000 7f6a3700e740 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93bc6a0 0x5590b9430c30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a240041b0 tx=0x7f6a24004a40 comp rx=0 tx=0).stop 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.595+0000 7f6a3700e740 1 -- 172.21.15.22:0/1415159133 shutdown_connections 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.595+0000 7f6a3700e740 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5590b9432b50 0x5590b9434f70 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.595+0000 7f6a3700e740 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590b93bc6a0 0x5590b9430c30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.595+0000 7f6a3700e740 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5590b93b3ac0 0x5590b93bfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.595+0000 7f6a3700e740 1 --2- 172.21.15.22:0/1415159133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b93b31a0 0x5590b93bf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.595+0000 7f6a13400640 0 client.25970 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.595+0000 7f6a3700e740 1 -- 172.21.15.22:0/1415159133 >> 172.21.15.22:0/1415159133 conn(0x5590b93ad8d0 msgr2=0x5590b93af560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.595+0000 7f6a3700e740 1 -- 172.21.15.22:0/1415159133 shutdown_connections 2024-04-20T21:38:18.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.595+0000 7f6a3700e740 1 -- 172.21.15.22:0/1415159133 wait complete. 2024-04-20T21:38:18.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.595+0000 7f6a3700e740 20 client.25970 trim_cache size 0 max 16384 2024-04-20T21:38:18.597 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:38:18.597 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:38:18.597 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:38:18.597 INFO:teuthology.orchestra.run.smithi022.stdout: "65f20df3-7c17-4f72-82fe-30774f04c5bf": { 2024-04-20T21:38:18.597 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:38:18.597 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "65f20df3-7c17-4f72-82fe-30774f04c5bf", 2024-04-20T21:38:18.597 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:38:18.597 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:38:18.597 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:38:18.597 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:38:18.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.596+0000 7f6a34400640 1 -- 172.21.15.22:0/2058351178 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f6a140a3da0 msgr2=0x7f6a140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.596+0000 7f6a34400640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f6a140a3da0 0x7f6a140a6260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a24005fd0 tx=0x7f6a24009500 comp rx=0 tx=0).stop 2024-04-20T21:38:18.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.596+0000 7f6a34400640 1 -- 172.21.15.22:0/2058351178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2c1072c0 msgr2=0x7f6a2c1a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:18.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.596+0000 7f6a34400640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2c1072c0 0x7f6a2c1a0c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a1802c240 tx=0x7f6a18004360 comp rx=0 tx=0).stop 2024-04-20T21:38:18.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.596+0000 7f6a34400640 1 -- 172.21.15.22:0/2058351178 shutdown_connections 2024-04-20T21:38:18.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.596+0000 7f6a34400640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f6a140a3da0 0x7f6a140a6260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.596+0000 7f6a34400640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a2c19a8c0 0x7f6a2c19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.596+0000 7f6a34400640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2c1072c0 0x7f6a2c1a0c50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.596+0000 7f6a34400640 1 --2- 172.21.15.22:0/2058351178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2c1021f0 0x7f6a2c1a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:18.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.596+0000 7f6a34400640 1 -- 172.21.15.22:0/2058351178 >> 172.21.15.22:0/2058351178 conn(0x7f6a2c0fda70 msgr2=0x7f6a2c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:18.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.596+0000 7f6a34400640 1 -- 172.21.15.22:0/2058351178 shutdown_connections 2024-04-20T21:38:18.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:18.596+0000 7f6a34400640 1 -- 172.21.15.22:0/2058351178 wait complete. 2024-04-20T21:38:18.614 INFO:tasks.cephfs.filesystem:scrub status for tag:65f20df3-7c17-4f72-82fe-30774f04c5bf - {'path': '/', 'tag': '65f20df3-7c17-4f72-82fe-30774f04c5bf', 'options': 'recursive,force'} 2024-04-20T21:38:18.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:18 smithi022 ceph-mon[16843]: pgmap v137: 97 pgs: 97 active+clean; 172 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-04-20T21:38:18.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:18 smithi122 ceph-mon[22147]: pgmap v137: 97 pgs: 97 active+clean; 172 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-04-20T21:38:18.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:18 smithi033 ceph-mon[21626]: pgmap v137: 97 pgs: 97 active+clean; 172 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-04-20T21:38:19.184 INFO:tasks.workunit.client.0.smithi022.stdout: 1 316812 9.70 MB/sec execute 47 sec latency 19.164 ms 2024-04-20T21:38:20.184 INFO:tasks.workunit.client.0.smithi022.stdout: 1 319078 9.76 MB/sec execute 48 sec latency 11.152 ms 2024-04-20T21:38:20.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:20 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:38:20.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:20 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:20.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:20 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:38:20.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:20 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:20.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:20 smithi122 ceph-mon[22147]: pgmap v138: 97 pgs: 97 active+clean; 172 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-04-20T21:38:20.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:20 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:38:20.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:20 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:20.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:20 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:38:20.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:20 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:20.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:20 smithi033 ceph-mon[21626]: pgmap v138: 97 pgs: 97 active+clean; 172 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-04-20T21:38:20.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:20 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:38:20.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:20 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:20.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:20 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:38:20.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:20 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:20.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:20 smithi022 ceph-mon[16843]: pgmap v138: 97 pgs: 97 active+clean; 172 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-04-20T21:38:21.184 INFO:tasks.workunit.client.0.smithi022.stdout: 1 320183 9.67 MB/sec execute 49 sec latency 62.038 ms 2024-04-20T21:38:22.184 INFO:tasks.workunit.client.0.smithi022.stdout: 1 322541 9.75 MB/sec execute 50 sec latency 24.839 ms 2024-04-20T21:38:22.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:22 smithi122 ceph-mon[22147]: pgmap v139: 97 pgs: 97 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 235 op/s 2024-04-20T21:38:22.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:22 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:38:22.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:22 smithi033 ceph-mon[21626]: pgmap v139: 97 pgs: 97 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 235 op/s 2024-04-20T21:38:22.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:22 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:38:22.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:22 smithi022 ceph-mon[16843]: pgmap v139: 97 pgs: 97 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 235 op/s 2024-04-20T21:38:22.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:22 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:38:23.184 INFO:tasks.workunit.client.0.smithi022.stdout: 1 323832 9.68 MB/sec execute 51 sec latency 17.272 ms 2024-04-20T21:38:24.184 INFO:tasks.workunit.client.0.smithi022.stdout: 1 326483 9.80 MB/sec execute 52 sec latency 10.082 ms 2024-04-20T21:38:24.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:24 smithi122 ceph-mon[22147]: pgmap v140: 97 pgs: 97 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 2.6 MiB/s wr, 158 op/s 2024-04-20T21:38:24.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:24 smithi033 ceph-mon[21626]: pgmap v140: 97 pgs: 97 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 2.6 MiB/s wr, 158 op/s 2024-04-20T21:38:24.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:24 smithi022 ceph-mon[16843]: pgmap v140: 97 pgs: 97 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 2.6 MiB/s wr, 158 op/s 2024-04-20T21:38:25.184 INFO:tasks.workunit.client.0.smithi022.stdout: 1 328199 9.75 MB/sec execute 53 sec latency 45.800 ms 2024-04-20T21:38:26.185 INFO:tasks.workunit.client.0.smithi022.stdout: 1 330367 9.82 MB/sec execute 54 sec latency 9.442 ms 2024-04-20T21:38:26.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:26 smithi122 ceph-mon[22147]: pgmap v141: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 242 op/s 2024-04-20T21:38:26.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:26 smithi033 ceph-mon[21626]: pgmap v141: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 242 op/s 2024-04-20T21:38:26.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:26 smithi022 ceph-mon[16843]: pgmap v141: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 242 op/s 2024-04-20T21:38:27.185 INFO:tasks.workunit.client.0.smithi022.stdout: 1 332145 9.75 MB/sec execute 55 sec latency 46.106 ms 2024-04-20T21:38:28.185 INFO:tasks.workunit.client.0.smithi022.stdout: 1 333811 9.81 MB/sec execute 56 sec latency 20.699 ms 2024-04-20T21:38:28.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:28 smithi122 ceph-mon[22147]: pgmap v142: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 2.6 MiB/s wr, 164 op/s 2024-04-20T21:38:28.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:28 smithi033 ceph-mon[21626]: pgmap v142: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 2.6 MiB/s wr, 164 op/s 2024-04-20T21:38:28.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:28 smithi022 ceph-mon[16843]: pgmap v142: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 2.6 MiB/s wr, 164 op/s 2024-04-20T21:38:29.185 INFO:tasks.workunit.client.0.smithi022.stdout: 1 335569 9.74 MB/sec execute 57 sec latency 24.689 ms 2024-04-20T21:38:30.185 INFO:tasks.workunit.client.0.smithi022.stdout: 1 337300 9.80 MB/sec execute 58 sec latency 31.052 ms 2024-04-20T21:38:30.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:30 smithi122 ceph-mon[22147]: pgmap v143: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 2.6 MiB/s wr, 164 op/s 2024-04-20T21:38:30.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:30 smithi033 ceph-mon[21626]: pgmap v143: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 2.6 MiB/s wr, 164 op/s 2024-04-20T21:38:30.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:30 smithi022 ceph-mon[16843]: pgmap v143: 97 pgs: 97 active+clean; 183 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 2.6 MiB/s wr, 164 op/s 2024-04-20T21:38:31.185 INFO:tasks.workunit.client.0.smithi022.stdout: 1 338853 9.73 MB/sec execute 59 sec latency 17.648 ms 2024-04-20T21:38:32.185 INFO:tasks.workunit.client.0.smithi022.stdout: 1 340767 9.79 MB/sec execute 60 sec latency 61.885 ms 2024-04-20T21:38:32.459 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:32 smithi022 ceph-mon[16843]: pgmap v144: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-04-20T21:38:32.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:32 smithi122 ceph-mon[22147]: pgmap v144: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-04-20T21:38:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:32 smithi033 ceph-mon[21626]: pgmap v144: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-04-20T21:38:33.185 INFO:tasks.workunit.client.0.smithi022.stdout: 1 342158 9.72 MB/sec execute 61 sec latency 21.409 ms 2024-04-20T21:38:34.185 INFO:tasks.workunit.client.0.smithi022.stdout: 1 344320 9.78 MB/sec execute 62 sec latency 10.687 ms 2024-04-20T21:38:34.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:34 smithi122 ceph-mon[22147]: pgmap v145: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 2.5 MiB/s wr, 158 op/s 2024-04-20T21:38:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:34 smithi033 ceph-mon[21626]: pgmap v145: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 2.5 MiB/s wr, 158 op/s 2024-04-20T21:38:34.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:34 smithi022 ceph-mon[16843]: pgmap v145: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 2.5 MiB/s wr, 158 op/s 2024-04-20T21:38:35.186 INFO:tasks.workunit.client.0.smithi022.stdout: 1 345573 9.71 MB/sec execute 63 sec latency 16.158 ms 2024-04-20T21:38:36.186 INFO:tasks.workunit.client.0.smithi022.stdout: 1 347756 9.76 MB/sec execute 64 sec latency 22.712 ms 2024-04-20T21:38:36.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:36 smithi122 ceph-mon[22147]: pgmap v146: 97 pgs: 97 active+clean; 193 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 232 op/s 2024-04-20T21:38:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:36 smithi033 ceph-mon[21626]: pgmap v146: 97 pgs: 97 active+clean; 193 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 232 op/s 2024-04-20T21:38:36.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:36 smithi022 ceph-mon[16843]: pgmap v146: 97 pgs: 97 active+clean; 193 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 232 op/s 2024-04-20T21:38:37.186 INFO:tasks.workunit.client.0.smithi022.stdout: 1 348600 9.64 MB/sec execute 65 sec latency 56.534 ms 2024-04-20T21:38:38.186 INFO:tasks.workunit.client.0.smithi022.stdout: 1 350796 9.68 MB/sec execute 66 sec latency 24.045 ms 2024-04-20T21:38:38.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:38 smithi122 ceph-mon[22147]: pgmap v147: 97 pgs: 97 active+clean; 193 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 2.3 MiB/s wr, 149 op/s 2024-04-20T21:38:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:38 smithi033 ceph-mon[21626]: pgmap v147: 97 pgs: 97 active+clean; 193 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 2.3 MiB/s wr, 149 op/s 2024-04-20T21:38:38.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:38 smithi022 ceph-mon[16843]: pgmap v147: 97 pgs: 97 active+clean; 193 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 2.3 MiB/s wr, 149 op/s 2024-04-20T21:38:39.186 INFO:tasks.workunit.client.0.smithi022.stdout: 1 351912 9.63 MB/sec execute 67 sec latency 19.434 ms 2024-04-20T21:38:40.186 INFO:tasks.workunit.client.0.smithi022.stdout: 1 353916 9.65 MB/sec execute 68 sec latency 13.465 ms 2024-04-20T21:38:40.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:40 smithi122 ceph-mon[22147]: pgmap v148: 97 pgs: 97 active+clean; 193 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 2.3 MiB/s wr, 149 op/s 2024-04-20T21:38:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:40 smithi033 ceph-mon[21626]: pgmap v148: 97 pgs: 97 active+clean; 193 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 2.3 MiB/s wr, 149 op/s 2024-04-20T21:38:40.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:40 smithi022 ceph-mon[16843]: pgmap v148: 97 pgs: 97 active+clean; 193 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 2.3 MiB/s wr, 149 op/s 2024-04-20T21:38:41.186 INFO:tasks.workunit.client.0.smithi022.stdout: 1 355361 9.63 MB/sec execute 69 sec latency 24.554 ms 2024-04-20T21:38:42.186 INFO:tasks.workunit.client.0.smithi022.stdout: 1 357130 9.59 MB/sec execute 70 sec latency 49.296 ms 2024-04-20T21:38:42.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:42 smithi122 ceph-mon[22147]: pgmap v149: 97 pgs: 97 active+clean; 198 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 219 op/s 2024-04-20T21:38:42.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:42 smithi033 ceph-mon[21626]: pgmap v149: 97 pgs: 97 active+clean; 198 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 219 op/s 2024-04-20T21:38:42.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:42 smithi022 ceph-mon[16843]: pgmap v149: 97 pgs: 97 active+clean; 198 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 219 op/s 2024-04-20T21:38:43.186 INFO:tasks.workunit.client.0.smithi022.stdout: 1 358707 9.62 MB/sec execute 71 sec latency 33.664 ms 2024-04-20T21:38:44.187 INFO:tasks.workunit.client.0.smithi022.stdout: 1 361022 9.64 MB/sec execute 72 sec latency 10.730 ms 2024-04-20T21:38:44.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:44 smithi122 ceph-mon[22147]: pgmap v150: 97 pgs: 97 active+clean; 198 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.3 MiB/s wr, 143 op/s 2024-04-20T21:38:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:44 smithi033 ceph-mon[21626]: pgmap v150: 97 pgs: 97 active+clean; 198 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.3 MiB/s wr, 143 op/s 2024-04-20T21:38:44.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:44 smithi022 ceph-mon[16843]: pgmap v150: 97 pgs: 97 active+clean; 198 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.3 MiB/s wr, 143 op/s 2024-04-20T21:38:45.187 INFO:tasks.workunit.client.0.smithi022.stdout: 1 362675 9.63 MB/sec execute 73 sec latency 15.229 ms 2024-04-20T21:38:46.187 INFO:tasks.workunit.client.0.smithi022.stdout: 1 365155 9.69 MB/sec execute 74 sec latency 18.777 ms 2024-04-20T21:38:46.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:46 smithi122 ceph-mon[22147]: pgmap v151: 97 pgs: 97 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 220 op/s 2024-04-20T21:38:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:46 smithi033 ceph-mon[21626]: pgmap v151: 97 pgs: 97 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 220 op/s 2024-04-20T21:38:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:46 smithi022 ceph-mon[16843]: pgmap v151: 97 pgs: 97 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 220 op/s 2024-04-20T21:38:47.187 INFO:tasks.workunit.client.0.smithi022.stdout: 1 366252 9.63 MB/sec execute 75 sec latency 83.753 ms 2024-04-20T21:38:48.187 INFO:tasks.workunit.client.0.smithi022.stdout: 1 368352 9.66 MB/sec execute 76 sec latency 32.962 ms 2024-04-20T21:38:48.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:48 smithi122 ceph-mon[22147]: pgmap v152: 97 pgs: 97 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-04-20T21:38:48.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:48 smithi033 ceph-mon[21626]: pgmap v152: 97 pgs: 97 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-04-20T21:38:48.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:38:48.615 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:38:48.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:48 smithi022 ceph-mon[16843]: pgmap v152: 97 pgs: 97 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-04-20T21:38:48.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.842+0000 7f7fddc00640 1 -- 172.21.15.22:0/3046991913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd810e6a0 msgr2=0x7f7fd8110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:48.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.842+0000 7f7fddc00640 1 --2- 172.21.15.22:0/3046991913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd810e6a0 0x7f7fd8110ab0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009900 tx=0x7f7fd002bcc0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.843+0000 7f7fddc00640 1 -- 172.21.15.22:0/3046991913 shutdown_connections 2024-04-20T21:38:48.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.843+0000 7f7fddc00640 1 --2- 172.21.15.22:0/3046991913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd810e6a0 0x7f7fd8110ab0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.843+0000 7f7fddc00640 1 --2- 172.21.15.22:0/3046991913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd8074c50 0x7f7fd810e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.843+0000 7f7fddc00640 1 --2- 172.21.15.22:0/3046991913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd8074330 0x7f7fd8074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.843+0000 7f7fddc00640 1 -- 172.21.15.22:0/3046991913 >> 172.21.15.22:0/3046991913 conn(0x7f7fd80fd930 msgr2=0x7f7fd80ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:48.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.843+0000 7f7fddc00640 1 -- 172.21.15.22:0/3046991913 shutdown_connections 2024-04-20T21:38:48.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.843+0000 7f7fddc00640 1 -- 172.21.15.22:0/3046991913 wait complete. 2024-04-20T21:38:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.844+0000 7f7fddc00640 1 Processor -- start 2024-04-20T21:38:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.844+0000 7f7fddc00640 1 -- start start 2024-04-20T21:38:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.844+0000 7f7fddc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd8074330 0x7f7fd81a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.844+0000 7f7fddc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd8074c50 0x7f7fd81a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.844+0000 7f7fddc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd810e6a0 0x7f7fd819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.844+0000 7f7fddc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8114410 con 0x7f7fd8074330 2024-04-20T21:38:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.844+0000 7f7fddc00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8114290 con 0x7f7fd810e6a0 2024-04-20T21:38:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.844+0000 7f7fddc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8114590 con 0x7f7fd8074c50 2024-04-20T21:38:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.845+0000 7f7fd7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd8074c50 0x7f7fd81a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.845+0000 7f7fd7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd8074c50 0x7f7fd81a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:40960/0 (socket says 172.21.15.22:40960) 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.845+0000 7f7fdd200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd810e6a0 0x7f7fd819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.845+0000 7f7fd7400640 1 -- 172.21.15.22:0/2272768916 learned_addr learned my addr 172.21.15.22:0/2272768916 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.845+0000 7f7fd7e00640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd8074330 0x7f7fd81a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.845+0000 7f7fd7400640 1 -- 172.21.15.22:0/2272768916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd810e6a0 msgr2=0x7f7fd819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.845+0000 7f7fd7400640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd810e6a0 0x7f7fd819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.845+0000 7f7fd7400640 1 -- 172.21.15.22:0/2272768916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd8074330 msgr2=0x7f7fd81a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.845+0000 7f7fd7400640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd8074330 0x7f7fd81a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.845+0000 7f7fd7400640 1 -- 172.21.15.22:0/2272768916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd819afb0 con 0x7f7fd8074c50 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.846+0000 7f7fd7400640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd8074c50 0x7f7fd81a0b70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000e9f0 tx=0x7f7fc000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.846+0000 7f7fd4c00640 1 -- 172.21.15.22:0/2272768916 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000cd80 con 0x7f7fd8074c50 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.846+0000 7f7fd4c00640 1 -- 172.21.15.22:0/2272768916 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc0010910 con 0x7f7fd8074c50 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.846+0000 7f7fddc00640 1 -- 172.21.15.22:0/2272768916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd819b2a0 con 0x7f7fd8074c50 2024-04-20T21:38:48.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.846+0000 7f7fd4c00640 1 -- 172.21.15.22:0/2272768916 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0018990 con 0x7f7fd8074c50 2024-04-20T21:38:48.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.846+0000 7f7fddc00640 1 -- 172.21.15.22:0/2272768916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd81a75b0 con 0x7f7fd8074c50 2024-04-20T21:38:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.851+0000 7f7fe1e34740 1 Processor -- start 2024-04-20T21:38:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.851+0000 7f7fe1e34740 1 -- start start 2024-04-20T21:38:48.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.851+0000 7f7fe1e34740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 0x5597fb8c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.852+0000 7f7fe1e34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.852+0000 7f7fe1e34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 0x5597fb8cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.852+0000 7f7fe1e34740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5597fb82a150 con 0x5597fb8c4380 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.852+0000 7f7fe1e34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5597fb7d0370 con 0x5597fb8cd880 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.852+0000 7f7fe1e34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5597fb826110 con 0x5597fb8c4ca0 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.852+0000 7f7fd4c00640 1 -- 172.21.15.22:0/2272768916 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7fc0010ac0 con 0x7f7fd8074c50 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.852+0000 7f7fdd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 0x5597fb8c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.852+0000 7f7fdd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 0x5597fb8c4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54430/0 (socket says 172.21.15.22:54430) 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.852+0000 7f7fdd200640 1 -- 172.21.15.22:0/1860862979 learned_addr learned my addr 172.21.15.22:0/1860862979 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.852+0000 7f7fd7e00640 1 --2- 172.21.15.22:0/1860862979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.852+0000 7f7fd7400640 1 --2- 172.21.15.22:0/1860862979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 0x5597fb8cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.853+0000 7f7fd7e00640 1 -- 172.21.15.22:0/1860862979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 msgr2=0x5597fb8cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.853+0000 7f7fd7e00640 1 --2- 172.21.15.22:0/1860862979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 0x5597fb8cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.853+0000 7f7fd7e00640 1 -- 172.21.15.22:0/1860862979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 msgr2=0x5597fb8c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.853+0000 7f7fd7e00640 1 --2- 172.21.15.22:0/1860862979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 0x5597fb8c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.853+0000 7f7fd7e00640 1 -- 172.21.15.22:0/1860862979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597fb6067d0 con 0x5597fb8c4ca0 2024-04-20T21:38:48.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.853+0000 7f7fd7e00640 1 --2- 172.21.15.22:0/1860862979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8cd340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009f50 tx=0x7f7fc8010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8987341f70c38838 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:48.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.853+0000 7f7fcd600640 1 -- 172.21.15.22:0/1860862979 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8003b70 con 0x5597fb8c4ca0 2024-04-20T21:38:48.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.853+0000 7f7fcd600640 1 -- 172.21.15.22:0/1860862979 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc8015070 con 0x5597fb8c4ca0 2024-04-20T21:38:48.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.853+0000 7f7fcd600640 1 -- 172.21.15.22:0/1860862979 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80119a0 con 0x5597fb8c4ca0 2024-04-20T21:38:48.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.854+0000 7f7fd4c00640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7fbc0a3da0 0x7f7fbc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.854+0000 7f7fd4c00640 1 -- 172.21.15.22:0/2272768916 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7fc00d0ab0 con 0x7f7fd8074c50 2024-04-20T21:38:48.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.854+0000 7f7fdd200640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7fbc0a3da0 0x7f7fbc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.854+0000 7f7fe1e34740 1 -- 172.21.15.22:0/1860862979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 msgr2=0x5597fb8cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:48.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.854+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/1860862979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8cd340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009f50 tx=0x7f7fc8010860 comp rx=0 tx=0).stop 2024-04-20T21:38:48.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.854+0000 7f7fe1e34740 1 -- 172.21.15.22:0/1860862979 shutdown_connections 2024-04-20T21:38:48.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.854+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/1860862979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 0x5597fb8cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.854+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/1860862979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8cd340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.854+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/1860862979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 0x5597fb8c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.854+0000 7f7fe1e34740 1 -- 172.21.15.22:0/1860862979 >> 172.21.15.22:0/1860862979 conn(0x5597fb8beab0 msgr2=0x5597fb8bee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:48.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.854+0000 7f7fe1e34740 1 -- 172.21.15.22:0/1860862979 shutdown_connections 2024-04-20T21:38:48.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.855+0000 7f7fe1e34740 1 -- 172.21.15.22:0/1860862979 wait complete. 2024-04-20T21:38:48.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.855+0000 7f7fdd200640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7fbc0a3da0 0x7f7fbc0a6260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00096c0 tx=0x7f7fd002e040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:48.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.855+0000 7f7fe1e34740 1 -- start start 2024-04-20T21:38:48.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.855+0000 7f7fe1e34740 1 Processor -- start 2024-04-20T21:38:48.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fe1e34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8c4380 0x5597fb8ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fe1e34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fe1e34740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8cd880 0x5597fb8cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fe1e34740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5597fb8d0670 con 0x5597fb8cd880 2024-04-20T21:38:48.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fe1e34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5597fb7e3740 con 0x5597fb8c4380 2024-04-20T21:38:48.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fe1e34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5597fb8d07f0 con 0x5597fb8c4ca0 2024-04-20T21:38:48.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fe1e34740 10 client.0 _ll_register_callbacks cb 0x5597fb7deb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:48.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fd7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fd7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8c4380 0x5597fb8ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fd7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8c4380 0x5597fb8ca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44390/0 (socket says 172.21.15.22:44390) 2024-04-20T21:38:48.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fdd200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8cd880 0x5597fb8cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fd7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:40980/0 (socket says 172.21.15.22:40980) 2024-04-20T21:38:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.856+0000 7f7fd7e00640 1 -- 172.21.15.22:0/422489698 learned_addr learned my addr 172.21.15.22:0/422489698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fd7400640 1 -- 172.21.15.22:0/422489698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8c4380 msgr2=0x5597fb8ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fd7400640 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8c4380 0x5597fb8ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fd7400640 1 -- 172.21.15.22:0/422489698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8cd880 msgr2=0x5597fb8cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fd7400640 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8cd880 0x5597fb8cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fd7400640 1 -- 172.21.15.22:0/422489698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597fb53c3f0 con 0x5597fb8c4ca0 2024-04-20T21:38:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fdd200640 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8cd880 0x5597fb8cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fd7e00640 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8c4380 0x5597fb8ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fd7400640 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8cae60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00ce1e0 tx=0x7f7fc00d41f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5034da3c3af52236 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fc6000640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:38:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fc6000640 1 -- 172.21.15.22:0/422489698 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc009c570 con 0x5597fb8c4ca0 2024-04-20T21:38:48.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fc6000640 1 -- 172.21.15.22:0/422489698 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7fc0087310 con 0x5597fb8c4ca0 2024-04-20T21:38:48.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fe1e34740 1 -- 172.21.15.22:0/422489698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc8015070 con 0x5597fb8c4ca0 2024-04-20T21:38:48.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fc6000640 1 -- 172.21.15.22:0/422489698 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7fc00bd0b0 con 0x5597fb8c4ca0 2024-04-20T21:38:48.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.857+0000 7f7fe1e34740 10 client.26005 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:48.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.858+0000 7f7fe1e34740 1 -- 172.21.15.22:0/422489698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5597fb4fc0f0 con 0x5597fb8c4ca0 2024-04-20T21:38:48.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.858+0000 7f7fc6000640 1 -- 172.21.15.22:0/422489698 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc00d6080 con 0x5597fb8c4ca0 2024-04-20T21:38:48.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.858+0000 7f7fc6000640 1 -- 172.21.15.22:0/422489698 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f7fc009bfa0 con 0x5597fb8c4ca0 2024-04-20T21:38:48.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.858+0000 7f7fe1e34740 10 client.26005 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:48.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.858+0000 7f7fe1e34740 10 client.26005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.858+0000 7f7fe1e34740 20 client.26005 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.858+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5597fb8ddbb0 0x5597fb8dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.858+0000 7f7fe1e34740 4 client.26005 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.858+0000 7f7fe1e34740 1 -- 172.21.15.22:0/422489698 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597fb837b70 con 0x5597fb8ddbb0 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.858+0000 7f7fd7e00640 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5597fb8ddbb0 0x5597fb8dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.859+0000 7f7fd7e00640 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5597fb8ddbb0 0x5597fb8dffe0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef57a23d07784e1 server_cookie=fb8dc07ec07e75e1 in_seq=0 out_seq=0 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.859+0000 7f7fc6000640 10 client.26005 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.860+0000 7f7fc6000640 1 -- 172.21.15.22:0/422489698 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597fb837b70 con 0x5597fb8ddbb0 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.860+0000 7f7fc6000640 10 client.26005 handle_command_reply: tid=0 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.860+0000 7f7fe1e34740 1 client.26005 shutdown 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.861+0000 7f7fe1e34740 1 -- 172.21.15.22:0/422489698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 msgr2=0x5597fb8cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.861+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8cae60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00ce1e0 tx=0x7f7fc00d41f0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.861+0000 7f7fe1e34740 1 -- 172.21.15.22:0/422489698 shutdown_connections 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.861+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8cd880 0x5597fb8cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.861+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8cae60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.861+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5597fb8ddbb0 0x5597fb8dffe0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.861+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/422489698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8c4380 0x5597fb8ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.861+0000 7f7fe1e34740 1 -- 172.21.15.22:0/422489698 >> 172.21.15.22:0/422489698 conn(0x5597fb8beab0 msgr2=0x5597fb8c0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:48.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.861+0000 7f7fc6000640 0 client.26005 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:48.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.861+0000 7f7fe1e34740 1 -- 172.21.15.22:0/422489698 shutdown_connections 2024-04-20T21:38:48.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.861+0000 7f7fe1e34740 1 -- 172.21.15.22:0/422489698 wait complete. 2024-04-20T21:38:48.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.861+0000 7f7fe1e34740 20 client.26005 trim_cache size 0 max 16384 2024-04-20T21:38:48.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.881+0000 7f7fe1e34740 1 Processor -- start 2024-04-20T21:38:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.881+0000 7f7fe1e34740 1 -- start start 2024-04-20T21:38:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.881+0000 7f7fe1e34740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 0x5597fb8cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.881+0000 7f7fe1e34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8d9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.881+0000 7f7fe1e34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 0x5597fb8b99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.881+0000 7f7fe1e34740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81ade00 con 0x5597fb8c4380 2024-04-20T21:38:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.881+0000 7f7fe1e34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81b0680 con 0x5597fb8cd880 2024-04-20T21:38:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.881+0000 7f7fe1e34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81ae470 con 0x5597fb8c4ca0 2024-04-20T21:38:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.882+0000 7f7fd7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 0x5597fb8cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.882+0000 7f7fdd200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 0x5597fb8b99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.882+0000 7f7fd7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8d9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.882+0000 7f7fd7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 0x5597fb8cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54440/0 (socket says 172.21.15.22:54440) 2024-04-20T21:38:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.882+0000 7f7fd7e00640 1 -- 172.21.15.22:0/3395009276 learned_addr learned my addr 172.21.15.22:0/3395009276 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.882+0000 7f7fd7400640 1 -- 172.21.15.22:0/3395009276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 msgr2=0x5597fb8b99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.882+0000 7f7fd7400640 1 --2- 172.21.15.22:0/3395009276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 0x5597fb8b99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.882+0000 7f7fd7400640 1 -- 172.21.15.22:0/3395009276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 msgr2=0x5597fb8cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.882+0000 7f7fd7400640 1 --2- 172.21.15.22:0/3395009276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 0x5597fb8cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.882+0000 7f7fd7400640 1 -- 172.21.15.22:0/3395009276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597fb8c98f0 con 0x5597fb8c4ca0 2024-04-20T21:38:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.882+0000 7f7fd7e00640 1 --2- 172.21.15.22:0/3395009276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 0x5597fb8cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.882+0000 7f7fd7400640 1 --2- 172.21.15.22:0/3395009276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8d9530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7fc009c9c0 tx=0x7f7fc00d0e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e22727b868a85d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.883+0000 7f7fccc00640 1 -- 172.21.15.22:0/3395009276 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc009c390 con 0x5597fb8c4ca0 2024-04-20T21:38:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.883+0000 7f7fccc00640 1 -- 172.21.15.22:0/3395009276 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc8015070 con 0x5597fb8c4ca0 2024-04-20T21:38:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.883+0000 7f7fccc00640 1 -- 172.21.15.22:0/3395009276 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc009ca30 con 0x5597fb8c4ca0 2024-04-20T21:38:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.884+0000 7f7fe1e34740 1 -- 172.21.15.22:0/3395009276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 msgr2=0x5597fb8d9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.884+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/3395009276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8d9530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7fc009c9c0 tx=0x7f7fc00d0e40 comp rx=0 tx=0).stop 2024-04-20T21:38:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.884+0000 7f7fe1e34740 1 -- 172.21.15.22:0/3395009276 shutdown_connections 2024-04-20T21:38:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.884+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/3395009276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 0x5597fb8b99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.884+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/3395009276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4ca0 0x5597fb8d9530 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.884+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/3395009276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4380 0x5597fb8cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.884+0000 7f7fe1e34740 1 -- 172.21.15.22:0/3395009276 >> 172.21.15.22:0/3395009276 conn(0x5597fb8beab0 msgr2=0x5597fb8ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.884+0000 7f7fe1e34740 1 -- 172.21.15.22:0/3395009276 shutdown_connections 2024-04-20T21:38:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.884+0000 7f7fe1e34740 1 -- 172.21.15.22:0/3395009276 wait complete. 2024-04-20T21:38:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.884+0000 7f7fe1e34740 1 -- start start 2024-04-20T21:38:48.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.884+0000 7f7fe1e34740 1 Processor -- start 2024-04-20T21:38:48.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.885+0000 7f7fe1e34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4380 0x5597fb8c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.885+0000 7f7fe1e34740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4ca0 0x5597fb8c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.885+0000 7f7fe1e34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 0x5597fb8c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.885+0000 7f7fe1e34740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81aeb40 con 0x5597fb8c4ca0 2024-04-20T21:38:48.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.885+0000 7f7fe1e34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81affb0 con 0x5597fb8cd880 2024-04-20T21:38:48.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.885+0000 7f7fe1e34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81af210 con 0x5597fb8c4380 2024-04-20T21:38:48.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.885+0000 7f7fe1e34740 10 client.0 _ll_register_callbacks cb 0x7f7fc00d42f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:48.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.885+0000 7f7fd7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4380 0x5597fb8c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.885+0000 7f7fd7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4ca0 0x5597fb8c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.885+0000 7f7fd7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4380 0x5597fb8c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:40986/0 (socket says 172.21.15.22:40986) 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.885+0000 7f7fd7e00640 1 -- 172.21.15.22:0/2368178671 learned_addr learned my addr 172.21.15.22:0/2368178671 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.885+0000 7f7fdd200640 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 0x5597fb8c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.886+0000 7f7fd7400640 1 -- 172.21.15.22:0/2368178671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4380 msgr2=0x5597fb8c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.886+0000 7f7fd7400640 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4380 0x5597fb8c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.886+0000 7f7fd7400640 1 -- 172.21.15.22:0/2368178671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 msgr2=0x5597fb8c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.886+0000 7f7fd7400640 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 0x5597fb8c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.886+0000 7f7fd7400640 1 -- 172.21.15.22:0/2368178671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597fb9431d0 con 0x5597fb8c4ca0 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.886+0000 7f7fd7e00640 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4380 0x5597fb8c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.886+0000 7f7fd7400640 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4ca0 0x5597fb8c7bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x5597fb8d0340 tx=0x7f7fc0087340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124991ed6ef10b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.886+0000 7f7fc6000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.886+0000 7f7fc6000640 1 -- 172.21.15.22:0/2368178671 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc009c390 con 0x5597fb8c4ca0 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.886+0000 7f7fc6000640 1 -- 172.21.15.22:0/2368178671 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7fc0010430 con 0x5597fb8c4ca0 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.886+0000 7f7fe1e34740 1 -- 172.21.15.22:0/2368178671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc8015070 con 0x5597fb8c4ca0 2024-04-20T21:38:48.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.887+0000 7f7fc6000640 1 -- 172.21.15.22:0/2368178671 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7fc009d130 con 0x5597fb8c4ca0 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.887+0000 7f7fe1e34740 10 client.15858 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.887+0000 7f7fe1e34740 1 -- 172.21.15.22:0/2368178671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5597fb9431d0 con 0x5597fb8c4ca0 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.887+0000 7f7fc6000640 1 -- 172.21.15.22:0/2368178671 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc8015070 con 0x5597fb8c4ca0 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.887+0000 7f7fc6000640 1 -- 172.21.15.22:0/2368178671 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f7fc00a1f70 con 0x5597fb8c4ca0 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.887+0000 7f7fe1e34740 10 client.15858 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.887+0000 7f7fe1e34740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.887+0000 7f7fe1e34740 20 client.15858 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.887+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5597fb944760 0x5597fb946b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.887+0000 7f7fe1e34740 4 client.15858 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.887+0000 7f7fe1e34740 1 -- 172.21.15.22:0/2368178671 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597fb8ba170 con 0x5597fb944760 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.887+0000 7f7fd7e00640 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5597fb944760 0x5597fb946b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.888+0000 7f7fd7e00640 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5597fb944760 0x5597fb946b50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c6c524d059ae37 server_cookie=54bff0eeb7cae6c in_seq=0 out_seq=0 2024-04-20T21:38:48.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.888+0000 7f7fc6000640 10 client.15858 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:48.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fc6000640 1 -- 172.21.15.22:0/2368178671 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5597fb8ba170 con 0x5597fb944760 2024-04-20T21:38:48.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fc6000640 10 client.15858 handle_command_reply: tid=0 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fe1e34740 1 client.15858 shutdown 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fe1e34740 1 -- 172.21.15.22:0/2368178671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4ca0 msgr2=0x5597fb8c7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4ca0 0x5597fb8c7bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x5597fb8d0340 tx=0x7f7fc0087340 comp rx=0 tx=0).stop 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fe1e34740 1 -- 172.21.15.22:0/2368178671 shutdown_connections 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5597fb944760 0x5597fb946b50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597fb8cd880 0x5597fb8c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597fb8c4ca0 0x5597fb8c7bf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fe1e34740 1 --2- 172.21.15.22:0/2368178671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5597fb8c4380 0x5597fb8c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fc6000640 0 client.15858 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fe1e34740 1 -- 172.21.15.22:0/2368178671 >> 172.21.15.22:0/2368178671 conn(0x5597fb8beab0 msgr2=0x5597fb8cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fe1e34740 1 -- 172.21.15.22:0/2368178671 shutdown_connections 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.889+0000 7f7fe1e34740 1 -- 172.21.15.22:0/2368178671 wait complete. 2024-04-20T21:38:48.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.890+0000 7f7fe1e34740 20 client.15858 trim_cache size 0 max 16384 2024-04-20T21:38:48.892 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:38:48.892 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:38:48.892 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:38:48.892 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:38:48.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.890+0000 7f7fddc00640 1 -- 172.21.15.22:0/2272768916 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7fbc0a3da0 msgr2=0x7f7fbc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:48.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.890+0000 7f7fddc00640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7fbc0a3da0 0x7f7fbc0a6260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00096c0 tx=0x7f7fd002e040 comp rx=0 tx=0).stop 2024-04-20T21:38:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.890+0000 7f7fddc00640 1 -- 172.21.15.22:0/2272768916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd8074c50 msgr2=0x7f7fd81a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.890+0000 7f7fddc00640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd8074c50 0x7f7fd81a0b70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000e9f0 tx=0x7f7fc000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.890+0000 7f7fddc00640 1 -- 172.21.15.22:0/2272768916 shutdown_connections 2024-04-20T21:38:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.890+0000 7f7fddc00640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7fbc0a3da0 0x7f7fbc0a6260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.890+0000 7f7fddc00640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd810e6a0 0x7f7fd819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.890+0000 7f7fddc00640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd8074c50 0x7f7fd81a0b70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.890+0000 7f7fddc00640 1 --2- 172.21.15.22:0/2272768916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fd8074330 0x7f7fd81a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.890+0000 7f7fddc00640 1 -- 172.21.15.22:0/2272768916 >> 172.21.15.22:0/2272768916 conn(0x7f7fd80fd930 msgr2=0x7f7fd80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.890+0000 7f7fddc00640 1 -- 172.21.15.22:0/2272768916 shutdown_connections 2024-04-20T21:38:48.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:48.891+0000 7f7fddc00640 1 -- 172.21.15.22:0/2272768916 wait complete. 2024-04-20T21:38:48.907 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:38:48.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:49.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.180+0000 7f9a77a00640 1 -- 172.21.15.22:0/4135829583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a70100d70 msgr2=0x7f9a70101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.180+0000 7f9a77a00640 1 --2- 172.21.15.22:0/4135829583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a70100d70 0x7f9a70101150 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009900 tx=0x7f9a6402bc30 comp rx=0 tx=0).stop 2024-04-20T21:38:49.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.180+0000 7f9a77a00640 1 -- 172.21.15.22:0/4135829583 shutdown_connections 2024-04-20T21:38:49.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.180+0000 7f9a77a00640 1 --2- 172.21.15.22:0/4135829583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a7010e920 0x7f9a70110d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.180+0000 7f9a77a00640 1 --2- 172.21.15.22:0/4135829583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a70101690 0x7f9a7010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.180+0000 7f9a77a00640 1 --2- 172.21.15.22:0/4135829583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a70100d70 0x7f9a70101150 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.180+0000 7f9a77a00640 1 -- 172.21.15.22:0/4135829583 >> 172.21.15.22:0/4135829583 conn(0x7f9a70077f60 msgr2=0x7f9a70078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:49.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.180+0000 7f9a77a00640 1 -- 172.21.15.22:0/4135829583 shutdown_connections 2024-04-20T21:38:49.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.181+0000 7f9a77a00640 1 -- 172.21.15.22:0/4135829583 wait complete. 2024-04-20T21:38:49.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.181+0000 7f9a77a00640 1 Processor -- start 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.182+0000 7f9a77a00640 1 -- start start 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.182+0000 7f9a77a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a70100d70 0x7f9a701a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.182+0000 7f9a77a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a70101690 0x7f9a701a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.182+0000 7f9a77a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a7010e920 0x7f9a7019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.182+0000 7f9a77a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9a70114590 con 0x7f9a7010e920 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.182+0000 7f9a77a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a70114410 con 0x7f9a70100d70 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.182+0000 7f9a77a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9a70114710 con 0x7f9a70101690 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.182+0000 7f9a75000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a70100d70 0x7f9a701a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.182+0000 7f9a75000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a70100d70 0x7f9a701a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44428/0 (socket says 172.21.15.22:44428) 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.182+0000 7f9a75000640 1 -- 172.21.15.22:0/3305409095 learned_addr learned my addr 172.21.15.22:0/3305409095 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.182+0000 7f9a75a00640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a7010e920 0x7f9a7019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.182+0000 7f9a6fe00640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a70101690 0x7f9a701a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.183+0000 7f9a75000640 1 -- 172.21.15.22:0/3305409095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a70101690 msgr2=0x7f9a701a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.183+0000 7f9a75000640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a70101690 0x7f9a701a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.183+0000 7f9a75000640 1 -- 172.21.15.22:0/3305409095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a7010e920 msgr2=0x7f9a7019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.183+0000 7f9a75000640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a7010e920 0x7f9a7019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.183+0000 7f9a75000640 1 -- 172.21.15.22:0/3305409095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7019b010 con 0x7f9a70100d70 2024-04-20T21:38:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.183+0000 7f9a75000640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a70100d70 0x7f9a701a06e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9a6402c240 tx=0x7f9a640042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.183+0000 7f9a6d600640 1 -- 172.21.15.22:0/3305409095 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a64043020 con 0x7f9a70100d70 2024-04-20T21:38:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.183+0000 7f9a6d600640 1 -- 172.21.15.22:0/3305409095 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a640054e0 con 0x7f9a70100d70 2024-04-20T21:38:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.183+0000 7f9a6d600640 1 -- 172.21.15.22:0/3305409095 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a64003b50 con 0x7f9a70100d70 2024-04-20T21:38:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.183+0000 7f9a77a00640 1 -- 172.21.15.22:0/3305409095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7019b2a0 con 0x7f9a70100d70 2024-04-20T21:38:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.183+0000 7f9a77a00640 1 -- 172.21.15.22:0/3305409095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a701a75a0 con 0x7f9a70100d70 2024-04-20T21:38:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.184+0000 7f9a77a00640 1 -- 172.21.15.22:0/3305409095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a70068d80 con 0x7f9a70100d70 2024-04-20T21:38:49.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.187+0000 7f9a6d600640 1 -- 172.21.15.22:0/3305409095 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9a64046020 con 0x7f9a70100d70 2024-04-20T21:38:49.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.188+0000 7f9a6d600640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9a580a3d20 0x7f9a580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.187 INFO:tasks.workunit.client.0.smithi022.stdout: 1 369757 9.63 MB/sec execute 77 sec latency 13.291 ms 2024-04-20T21:38:49.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.188+0000 7f9a6d600640 1 -- 172.21.15.22:0/3305409095 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f9a640f2760 con 0x7f9a70100d70 2024-04-20T21:38:49.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.188+0000 7f9a6fe00640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9a580a3d20 0x7f9a580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.189+0000 7f9a6fe00640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9a580a3d20 0x7f9a580a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9a70100880 tx=0x7f9a5c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:49.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.192+0000 7f9a6d600640 1 -- 172.21.15.22:0/3305409095 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9a6404d020 con 0x7f9a70100d70 2024-04-20T21:38:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.454+0000 7f9a77a00640 1 -- 172.21.15.22:0/3305409095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a70109ab0 con 0x7f9a70100d70 2024-04-20T21:38:49.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.455+0000 7f9a6d600640 1 -- 172.21.15.22:0/3305409095 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f9a640beef0 con 0x7f9a70100d70 2024-04-20T21:38:49.454 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:38:49.455 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:38:49.455 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:38:49.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.458+0000 7f9a77a00640 1 -- 172.21.15.22:0/3305409095 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9a580a3d20 msgr2=0x7f9a580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.458+0000 7f9a77a00640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9a580a3d20 0x7f9a580a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9a70100880 tx=0x7f9a5c005f90 comp rx=0 tx=0).stop 2024-04-20T21:38:49.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.458+0000 7f9a77a00640 1 -- 172.21.15.22:0/3305409095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a70100d70 msgr2=0x7f9a701a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.458+0000 7f9a77a00640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a70100d70 0x7f9a701a06e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9a6402c240 tx=0x7f9a640042f0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.458+0000 7f9a77a00640 1 -- 172.21.15.22:0/3305409095 shutdown_connections 2024-04-20T21:38:49.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.458+0000 7f9a77a00640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9a580a3d20 0x7f9a580a61e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.458+0000 7f9a77a00640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a7010e920 0x7f9a7019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.458+0000 7f9a77a00640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a70101690 0x7f9a701a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.458+0000 7f9a77a00640 1 --2- 172.21.15.22:0/3305409095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a70100d70 0x7f9a701a06e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.458+0000 7f9a77a00640 1 -- 172.21.15.22:0/3305409095 >> 172.21.15.22:0/3305409095 conn(0x7f9a70077f60 msgr2=0x7f9a7010fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:49.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.458+0000 7f9a77a00640 1 -- 172.21.15.22:0/3305409095 shutdown_connections 2024-04-20T21:38:49.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.458+0000 7f9a77a00640 1 -- 172.21.15.22:0/3305409095 wait complete. 2024-04-20T21:38:49.477 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:38:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.692+0000 7ff43b200640 1 -- 172.21.15.22:0/3115732045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff43c104280 msgr2=0x7ff43c104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.692+0000 7ff43b200640 1 --2- 172.21.15.22:0/3115732045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff43c104280 0x7ff43c104660 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009900 tx=0x7ff42c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:38:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.693+0000 7ff43b200640 1 -- 172.21.15.22:0/3115732045 shutdown_connections 2024-04-20T21:38:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.693+0000 7ff43b200640 1 --2- 172.21.15.22:0/3115732045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c109350 0x7ff43c110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.693+0000 7ff43b200640 1 --2- 172.21.15.22:0/3115732045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c104c30 0x7ff43c108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.693+0000 7ff43b200640 1 --2- 172.21.15.22:0/3115732045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff43c104280 0x7ff43c104660 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.693+0000 7ff43b200640 1 -- 172.21.15.22:0/3115732045 >> 172.21.15.22:0/3115732045 conn(0x7ff43c0ffaa0 msgr2=0x7ff43c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.693+0000 7ff43b200640 1 -- 172.21.15.22:0/3115732045 shutdown_connections 2024-04-20T21:38:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.693+0000 7ff43b200640 1 -- 172.21.15.22:0/3115732045 wait complete. 2024-04-20T21:38:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.694+0000 7ff43b200640 1 Processor -- start 2024-04-20T21:38:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.694+0000 7ff43b200640 1 -- start start 2024-04-20T21:38:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.694+0000 7ff43b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c104280 0x7ff43c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.694+0000 7ff43b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c109350 0x7ff43c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.694+0000 7ff43b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff43c19a790 0x7ff43c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.694+0000 7ff43b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c114450 con 0x7ff43c104280 2024-04-20T21:38:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.694+0000 7ff43b200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1142d0 con 0x7ff43c109350 2024-04-20T21:38:49.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.694+0000 7ff43b200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1145d0 con 0x7ff43c19a790 2024-04-20T21:38:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.695+0000 7ff439e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c104280 0x7ff43c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.695+0000 7ff43a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff43c19a790 0x7ff43c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.695+0000 7ff439e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c104280 0x7ff43c1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54492/0 (socket says 172.21.15.22:54492) 2024-04-20T21:38:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.695+0000 7ff439e00640 1 -- 172.21.15.22:0/3679054987 learned_addr learned my addr 172.21.15.22:0/3679054987 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.695+0000 7ff439400640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c109350 0x7ff43c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.695+0000 7ff43a800640 1 -- 172.21.15.22:0/3679054987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c109350 msgr2=0x7ff43c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.695+0000 7ff43a800640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c109350 0x7ff43c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.695+0000 7ff43a800640 1 -- 172.21.15.22:0/3679054987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c104280 msgr2=0x7ff43c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.695+0000 7ff43a800640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c104280 0x7ff43c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.695+0000 7ff43a800640 1 -- 172.21.15.22:0/3679054987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c19b320 con 0x7ff43c19a790 2024-04-20T21:38:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.695+0000 7ff439e00640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c104280 0x7ff43c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.696+0000 7ff43a800640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff43c19a790 0x7ff43c19ac40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff434009fd0 tx=0x7ff43400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.696+0000 7ff432a00640 1 -- 172.21.15.22:0/3679054987 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43400ce20 con 0x7ff43c19a790 2024-04-20T21:38:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.696+0000 7ff432a00640 1 -- 172.21.15.22:0/3679054987 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff434005020 con 0x7ff43c19a790 2024-04-20T21:38:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.696+0000 7ff43b200640 1 -- 172.21.15.22:0/3679054987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c19b610 con 0x7ff43c19a790 2024-04-20T21:38:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.696+0000 7ff432a00640 1 -- 172.21.15.22:0/3679054987 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434010720 con 0x7ff43c19a790 2024-04-20T21:38:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.696+0000 7ff43b200640 1 -- 172.21.15.22:0/3679054987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c1a77f0 con 0x7ff43c19a790 2024-04-20T21:38:49.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.699+0000 7ff432a00640 1 -- 172.21.15.22:0/3679054987 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff434005540 con 0x7ff43c19a790 2024-04-20T21:38:49.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.699+0000 7ff4439c3740 1 Processor -- start 2024-04-20T21:38:49.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.700+0000 7ff4439c3740 1 -- start start 2024-04-20T21:38:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.700+0000 7ff4439c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cebe170 0x562c0cebe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.700+0000 7ff4439c3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebea90 0x562c0cec7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.700+0000 7ff4439c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 0x562c0cec9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.700+0000 7ff4439c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562c0ce23050 con 0x562c0cebe170 2024-04-20T21:38:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.700+0000 7ff4439c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562c0cdc9490 con 0x562c0cec7670 2024-04-20T21:38:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.700+0000 7ff4439c3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562c0ce1f010 con 0x562c0cebea90 2024-04-20T21:38:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.700+0000 7ff439e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebea90 0x562c0cec7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.700+0000 7ff439e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebea90 0x562c0cec7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:41036/0 (socket says 172.21.15.22:41036) 2024-04-20T21:38:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.700+0000 7ff439e00640 1 -- 172.21.15.22:0/2737913190 learned_addr learned my addr 172.21.15.22:0/2737913190 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.701+0000 7ff439400640 1 --2- 172.21.15.22:0/2737913190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cebe170 0x562c0cebe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.701+0000 7ff439e00640 1 -- 172.21.15.22:0/2737913190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 msgr2=0x562c0cec9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.701+0000 7ff439400640 1 --2- 172.21.15.22:0/2737913190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 0x562c0cec9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.701+0000 7ff439e00640 1 --2- 172.21.15.22:0/2737913190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 0x562c0cec9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.701+0000 7ff439e00640 1 -- 172.21.15.22:0/2737913190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cebe170 msgr2=0x562c0cebe550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.701+0000 7ff439e00640 1 --2- 172.21.15.22:0/2737913190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cebe170 0x562c0cebe550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.701+0000 7ff439e00640 1 -- 172.21.15.22:0/2737913190 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c0cdd1070 con 0x562c0cebea90 2024-04-20T21:38:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.701+0000 7ff439400640 1 --2- 172.21.15.22:0/2737913190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 0x562c0cec9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:49.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.701+0000 7ff432a00640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff4200a3da0 0x7ff4200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.701+0000 7ff439e00640 1 --2- 172.21.15.22:0/2737913190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebea90 0x562c0cec7130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009620 tx=0x7ff42c009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e60bf7251ef11dcd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.701+0000 7ff432a00640 1 -- 172.21.15.22:0/3679054987 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7ff4340d0e50 con 0x7ff43c19a790 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.702+0000 7ff439400640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff4200a3da0 0x7ff4200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.702+0000 7ff427000640 1 -- 172.21.15.22:0/2737913190 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c045070 con 0x562c0cebea90 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.702+0000 7ff427000640 1 -- 172.21.15.22:0/2737913190 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff42c02c690 con 0x562c0cebea90 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.702+0000 7ff427000640 1 -- 172.21.15.22:0/2737913190 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c004f60 con 0x562c0cebea90 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.703+0000 7ff4439c3740 1 -- 172.21.15.22:0/2737913190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebea90 msgr2=0x562c0cec7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.703+0000 7ff4439c3740 1 --2- 172.21.15.22:0/2737913190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebea90 0x562c0cec7130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009620 tx=0x7ff42c009650 comp rx=0 tx=0).stop 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.703+0000 7ff439400640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff4200a3da0 0x7ff4200a6260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff428009710 tx=0x7ff428005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.703+0000 7ff4439c3740 1 -- 172.21.15.22:0/2737913190 shutdown_connections 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.703+0000 7ff4439c3740 1 --2- 172.21.15.22:0/2737913190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 0x562c0cec9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.703+0000 7ff4439c3740 1 --2- 172.21.15.22:0/2737913190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebea90 0x562c0cec7130 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.703+0000 7ff4439c3740 1 --2- 172.21.15.22:0/2737913190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cebe170 0x562c0cebe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.703+0000 7ff4439c3740 1 -- 172.21.15.22:0/2737913190 >> 172.21.15.22:0/2737913190 conn(0x562c0ceb88a0 msgr2=0x562c0ceb8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:49.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.703+0000 7ff4439c3740 1 -- 172.21.15.22:0/2737913190 shutdown_connections 2024-04-20T21:38:49.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.703+0000 7ff4439c3740 1 -- 172.21.15.22:0/2737913190 wait complete. 2024-04-20T21:38:49.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.704+0000 7ff4439c3740 1 -- start start 2024-04-20T21:38:49.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.704+0000 7ff4439c3740 1 Processor -- start 2024-04-20T21:38:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.704+0000 7ff4439c3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cec4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.704+0000 7ff4439c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cebea90 0x562c0cec4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.704+0000 7ff4439c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 0x562c0cec5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.704+0000 7ff4439c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562c0ceca400 con 0x562c0cebea90 2024-04-20T21:38:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.704+0000 7ff4439c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562c0cddc860 con 0x562c0cec7670 2024-04-20T21:38:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.704+0000 7ff4439c3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562c0ceca580 con 0x562c0cebe170 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.704+0000 7ff4439c3740 10 client.0 _ll_register_callbacks cb 0x562c0cdaafe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff43a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 0x562c0cec5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff439e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cec4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff43a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 0x562c0cec5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44482/0 (socket says 172.21.15.22:44482) 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff43a800640 1 -- 172.21.15.22:0/3876562885 learned_addr learned my addr 172.21.15.22:0/3876562885 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff439400640 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cebea90 0x562c0cec4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff439e00640 1 -- 172.21.15.22:0/3876562885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 msgr2=0x562c0cec5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff439e00640 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 0x562c0cec5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff439e00640 1 -- 172.21.15.22:0/3876562885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cebea90 msgr2=0x562c0cec4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff439e00640 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cebea90 0x562c0cec4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff439e00640 1 -- 172.21.15.22:0/3876562885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c0cde0950 con 0x562c0cebe170 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff43a800640 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 0x562c0cec5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff439e00640 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cec4690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0095f0 tx=0x7ff42c02c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7bfe815313e8e54 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:49.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.705+0000 7ff41b400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.706+0000 7ff41b400640 1 -- 172.21.15.22:0/3876562885 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c045040 con 0x562c0cebe170 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.706+0000 7ff41b400640 1 -- 172.21.15.22:0/3876562885 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7ff42c005530 con 0x562c0cebe170 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.706+0000 7ff4439c3740 1 -- 172.21.15.22:0/3876562885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562c0ce30a70 con 0x562c0cebe170 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.706+0000 7ff41b400640 1 -- 172.21.15.22:0/3876562885 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff42c05b050 con 0x562c0cebe170 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.706+0000 7ff4439c3740 10 client.26029 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.706+0000 7ff4439c3740 1 -- 172.21.15.22:0/3876562885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562c0cdae820 con 0x562c0cebe170 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.706+0000 7ff41b400640 1 -- 172.21.15.22:0/3876562885 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562c0ce30a70 con 0x562c0cebe170 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.707+0000 7ff41b400640 1 -- 172.21.15.22:0/3876562885 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7ff42c03f560 con 0x562c0cebe170 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.707+0000 7ff4439c3740 10 client.26029 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.707+0000 7ff4439c3740 10 client.26029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.707+0000 7ff4439c3740 20 client.26029 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.707+0000 7ff4439c3740 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562c0ced73e0 0x562c0ced97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.707+0000 7ff4439c3740 4 client.26029 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.707+0000 7ff4439c3740 1 -- 172.21.15.22:0/3876562885 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c0ce2a8e0 con 0x562c0ced73e0 2024-04-20T21:38:49.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.707+0000 7ff439400640 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562c0ced73e0 0x562c0ced97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.708+0000 7ff439400640 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562c0ced73e0 0x562c0ced97d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8de2fe862c289f server_cookie=b4deb73740252343 in_seq=0 out_seq=0 2024-04-20T21:38:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.708+0000 7ff41b400640 10 client.26029 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff41b400640 1 -- 172.21.15.22:0/3876562885 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c0ce2a8e0 con 0x562c0ced73e0 2024-04-20T21:38:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff41b400640 10 client.26029 handle_command_reply: tid=0 2024-04-20T21:38:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff4439c3740 1 client.26029 shutdown 2024-04-20T21:38:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff4439c3740 1 -- 172.21.15.22:0/3876562885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 msgr2=0x562c0cec4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff4439c3740 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cec4690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0095f0 tx=0x7ff42c02c950 comp rx=0 tx=0).stop 2024-04-20T21:38:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff4439c3740 1 -- 172.21.15.22:0/3876562885 shutdown_connections 2024-04-20T21:38:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff4439c3740 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562c0ced73e0 0x562c0ced97d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff4439c3740 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cec7670 0x562c0cec5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff4439c3740 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cebea90 0x562c0cec4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff41b400640 0 client.26029 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff4439c3740 1 --2- 172.21.15.22:0/3876562885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cec4690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff4439c3740 1 -- 172.21.15.22:0/3876562885 >> 172.21.15.22:0/3876562885 conn(0x562c0ceb88a0 msgr2=0x562c0ceba5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff4439c3740 1 -- 172.21.15.22:0/3876562885 shutdown_connections 2024-04-20T21:38:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff4439c3740 1 -- 172.21.15.22:0/3876562885 wait complete. 2024-04-20T21:38:49.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.710+0000 7ff4439c3740 20 client.26029 trim_cache size 0 max 16384 2024-04-20T21:38:49.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.726+0000 7ff4439c3740 1 Processor -- start 2024-04-20T21:38:49.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.726+0000 7ff4439c3740 1 -- start start 2024-04-20T21:38:49.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.726+0000 7ff4439c3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cec6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.726+0000 7ff4439c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 0x562c0ced32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.726+0000 7ff4439c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cec7670 0x562c0cec36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.726+0000 7ff4439c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1adda0 con 0x562c0cec7670 2024-04-20T21:38:49.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.726+0000 7ff4439c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1b0620 con 0x562c0cebea90 2024-04-20T21:38:49.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.726+0000 7ff4439c3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1ae410 con 0x562c0cebe170 2024-04-20T21:38:49.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.727+0000 7ff439400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 0x562c0ced32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.727+0000 7ff439400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 0x562c0ced32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44498/0 (socket says 172.21.15.22:44498) 2024-04-20T21:38:49.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.727+0000 7ff439400640 1 -- 172.21.15.22:0/1422640654 learned_addr learned my addr 172.21.15.22:0/1422640654 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:49.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.727+0000 7ff43a800640 1 --2- 172.21.15.22:0/1422640654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cec7670 0x562c0cec36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.727+0000 7ff439e00640 1 --2- 172.21.15.22:0/1422640654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cec6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.727+0000 7ff439400640 1 -- 172.21.15.22:0/1422640654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 msgr2=0x562c0cec6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:49.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.727+0000 7ff439400640 1 --2- 172.21.15.22:0/1422640654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cec6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.727+0000 7ff439400640 1 -- 172.21.15.22:0/1422640654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cec7670 msgr2=0x562c0cec36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:49.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.727+0000 7ff439400640 1 --2- 172.21.15.22:0/1422640654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cec7670 0x562c0cec36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.727+0000 7ff439400640 1 -- 172.21.15.22:0/1422640654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c0cb353f0 con 0x562c0cebea90 2024-04-20T21:38:49.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.727+0000 7ff439e00640 1 --2- 172.21.15.22:0/1422640654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cec6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:49.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.728+0000 7ff439400640 1 --2- 172.21.15.22:0/1422640654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 0x562c0ced32c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff43c10a8a0 tx=0x7ff42801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=231840b00e1d6a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:49.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.728+0000 7ff426600640 1 -- 172.21.15.22:0/1422640654 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42802b470 con 0x562c0cebea90 2024-04-20T21:38:49.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.728+0000 7ff426600640 1 -- 172.21.15.22:0/1422640654 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff428030070 con 0x562c0cebea90 2024-04-20T21:38:49.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.728+0000 7ff426600640 1 -- 172.21.15.22:0/1422640654 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42801e400 con 0x562c0cebea90 2024-04-20T21:38:49.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.729+0000 7ff4439c3740 1 -- 172.21.15.22:0/1422640654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 msgr2=0x562c0ced32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.729+0000 7ff4439c3740 1 --2- 172.21.15.22:0/1422640654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 0x562c0ced32c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff43c10a8a0 tx=0x7ff42801b000 comp rx=0 tx=0).stop 2024-04-20T21:38:49.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.729+0000 7ff4439c3740 1 -- 172.21.15.22:0/1422640654 shutdown_connections 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.729+0000 7ff4439c3740 1 --2- 172.21.15.22:0/1422640654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cec7670 0x562c0cec36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.729+0000 7ff4439c3740 1 --2- 172.21.15.22:0/1422640654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 0x562c0ced32c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.729+0000 7ff4439c3740 1 --2- 172.21.15.22:0/1422640654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cec6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.729+0000 7ff4439c3740 1 -- 172.21.15.22:0/1422640654 >> 172.21.15.22:0/1422640654 conn(0x562c0ceb88a0 msgr2=0x562c0ceb8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.729+0000 7ff4439c3740 1 -- 172.21.15.22:0/1422640654 shutdown_connections 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.729+0000 7ff4439c3740 1 -- 172.21.15.22:0/1422640654 wait complete. 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.729+0000 7ff4439c3740 1 -- start start 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.730+0000 7ff4439c3740 1 Processor -- start 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.730+0000 7ff4439c3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cecf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.730+0000 7ff4439c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 0x562c0cecfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.730+0000 7ff4439c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cec7670 0x562c0cec0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.730+0000 7ff4439c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1aeae0 con 0x562c0cec7670 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.730+0000 7ff4439c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1aff50 con 0x562c0cebea90 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.730+0000 7ff4439c3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1af1b0 con 0x562c0cebe170 2024-04-20T21:38:49.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.730+0000 7ff4439c3740 10 client.0 _ll_register_callbacks cb 0x7ff42c02c880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:49.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.730+0000 7ff439400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 0x562c0cecfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.731+0000 7ff439400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 0x562c0cecfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44508/0 (socket says 172.21.15.22:44508) 2024-04-20T21:38:49.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.731+0000 7ff439400640 1 -- 172.21.15.22:0/905731817 learned_addr learned my addr 172.21.15.22:0/905731817 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:49.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.731+0000 7ff43a800640 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cec7670 0x562c0cec0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.731+0000 7ff439e00640 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cecf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.731+0000 7ff439400640 1 -- 172.21.15.22:0/905731817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 msgr2=0x562c0cecf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:49.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.731+0000 7ff439400640 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cecf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.731+0000 7ff439400640 1 -- 172.21.15.22:0/905731817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cec7670 msgr2=0x562c0cec0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.731+0000 7ff439400640 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cec7670 0x562c0cec0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.731+0000 7ff439400640 1 -- 172.21.15.22:0/905731817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c0cbff7d0 con 0x562c0cebea90 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.731+0000 7ff439400640 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 0x562c0cecfd10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff42801f5e0 tx=0x7ff42801f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9de73888f6f0acd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.731+0000 7ff41b400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.732+0000 7ff41b400640 1 -- 172.21.15.22:0/905731817 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42803b020 con 0x562c0cebea90 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.732+0000 7ff41b400640 1 -- 172.21.15.22:0/905731817 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7ff42802b470 con 0x562c0cebea90 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.732+0000 7ff4439c3740 1 -- 172.21.15.22:0/905731817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff428030070 con 0x562c0cebea90 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.732+0000 7ff41b400640 1 -- 172.21.15.22:0/905731817 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff428046050 con 0x562c0cebea90 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.732+0000 7ff4439c3740 10 client.26009 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.732+0000 7ff4439c3740 1 -- 172.21.15.22:0/905731817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562c0cf3cf10 con 0x562c0cebea90 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.732+0000 7ff41b400640 1 -- 172.21.15.22:0/905731817 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff428030070 con 0x562c0cebea90 2024-04-20T21:38:49.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.733+0000 7ff41b400640 1 -- 172.21.15.22:0/905731817 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7ff428005310 con 0x562c0cebea90 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.733+0000 7ff4439c3740 10 client.26009 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.733+0000 7ff4439c3740 10 client.26009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.733+0000 7ff4439c3740 20 client.26009 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.733+0000 7ff4439c3740 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562c0cf3d5c0 0x562c0cf3f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.733+0000 7ff4439c3740 4 client.26009 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.733+0000 7ff4439c3740 1 -- 172.21.15.22:0/905731817 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c0cda65a0 con 0x562c0cf3d5c0 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.733+0000 7ff439e00640 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562c0cf3d5c0 0x562c0cf3f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.733+0000 7ff439e00640 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562c0cf3d5c0 0x562c0cf3f9e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a84f9bb163d3aaf server_cookie=e739664f35f5bf26 in_seq=0 out_seq=0 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.733+0000 7ff41b400640 10 client.26009 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff41b400640 1 -- 172.21.15.22:0/905731817 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c0cda65a0 con 0x562c0cf3d5c0 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff41b400640 10 client.26009 handle_command_reply: tid=0 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff4439c3740 1 client.26009 shutdown 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff4439c3740 1 -- 172.21.15.22:0/905731817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 msgr2=0x562c0cecfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff4439c3740 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 0x562c0cecfd10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff42801f5e0 tx=0x7ff42801f410 comp rx=0 tx=0).stop 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff4439c3740 1 -- 172.21.15.22:0/905731817 shutdown_connections 2024-04-20T21:38:49.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff4439c3740 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562c0cec7670 0x562c0cec0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff4439c3740 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562c0cf3d5c0 0x562c0cf3f9e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff4439c3740 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562c0cebea90 0x562c0cecfd10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff4439c3740 1 --2- 172.21.15.22:0/905731817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c0cebe170 0x562c0cecf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff4439c3740 1 -- 172.21.15.22:0/905731817 >> 172.21.15.22:0/905731817 conn(0x562c0ceb88a0 msgr2=0x562c0ceb7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:49.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff41b400640 0 client.26009 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:49.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff4439c3740 1 -- 172.21.15.22:0/905731817 shutdown_connections 2024-04-20T21:38:49.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff4439c3740 1 -- 172.21.15.22:0/905731817 wait complete. 2024-04-20T21:38:49.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.734+0000 7ff4439c3740 20 client.26009 trim_cache size 0 max 16384 2024-04-20T21:38:49.736 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:38:49.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.735+0000 7ff43b200640 1 -- 172.21.15.22:0/3679054987 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff4200a3da0 msgr2=0x7ff4200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.735+0000 7ff43b200640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff4200a3da0 0x7ff4200a6260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff428009710 tx=0x7ff428005f90 comp rx=0 tx=0).stop 2024-04-20T21:38:49.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.735+0000 7ff43b200640 1 -- 172.21.15.22:0/3679054987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff43c19a790 msgr2=0x7ff43c19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:49.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.735+0000 7ff43b200640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff43c19a790 0x7ff43c19ac40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff434009fd0 tx=0x7ff43400ef10 comp rx=0 tx=0).stop 2024-04-20T21:38:49.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.735+0000 7ff43b200640 1 -- 172.21.15.22:0/3679054987 shutdown_connections 2024-04-20T21:38:49.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.735+0000 7ff43b200640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff4200a3da0 0x7ff4200a6260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.735+0000 7ff43b200640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff43c19a790 0x7ff43c19ac40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.735+0000 7ff43b200640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c109350 0x7ff43c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.735+0000 7ff43b200640 1 --2- 172.21.15.22:0/3679054987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff43c104280 0x7ff43c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:49.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.735+0000 7ff43b200640 1 -- 172.21.15.22:0/3679054987 >> 172.21.15.22:0/3679054987 conn(0x7ff43c0ffaa0 msgr2=0x7ff43c101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:49.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.735+0000 7ff43b200640 1 -- 172.21.15.22:0/3679054987 shutdown_connections 2024-04-20T21:38:49.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:49.735+0000 7ff43b200640 1 -- 172.21.15.22:0/3679054987 wait complete. 2024-04-20T21:38:50.187 INFO:tasks.workunit.client.0.smithi022.stdout: 1 372246 9.68 MB/sec execute 78 sec latency 9.501 ms 2024-04-20T21:38:50.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:50 smithi122 ceph-mon[22147]: pgmap v153: 97 pgs: 97 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-04-20T21:38:50.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:50 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3305409095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:38:50.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:50 smithi033 ceph-mon[21626]: pgmap v153: 97 pgs: 97 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-04-20T21:38:50.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:50 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3305409095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:38:50.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:50 smithi022 ceph-mon[16843]: pgmap v153: 97 pgs: 97 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-04-20T21:38:50.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:50 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3305409095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:38:50.754 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:38:50.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:38:50.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:38:50.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.981+0000 7fdc8d600640 1 -- 172.21.15.22:0/3233230724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88103950 msgr2=0x7fdc8810be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:50.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.981+0000 7fdc8d600640 1 --2- 172.21.15.22:0/3233230724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88103950 0x7fdc8810be30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009900 tx=0x7fdc7002bcc0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.981+0000 7fdc8d600640 1 -- 172.21.15.22:0/3233230724 shutdown_connections 2024-04-20T21:38:50.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.981+0000 7fdc8d600640 1 --2- 172.21.15.22:0/3233230724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc8810c450 0x7fdc8810e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.981+0000 7fdc8d600640 1 --2- 172.21.15.22:0/3233230724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88103950 0x7fdc8810be30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.981+0000 7fdc8d600640 1 --2- 172.21.15.22:0/3233230724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc88103030 0x7fdc88103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.981+0000 7fdc8d600640 1 -- 172.21.15.22:0/3233230724 >> 172.21.15.22:0/3233230724 conn(0x7fdc880fc850 msgr2=0x7fdc880fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:50.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.981+0000 7fdc8d600640 1 -- 172.21.15.22:0/3233230724 shutdown_connections 2024-04-20T21:38:50.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.981+0000 7fdc8d600640 1 -- 172.21.15.22:0/3233230724 wait complete. 2024-04-20T21:38:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.982+0000 7fdc8d600640 1 Processor -- start 2024-04-20T21:38:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.982+0000 7fdc8d600640 1 -- start start 2024-04-20T21:38:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.983+0000 7fdc8d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc88103030 0x7fdc881a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.983+0000 7fdc8d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc88103950 0x7fdc881a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.983+0000 7fdc8d600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8810c450 0x7fdc8819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.983+0000 7fdc8d600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88112260 con 0x7fdc88103030 2024-04-20T21:38:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.983+0000 7fdc8d600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881120e0 con 0x7fdc8810c450 2024-04-20T21:38:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.983+0000 7fdc8d600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881123e0 con 0x7fdc88103950 2024-04-20T21:38:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.983+0000 7fdc87e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc88103030 0x7fdc881a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.983+0000 7fdc87e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc88103030 0x7fdc881a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54564/0 (socket says 172.21.15.22:54564) 2024-04-20T21:38:50.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.983+0000 7fdc87e00640 1 -- 172.21.15.22:0/3758352396 learned_addr learned my addr 172.21.15.22:0/3758352396 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:50.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.983+0000 7fdc7f400640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc88103950 0x7fdc881a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:50.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.983+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8810c450 0x7fdc8819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:50.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.984+0000 7fdc87e00640 1 -- 172.21.15.22:0/3758352396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc88103950 msgr2=0x7fdc881a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:50.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.984+0000 7fdc87e00640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc88103950 0x7fdc881a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.984+0000 7fdc87e00640 1 -- 172.21.15.22:0/3758352396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8810c450 msgr2=0x7fdc8819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:50.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.984+0000 7fdc87e00640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8810c450 0x7fdc8819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.984+0000 7fdc87e00640 1 -- 172.21.15.22:0/3758352396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8819af70 con 0x7fdc88103030 2024-04-20T21:38:50.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.984+0000 7fdc7f400640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc88103950 0x7fdc881a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:50.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.984+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8810c450 0x7fdc8819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:50.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.984+0000 7fdc87e00640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc88103030 0x7fdc881a0630 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009d50 tx=0x7fdc7800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:50.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.984+0000 7fdc85600640 1 -- 172.21.15.22:0/3758352396 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78003b70 con 0x7fdc88103030 2024-04-20T21:38:50.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.984+0000 7fdc85600640 1 -- 172.21.15.22:0/3758352396 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc78003d10 con 0x7fdc88103030 2024-04-20T21:38:50.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.985+0000 7fdc8d600640 1 -- 172.21.15.22:0/3758352396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8819b260 con 0x7fdc88103030 2024-04-20T21:38:50.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.985+0000 7fdc85600640 1 -- 172.21.15.22:0/3758352396 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78010a20 con 0x7fdc88103030 2024-04-20T21:38:50.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.985+0000 7fdc8d600640 1 -- 172.21.15.22:0/3758352396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc881a7480 con 0x7fdc88103030 2024-04-20T21:38:50.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.988+0000 7fdc91972740 1 Processor -- start 2024-04-20T21:38:50.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.988+0000 7fdc91972740 1 -- start start 2024-04-20T21:38:50.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.988+0000 7fdc91972740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36abe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:50.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.988+0000 7fdc91972740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ac70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc85600640 1 -- 172.21.15.22:0/3758352396 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fdc78003ec0 con 0x7fdc88103030 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc91972740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36ac99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc91972740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560a36a22d20 con 0x560a36ac75f0 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc91972740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560a369c93e0 con 0x560a36abea10 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc91972740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560a36a1ece0 con 0x560a36abe0f0 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc8cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36abe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc87e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ac70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc8cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36abe4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:41064/0 (socket says 172.21.15.22:41064) 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc8cc00640 1 -- 172.21.15.22:0/518106273 learned_addr learned my addr 172.21.15.22:0/518106273 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc7f400640 1 --2- 172.21.15.22:0/518106273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36ac99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc8cc00640 1 -- 172.21.15.22:0/518106273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 msgr2=0x560a36ac70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/518106273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ac70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc8cc00640 1 -- 172.21.15.22:0/518106273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 msgr2=0x560a36ac99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:50.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/518106273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36ac99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc8cc00640 1 -- 172.21.15.22:0/518106273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a369e8980 con 0x560a36abe0f0 2024-04-20T21:38:50.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.989+0000 7fdc7f400640 1 --2- 172.21.15.22:0/518106273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36ac99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:50.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.990+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/518106273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36abe4d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdc80011440 tx=0x7fdc80011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfe42823388096c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:50.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.990+0000 7fdc7d200640 1 -- 172.21.15.22:0/518106273 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80004020 con 0x560a36abe0f0 2024-04-20T21:38:50.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.990+0000 7fdc7d200640 1 -- 172.21.15.22:0/518106273 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc800041c0 con 0x560a36abe0f0 2024-04-20T21:38:50.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.990+0000 7fdc7d200640 1 -- 172.21.15.22:0/518106273 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8000f940 con 0x560a36abe0f0 2024-04-20T21:38:50.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.990+0000 7fdc85600640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdc6c0a3d20 0x7fdc6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:50.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.991+0000 7fdc85600640 1 -- 172.21.15.22:0/3758352396 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fdc780d0ea0 con 0x7fdc88103030 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.991+0000 7fdc87e00640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdc6c0a3d20 0x7fdc6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.991+0000 7fdc91972740 1 -- 172.21.15.22:0/518106273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 msgr2=0x560a36abe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.991+0000 7fdc91972740 1 --2- 172.21.15.22:0/518106273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36abe4d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdc80011440 tx=0x7fdc80011900 comp rx=0 tx=0).stop 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.991+0000 7fdc91972740 1 -- 172.21.15.22:0/518106273 shutdown_connections 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.991+0000 7fdc91972740 1 --2- 172.21.15.22:0/518106273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36ac99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.991+0000 7fdc91972740 1 --2- 172.21.15.22:0/518106273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ac70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.991+0000 7fdc91972740 1 --2- 172.21.15.22:0/518106273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36abe4d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.991+0000 7fdc91972740 1 -- 172.21.15.22:0/518106273 >> 172.21.15.22:0/518106273 conn(0x560a36ab8820 msgr2=0x560a36ab8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.991+0000 7fdc91972740 1 -- 172.21.15.22:0/518106273 shutdown_connections 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.991+0000 7fdc91972740 1 -- 172.21.15.22:0/518106273 wait complete. 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.991+0000 7fdc87e00640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdc6c0a3d20 0x7fdc6c0a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdc780ce680 tx=0x7fdc780d49f0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.992+0000 7fdc91972740 1 -- start start 2024-04-20T21:38:50.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.992+0000 7fdc91972740 1 Processor -- start 2024-04-20T21:38:50.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc91972740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36abe0f0 0x560a36ac4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:50.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc91972740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ac4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:50.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc91972740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36ac75f0 0x560a36ac5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:50.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc91972740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560a36aca3e0 con 0x560a36abe0f0 2024-04-20T21:38:50.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc91972740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560a369dc7a0 con 0x560a36abea10 2024-04-20T21:38:50.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc91972740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560a36aca560 con 0x560a36ac75f0 2024-04-20T21:38:50.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc91972740 10 client.0 _ll_register_callbacks cb 0x560a369deb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:50.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc7f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ac4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc8cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36abe0f0 0x560a36ac4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc8cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36abe0f0 0x560a36ac4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54574/0 (socket says 172.21.15.22:54574) 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc8cc00640 1 -- 172.21.15.22:0/630037003 learned_addr learned my addr 172.21.15.22:0/630037003 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc7f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ac4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44554/0 (socket says 172.21.15.22:44554) 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.993+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/630037003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36ac75f0 0x560a36ac5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.994+0000 7fdc7f400640 1 -- 172.21.15.22:0/630037003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36ac75f0 msgr2=0x560a36ac5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.994+0000 7fdc7f400640 1 --2- 172.21.15.22:0/630037003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36ac75f0 0x560a36ac5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.994+0000 7fdc7f400640 1 -- 172.21.15.22:0/630037003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36abe0f0 msgr2=0x560a36ac4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.994+0000 7fdc7f400640 1 --2- 172.21.15.22:0/630037003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36abe0f0 0x560a36ac4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.994+0000 7fdc7f400640 1 -- 172.21.15.22:0/630037003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a367fd7e0 con 0x560a36abea10 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.994+0000 7fdc7f400640 1 --2- 172.21.15.22:0/630037003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ac4bc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc7002c200 tx=0x7fdc70009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5399ae654eabb631 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.994+0000 7fdc75600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.994+0000 7fdc75600640 1 -- 172.21.15.22:0/630037003 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70043070 con 0x560a36abea10 2024-04-20T21:38:50.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.994+0000 7fdc91972740 1 -- 172.21.15.22:0/630037003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc800041c0 con 0x560a36abea10 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.995+0000 7fdc75600640 1 -- 172.21.15.22:0/630037003 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fdc7002c910 con 0x560a36abea10 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.995+0000 7fdc75600640 1 -- 172.21.15.22:0/630037003 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdc7002cb10 con 0x560a36abea10 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.995+0000 7fdc91972740 10 client.26021 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.995+0000 7fdc91972740 1 -- 172.21.15.22:0/630037003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560a3685df70 con 0x560a36abea10 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.995+0000 7fdc75600640 1 -- 172.21.15.22:0/630037003 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc800041c0 con 0x560a36abea10 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.995+0000 7fdc75600640 1 -- 172.21.15.22:0/630037003 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fdc700035f0 con 0x560a36abea10 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.995+0000 7fdc91972740 10 client.26021 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.995+0000 7fdc91972740 10 client.26021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.995+0000 7fdc91972740 20 client.26021 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.995+0000 7fdc91972740 1 --2- 172.21.15.22:0/630037003 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560a36ad7970 0x560a36ad9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.995+0000 7fdc91972740 4 client.26021 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.995+0000 7fdc91972740 1 -- 172.21.15.22:0/630037003 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a36a30740 con 0x560a36ad7970 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.996+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/630037003 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560a36ad7970 0x560a36ad9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:50.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.996+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/630037003 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560a36ad7970 0x560a36ad9d60 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce378b93887e3727 server_cookie=9586dca9bbe92c10 in_seq=0 out_seq=0 2024-04-20T21:38:50.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.996+0000 7fdc75600640 10 client.26021 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:50.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.997+0000 7fdc75600640 1 -- 172.21.15.22:0/630037003 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a36a30740 con 0x560a36ad7970 2024-04-20T21:38:50.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.997+0000 7fdc75600640 10 client.26021 handle_command_reply: tid=0 2024-04-20T21:38:50.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc91972740 1 client.26021 shutdown 2024-04-20T21:38:50.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc91972740 1 -- 172.21.15.22:0/630037003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 msgr2=0x560a36ac4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:50.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc91972740 1 --2- 172.21.15.22:0/630037003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ac4bc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc7002c200 tx=0x7fdc70009680 comp rx=0 tx=0).stop 2024-04-20T21:38:50.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc91972740 1 -- 172.21.15.22:0/630037003 shutdown_connections 2024-04-20T21:38:50.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc91972740 1 --2- 172.21.15.22:0/630037003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36ac75f0 0x560a36ac5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc91972740 1 --2- 172.21.15.22:0/630037003 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560a36ad7970 0x560a36ad9d60 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc91972740 1 --2- 172.21.15.22:0/630037003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ac4bc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc91972740 1 --2- 172.21.15.22:0/630037003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36abe0f0 0x560a36ac4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:50.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc91972740 1 -- 172.21.15.22:0/630037003 >> 172.21.15.22:0/630037003 conn(0x560a36ab8820 msgr2=0x560a36aba570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc75600640 0 client.26021 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:51.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc91972740 1 -- 172.21.15.22:0/630037003 shutdown_connections 2024-04-20T21:38:51.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc91972740 1 -- 172.21.15.22:0/630037003 wait complete. 2024-04-20T21:38:51.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:50.998+0000 7fdc91972740 20 client.26021 trim_cache size 0 max 16384 2024-04-20T21:38:51.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.021+0000 7fdc91972740 1 Processor -- start 2024-04-20T21:38:51.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.022+0000 7fdc91972740 1 -- start start 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.022+0000 7fdc91972740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36ac6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.022+0000 7fdc91972740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ad32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.022+0000 7fdc91972740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36adf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.022+0000 7fdc91972740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881add70 con 0x560a36ac75f0 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.022+0000 7fdc91972740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881b05f0 con 0x560a36abea10 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.022+0000 7fdc91972740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881ae3e0 con 0x560a36abe0f0 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.022+0000 7fdc7f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ad32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.022+0000 7fdc7f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ad32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44564/0 (socket says 172.21.15.22:44564) 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.022+0000 7fdc7f400640 1 -- 172.21.15.22:0/3916449538 learned_addr learned my addr 172.21.15.22:0/3916449538 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.023+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/3916449538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36ac6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.023+0000 7fdc7f400640 1 -- 172.21.15.22:0/3916449538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 msgr2=0x560a36ac6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.023+0000 7fdc7f400640 1 --2- 172.21.15.22:0/3916449538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36ac6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.023+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/3916449538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36adf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.023+0000 7fdc7f400640 1 -- 172.21.15.22:0/3916449538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 msgr2=0x560a36adf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.023+0000 7fdc7f400640 1 --2- 172.21.15.22:0/3916449538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36adf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.023+0000 7fdc7f400640 1 -- 172.21.15.22:0/3916449538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a36abe840 con 0x560a36abea10 2024-04-20T21:38:51.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.023+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/3916449538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36adf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:51.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.023+0000 7fdc7f400640 1 --2- 172.21.15.22:0/3916449538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ad32a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdc7002bd00 tx=0x7fdc700058f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a94f7c161d7bdb38 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.023+0000 7fdc77e00640 1 -- 172.21.15.22:0/3916449538 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70054020 con 0x560a36abea10 2024-04-20T21:38:51.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.023+0000 7fdc77e00640 1 -- 172.21.15.22:0/3916449538 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc70009780 con 0x560a36abea10 2024-04-20T21:38:51.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.023+0000 7fdc77e00640 1 -- 172.21.15.22:0/3916449538 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70043040 con 0x560a36abea10 2024-04-20T21:38:51.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.024+0000 7fdc91972740 1 -- 172.21.15.22:0/3916449538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 msgr2=0x560a36ad32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.024+0000 7fdc91972740 1 --2- 172.21.15.22:0/3916449538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ad32a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdc7002bd00 tx=0x7fdc700058f0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.024+0000 7fdc91972740 1 -- 172.21.15.22:0/3916449538 shutdown_connections 2024-04-20T21:38:51.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.024+0000 7fdc91972740 1 --2- 172.21.15.22:0/3916449538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36adf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.024+0000 7fdc91972740 1 --2- 172.21.15.22:0/3916449538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36ad32a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.024+0000 7fdc91972740 1 --2- 172.21.15.22:0/3916449538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36ac6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.024+0000 7fdc91972740 1 -- 172.21.15.22:0/3916449538 >> 172.21.15.22:0/3916449538 conn(0x560a36ab8820 msgr2=0x560a36ab7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.024+0000 7fdc91972740 1 -- 172.21.15.22:0/3916449538 shutdown_connections 2024-04-20T21:38:51.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.024+0000 7fdc91972740 1 -- 172.21.15.22:0/3916449538 wait complete. 2024-04-20T21:38:51.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.025+0000 7fdc91972740 1 -- start start 2024-04-20T21:38:51.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.025+0000 7fdc91972740 1 Processor -- start 2024-04-20T21:38:51.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.025+0000 7fdc91972740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36acb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.025+0000 7fdc91972740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36acbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.025+0000 7fdc91972740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36b3ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc91972740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881aeab0 con 0x560a36ac75f0 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc91972740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881aff20 con 0x560a36abea10 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc91972740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881af180 con 0x560a36abe0f0 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc91972740 10 client.0 _ll_register_callbacks cb 0x7fdc80005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc7f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36acbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc7f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36acbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44572/0 (socket says 172.21.15.22:44572) 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc7f400640 1 -- 172.21.15.22:0/3622803196 learned_addr learned my addr 172.21.15.22:0/3622803196 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36acb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36b3ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc7f400640 1 -- 172.21.15.22:0/3622803196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 msgr2=0x560a36acb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc7f400640 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36acb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc7f400640 1 -- 172.21.15.22:0/3622803196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 msgr2=0x560a36b3ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc7f400640 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36b3ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc7f400640 1 -- 172.21.15.22:0/3622803196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a366f5170 con 0x560a36abea10 2024-04-20T21:38:51.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.026+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36b3ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:51.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.027+0000 7fdc7f400640 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36acbe30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdc7002c200 tx=0x7fdc70003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b758c9096ea3954 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.027+0000 7fdc75600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.027+0000 7fdc75600640 1 -- 172.21.15.22:0/3622803196 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7005d020 con 0x560a36abea10 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.027+0000 7fdc75600640 1 -- 172.21.15.22:0/3622803196 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fdc700050e0 con 0x560a36abea10 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.027+0000 7fdc91972740 1 -- 172.21.15.22:0/3622803196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc70009780 con 0x560a36abea10 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.027+0000 7fdc75600640 1 -- 172.21.15.22:0/3622803196 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdc70068050 con 0x560a36abea10 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.027+0000 7fdc91972740 10 client.26027 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.027+0000 7fdc91972740 1 -- 172.21.15.22:0/3622803196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560a36b3e440 con 0x560a36abea10 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.028+0000 7fdc75600640 1 -- 172.21.15.22:0/3622803196 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc70009780 con 0x560a36abea10 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.028+0000 7fdc75600640 1 -- 172.21.15.22:0/3622803196 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fdc70059ba0 con 0x560a36abea10 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.028+0000 7fdc91972740 10 client.26027 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.028+0000 7fdc91972740 10 client.26027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.028+0000 7fdc91972740 20 client.26027 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.028+0000 7fdc91972740 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560a36b3ea40 0x560a36b40e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.028+0000 7fdc91972740 4 client.26027 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:38:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.028+0000 7fdc91972740 1 -- 172.21.15.22:0/3622803196 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560a36a2a5b0 con 0x560a36b3ea40 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.028+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560a36b3ea40 0x560a36b40e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.029+0000 7fdc8cc00640 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560a36b3ea40 0x560a36b40e60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b4dcd8494b5972 server_cookie=df9ebfd7e012724e in_seq=0 out_seq=0 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.029+0000 7fdc75600640 10 client.26027 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc75600640 1 -- 172.21.15.22:0/3622803196 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560a36a2a5b0 con 0x560a36b3ea40 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc75600640 10 client.26027 handle_command_reply: tid=0 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc91972740 1 client.26027 shutdown 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc91972740 1 -- 172.21.15.22:0/3622803196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 msgr2=0x560a36acbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc91972740 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36acbe30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdc7002c200 tx=0x7fdc70003820 comp rx=0 tx=0).stop 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc91972740 1 -- 172.21.15.22:0/3622803196 shutdown_connections 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc91972740 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560a36ac75f0 0x560a36b3ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc91972740 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560a36b3ea40 0x560a36b40e60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc91972740 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560a36abea10 0x560a36acbe30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc91972740 1 --2- 172.21.15.22:0/3622803196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a36abe0f0 0x560a36acb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc91972740 1 -- 172.21.15.22:0/3622803196 >> 172.21.15.22:0/3622803196 conn(0x560a36ab8820 msgr2=0x560a36ac45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc75600640 0 client.26027 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.030+0000 7fdc91972740 1 -- 172.21.15.22:0/3622803196 shutdown_connections 2024-04-20T21:38:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.031+0000 7fdc91972740 1 -- 172.21.15.22:0/3622803196 wait complete. 2024-04-20T21:38:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.031+0000 7fdc91972740 20 client.26027 trim_cache size 0 max 16384 2024-04-20T21:38:51.034 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:38:51.034 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:38:51.034 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "95d71bf0-bed8-47c9-a994-2e2163cd0e90", 2024-04-20T21:38:51.034 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:38:51.034 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:38:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.031+0000 7fdc8d600640 1 -- 172.21.15.22:0/3758352396 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdc6c0a3d20 msgr2=0x7fdc6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.031+0000 7fdc8d600640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdc6c0a3d20 0x7fdc6c0a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdc780ce680 tx=0x7fdc780d49f0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.031+0000 7fdc8d600640 1 -- 172.21.15.22:0/3758352396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc88103030 msgr2=0x7fdc881a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.031+0000 7fdc8d600640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc88103030 0x7fdc881a0630 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009d50 tx=0x7fdc7800c6a0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.031+0000 7fdc8d600640 1 -- 172.21.15.22:0/3758352396 shutdown_connections 2024-04-20T21:38:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.031+0000 7fdc8d600640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdc6c0a3d20 0x7fdc6c0a61e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.031+0000 7fdc8d600640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8810c450 0x7fdc8819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.031+0000 7fdc8d600640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc88103950 0x7fdc881a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.032+0000 7fdc8d600640 1 --2- 172.21.15.22:0/3758352396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc88103030 0x7fdc881a0630 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.032+0000 7fdc8d600640 1 -- 172.21.15.22:0/3758352396 >> 172.21.15.22:0/3758352396 conn(0x7fdc880fc850 msgr2=0x7fdc88100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.032+0000 7fdc8d600640 1 -- 172.21.15.22:0/3758352396 shutdown_connections 2024-04-20T21:38:51.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.032+0000 7fdc8d600640 1 -- 172.21.15.22:0/3758352396 wait complete. 2024-04-20T21:38:51.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:38:51.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:38:51.188 INFO:tasks.workunit.client.0.smithi022.stdout: 1 373557 9.64 MB/sec execute 79 sec latency 23.569 ms 2024-04-20T21:38:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.267+0000 7f0c77e00640 1 -- 172.21.15.22:0/1365899404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c78104c30 msgr2=0x7f0c78108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.267+0000 7f0c77e00640 1 --2- 172.21.15.22:0/1365899404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c78104c30 0x7f0c78108c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009860 tx=0x7f0c6402bb70 comp rx=0 tx=0).stop 2024-04-20T21:38:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.268+0000 7f0c77e00640 1 -- 172.21.15.22:0/1365899404 shutdown_connections 2024-04-20T21:38:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.268+0000 7f0c77e00640 1 --2- 172.21.15.22:0/1365899404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78109350 0x7f0c78110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.268+0000 7f0c77e00640 1 --2- 172.21.15.22:0/1365899404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c78104c30 0x7f0c78108c70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.268+0000 7f0c77e00640 1 --2- 172.21.15.22:0/1365899404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c78104280 0x7f0c78104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.268+0000 7f0c77e00640 1 -- 172.21.15.22:0/1365899404 >> 172.21.15.22:0/1365899404 conn(0x7f0c780ffaa0 msgr2=0x7f0c78101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.268+0000 7f0c77e00640 1 -- 172.21.15.22:0/1365899404 shutdown_connections 2024-04-20T21:38:51.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.268+0000 7f0c77e00640 1 -- 172.21.15.22:0/1365899404 wait complete. 2024-04-20T21:38:51.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.269+0000 7f0c77e00640 1 Processor -- start 2024-04-20T21:38:51.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.269+0000 7f0c77e00640 1 -- start start 2024-04-20T21:38:51.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.269+0000 7f0c77e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78104280 0x7f0c781a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.269+0000 7f0c77e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c78109350 0x7f0c781a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.269+0000 7f0c77e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c7819a760 0x7f0c7819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.269+0000 7f0c77e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781143c0 con 0x7f0c78109350 2024-04-20T21:38:51.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.269+0000 7f0c77e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c78114240 con 0x7f0c78104280 2024-04-20T21:38:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.269+0000 7f0c77e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c78114540 con 0x7f0c7819a760 2024-04-20T21:38:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.270+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78104280 0x7f0c781a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.270+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78104280 0x7f0c781a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44588/0 (socket says 172.21.15.22:44588) 2024-04-20T21:38:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.270+0000 7f0c76a00640 1 -- 172.21.15.22:0/4249028091 learned_addr learned my addr 172.21.15.22:0/4249028091 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.270+0000 7f0c77400640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c7819a760 0x7f0c7819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.270+0000 7f0c76000640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c78109350 0x7f0c781a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.270+0000 7f0c76a00640 1 -- 172.21.15.22:0/4249028091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c7819a760 msgr2=0x7f0c7819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.270+0000 7f0c76a00640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c7819a760 0x7f0c7819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.270+0000 7f0c76a00640 1 -- 172.21.15.22:0/4249028091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c78109350 msgr2=0x7f0c781a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.270+0000 7f0c76a00640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c78109350 0x7f0c781a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.270+0000 7f0c76a00640 1 -- 172.21.15.22:0/4249028091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7819b360 con 0x7f0c78104280 2024-04-20T21:38:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.270+0000 7f0c76a00640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78104280 0x7f0c781a06a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800e9c0 tx=0x7f0c6800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.271+0000 7f0c6f400640 1 -- 172.21.15.22:0/4249028091 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6800ccc0 con 0x7f0c78104280 2024-04-20T21:38:51.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.271+0000 7f0c77e00640 1 -- 172.21.15.22:0/4249028091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c781a7440 con 0x7f0c78104280 2024-04-20T21:38:51.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.271+0000 7f0c6f400640 1 -- 172.21.15.22:0/4249028091 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c68005500 con 0x7f0c78104280 2024-04-20T21:38:51.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.271+0000 7f0c6f400640 1 -- 172.21.15.22:0/4249028091 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68010430 con 0x7f0c78104280 2024-04-20T21:38:51.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.271+0000 7f0c77e00640 1 -- 172.21.15.22:0/4249028091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c781a7840 con 0x7f0c78104280 2024-04-20T21:38:51.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.274+0000 7f0c805d5740 1 Processor -- start 2024-04-20T21:38:51.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.274+0000 7f0c805d5740 1 -- start start 2024-04-20T21:38:51.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.274+0000 7f0c6f400640 1 -- 172.21.15.22:0/4249028091 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0c680046c0 con 0x7f0c78104280 2024-04-20T21:38:51.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.275+0000 7f0c805d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 0x55a631ba1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.275+0000 7f0c805d5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631ba1ca0 0x55a631baa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.275+0000 7f0c805d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 0x55a631bacc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.275+0000 7f0c805d5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a631b07150 con 0x55a631ba1ca0 2024-04-20T21:38:51.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.275+0000 7f0c805d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a631aad370 con 0x55a631ba1380 2024-04-20T21:38:51.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.275+0000 7f0c805d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a631b03110 con 0x55a631baa880 2024-04-20T21:38:51.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.275+0000 7f0c77400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 0x55a631ba1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.275+0000 7f0c77400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 0x55a631ba1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44596/0 (socket says 172.21.15.22:44596) 2024-04-20T21:38:51.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.275+0000 7f0c77400640 1 -- 172.21.15.22:0/738884526 learned_addr learned my addr 172.21.15.22:0/738884526 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:51.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.275+0000 7f0c76000640 1 --2- 172.21.15.22:0/738884526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 0x55a631bacc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.275+0000 7f0c76a00640 1 --2- 172.21.15.22:0/738884526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631ba1ca0 0x55a631baa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.276+0000 7f0c76a00640 1 -- 172.21.15.22:0/738884526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 msgr2=0x55a631bacc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.276+0000 7f0c76a00640 1 --2- 172.21.15.22:0/738884526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 0x55a631bacc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.276+0000 7f0c76a00640 1 -- 172.21.15.22:0/738884526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 msgr2=0x55a631ba1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.276+0000 7f0c76a00640 1 --2- 172.21.15.22:0/738884526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 0x55a631ba1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.276+0000 7f0c76a00640 1 -- 172.21.15.22:0/738884526 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6318e37d0 con 0x55a631ba1ca0 2024-04-20T21:38:51.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.276+0000 7f0c76000640 1 --2- 172.21.15.22:0/738884526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 0x55a631bacc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:51.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.276+0000 7f0c76a00640 1 --2- 172.21.15.22:0/738884526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631ba1ca0 0x55a631baa340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0c680ce220 tx=0x7f0c680d4bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bae669f1eef61c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.277+0000 7f0c6f400640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0c5c0a3d20 0x7f0c5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.277+0000 7f0c63e00640 1 -- 172.21.15.22:0/738884526 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c680d9c00 con 0x55a631ba1ca0 2024-04-20T21:38:51.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.277+0000 7f0c63e00640 1 -- 172.21.15.22:0/738884526 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c680e2070 con 0x55a631ba1ca0 2024-04-20T21:38:51.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.277+0000 7f0c63e00640 1 -- 172.21.15.22:0/738884526 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c680990e0 con 0x55a631ba1ca0 2024-04-20T21:38:51.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.277+0000 7f0c6f400640 1 -- 172.21.15.22:0/4249028091 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f0c680d0b50 con 0x7f0c78104280 2024-04-20T21:38:51.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.277+0000 7f0c77400640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0c5c0a3d20 0x7f0c5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.278+0000 7f0c805d5740 1 -- 172.21.15.22:0/738884526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631ba1ca0 msgr2=0x55a631baa340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.278+0000 7f0c805d5740 1 --2- 172.21.15.22:0/738884526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631ba1ca0 0x55a631baa340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0c680ce220 tx=0x7f0c680d4bf0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.278+0000 7f0c77400640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0c5c0a3d20 0x7f0c5c0a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0c70012560 tx=0x7f0c70009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.278+0000 7f0c805d5740 1 -- 172.21.15.22:0/738884526 shutdown_connections 2024-04-20T21:38:51.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.278+0000 7f0c805d5740 1 --2- 172.21.15.22:0/738884526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 0x55a631bacc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.278+0000 7f0c805d5740 1 --2- 172.21.15.22:0/738884526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631ba1ca0 0x55a631baa340 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.278+0000 7f0c805d5740 1 --2- 172.21.15.22:0/738884526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 0x55a631ba1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.278+0000 7f0c805d5740 1 -- 172.21.15.22:0/738884526 >> 172.21.15.22:0/738884526 conn(0x55a631b9bab0 msgr2=0x55a631b9be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.279+0000 7f0c805d5740 1 -- 172.21.15.22:0/738884526 shutdown_connections 2024-04-20T21:38:51.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.279+0000 7f0c805d5740 1 -- 172.21.15.22:0/738884526 wait complete. 2024-04-20T21:38:51.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.279+0000 7f0c805d5740 1 -- start start 2024-04-20T21:38:51.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.280+0000 7f0c805d5740 1 Processor -- start 2024-04-20T21:38:51.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.280+0000 7f0c805d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 0x55a631ba78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.280+0000 7f0c805d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631ba1ca0 0x55a631ba7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.280+0000 7f0c805d5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631baa880 0x55a631ba8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.280+0000 7f0c805d5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a631bad610 con 0x55a631baa880 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.280+0000 7f0c805d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a631ac0740 con 0x55a631ba1380 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.280+0000 7f0c805d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a631bad790 con 0x55a631ba1ca0 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.280+0000 7f0c805d5740 10 client.0 _ll_register_callbacks cb 0x55a631abbb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.281+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 0x55a631ba78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.281+0000 7f0c77400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631baa880 0x55a631ba8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.281+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 0x55a631ba78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44608/0 (socket says 172.21.15.22:44608) 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.281+0000 7f0c76000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631ba1ca0 0x55a631ba7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.281+0000 7f0c76a00640 1 -- 172.21.15.22:0/1668812095 learned_addr learned my addr 172.21.15.22:0/1668812095 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.281+0000 7f0c76a00640 1 -- 172.21.15.22:0/1668812095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631ba1ca0 msgr2=0x55a631ba7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.281+0000 7f0c76a00640 1 --2- 172.21.15.22:0/1668812095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631ba1ca0 0x55a631ba7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.281+0000 7f0c76a00640 1 -- 172.21.15.22:0/1668812095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631baa880 msgr2=0x55a631ba8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.281+0000 7f0c76a00640 1 --2- 172.21.15.22:0/1668812095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631baa880 0x55a631ba8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.281+0000 7f0c76a00640 1 -- 172.21.15.22:0/1668812095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6318193f0 con 0x55a631ba1380 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.281+0000 7f0c76a00640 1 --2- 172.21.15.22:0/1668812095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 0x55a631ba78f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0c68098ed0 tx=0x7f0c680d4580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd5299473e8d8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.281+0000 7f0c60c00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c60c00640 1 -- 172.21.15.22:0/1668812095 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6809c3d0 con 0x55a631ba1380 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c60c00640 1 -- 172.21.15.22:0/1668812095 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f0c680ce4a0 con 0x55a631ba1380 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c805d5740 1 -- 172.21.15.22:0/1668812095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c680e2070 con 0x55a631ba1380 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c60c00640 1 -- 172.21.15.22:0/1668812095 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0c6809c5b0 con 0x55a631ba1380 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c805d5740 10 client.26039 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c805d5740 1 -- 172.21.15.22:0/1668812095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a6317d90f0 con 0x55a631ba1380 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c60c00640 1 -- 172.21.15.22:0/1668812095 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c680e2070 con 0x55a631ba1380 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c60c00640 1 -- 172.21.15.22:0/1668812095 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f0c6809ccd0 con 0x55a631ba1380 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c805d5740 10 client.26039 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c805d5740 10 client.26039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c805d5740 20 client.26039 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:51.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c805d5740 1 --2- 172.21.15.22:0/1668812095 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a631bbaa80 0x55a631bbce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c805d5740 4 client.26039 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:51.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.282+0000 7f0c805d5740 1 -- 172.21.15.22:0/1668812095 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a631b14b70 con 0x55a631bbaa80 2024-04-20T21:38:51.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.283+0000 7f0c76000640 1 --2- 172.21.15.22:0/1668812095 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a631bbaa80 0x55a631bbce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.284+0000 7f0c76000640 1 --2- 172.21.15.22:0/1668812095 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a631bbaa80 0x55a631bbce70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a4dd8c01086c4f server_cookie=266c11296805dd35 in_seq=0 out_seq=0 2024-04-20T21:38:51.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.284+0000 7f0c60c00640 10 client.26039 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:51.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c60c00640 1 -- 172.21.15.22:0/1668812095 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a631b14b70 con 0x55a631bbaa80 2024-04-20T21:38:51.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c60c00640 10 client.26039 handle_command_reply: tid=0 2024-04-20T21:38:51.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c805d5740 1 client.26039 shutdown 2024-04-20T21:38:51.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c805d5740 1 -- 172.21.15.22:0/1668812095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 msgr2=0x55a631ba78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c805d5740 1 --2- 172.21.15.22:0/1668812095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 0x55a631ba78f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0c68098ed0 tx=0x7f0c680d4580 comp rx=0 tx=0).stop 2024-04-20T21:38:51.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c805d5740 1 -- 172.21.15.22:0/1668812095 shutdown_connections 2024-04-20T21:38:51.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c805d5740 1 --2- 172.21.15.22:0/1668812095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631baa880 0x55a631ba8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c805d5740 1 --2- 172.21.15.22:0/1668812095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631ba1ca0 0x55a631ba7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c805d5740 1 --2- 172.21.15.22:0/1668812095 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a631bbaa80 0x55a631bbce70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c805d5740 1 --2- 172.21.15.22:0/1668812095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1380 0x55a631ba78f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c805d5740 1 -- 172.21.15.22:0/1668812095 >> 172.21.15.22:0/1668812095 conn(0x55a631b9bab0 msgr2=0x55a631b9d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c60c00640 0 client.26039 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:51.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c805d5740 1 -- 172.21.15.22:0/1668812095 shutdown_connections 2024-04-20T21:38:51.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c805d5740 1 -- 172.21.15.22:0/1668812095 wait complete. 2024-04-20T21:38:51.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.285+0000 7f0c805d5740 20 client.26039 trim_cache size 0 max 16384 2024-04-20T21:38:51.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.319+0000 7f0c805d5740 1 Processor -- start 2024-04-20T21:38:51.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.319+0000 7f0c805d5740 1 -- start start 2024-04-20T21:38:51.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.320+0000 7f0c805d5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631ba1380 0x55a631ba9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.320+0000 7f0c805d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1ca0 0x55a631b9d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.320+0000 7f0c805d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 0x55a631bbfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.320+0000 7f0c805d5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781ade60 con 0x55a631ba1380 2024-04-20T21:38:51.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.320+0000 7f0c805d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781b06e0 con 0x55a631ba1ca0 2024-04-20T21:38:51.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.320+0000 7f0c805d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781ae4d0 con 0x55a631baa880 2024-04-20T21:38:51.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.320+0000 7f0c77400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 0x55a631bbfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.320+0000 7f0c77400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 0x55a631bbfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:41162/0 (socket says 172.21.15.22:41162) 2024-04-20T21:38:51.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.320+0000 7f0c77400640 1 -- 172.21.15.22:0/1616464394 learned_addr learned my addr 172.21.15.22:0/1616464394 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:51.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.320+0000 7f0c76000640 1 --2- 172.21.15.22:0/1616464394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1ca0 0x55a631b9d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.321+0000 7f0c76a00640 1 --2- 172.21.15.22:0/1616464394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631ba1380 0x55a631ba9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.322+0000 7f0c77400640 1 -- 172.21.15.22:0/1616464394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1ca0 msgr2=0x55a631b9d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.322+0000 7f0c77400640 1 --2- 172.21.15.22:0/1616464394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1ca0 0x55a631b9d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.322+0000 7f0c77400640 1 -- 172.21.15.22:0/1616464394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631ba1380 msgr2=0x55a631ba9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.322+0000 7f0c77400640 1 --2- 172.21.15.22:0/1616464394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631ba1380 0x55a631ba9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.322+0000 7f0c77400640 1 -- 172.21.15.22:0/1616464394 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a631bad6d0 con 0x55a631baa880 2024-04-20T21:38:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.323+0000 7f0c76a00640 1 --2- 172.21.15.22:0/1616464394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631ba1380 0x55a631ba9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:51.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.325+0000 7f0c77400640 1 --2- 172.21.15.22:0/1616464394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 0x55a631bbfe90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0c70003eb0 tx=0x7f0c7001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dc4282bdb437716 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.326+0000 7f0c63400640 1 -- 172.21.15.22:0/1616464394 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7000f600 con 0x55a631baa880 2024-04-20T21:38:51.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.326+0000 7f0c63400640 1 -- 172.21.15.22:0/1616464394 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c7000fc80 con 0x55a631baa880 2024-04-20T21:38:51.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.326+0000 7f0c63400640 1 -- 172.21.15.22:0/1616464394 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7001fd10 con 0x55a631baa880 2024-04-20T21:38:51.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.327+0000 7f0c805d5740 1 -- 172.21.15.22:0/1616464394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 msgr2=0x55a631bbfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.327+0000 7f0c805d5740 1 --2- 172.21.15.22:0/1616464394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 0x55a631bbfe90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0c70003eb0 tx=0x7f0c7001b000 comp rx=0 tx=0).stop 2024-04-20T21:38:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.328+0000 7f0c805d5740 1 -- 172.21.15.22:0/1616464394 shutdown_connections 2024-04-20T21:38:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.328+0000 7f0c805d5740 1 --2- 172.21.15.22:0/1616464394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631baa880 0x55a631bbfe90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.328+0000 7f0c805d5740 1 --2- 172.21.15.22:0/1616464394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1ca0 0x55a631b9d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.328+0000 7f0c805d5740 1 --2- 172.21.15.22:0/1616464394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631ba1380 0x55a631ba9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.328+0000 7f0c805d5740 1 -- 172.21.15.22:0/1616464394 >> 172.21.15.22:0/1616464394 conn(0x55a631b9bab0 msgr2=0x55a631b9cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.328+0000 7f0c805d5740 1 -- 172.21.15.22:0/1616464394 shutdown_connections 2024-04-20T21:38:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.328+0000 7f0c805d5740 1 -- 172.21.15.22:0/1616464394 wait complete. 2024-04-20T21:38:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.329+0000 7f0c805d5740 1 -- start start 2024-04-20T21:38:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.329+0000 7f0c805d5740 1 Processor -- start 2024-04-20T21:38:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c805d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631ba1380 0x55a631bbd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c805d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1ca0 0x55a631bbdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c805d5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631baa880 0x55a631bbe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c805d5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781aeba0 con 0x55a631baa880 2024-04-20T21:38:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c805d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781b0010 con 0x55a631ba1ca0 2024-04-20T21:38:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c805d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781af270 con 0x55a631ba1380 2024-04-20T21:38:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c805d5740 10 client.0 _ll_register_callbacks cb 0x7f0c680d49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c76a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631ba1380 0x55a631bbd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c76000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1ca0 0x55a631bbdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c76000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1ca0 0x55a631bbdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44634/0 (socket says 172.21.15.22:44634) 2024-04-20T21:38:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c76000640 1 -- 172.21.15.22:0/4104033037 learned_addr learned my addr 172.21.15.22:0/4104033037 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:51.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c77400640 1 --2- 172.21.15.22:0/4104033037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631baa880 0x55a631bbe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c76000640 1 -- 172.21.15.22:0/4104033037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631ba1380 msgr2=0x55a631bbd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c76000640 1 --2- 172.21.15.22:0/4104033037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631ba1380 0x55a631bbd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c76000640 1 -- 172.21.15.22:0/4104033037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631baa880 msgr2=0x55a631bbe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:51.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c76000640 1 --2- 172.21.15.22:0/4104033037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631baa880 0x55a631bbe290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.330+0000 7f0c76000640 1 -- 172.21.15.22:0/4104033037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a631c1fc60 con 0x55a631ba1ca0 2024-04-20T21:38:51.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.331+0000 7f0c76000640 1 --2- 172.21.15.22:0/4104033037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1ca0 0x55a631bbdd50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0c6402c180 tx=0x7f0c64004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8aa3ea15f457962 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.331+0000 7f0c60c00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.331+0000 7f0c60c00640 1 -- 172.21.15.22:0/4104033037 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64039040 con 0x55a631ba1ca0 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.331+0000 7f0c805d5740 1 -- 172.21.15.22:0/4104033037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c7000fc80 con 0x55a631ba1ca0 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.331+0000 7f0c60c00640 1 -- 172.21.15.22:0/4104033037 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f0c640041f0 con 0x55a631ba1ca0 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.331+0000 7f0c60c00640 1 -- 172.21.15.22:0/4104033037 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0c640043f0 con 0x55a631ba1ca0 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.331+0000 7f0c805d5740 10 client.26045 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.331+0000 7f0c805d5740 1 -- 172.21.15.22:0/4104033037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a631c20f40 con 0x55a631ba1ca0 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.332+0000 7f0c60c00640 1 -- 172.21.15.22:0/4104033037 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c7000fc80 con 0x55a631ba1ca0 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.332+0000 7f0c60c00640 1 -- 172.21.15.22:0/4104033037 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f0c64032640 con 0x55a631ba1ca0 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.332+0000 7f0c805d5740 10 client.26045 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.332+0000 7f0c805d5740 10 client.26045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.332+0000 7f0c805d5740 20 client.26045 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.332+0000 7f0c805d5740 1 --2- 172.21.15.22:0/4104033037 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a631c21660 0x55a631c23a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.332+0000 7f0c805d5740 4 client.26045 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:38:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.332+0000 7f0c805d5740 1 -- 172.21.15.22:0/4104033037 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a631b0e9e0 con 0x55a631c21660 2024-04-20T21:38:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.333+0000 7f0c76a00640 1 --2- 172.21.15.22:0/4104033037 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a631c21660 0x55a631c23a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.333+0000 7f0c76a00640 1 --2- 172.21.15.22:0/4104033037 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a631c21660 0x55a631c23a80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d88c97e08c2d10 server_cookie=ff8cca41ea361558 in_seq=0 out_seq=0 2024-04-20T21:38:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.333+0000 7f0c60c00640 10 client.26045 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.335+0000 7f0c60c00640 1 -- 172.21.15.22:0/4104033037 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a631b0e9e0 con 0x55a631c21660 2024-04-20T21:38:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.335+0000 7f0c60c00640 10 client.26045 handle_command_reply: tid=0 2024-04-20T21:38:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.335+0000 7f0c805d5740 1 client.26045 shutdown 2024-04-20T21:38:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.336+0000 7f0c805d5740 1 -- 172.21.15.22:0/4104033037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1ca0 msgr2=0x55a631bbdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.336+0000 7f0c805d5740 1 --2- 172.21.15.22:0/4104033037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1ca0 0x55a631bbdd50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0c6402c180 tx=0x7f0c64004040 comp rx=0 tx=0).stop 2024-04-20T21:38:51.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.336+0000 7f0c805d5740 1 -- 172.21.15.22:0/4104033037 shutdown_connections 2024-04-20T21:38:51.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.336+0000 7f0c805d5740 1 --2- 172.21.15.22:0/4104033037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a631baa880 0x55a631bbe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.336+0000 7f0c805d5740 1 --2- 172.21.15.22:0/4104033037 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a631c21660 0x55a631c23a80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.336+0000 7f0c805d5740 1 --2- 172.21.15.22:0/4104033037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a631ba1ca0 0x55a631bbdd50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.336+0000 7f0c805d5740 1 --2- 172.21.15.22:0/4104033037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a631ba1380 0x55a631bbd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.336+0000 7f0c805d5740 1 -- 172.21.15.22:0/4104033037 >> 172.21.15.22:0/4104033037 conn(0x55a631b9bab0 msgr2=0x55a631b9d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.336+0000 7f0c60c00640 0 client.26045 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:51.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.336+0000 7f0c805d5740 1 -- 172.21.15.22:0/4104033037 shutdown_connections 2024-04-20T21:38:51.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.336+0000 7f0c805d5740 1 -- 172.21.15.22:0/4104033037 wait complete. 2024-04-20T21:38:51.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.336+0000 7f0c805d5740 20 client.26045 trim_cache size 0 max 16384 2024-04-20T21:38:51.336 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:38:51.336 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:38:51.336 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:38:51.337 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:38:51.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.337+0000 7f0c77e00640 1 -- 172.21.15.22:0/4249028091 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0c5c0a3d20 msgr2=0x7f0c5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.337+0000 7f0c77e00640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0c5c0a3d20 0x7f0c5c0a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0c70012560 tx=0x7f0c70009500 comp rx=0 tx=0).stop 2024-04-20T21:38:51.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.337+0000 7f0c77e00640 1 -- 172.21.15.22:0/4249028091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78104280 msgr2=0x7f0c781a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.337+0000 7f0c77e00640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78104280 0x7f0c781a06a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800e9c0 tx=0x7f0c6800ee80 comp rx=0 tx=0).stop 2024-04-20T21:38:51.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.337+0000 7f0c77e00640 1 -- 172.21.15.22:0/4249028091 shutdown_connections 2024-04-20T21:38:51.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.337+0000 7f0c77e00640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0c5c0a3d20 0x7f0c5c0a61e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.337+0000 7f0c77e00640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c7819a760 0x7f0c7819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.337+0000 7f0c77e00640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c78109350 0x7f0c781a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.337+0000 7f0c77e00640 1 --2- 172.21.15.22:0/4249028091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78104280 0x7f0c781a06a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.337+0000 7f0c77e00640 1 -- 172.21.15.22:0/4249028091 >> 172.21.15.22:0/4249028091 conn(0x7f0c780ffaa0 msgr2=0x7f0c78100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.337+0000 7f0c77e00640 1 -- 172.21.15.22:0/4249028091 shutdown_connections 2024-04-20T21:38:51.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.337+0000 7f0c77e00640 1 -- 172.21.15.22:0/4249028091 wait complete. 2024-04-20T21:38:51.355 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:38:51.356 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:51.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.545+0000 7fb1f2600640 1 -- 172.21.15.22:0/992422674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ec1070c0 msgr2=0x7fb1ec10ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.545+0000 7fb1f2600640 1 --2- 172.21.15.22:0/992422674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ec1070c0 0x7fb1ec10ec90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009900 tx=0x7fb1e402bc40 comp rx=0 tx=0).stop 2024-04-20T21:38:51.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.545+0000 7fb1f2600640 1 -- 172.21.15.22:0/992422674 shutdown_connections 2024-04-20T21:38:51.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.545+0000 7fb1f2600640 1 --2- 172.21.15.22:0/992422674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ec1070c0 0x7fb1ec10ec90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.545+0000 7fb1f2600640 1 --2- 172.21.15.22:0/992422674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1ec1029a0 0x7fb1ec1069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.545+0000 7fb1f2600640 1 --2- 172.21.15.22:0/992422674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1ec101ff0 0x7fb1ec1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.545+0000 7fb1f2600640 1 -- 172.21.15.22:0/992422674 >> 172.21.15.22:0/992422674 conn(0x7fb1ec0fd890 msgr2=0x7fb1ec0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.545+0000 7fb1f2600640 1 -- 172.21.15.22:0/992422674 shutdown_connections 2024-04-20T21:38:51.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.545+0000 7fb1f2600640 1 -- 172.21.15.22:0/992422674 wait complete. 2024-04-20T21:38:51.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.546+0000 7fb1f2600640 1 Processor -- start 2024-04-20T21:38:51.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.546+0000 7fb1f2600640 1 -- start start 2024-04-20T21:38:51.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1ec101ff0 0x7fb1ec074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f2600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1ec1070c0 0x7fb1ec0747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f2600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ec074ce0 0x7fb1ec1a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f2600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1121b0 con 0x7fb1ec1070c0 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f2600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec112030 con 0x7fb1ec074ce0 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f2600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec112330 con 0x7fb1ec101ff0 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f1c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ec074ce0 0x7fb1ec1a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1ebe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1ec1070c0 0x7fb1ec0747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f1c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ec074ce0 0x7fb1ec1a6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44656/0 (socket says 172.21.15.22:44656) 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f1c00640 1 -- 172.21.15.22:0/3882209115 learned_addr learned my addr 172.21.15.22:0/3882209115 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f1200640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1ec101ff0 0x7fb1ec074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f1c00640 1 -- 172.21.15.22:0/3882209115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1ec101ff0 msgr2=0x7fb1ec074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f1c00640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1ec101ff0 0x7fb1ec074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f1c00640 1 -- 172.21.15.22:0/3882209115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1ec1070c0 msgr2=0x7fb1ec0747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.547+0000 7fb1f1c00640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1ec1070c0 0x7fb1ec0747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.548+0000 7fb1f1c00640 1 -- 172.21.15.22:0/3882209115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ec1a6ee0 con 0x7fb1ec074ce0 2024-04-20T21:38:51.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.548+0000 7fb1ebe00640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1ec1070c0 0x7fb1ec0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:51.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.548+0000 7fb1f1c00640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ec074ce0 0x7fb1ec1a6840 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb1e402efd0 tx=0x7fb1e40042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.548+0000 7fb1e9600640 1 -- 172.21.15.22:0/3882209115 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4041070 con 0x7fb1ec074ce0 2024-04-20T21:38:51.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.548+0000 7fb1e9600640 1 -- 172.21.15.22:0/3882209115 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1e402cca0 con 0x7fb1ec074ce0 2024-04-20T21:38:51.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.548+0000 7fb1e9600640 1 -- 172.21.15.22:0/3882209115 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4003d60 con 0x7fb1ec074ce0 2024-04-20T21:38:51.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.548+0000 7fb1f2600640 1 -- 172.21.15.22:0/3882209115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ec1a7170 con 0x7fb1ec074ce0 2024-04-20T21:38:51.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.548+0000 7fb1f2600640 1 -- 172.21.15.22:0/3882209115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ec1a7610 con 0x7fb1ec074ce0 2024-04-20T21:38:51.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.550+0000 7fb1f2600640 1 -- 172.21.15.22:0/3882209115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ec068ca0 con 0x7fb1ec074ce0 2024-04-20T21:38:51.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.551+0000 7fb1e9600640 1 -- 172.21.15.22:0/3882209115 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb1e4044070 con 0x7fb1ec074ce0 2024-04-20T21:38:51.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.553+0000 7fb1e9600640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb1d40a3da0 0x7fb1d40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:51.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.553+0000 7fb1e9600640 1 -- 172.21.15.22:0/3882209115 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fb1e40f23e0 con 0x7fb1ec074ce0 2024-04-20T21:38:51.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.553+0000 7fb1f1200640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb1d40a3da0 0x7fb1d40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:51.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.554+0000 7fb1f1200640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb1d40a3da0 0x7fb1d40a6260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00099a0 tx=0x7fb1e0009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:51.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.557+0000 7fb1e9600640 1 -- 172.21.15.22:0/3882209115 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb1e40be4c0 con 0x7fb1ec074ce0 2024-04-20T21:38:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.868+0000 7fb1f2600640 1 -- 172.21.15.22:0/3882209115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1ec10a2f0 con 0x7fb1ec074ce0 2024-04-20T21:38:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.870+0000 7fb1e9600640 1 -- 172.21.15.22:0/3882209115 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7fb1e40c27c0 con 0x7fb1ec074ce0 2024-04-20T21:38:51.870 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:38:51.870 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:38:51.870 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:38:51.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.872+0000 7fb1f2600640 1 -- 172.21.15.22:0/3882209115 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb1d40a3da0 msgr2=0x7fb1d40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.872+0000 7fb1f2600640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb1d40a3da0 0x7fb1d40a6260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00099a0 tx=0x7fb1e0009500 comp rx=0 tx=0).stop 2024-04-20T21:38:51.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.873+0000 7fb1f2600640 1 -- 172.21.15.22:0/3882209115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ec074ce0 msgr2=0x7fb1ec1a6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:51.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.873+0000 7fb1f2600640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ec074ce0 0x7fb1ec1a6840 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb1e402efd0 tx=0x7fb1e40042e0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.873+0000 7fb1f2600640 1 -- 172.21.15.22:0/3882209115 shutdown_connections 2024-04-20T21:38:51.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.873+0000 7fb1f2600640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb1d40a3da0 0x7fb1d40a6260 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.873+0000 7fb1f2600640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ec074ce0 0x7fb1ec1a6840 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.873+0000 7fb1f2600640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1ec1070c0 0x7fb1ec0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.873+0000 7fb1f2600640 1 --2- 172.21.15.22:0/3882209115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1ec101ff0 0x7fb1ec074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:51.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.873+0000 7fb1f2600640 1 -- 172.21.15.22:0/3882209115 >> 172.21.15.22:0/3882209115 conn(0x7fb1ec0fd890 msgr2=0x7fb1ec0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:51.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.873+0000 7fb1f2600640 1 -- 172.21.15.22:0/3882209115 shutdown_connections 2024-04-20T21:38:51.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:51.873+0000 7fb1f2600640 1 -- 172.21.15.22:0/3882209115 wait complete. 2024-04-20T21:38:51.890 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:38:52.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.080+0000 7f41ffe00640 1 -- 172.21.15.22:0/3410058295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42001016d0 msgr2=0x7f4200105b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:52.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.080+0000 7f41ffe00640 1 --2- 172.21.15.22:0/3410058295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42001016d0 0x7f4200105b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009900 tx=0x7f41ec02bcd0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.080+0000 7f41ffe00640 1 -- 172.21.15.22:0/3410058295 shutdown_connections 2024-04-20T21:38:52.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.080+0000 7f41ffe00640 1 --2- 172.21.15.22:0/3410058295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42001061b0 0x7f4200106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.080+0000 7f41ffe00640 1 --2- 172.21.15.22:0/3410058295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42001016d0 0x7f4200105b90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.080+0000 7f41ffe00640 1 --2- 172.21.15.22:0/3410058295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4200100db0 0x7f4200101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.080+0000 7f41ffe00640 1 -- 172.21.15.22:0/3410058295 >> 172.21.15.22:0/3410058295 conn(0x7f42000fc7e0 msgr2=0x7f42000fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:52.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.080+0000 7f41ffe00640 1 -- 172.21.15.22:0/3410058295 shutdown_connections 2024-04-20T21:38:52.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.080+0000 7f41ffe00640 1 -- 172.21.15.22:0/3410058295 wait complete. 2024-04-20T21:38:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.081+0000 7f41ffe00640 1 Processor -- start 2024-04-20T21:38:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.081+0000 7f41ffe00640 1 -- start start 2024-04-20T21:38:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.081+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4200100db0 0x7f42001a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.081+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42001016d0 0x7f42001a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.081+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42001061b0 0x7f420019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.081+0000 7f41ffe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4200118740 con 0x7f42001061b0 2024-04-20T21:38:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.081+0000 7f41ffe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42001185c0 con 0x7f4200100db0 2024-04-20T21:38:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.081+0000 7f41ffe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f42001188c0 con 0x7f42001016d0 2024-04-20T21:38:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41fea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4200100db0 0x7f42001a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41fe000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42001016d0 0x7f42001a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41ff400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42001061b0 0x7f420019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41fea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4200100db0 0x7f42001a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44678/0 (socket says 172.21.15.22:44678) 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41fea00640 1 -- 172.21.15.22:0/2103383116 learned_addr learned my addr 172.21.15.22:0/2103383116 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41fea00640 1 -- 172.21.15.22:0/2103383116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42001016d0 msgr2=0x7f42001a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41fea00640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42001016d0 0x7f42001a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41fea00640 1 -- 172.21.15.22:0/2103383116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42001061b0 msgr2=0x7f420019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41fea00640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42001061b0 0x7f420019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41fea00640 1 -- 172.21.15.22:0/2103383116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420019f220 con 0x7f4200100db0 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41fe000640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42001016d0 0x7f42001a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41ff400640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42001061b0 0x7f420019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.082+0000 7f41fea00640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4200100db0 0x7f42001a48f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f41f000e9f0 tx=0x7f41f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.083+0000 7f4204a00640 1 -- 172.21.15.22:0/2103383116 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f000cd80 con 0x7f4200100db0 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.083+0000 7f4204a00640 1 -- 172.21.15.22:0/2103383116 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f0010910 con 0x7f4200100db0 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.083+0000 7f4204a00640 1 -- 172.21.15.22:0/2103383116 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0018990 con 0x7f4200100db0 2024-04-20T21:38:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.083+0000 7f41ffe00640 1 -- 172.21.15.22:0/2103383116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420019f510 con 0x7f4200100db0 2024-04-20T21:38:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.083+0000 7f41ffe00640 1 -- 172.21.15.22:0/2103383116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42001ab7b0 con 0x7f4200100db0 2024-04-20T21:38:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.086+0000 7f4208a80740 1 Processor -- start 2024-04-20T21:38:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.086+0000 7f4208a80740 1 -- start start 2024-04-20T21:38:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.086+0000 7f4204a00640 1 -- 172.21.15.22:0/2103383116 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f41f0010ac0 con 0x7f4200100db0 2024-04-20T21:38:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f4208a80740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f4208a80740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 0x562b4d7b8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f4208a80740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7b8670 0x562b4d7baa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f4208a80740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562b4d714050 con 0x562b4d7b8670 2024-04-20T21:38:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f4208a80740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562b4d6ba490 con 0x562b4d7afa90 2024-04-20T21:38:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f4208a80740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562b4d710010 con 0x562b4d7af170 2024-04-20T21:38:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f41fea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 0x562b4d7b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f41fe000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7b8670 0x562b4d7baa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f41fea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 0x562b4d7b8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44682/0 (socket says 172.21.15.22:44682) 2024-04-20T21:38:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f41fea00640 1 -- 172.21.15.22:0/3992453940 learned_addr learned my addr 172.21.15.22:0/3992453940 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f41ff400640 1 --2- 172.21.15.22:0/3992453940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f41fea00640 1 -- 172.21.15.22:0/3992453940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 msgr2=0x562b4d7af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f41fea00640 1 --2- 172.21.15.22:0/3992453940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.087+0000 7f41fea00640 1 -- 172.21.15.22:0/3992453940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7b8670 msgr2=0x562b4d7baa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.088+0000 7f41fea00640 1 --2- 172.21.15.22:0/3992453940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7b8670 0x562b4d7baa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.088+0000 7f41fea00640 1 -- 172.21.15.22:0/3992453940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4d6c2070 con 0x562b4d7afa90 2024-04-20T21:38:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.088+0000 7f41fe000640 1 --2- 172.21.15.22:0/3992453940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7b8670 0x562b4d7baa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.088+0000 7f41fea00640 1 --2- 172.21.15.22:0/3992453940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 0x562b4d7b8130 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f41f00ce110 tx=0x7f41f00d95c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=623e77bd5d5600a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:52.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.088+0000 7f41ebe00640 1 -- 172.21.15.22:0/3992453940 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00d8040 con 0x562b4d7afa90 2024-04-20T21:38:52.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.088+0000 7f4204a00640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f41e40a3da0 0x7f41e40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.088+0000 7f41ebe00640 1 -- 172.21.15.22:0/3992453940 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f00e2070 con 0x562b4d7afa90 2024-04-20T21:38:52.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.088+0000 7f41ebe00640 1 -- 172.21.15.22:0/3992453940 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f009c610 con 0x562b4d7afa90 2024-04-20T21:38:52.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.089+0000 7f4204a00640 1 -- 172.21.15.22:0/2103383116 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f41f00d0b00 con 0x7f4200100db0 2024-04-20T21:38:52.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.089+0000 7f41ff400640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f41e40a3da0 0x7f41e40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.089+0000 7f4208a80740 1 -- 172.21.15.22:0/3992453940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 msgr2=0x562b4d7b8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:52.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.089+0000 7f4208a80740 1 --2- 172.21.15.22:0/3992453940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 0x562b4d7b8130 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f41f00ce110 tx=0x7f41f00d95c0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.089+0000 7f4208a80740 1 -- 172.21.15.22:0/3992453940 shutdown_connections 2024-04-20T21:38:52.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.089+0000 7f4208a80740 1 --2- 172.21.15.22:0/3992453940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7b8670 0x562b4d7baa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.089+0000 7f4208a80740 1 --2- 172.21.15.22:0/3992453940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 0x562b4d7b8130 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.089+0000 7f4208a80740 1 --2- 172.21.15.22:0/3992453940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.089+0000 7f4208a80740 1 -- 172.21.15.22:0/3992453940 >> 172.21.15.22:0/3992453940 conn(0x562b4d7a98a0 msgr2=0x562b4d7a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.089+0000 7f4208a80740 1 -- 172.21.15.22:0/3992453940 shutdown_connections 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.089+0000 7f4208a80740 1 -- 172.21.15.22:0/3992453940 wait complete. 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.090+0000 7f41ff400640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f41e40a3da0 0x7f41e40a6260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f41f8005fd0 tx=0x7f41f8009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.090+0000 7f4208a80740 1 -- start start 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.090+0000 7f4208a80740 1 Processor -- start 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f4208a80740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7af170 0x562b4d7b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f4208a80740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7afa90 0x562b4d7b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f4208a80740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7b8670 0x562b4d7b6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f4208a80740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562b4d7bb400 con 0x562b4d7af170 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f4208a80740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562b4d6cd860 con 0x562b4d7b8670 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f4208a80740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562b4d7bb580 con 0x562b4d7afa90 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f4208a80740 10 client.0 _ll_register_callbacks cb 0x562b4d69bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f41fe000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7afa90 0x562b4d7b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f41ff400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7b8670 0x562b4d7b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f41fe000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7afa90 0x562b4d7b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:41202/0 (socket says 172.21.15.22:41202) 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f41fe000640 1 -- 172.21.15.22:0/2943923737 learned_addr learned my addr 172.21.15.22:0/2943923737 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f41fea00640 1 --2- 172.21.15.22:0/2943923737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7af170 0x562b4d7b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f41fe000640 1 -- 172.21.15.22:0/2943923737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7b8670 msgr2=0x562b4d7b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f41fe000640 1 --2- 172.21.15.22:0/2943923737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7b8670 0x562b4d7b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f41fe000640 1 -- 172.21.15.22:0/2943923737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7af170 msgr2=0x562b4d7b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f41fe000640 1 --2- 172.21.15.22:0/2943923737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7af170 0x562b4d7b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.091+0000 7f41fe000640 1 -- 172.21.15.22:0/2943923737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4d6d1950 con 0x562b4d7afa90 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.092+0000 7f41fe000640 1 --2- 172.21.15.22:0/2943923737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7afa90 0x562b4d7b5c10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f41ec035000 tx=0x7f41ec02c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=740e20d146e27059 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.092+0000 7f41e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.092+0000 7f41e8c00640 1 -- 172.21.15.22:0/2943923737 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0048d0 con 0x562b4d7afa90 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.092+0000 7f4208a80740 1 -- 172.21.15.22:0/2943923737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f00e2070 con 0x562b4d7afa90 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.092+0000 7f41e8c00640 1 -- 172.21.15.22:0/2943923737 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f41ec004a70 con 0x562b4d7afa90 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.092+0000 7f41e8c00640 1 -- 172.21.15.22:0/2943923737 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f41ec004c70 con 0x562b4d7afa90 2024-04-20T21:38:52.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.092+0000 7f4208a80740 10 client.26089 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.092+0000 7f4208a80740 1 -- 172.21.15.22:0/2943923737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562b4d69f820 con 0x562b4d7afa90 2024-04-20T21:38:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.093+0000 7f41e8c00640 1 -- 172.21.15.22:0/2943923737 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f00e2070 con 0x562b4d7afa90 2024-04-20T21:38:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.093+0000 7f41e8c00640 1 -- 172.21.15.22:0/2943923737 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f41ec003980 con 0x562b4d7afa90 2024-04-20T21:38:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.093+0000 7f4208a80740 10 client.26089 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.093+0000 7f4208a80740 10 client.26089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.093+0000 7f4208a80740 20 client.26089 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.093+0000 7f4208a80740 1 --2- 172.21.15.22:0/2943923737 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562b4d7c85d0 0x562b4d7ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.093+0000 7f4208a80740 4 client.26089 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.093+0000 7f4208a80740 1 -- 172.21.15.22:0/2943923737 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b4d721a70 con 0x562b4d7c85d0 2024-04-20T21:38:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.093+0000 7f41fea00640 1 --2- 172.21.15.22:0/2943923737 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562b4d7c85d0 0x562b4d7ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.094+0000 7f41fea00640 1 --2- 172.21.15.22:0/2943923737 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562b4d7c85d0 0x562b4d7ca9e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c68da723e1b1ed server_cookie=e0744cab64dd62cc in_seq=0 out_seq=0 2024-04-20T21:38:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.094+0000 7f41e8c00640 10 client.26089 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:52.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f41e8c00640 1 -- 172.21.15.22:0/2943923737 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b4d721a70 con 0x562b4d7c85d0 2024-04-20T21:38:52.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f41e8c00640 10 client.26089 handle_command_reply: tid=0 2024-04-20T21:38:52.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f4208a80740 1 client.26089 shutdown 2024-04-20T21:38:52.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f4208a80740 1 -- 172.21.15.22:0/2943923737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7afa90 msgr2=0x562b4d7b5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:52.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f4208a80740 1 --2- 172.21.15.22:0/2943923737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7afa90 0x562b4d7b5c10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f41ec035000 tx=0x7f41ec02c9c0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f4208a80740 1 -- 172.21.15.22:0/2943923737 shutdown_connections 2024-04-20T21:38:52.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f4208a80740 1 --2- 172.21.15.22:0/2943923737 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562b4d7c85d0 0x562b4d7ca9e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f4208a80740 1 --2- 172.21.15.22:0/2943923737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7b8670 0x562b4d7b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f4208a80740 1 --2- 172.21.15.22:0/2943923737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7afa90 0x562b4d7b5c10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f4208a80740 1 --2- 172.21.15.22:0/2943923737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7af170 0x562b4d7b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f41e8c00640 0 client.26089 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:52.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f4208a80740 1 -- 172.21.15.22:0/2943923737 >> 172.21.15.22:0/2943923737 conn(0x562b4d7a98a0 msgr2=0x562b4d7ab5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:52.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f4208a80740 1 -- 172.21.15.22:0/2943923737 shutdown_connections 2024-04-20T21:38:52.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f4208a80740 1 -- 172.21.15.22:0/2943923737 wait complete. 2024-04-20T21:38:52.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.096+0000 7f4208a80740 20 client.26089 trim_cache size 0 max 16384 2024-04-20T21:38:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.112+0000 7f4208a80740 1 Processor -- start 2024-04-20T21:38:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.112+0000 7f4208a80740 1 -- start start 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.112+0000 7f4208a80740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.112+0000 7f4208a80740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 0x562b4d7c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f4208a80740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7b8670 0x562b4d7d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f4208a80740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b21e0 con 0x562b4d7b8670 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f4208a80740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b4a60 con 0x562b4d7afa90 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f4208a80740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b2850 con 0x562b4d7af170 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f41fe000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 0x562b4d7c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f41ff400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7b8670 0x562b4d7d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f41fe000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 0x562b4d7c42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44696/0 (socket says 172.21.15.22:44696) 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f41fe000640 1 -- 172.21.15.22:0/2406024741 learned_addr learned my addr 172.21.15.22:0/2406024741 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f41fea00640 1 --2- 172.21.15.22:0/2406024741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f41fe000640 1 -- 172.21.15.22:0/2406024741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 msgr2=0x562b4d7b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:52.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f41fe000640 1 --2- 172.21.15.22:0/2406024741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f41fe000640 1 -- 172.21.15.22:0/2406024741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7b8670 msgr2=0x562b4d7d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f41fe000640 1 --2- 172.21.15.22:0/2406024741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7b8670 0x562b4d7d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f41fe000640 1 -- 172.21.15.22:0/2406024741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4d4263f0 con 0x562b4d7afa90 2024-04-20T21:38:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.113+0000 7f41fea00640 1 --2- 172.21.15.22:0/2406024741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.114+0000 7f41fe000640 1 --2- 172.21.15.22:0/2406024741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 0x562b4d7c42c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f41ec036040 tx=0x7f41ec005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89992eb8a9324384 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.114+0000 7f41eb400640 1 -- 172.21.15.22:0/2406024741 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec056070 con 0x562b4d7afa90 2024-04-20T21:38:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.114+0000 7f41eb400640 1 -- 172.21.15.22:0/2406024741 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41ec009520 con 0x562b4d7afa90 2024-04-20T21:38:52.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.114+0000 7f41eb400640 1 -- 172.21.15.22:0/2406024741 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0039e0 con 0x562b4d7afa90 2024-04-20T21:38:52.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.115+0000 7f4208a80740 1 -- 172.21.15.22:0/2406024741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 msgr2=0x562b4d7c42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:52.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.115+0000 7f4208a80740 1 --2- 172.21.15.22:0/2406024741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 0x562b4d7c42c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f41ec036040 tx=0x7f41ec005e10 comp rx=0 tx=0).stop 2024-04-20T21:38:52.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.115+0000 7f4208a80740 1 -- 172.21.15.22:0/2406024741 shutdown_connections 2024-04-20T21:38:52.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.115+0000 7f4208a80740 1 --2- 172.21.15.22:0/2406024741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7b8670 0x562b4d7d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.115+0000 7f4208a80740 1 --2- 172.21.15.22:0/2406024741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7afa90 0x562b4d7c42c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.115+0000 7f4208a80740 1 --2- 172.21.15.22:0/2406024741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.115+0000 7f4208a80740 1 -- 172.21.15.22:0/2406024741 >> 172.21.15.22:0/2406024741 conn(0x562b4d7a98a0 msgr2=0x562b4d7b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.115+0000 7f4208a80740 1 -- 172.21.15.22:0/2406024741 shutdown_connections 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.115+0000 7f4208a80740 1 -- 172.21.15.22:0/2406024741 wait complete. 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.115+0000 7f4208a80740 1 -- start start 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f4208a80740 1 Processor -- start 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f4208a80740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f4208a80740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7afa90 0x562b4d7bbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f4208a80740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7b8670 0x562b4d7bc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f4208a80740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b2f20 con 0x562b4d7afa90 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f4208a80740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b4390 con 0x562b4d7b8670 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f4208a80740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b35f0 con 0x562b4d7af170 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f4208a80740 10 client.0 _ll_register_callbacks cb 0x7f41ec004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f41fea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f41fe000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7afa90 0x562b4d7bbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f41ff400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7b8670 0x562b4d7bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f41fea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7c0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:41228/0 (socket says 172.21.15.22:41228) 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.116+0000 7f41fea00640 1 -- 172.21.15.22:0/2263559138 learned_addr learned my addr 172.21.15.22:0/2263559138 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f41fea00640 1 -- 172.21.15.22:0/2263559138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7b8670 msgr2=0x562b4d7bc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f41fea00640 1 --2- 172.21.15.22:0/2263559138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7b8670 0x562b4d7bc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f41fea00640 1 -- 172.21.15.22:0/2263559138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7afa90 msgr2=0x562b4d7bbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f41fea00640 1 --2- 172.21.15.22:0/2263559138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7afa90 0x562b4d7bbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f41fea00640 1 -- 172.21.15.22:0/2263559138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4d4f07d0 con 0x562b4d7af170 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f41fea00640 1 --2- 172.21.15.22:0/2263559138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7c0a60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f41f00d9ac0 tx=0x7f41f0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15983526a132686e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f41e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f41e8c00640 1 -- 172.21.15.22:0/2263559138 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f009b090 con 0x562b4d7af170 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f41e8c00640 1 -- 172.21.15.22:0/2263559138 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f41f00d46a0 con 0x562b4d7af170 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f4208a80740 1 -- 172.21.15.22:0/2263559138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41ec009520 con 0x562b4d7af170 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f41e8c00640 1 -- 172.21.15.22:0/2263559138 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f41f009b280 con 0x562b4d7af170 2024-04-20T21:38:52.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f4208a80740 10 client.26095 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:52.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.117+0000 7f4208a80740 1 -- 172.21.15.22:0/2263559138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562b4d82e870 con 0x562b4d7af170 2024-04-20T21:38:52.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.118+0000 7f41e8c00640 1 -- 172.21.15.22:0/2263559138 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41ec009520 con 0x562b4d7af170 2024-04-20T21:38:52.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.118+0000 7f41e8c00640 1 -- 172.21.15.22:0/2263559138 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f41f0099140 con 0x562b4d7af170 2024-04-20T21:38:52.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.118+0000 7f4208a80740 10 client.26095 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:52.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.118+0000 7f4208a80740 10 client.26095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:52.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.118+0000 7f4208a80740 20 client.26095 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:52.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.118+0000 7f4208a80740 1 --2- 172.21.15.22:0/2263559138 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562b4d82eb60 0x562b4d830f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:52.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.118+0000 7f4208a80740 4 client.26095 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:38:52.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.118+0000 7f4208a80740 1 -- 172.21.15.22:0/2263559138 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b4d71b8e0 con 0x562b4d82eb60 2024-04-20T21:38:52.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.118+0000 7f41fe000640 1 --2- 172.21.15.22:0/2263559138 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562b4d82eb60 0x562b4d830f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:52.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.119+0000 7f41fe000640 1 --2- 172.21.15.22:0/2263559138 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562b4d82eb60 0x562b4d830f50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38017b75f29bcedb server_cookie=d1e06679ff517509 in_seq=0 out_seq=0 2024-04-20T21:38:52.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.119+0000 7f41e8c00640 10 client.26095 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:52.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.119+0000 7f41e8c00640 1 -- 172.21.15.22:0/2263559138 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b4d71b8e0 con 0x562b4d82eb60 2024-04-20T21:38:52.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.119+0000 7f41e8c00640 10 client.26095 handle_command_reply: tid=0 2024-04-20T21:38:52.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.119+0000 7f4208a80740 1 client.26095 shutdown 2024-04-20T21:38:52.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.119+0000 7f4208a80740 1 -- 172.21.15.22:0/2263559138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 msgr2=0x562b4d7c0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:52.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.119+0000 7f4208a80740 1 --2- 172.21.15.22:0/2263559138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7c0a60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f41f00d9ac0 tx=0x7f41f0009310 comp rx=0 tx=0).stop 2024-04-20T21:38:52.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.119+0000 7f4208a80740 1 -- 172.21.15.22:0/2263559138 shutdown_connections 2024-04-20T21:38:52.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.119+0000 7f4208a80740 1 --2- 172.21.15.22:0/2263559138 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x562b4d82eb60 0x562b4d830f50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.119+0000 7f4208a80740 1 --2- 172.21.15.22:0/2263559138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4d7b8670 0x562b4d7bc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.119+0000 7f4208a80740 1 --2- 172.21.15.22:0/2263559138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b4d7afa90 0x562b4d7bbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.120+0000 7f4208a80740 1 --2- 172.21.15.22:0/2263559138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4d7af170 0x562b4d7c0a60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.120+0000 7f41e8c00640 0 client.26095 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:52.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.120+0000 7f4208a80740 1 -- 172.21.15.22:0/2263559138 >> 172.21.15.22:0/2263559138 conn(0x562b4d7a98a0 msgr2=0x562b4d7aacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:52.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.120+0000 7f4208a80740 1 -- 172.21.15.22:0/2263559138 shutdown_connections 2024-04-20T21:38:52.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.120+0000 7f4208a80740 1 -- 172.21.15.22:0/2263559138 wait complete. 2024-04-20T21:38:52.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.120+0000 7f4208a80740 20 client.26095 trim_cache size 0 max 16384 2024-04-20T21:38:52.123 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:38:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.120+0000 7f41ffe00640 1 -- 172.21.15.22:0/2103383116 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f41e40a3da0 msgr2=0x7f41e40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.120+0000 7f41ffe00640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f41e40a3da0 0x7f41e40a6260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f41f8005fd0 tx=0x7f41f8009500 comp rx=0 tx=0).stop 2024-04-20T21:38:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.120+0000 7f41ffe00640 1 -- 172.21.15.22:0/2103383116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4200100db0 msgr2=0x7f42001a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.120+0000 7f41ffe00640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4200100db0 0x7f42001a48f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f41f000e9f0 tx=0x7f41f000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.120+0000 7f41ffe00640 1 -- 172.21.15.22:0/2103383116 shutdown_connections 2024-04-20T21:38:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.121+0000 7f41ffe00640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f41e40a3da0 0x7f41e40a6260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.121+0000 7f41ffe00640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42001061b0 0x7f420019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.121+0000 7f41ffe00640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42001016d0 0x7f42001a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.121+0000 7f41ffe00640 1 --2- 172.21.15.22:0/2103383116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4200100db0 0x7f42001a48f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:52.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.121+0000 7f41ffe00640 1 -- 172.21.15.22:0/2103383116 >> 172.21.15.22:0/2103383116 conn(0x7f42000fc7e0 msgr2=0x7f42000fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:52.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.121+0000 7f41ffe00640 1 -- 172.21.15.22:0/2103383116 shutdown_connections 2024-04-20T21:38:52.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:52.121+0000 7f41ffe00640 1 -- 172.21.15.22:0/2103383116 wait complete. 2024-04-20T21:38:52.187 INFO:tasks.workunit.client.0.smithi022.stdout: 1 376114 9.72 MB/sec execute 80 sec latency 37.930 ms 2024-04-20T21:38:52.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:52 smithi122 ceph-mon[22147]: pgmap v154: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 225 op/s 2024-04-20T21:38:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:52 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:38:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:52 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:52 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:38:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:52 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:52 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:38:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:52 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3882209115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:38:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:52 smithi033 ceph-mon[21626]: pgmap v154: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 225 op/s 2024-04-20T21:38:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:52 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:38:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:52 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:52 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:38:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:52 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:52 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:38:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:52 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3882209115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:38:52.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:52 smithi022 ceph-mon[16843]: pgmap v154: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 225 op/s 2024-04-20T21:38:52.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:52 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:38:52.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:52 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:52.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:52 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:38:52.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:52 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:52.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:52 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:38:52.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:52 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3882209115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:38:53.138 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:38:53.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:38:53.138 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:38:53.188 INFO:tasks.workunit.client.0.smithi022.stdout: 1 377583 9.68 MB/sec execute 81 sec latency 16.799 ms 2024-04-20T21:38:53.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.367+0000 7f0a49600640 1 -- 172.21.15.22:0/1346298946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a44104c30 msgr2=0x7f0a44108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.367+0000 7f0a49600640 1 --2- 172.21.15.22:0/1346298946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a44104c30 0x7f0a44108c70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009ca0 tx=0x7f0a3402c100 comp rx=0 tx=0).stop 2024-04-20T21:38:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.367+0000 7f0a49600640 1 -- 172.21.15.22:0/1346298946 shutdown_connections 2024-04-20T21:38:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.367+0000 7f0a49600640 1 --2- 172.21.15.22:0/1346298946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a44109350 0x7f0a44110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.367+0000 7f0a49600640 1 --2- 172.21.15.22:0/1346298946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a44104c30 0x7f0a44108c70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.367+0000 7f0a49600640 1 --2- 172.21.15.22:0/1346298946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a44104280 0x7f0a44104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.367+0000 7f0a49600640 1 -- 172.21.15.22:0/1346298946 >> 172.21.15.22:0/1346298946 conn(0x7f0a440ffaa0 msgr2=0x7f0a44101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.367+0000 7f0a49600640 1 -- 172.21.15.22:0/1346298946 shutdown_connections 2024-04-20T21:38:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.367+0000 7f0a49600640 1 -- 172.21.15.22:0/1346298946 wait complete. 2024-04-20T21:38:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.368+0000 7f0a49600640 1 Processor -- start 2024-04-20T21:38:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.369+0000 7f0a49600640 1 -- start start 2024-04-20T21:38:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.369+0000 7f0a49600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a44104280 0x7f0a441a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.369+0000 7f0a49600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a44109350 0x7f0a441a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.369+0000 7f0a49600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4419a6e0 0x7f0a4419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.369+0000 7f0a49600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441143c0 con 0x7f0a44104280 2024-04-20T21:38:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.369+0000 7f0a49600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a44114240 con 0x7f0a4419a6e0 2024-04-20T21:38:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.369+0000 7f0a49600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a44114540 con 0x7f0a44109350 2024-04-20T21:38:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.369+0000 7f0a48c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4419a6e0 0x7f0a4419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.369+0000 7f0a43e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a44104280 0x7f0a441a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.369+0000 7f0a48c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4419a6e0 0x7f0a4419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50844/0 (socket says 172.21.15.22:50844) 2024-04-20T21:38:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.369+0000 7f0a48c00640 1 -- 172.21.15.22:0/3720716997 learned_addr learned my addr 172.21.15.22:0/3720716997 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.369+0000 7f0a3b400640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a44109350 0x7f0a441a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.370+0000 7f0a48c00640 1 -- 172.21.15.22:0/3720716997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a44109350 msgr2=0x7f0a441a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.370+0000 7f0a48c00640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a44109350 0x7f0a441a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.370+0000 7f0a48c00640 1 -- 172.21.15.22:0/3720716997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a44104280 msgr2=0x7f0a441a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.370+0000 7f0a48c00640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a44104280 0x7f0a441a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.370+0000 7f0a48c00640 1 -- 172.21.15.22:0/3720716997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4419b2e0 con 0x7f0a4419a6e0 2024-04-20T21:38:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.370+0000 7f0a48c00640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4419a6e0 0x7f0a4419ab60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00c970 tx=0x7f0a3c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:53.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.370+0000 7f0a41600640 1 -- 172.21.15.22:0/3720716997 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c007bb0 con 0x7f0a4419a6e0 2024-04-20T21:38:53.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.371+0000 7f0a49600640 1 -- 172.21.15.22:0/3720716997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a441a73c0 con 0x7f0a4419a6e0 2024-04-20T21:38:53.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.371+0000 7f0a41600640 1 -- 172.21.15.22:0/3720716997 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3c007d50 con 0x7f0a4419a6e0 2024-04-20T21:38:53.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.371+0000 7f0a41600640 1 -- 172.21.15.22:0/3720716997 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c018800 con 0x7f0a4419a6e0 2024-04-20T21:38:53.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.371+0000 7f0a49600640 1 -- 172.21.15.22:0/3720716997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a441a7770 con 0x7f0a4419a6e0 2024-04-20T21:38:53.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.374+0000 7f0a4da36740 1 Processor -- start 2024-04-20T21:38:53.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.374+0000 7f0a41600640 1 -- 172.21.15.22:0/3720716997 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0a3c00fa50 con 0x7f0a4419a6e0 2024-04-20T21:38:53.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.374+0000 7f0a4da36740 1 -- start start 2024-04-20T21:38:53.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a4da36740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae340f0 0x55c96ae344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a4da36740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 0x55c96ae3d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a4da36740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae3f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a4da36740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c96ad98d20 con 0x55c96ae3d5f0 2024-04-20T21:38:53.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a4da36740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c96ad3f3e0 con 0x55c96ae34a10 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a4da36740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c96ad94ce0 con 0x55c96ae340f0 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a43e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 0x55c96ae3d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a43e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 0x55c96ae3d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50856/0 (socket says 172.21.15.22:50856) 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a43e00640 1 -- 172.21.15.22:0/1826897647 learned_addr learned my addr 172.21.15.22:0/1826897647 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a3b400640 1 --2- 172.21.15.22:0/1826897647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae340f0 0x55c96ae344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a3b400640 1 --2- 172.21.15.22:0/1826897647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae3f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a43e00640 1 -- 172.21.15.22:0/1826897647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae340f0 msgr2=0x55c96ae344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a43e00640 1 --2- 172.21.15.22:0/1826897647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae340f0 0x55c96ae344d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a43e00640 1 -- 172.21.15.22:0/1826897647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 msgr2=0x55c96ae3f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a43e00640 1 --2- 172.21.15.22:0/1826897647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae3f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a43e00640 1 -- 172.21.15.22:0/1826897647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96ad5e980 con 0x55c96ae34a10 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.375+0000 7f0a3b400640 1 --2- 172.21.15.22:0/1826897647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae3f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.376+0000 7f0a43e00640 1 --2- 172.21.15.22:0/1826897647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 0x55c96ae3d0b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009cf0 tx=0x7f0a2c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f7158fb5795c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.376+0000 7f0a39200640 1 -- 172.21.15.22:0/1826897647 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c004230 con 0x55c96ae34a10 2024-04-20T21:38:53.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.376+0000 7f0a41600640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0a280a3d20 0x7f0a280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.376+0000 7f0a39200640 1 -- 172.21.15.22:0/1826897647 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c0043d0 con 0x55c96ae34a10 2024-04-20T21:38:53.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.376+0000 7f0a39200640 1 -- 172.21.15.22:0/1826897647 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c019960 con 0x55c96ae34a10 2024-04-20T21:38:53.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.376+0000 7f0a41600640 1 -- 172.21.15.22:0/3720716997 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f0a3c0d07f0 con 0x7f0a4419a6e0 2024-04-20T21:38:53.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.376+0000 7f0a3b400640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0a280a3d20 0x7f0a280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.377+0000 7f0a4da36740 1 -- 172.21.15.22:0/1826897647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 msgr2=0x55c96ae3d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.377+0000 7f0a4da36740 1 --2- 172.21.15.22:0/1826897647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 0x55c96ae3d0b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009cf0 tx=0x7f0a2c011810 comp rx=0 tx=0).stop 2024-04-20T21:38:53.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.377+0000 7f0a4da36740 1 -- 172.21.15.22:0/1826897647 shutdown_connections 2024-04-20T21:38:53.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.377+0000 7f0a4da36740 1 --2- 172.21.15.22:0/1826897647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae3f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.377+0000 7f0a4da36740 1 --2- 172.21.15.22:0/1826897647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 0x55c96ae3d0b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.377+0000 7f0a4da36740 1 --2- 172.21.15.22:0/1826897647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae340f0 0x55c96ae344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.377+0000 7f0a4da36740 1 -- 172.21.15.22:0/1826897647 >> 172.21.15.22:0/1826897647 conn(0x55c96ae2e820 msgr2=0x55c96ae2ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:53.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.377+0000 7f0a4da36740 1 -- 172.21.15.22:0/1826897647 shutdown_connections 2024-04-20T21:38:53.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.377+0000 7f0a3b400640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0a280a3d20 0x7f0a280a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0a3402c710 tx=0x7f0a3402c100 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:53.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.377+0000 7f0a4da36740 1 -- 172.21.15.22:0/1826897647 wait complete. 2024-04-20T21:38:53.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.378+0000 7f0a4da36740 1 -- start start 2024-04-20T21:38:53.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.378+0000 7f0a4da36740 1 Processor -- start 2024-04-20T21:38:53.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.378+0000 7f0a4da36740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 0x55c96ae3a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.378+0000 7f0a4da36740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae34a10 0x55c96ae3aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.378+0000 7f0a4da36740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae3b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.378+0000 7f0a4da36740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c96ae403e0 con 0x55c96ae3d5f0 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.378+0000 7f0a4da36740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c96ad527a0 con 0x55c96ae340f0 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.378+0000 7f0a4da36740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c96ae40560 con 0x55c96ae34a10 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.379+0000 7f0a43e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 0x55c96ae3a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.379+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae34a10 0x55c96ae3aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.379+0000 7f0a43e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 0x55c96ae3a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50858/0 (socket says 172.21.15.22:50858) 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.379+0000 7f0a4da36740 10 client.0 _ll_register_callbacks cb 0x55c96ad54b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.379+0000 7f0a43e00640 1 -- 172.21.15.22:0/3532935182 learned_addr learned my addr 172.21.15.22:0/3532935182 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.379+0000 7f0a48c00640 1 --2- 172.21.15.22:0/3532935182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae3b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.379+0000 7f0a43e00640 1 -- 172.21.15.22:0/3532935182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae34a10 msgr2=0x55c96ae3aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.379+0000 7f0a43e00640 1 --2- 172.21.15.22:0/3532935182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae34a10 0x55c96ae3aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.379+0000 7f0a43e00640 1 -- 172.21.15.22:0/3532935182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 msgr2=0x55c96ae3b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.379+0000 7f0a43e00640 1 --2- 172.21.15.22:0/3532935182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae3b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.379+0000 7f0a43e00640 1 -- 172.21.15.22:0/3532935182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96ab737e0 con 0x55c96ae340f0 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.379+0000 7f0a43e00640 1 --2- 172.21.15.22:0/3532935182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 0x55c96ae3a660 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c003820 tx=0x7f0a2c003850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e98e2057bda5e17f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.380+0000 7f0a31600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.380+0000 7f0a31600640 1 -- 172.21.15.22:0/3532935182 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c027070 con 0x55c96ae340f0 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.380+0000 7f0a31600640 1 -- 172.21.15.22:0/3532935182 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f0a2c005240 con 0x55c96ae340f0 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.380+0000 7f0a4da36740 1 -- 172.21.15.22:0/3532935182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a2c0043d0 con 0x55c96ae340f0 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.380+0000 7f0a31600640 1 -- 172.21.15.22:0/3532935182 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0a2c033050 con 0x55c96ae340f0 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.380+0000 7f0a4da36740 10 client.26081 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.380+0000 7f0a4da36740 1 -- 172.21.15.22:0/3532935182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c96abd3f70 con 0x55c96ae340f0 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.380+0000 7f0a31600640 1 -- 172.21.15.22:0/3532935182 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c0043d0 con 0x55c96ae340f0 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.381+0000 7f0a31600640 1 -- 172.21.15.22:0/3532935182 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f0a2c004ba0 con 0x55c96ae340f0 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.381+0000 7f0a4da36740 10 client.26081 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:53.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.381+0000 7f0a4da36740 10 client.26081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:53.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.381+0000 7f0a4da36740 20 client.26081 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:53.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.381+0000 7f0a4da36740 1 --2- 172.21.15.22:0/3532935182 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c96ae4d880 0x55c96ae4fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.381+0000 7f0a4da36740 4 client.26081 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:53.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.381+0000 7f0a4da36740 1 -- 172.21.15.22:0/3532935182 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c96ada6740 con 0x55c96ae4d880 2024-04-20T21:38:53.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.381+0000 7f0a3b400640 1 --2- 172.21.15.22:0/3532935182 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c96ae4d880 0x55c96ae4fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.382+0000 7f0a3b400640 1 --2- 172.21.15.22:0/3532935182 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c96ae4d880 0x55c96ae4fc70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28ec953264bca560 server_cookie=af72acbde58dea14 in_seq=0 out_seq=0 2024-04-20T21:38:53.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.382+0000 7f0a31600640 10 client.26081 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:53.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.382+0000 7f0a31600640 1 -- 172.21.15.22:0/3532935182 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c96ada6740 con 0x55c96ae4d880 2024-04-20T21:38:53.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.383+0000 7f0a31600640 10 client.26081 handle_command_reply: tid=0 2024-04-20T21:38:53.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.383+0000 7f0a4da36740 1 client.26081 shutdown 2024-04-20T21:38:53.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.383+0000 7f0a4da36740 1 -- 172.21.15.22:0/3532935182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 msgr2=0x55c96ae3a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.383+0000 7f0a4da36740 1 --2- 172.21.15.22:0/3532935182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 0x55c96ae3a660 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c003820 tx=0x7f0a2c003850 comp rx=0 tx=0).stop 2024-04-20T21:38:53.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.383+0000 7f0a4da36740 1 -- 172.21.15.22:0/3532935182 shutdown_connections 2024-04-20T21:38:53.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.383+0000 7f0a4da36740 1 --2- 172.21.15.22:0/3532935182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae3b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.383+0000 7f0a4da36740 1 --2- 172.21.15.22:0/3532935182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae34a10 0x55c96ae3aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.383+0000 7f0a4da36740 1 --2- 172.21.15.22:0/3532935182 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c96ae4d880 0x55c96ae4fc70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.383+0000 7f0a4da36740 1 --2- 172.21.15.22:0/3532935182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 0x55c96ae3a660 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.383+0000 7f0a4da36740 1 -- 172.21.15.22:0/3532935182 >> 172.21.15.22:0/3532935182 conn(0x55c96ae2e820 msgr2=0x55c96ae30570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:53.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.383+0000 7f0a31600640 0 client.26081 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:53.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.384+0000 7f0a4da36740 1 -- 172.21.15.22:0/3532935182 shutdown_connections 2024-04-20T21:38:53.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.384+0000 7f0a4da36740 1 -- 172.21.15.22:0/3532935182 wait complete. 2024-04-20T21:38:53.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.384+0000 7f0a4da36740 20 client.26081 trim_cache size 0 max 16384 2024-04-20T21:38:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.404+0000 7f0a4da36740 1 Processor -- start 2024-04-20T21:38:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.405+0000 7f0a4da36740 1 -- start start 2024-04-20T21:38:53.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.405+0000 7f0a4da36740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae340f0 0x55c96ae3cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.405+0000 7f0a4da36740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 0x55c96ae492a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.405+0000 7f0a4da36740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae2d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.405+0000 7f0a4da36740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441adcf0 con 0x55c96ae3d5f0 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.405+0000 7f0a4da36740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441b0570 con 0x55c96ae34a10 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.405+0000 7f0a4da36740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441ae360 con 0x55c96ae340f0 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.405+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 0x55c96ae492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.405+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 0x55c96ae492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50860/0 (socket says 172.21.15.22:50860) 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.405+0000 7f0a3b400640 1 -- 172.21.15.22:0/4247868075 learned_addr learned my addr 172.21.15.22:0/4247868075 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.405+0000 7f0a48c00640 1 --2- 172.21.15.22:0/4247868075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae2d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.405+0000 7f0a43e00640 1 --2- 172.21.15.22:0/4247868075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae340f0 0x55c96ae3cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.406+0000 7f0a3b400640 1 -- 172.21.15.22:0/4247868075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae340f0 msgr2=0x55c96ae3cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.406+0000 7f0a3b400640 1 --2- 172.21.15.22:0/4247868075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae340f0 0x55c96ae3cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.406+0000 7f0a3b400640 1 -- 172.21.15.22:0/4247868075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 msgr2=0x55c96ae2d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.406+0000 7f0a3b400640 1 --2- 172.21.15.22:0/4247868075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae2d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.406+0000 7f0a3b400640 1 -- 172.21.15.22:0/4247868075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96aa6b170 con 0x55c96ae34a10 2024-04-20T21:38:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.406+0000 7f0a3b400640 1 --2- 172.21.15.22:0/4247868075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 0x55c96ae492a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0a34004af0 tx=0x7f0a340095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd487482cab4dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.406+0000 7f0a33e00640 1 -- 172.21.15.22:0/4247868075 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34057070 con 0x55c96ae34a10 2024-04-20T21:38:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.406+0000 7f0a33e00640 1 -- 172.21.15.22:0/4247868075 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a34003cf0 con 0x55c96ae34a10 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.406+0000 7f0a33e00640 1 -- 172.21.15.22:0/4247868075 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34051070 con 0x55c96ae34a10 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.407+0000 7f0a4da36740 1 -- 172.21.15.22:0/4247868075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 msgr2=0x55c96ae492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.407+0000 7f0a4da36740 1 --2- 172.21.15.22:0/4247868075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 0x55c96ae492a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0a34004af0 tx=0x7f0a340095c0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.407+0000 7f0a4da36740 1 -- 172.21.15.22:0/4247868075 shutdown_connections 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.407+0000 7f0a4da36740 1 --2- 172.21.15.22:0/4247868075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae3d5f0 0x55c96ae2d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.407+0000 7f0a4da36740 1 --2- 172.21.15.22:0/4247868075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae34a10 0x55c96ae492a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.407+0000 7f0a4da36740 1 --2- 172.21.15.22:0/4247868075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae340f0 0x55c96ae3cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.407+0000 7f0a4da36740 1 -- 172.21.15.22:0/4247868075 >> 172.21.15.22:0/4247868075 conn(0x55c96ae2e820 msgr2=0x55c96ae2d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.407+0000 7f0a4da36740 1 -- 172.21.15.22:0/4247868075 shutdown_connections 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.407+0000 7f0a4da36740 1 -- 172.21.15.22:0/4247868075 wait complete. 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.408+0000 7f0a4da36740 1 -- start start 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.408+0000 7f0a4da36740 1 Processor -- start 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.408+0000 7f0a4da36740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 0x55c96ae45ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.408+0000 7f0a4da36740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae34a10 0x55c96ae41010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.408+0000 7f0a4da36740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae3d5f0 0x55c96ae41620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.408+0000 7f0a4da36740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441aea30 con 0x55c96ae34a10 2024-04-20T21:38:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.408+0000 7f0a4da36740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441afea0 con 0x55c96ae340f0 2024-04-20T21:38:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.408+0000 7f0a4da36740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441af100 con 0x55c96ae3d5f0 2024-04-20T21:38:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.408+0000 7f0a4da36740 10 client.0 _ll_register_callbacks cb 0x7f0a2c009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae34a10 0x55c96ae41010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a43e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 0x55c96ae45ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a43e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 0x55c96ae45ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50870/0 (socket says 172.21.15.22:50870) 2024-04-20T21:38:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a43e00640 1 -- 172.21.15.22:0/2926344655 learned_addr learned my addr 172.21.15.22:0/2926344655 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a48c00640 1 --2- 172.21.15.22:0/2926344655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae3d5f0 0x55c96ae41620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a43e00640 1 -- 172.21.15.22:0/2926344655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae3d5f0 msgr2=0x55c96ae41620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a43e00640 1 --2- 172.21.15.22:0/2926344655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae3d5f0 0x55c96ae41620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a43e00640 1 -- 172.21.15.22:0/2926344655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae34a10 msgr2=0x55c96ae41010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a43e00640 1 --2- 172.21.15.22:0/2926344655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae34a10 0x55c96ae41010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a43e00640 1 -- 172.21.15.22:0/2926344655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96ae41e50 con 0x55c96ae340f0 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a43e00640 1 --2- 172.21.15.22:0/2926344655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 0x55c96ae45ca0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c011d10 tx=0x7f0a2c0099d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d11c323ddfe2377 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a31600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.409+0000 7f0a31600640 1 -- 172.21.15.22:0/2926344655 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c03c060 con 0x55c96ae340f0 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.410+0000 7f0a31600640 1 -- 172.21.15.22:0/2926344655 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f0a2c005200 con 0x55c96ae340f0 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.410+0000 7f0a4da36740 1 -- 172.21.15.22:0/2926344655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a34003cf0 con 0x55c96ae340f0 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.410+0000 7f0a31600640 1 -- 172.21.15.22:0/2926344655 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0a2c027070 con 0x55c96ae340f0 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.410+0000 7f0a4da36740 10 client.26087 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.410+0000 7f0a4da36740 1 -- 172.21.15.22:0/2926344655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c96aeb3e10 con 0x55c96ae340f0 2024-04-20T21:38:53.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.410+0000 7f0a31600640 1 -- 172.21.15.22:0/2926344655 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a34003cf0 con 0x55c96ae340f0 2024-04-20T21:38:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.410+0000 7f0a31600640 1 -- 172.21.15.22:0/2926344655 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f0a2c005400 con 0x55c96ae340f0 2024-04-20T21:38:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.410+0000 7f0a4da36740 10 client.26087 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.410+0000 7f0a4da36740 10 client.26087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.410+0000 7f0a4da36740 20 client.26087 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.411+0000 7f0a4da36740 1 --2- 172.21.15.22:0/2926344655 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c96aeb4470 0x55c96aeb6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.411+0000 7f0a4da36740 4 client.26087 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:38:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.411+0000 7f0a4da36740 1 -- 172.21.15.22:0/2926344655 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c96ada05b0 con 0x55c96aeb4470 2024-04-20T21:38:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.411+0000 7f0a3b400640 1 --2- 172.21.15.22:0/2926344655 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c96aeb4470 0x55c96aeb6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.411+0000 7f0a3b400640 1 --2- 172.21.15.22:0/2926344655 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c96aeb4470 0x55c96aeb6860 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5a2f885064b40a9 server_cookie=7868ce286fcea58 in_seq=0 out_seq=0 2024-04-20T21:38:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.411+0000 7f0a31600640 10 client.26087 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:53.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.412+0000 7f0a31600640 1 -- 172.21.15.22:0/2926344655 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c96ada05b0 con 0x55c96aeb4470 2024-04-20T21:38:53.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.412+0000 7f0a31600640 10 client.26087 handle_command_reply: tid=0 2024-04-20T21:38:53.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.412+0000 7f0a4da36740 1 client.26087 shutdown 2024-04-20T21:38:53.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.412+0000 7f0a4da36740 1 -- 172.21.15.22:0/2926344655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 msgr2=0x55c96ae45ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.412+0000 7f0a4da36740 1 --2- 172.21.15.22:0/2926344655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 0x55c96ae45ca0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c011d10 tx=0x7f0a2c0099d0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.413+0000 7f0a4da36740 1 -- 172.21.15.22:0/2926344655 shutdown_connections 2024-04-20T21:38:53.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.413+0000 7f0a4da36740 1 --2- 172.21.15.22:0/2926344655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c96ae3d5f0 0x55c96ae41620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.413+0000 7f0a4da36740 1 --2- 172.21.15.22:0/2926344655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c96ae34a10 0x55c96ae41010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.413+0000 7f0a4da36740 1 --2- 172.21.15.22:0/2926344655 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55c96aeb4470 0x55c96aeb6860 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.413+0000 7f0a4da36740 1 --2- 172.21.15.22:0/2926344655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c96ae340f0 0x55c96ae45ca0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.413+0000 7f0a4da36740 1 -- 172.21.15.22:0/2926344655 >> 172.21.15.22:0/2926344655 conn(0x55c96ae2e820 msgr2=0x55c96ae303b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:53.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.413+0000 7f0a31600640 0 client.26087 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:53.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.413+0000 7f0a4da36740 1 -- 172.21.15.22:0/2926344655 shutdown_connections 2024-04-20T21:38:53.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.413+0000 7f0a4da36740 1 -- 172.21.15.22:0/2926344655 wait complete. 2024-04-20T21:38:53.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.413+0000 7f0a4da36740 20 client.26087 trim_cache size 0 max 16384 2024-04-20T21:38:53.415 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:38:53.415 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:38:53.415 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "c66a1d41-80e3-4c32-984c-bfbbd7059f87", 2024-04-20T21:38:53.415 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:38:53.415 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:38:53.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.414+0000 7f0a49600640 1 -- 172.21.15.22:0/3720716997 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0a280a3d20 msgr2=0x7f0a280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.414+0000 7f0a49600640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0a280a3d20 0x7f0a280a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0a3402c710 tx=0x7f0a3402c100 comp rx=0 tx=0).stop 2024-04-20T21:38:53.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.414+0000 7f0a49600640 1 -- 172.21.15.22:0/3720716997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4419a6e0 msgr2=0x7f0a4419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.414+0000 7f0a49600640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4419a6e0 0x7f0a4419ab60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00c970 tx=0x7f0a3c00ce30 comp rx=0 tx=0).stop 2024-04-20T21:38:53.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.414+0000 7f0a49600640 1 -- 172.21.15.22:0/3720716997 shutdown_connections 2024-04-20T21:38:53.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.414+0000 7f0a49600640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0a280a3d20 0x7f0a280a61e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.414+0000 7f0a49600640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a4419a6e0 0x7f0a4419ab60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.414+0000 7f0a49600640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a44109350 0x7f0a441a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.414+0000 7f0a49600640 1 --2- 172.21.15.22:0/3720716997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a44104280 0x7f0a441a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.414+0000 7f0a49600640 1 -- 172.21.15.22:0/3720716997 >> 172.21.15.22:0/3720716997 conn(0x7f0a440ffaa0 msgr2=0x7f0a44101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:53.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.414+0000 7f0a49600640 1 -- 172.21.15.22:0/3720716997 shutdown_connections 2024-04-20T21:38:53.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.414+0000 7f0a49600640 1 -- 172.21.15.22:0/3720716997 wait complete. 2024-04-20T21:38:53.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:38:53.432 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:38:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.649+0000 7f3e4fe00640 1 -- 172.21.15.22:0/3695390552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e50102070 msgr2=0x7f3e50102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.649+0000 7f3e4fe00640 1 --2- 172.21.15.22:0/3695390552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e50102070 0x7f3e50102450 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009900 tx=0x7f3e4002bc30 comp rx=0 tx=0).stop 2024-04-20T21:38:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.649+0000 7f3e4fe00640 1 -- 172.21.15.22:0/3695390552 shutdown_connections 2024-04-20T21:38:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.649+0000 7f3e4fe00640 1 --2- 172.21.15.22:0/3695390552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e50107140 0x7f3e5010ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.649+0000 7f3e4fe00640 1 --2- 172.21.15.22:0/3695390552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e50102a20 0x7f3e50106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.649+0000 7f3e4fe00640 1 --2- 172.21.15.22:0/3695390552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e50102070 0x7f3e50102450 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.649+0000 7f3e4fe00640 1 -- 172.21.15.22:0/3695390552 >> 172.21.15.22:0/3695390552 conn(0x7f3e500fd930 msgr2=0x7f3e500ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.649+0000 7f3e4fe00640 1 -- 172.21.15.22:0/3695390552 shutdown_connections 2024-04-20T21:38:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.649+0000 7f3e4fe00640 1 -- 172.21.15.22:0/3695390552 wait complete. 2024-04-20T21:38:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.650+0000 7f3e4fe00640 1 Processor -- start 2024-04-20T21:38:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.650+0000 7f3e4fe00640 1 -- start start 2024-04-20T21:38:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e50102070 0x7f3e50108d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e50107140 0x7f3e50109270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e501097b0 0x7f3e501aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4fe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501121f0 con 0x7f3e501097b0 2024-04-20T21:38:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4fe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50112070 con 0x7f3e50102070 2024-04-20T21:38:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4fe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50112370 con 0x7f3e50107140 2024-04-20T21:38:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4ea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e50102070 0x7f3e50108d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4ea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e50102070 0x7f3e50108d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50874/0 (socket says 172.21.15.22:50874) 2024-04-20T21:38:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4ea00640 1 -- 172.21.15.22:0/2510904555 learned_addr learned my addr 172.21.15.22:0/2510904555 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4e000640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e50107140 0x7f3e50109270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4f400640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e501097b0 0x7f3e501aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4ea00640 1 -- 172.21.15.22:0/2510904555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e50107140 msgr2=0x7f3e50109270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4ea00640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e50107140 0x7f3e50109270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4ea00640 1 -- 172.21.15.22:0/2510904555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e501097b0 msgr2=0x7f3e501aaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4ea00640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e501097b0 0x7f3e501aaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.651+0000 7f3e4ea00640 1 -- 172.21.15.22:0/2510904555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e501ab420 con 0x7f3e50102070 2024-04-20T21:38:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.652+0000 7f3e4ea00640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e50102070 0x7f3e50108d30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3e40006010 tx=0x7f3e4002cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.652+0000 7f3e47400640 1 -- 172.21.15.22:0/2510904555 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40041070 con 0x7f3e50102070 2024-04-20T21:38:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.652+0000 7f3e47400640 1 -- 172.21.15.22:0/2510904555 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e40004450 con 0x7f3e50102070 2024-04-20T21:38:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.652+0000 7f3e4fe00640 1 -- 172.21.15.22:0/2510904555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e501ab6b0 con 0x7f3e50102070 2024-04-20T21:38:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.652+0000 7f3e47400640 1 -- 172.21.15.22:0/2510904555 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40003e00 con 0x7f3e50102070 2024-04-20T21:38:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.652+0000 7f3e4fe00640 1 -- 172.21.15.22:0/2510904555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e501abb70 con 0x7f3e50102070 2024-04-20T21:38:53.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.655+0000 7f3e5896f740 1 Processor -- start 2024-04-20T21:38:53.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.655+0000 7f3e47400640 1 -- 172.21.15.22:0/2510904555 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3e40044070 con 0x7f3e50102070 2024-04-20T21:38:53.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.656+0000 7f3e5896f740 1 -- start start 2024-04-20T21:38:53.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.656+0000 7f3e5896f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 0x564b00d2f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.656+0000 7f3e5896f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d38340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.656+0000 7f3e5896f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d38880 0x564b00d3ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.656+0000 7f3e5896f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564b00c95150 con 0x564b00d2fca0 2024-04-20T21:38:53.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.656+0000 7f3e5896f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564b00c3b370 con 0x564b00d2f380 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.656+0000 7f3e5896f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564b00c91110 con 0x564b00d38880 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.656+0000 7f3e4f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 0x564b00d2f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.656+0000 7f3e4f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 0x564b00d2f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50884/0 (socket says 172.21.15.22:50884) 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.656+0000 7f3e4f400640 1 -- 172.21.15.22:0/851585412 learned_addr learned my addr 172.21.15.22:0/851585412 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.656+0000 7f3e4e000640 1 --2- 172.21.15.22:0/851585412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d38880 0x564b00d3ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.656+0000 7f3e4ea00640 1 --2- 172.21.15.22:0/851585412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d38340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.657+0000 7f3e4f400640 1 -- 172.21.15.22:0/851585412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d38880 msgr2=0x564b00d3ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.657+0000 7f3e4f400640 1 --2- 172.21.15.22:0/851585412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d38880 0x564b00d3ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.657+0000 7f3e4f400640 1 -- 172.21.15.22:0/851585412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 msgr2=0x564b00d38340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.657+0000 7f3e4f400640 1 --2- 172.21.15.22:0/851585412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d38340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.657+0000 7f3e4f400640 1 -- 172.21.15.22:0/851585412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b00a717d0 con 0x564b00d2f380 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.657+0000 7f3e4f400640 1 --2- 172.21.15.22:0/851585412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 0x564b00d2f760 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009f40 tx=0x7f3e48010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5496400e56374d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.657+0000 7f3e47400640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3e340a3d20 0x7f3e340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.657+0000 7f3e3be00640 1 -- 172.21.15.22:0/851585412 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48003b70 con 0x564b00d2f380 2024-04-20T21:38:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.657+0000 7f3e3be00640 1 -- 172.21.15.22:0/851585412 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e48003d10 con 0x564b00d2f380 2024-04-20T21:38:53.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.657+0000 7f3e3be00640 1 -- 172.21.15.22:0/851585412 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e480119d0 con 0x564b00d2f380 2024-04-20T21:38:53.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.657+0000 7f3e47400640 1 -- 172.21.15.22:0/2510904555 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f3e400f2260 con 0x7f3e50102070 2024-04-20T21:38:53.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.658+0000 7f3e4ea00640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3e340a3d20 0x7f3e340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.658+0000 7f3e5896f740 1 -- 172.21.15.22:0/851585412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 msgr2=0x564b00d2f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.658+0000 7f3e5896f740 1 --2- 172.21.15.22:0/851585412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 0x564b00d2f760 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009f40 tx=0x7f3e48010860 comp rx=0 tx=0).stop 2024-04-20T21:38:53.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.658+0000 7f3e4ea00640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3e340a3d20 0x7f3e340a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x564b00d2ee50 tx=0x7f3e4002bc30 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:53.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.658+0000 7f3e5896f740 1 -- 172.21.15.22:0/851585412 shutdown_connections 2024-04-20T21:38:53.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.658+0000 7f3e5896f740 1 --2- 172.21.15.22:0/851585412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d38880 0x564b00d3ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.658+0000 7f3e5896f740 1 --2- 172.21.15.22:0/851585412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d38340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.658+0000 7f3e5896f740 1 --2- 172.21.15.22:0/851585412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 0x564b00d2f760 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.658+0000 7f3e5896f740 1 -- 172.21.15.22:0/851585412 >> 172.21.15.22:0/851585412 conn(0x564b00d29ab0 msgr2=0x564b00d29e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.658+0000 7f3e5896f740 1 -- 172.21.15.22:0/851585412 shutdown_connections 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.658+0000 7f3e5896f740 1 -- 172.21.15.22:0/851585412 wait complete. 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.659+0000 7f3e5896f740 1 -- start start 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.659+0000 7f3e5896f740 1 Processor -- start 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e5896f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d2f380 0x564b00d35940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e5896f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d35e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e5896f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d38880 0x564b00d363c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e5896f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564b00d3b670 con 0x564b00d2fca0 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e5896f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564b00c4e740 con 0x564b00d38880 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e5896f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564b00d3b7f0 con 0x564b00d2f380 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e5896f740 10 client.0 _ll_register_callbacks cb 0x564b00c49b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e4e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d35e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e4f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d2f380 0x564b00d35940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e4e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d35e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51330/0 (socket says 172.21.15.22:51330) 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e4e000640 1 -- 172.21.15.22:0/642509007 learned_addr learned my addr 172.21.15.22:0/642509007 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e4f400640 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d38880 0x564b00d363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e4e000640 1 -- 172.21.15.22:0/642509007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d2f380 msgr2=0x564b00d35940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e4e000640 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d2f380 0x564b00d35940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e4e000640 1 -- 172.21.15.22:0/642509007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d38880 msgr2=0x564b00d363c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e4e000640 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d38880 0x564b00d363c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e4e000640 1 -- 172.21.15.22:0/642509007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b009a73f0 con 0x564b00d2fca0 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.660+0000 7f3e4f400640 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d38880 0x564b00d363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.661+0000 7f3e4e000640 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d35e80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c012700 tx=0x7f3e3c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485be69771917f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.661+0000 7f3e38c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.661+0000 7f3e38c00640 1 -- 172.21.15.22:0/642509007 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c004020 con 0x564b00d2fca0 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.661+0000 7f3e5896f740 1 -- 172.21.15.22:0/642509007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e48003d10 con 0x564b00d2fca0 2024-04-20T21:38:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.661+0000 7f3e38c00640 1 -- 172.21.15.22:0/642509007 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f3e3c0041c0 con 0x564b00d2fca0 2024-04-20T21:38:53.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.661+0000 7f3e38c00640 1 -- 172.21.15.22:0/642509007 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3e3c0043c0 con 0x564b00d2fca0 2024-04-20T21:38:53.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.661+0000 7f3e5896f740 10 client.15966 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:53.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.661+0000 7f3e5896f740 1 -- 172.21.15.22:0/642509007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564b009670f0 con 0x564b00d2fca0 2024-04-20T21:38:53.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.662+0000 7f3e38c00640 1 -- 172.21.15.22:0/642509007 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e48003d10 con 0x564b00d2fca0 2024-04-20T21:38:53.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.662+0000 7f3e38c00640 1 -- 172.21.15.22:0/642509007 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f3e3c003bb0 con 0x564b00d2fca0 2024-04-20T21:38:53.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.662+0000 7f3e5896f740 10 client.15966 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:53.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.662+0000 7f3e5896f740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:53.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.662+0000 7f3e5896f740 20 client.15966 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:53.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.662+0000 7f3e5896f740 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x564b00d48c90 0x564b00d4b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.662+0000 7f3e5896f740 4 client.15966 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:53.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.662+0000 7f3e5896f740 1 -- 172.21.15.22:0/642509007 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b00ca2b70 con 0x564b00d48c90 2024-04-20T21:38:53.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.663+0000 7f3e4f400640 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x564b00d48c90 0x564b00d4b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.663+0000 7f3e4f400640 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x564b00d48c90 0x564b00d4b0f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d87f7f97d6da55 server_cookie=23aaa69b229621b0 in_seq=0 out_seq=0 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.663+0000 7f3e38c00640 10 client.15966 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.664+0000 7f3e38c00640 1 -- 172.21.15.22:0/642509007 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b00ca2b70 con 0x564b00d48c90 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.664+0000 7f3e38c00640 10 client.15966 handle_command_reply: tid=0 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.664+0000 7f3e5896f740 1 client.15966 shutdown 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.664+0000 7f3e5896f740 1 -- 172.21.15.22:0/642509007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 msgr2=0x564b00d35e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.664+0000 7f3e5896f740 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d35e80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c012700 tx=0x7f3e3c012bc0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.664+0000 7f3e5896f740 1 -- 172.21.15.22:0/642509007 shutdown_connections 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.664+0000 7f3e5896f740 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x564b00d48c90 0x564b00d4b0f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.665+0000 7f3e5896f740 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d38880 0x564b00d363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.665+0000 7f3e5896f740 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d35e80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.665+0000 7f3e5896f740 1 --2- 172.21.15.22:0/642509007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d2f380 0x564b00d35940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.665+0000 7f3e5896f740 1 -- 172.21.15.22:0/642509007 >> 172.21.15.22:0/642509007 conn(0x564b00d29ab0 msgr2=0x564b00d2b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.665+0000 7f3e38c00640 0 client.15966 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.665+0000 7f3e5896f740 1 -- 172.21.15.22:0/642509007 shutdown_connections 2024-04-20T21:38:53.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.665+0000 7f3e5896f740 1 -- 172.21.15.22:0/642509007 wait complete. 2024-04-20T21:38:53.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.665+0000 7f3e5896f740 20 client.15966 trim_cache size 0 max 16384 2024-04-20T21:38:53.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.684+0000 7f3e5896f740 1 Processor -- start 2024-04-20T21:38:53.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.685+0000 7f3e5896f740 1 -- start start 2024-04-20T21:38:53.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.685+0000 7f3e5896f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d2f380 0x564b00d38070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.685+0000 7f3e5896f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2fca0 0x564b00d44530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.685+0000 7f3e5896f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d38880 0x564b00d34880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.685+0000 7f3e5896f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501b21b0 con 0x564b00d38880 2024-04-20T21:38:53.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.685+0000 7f3e5896f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501b4a30 con 0x564b00d2fca0 2024-04-20T21:38:53.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.685+0000 7f3e5896f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501b2820 con 0x564b00d2f380 2024-04-20T21:38:53.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.685+0000 7f3e4e000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2fca0 0x564b00d44530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.685+0000 7f3e4e000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2fca0 0x564b00d44530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50912/0 (socket says 172.21.15.22:50912) 2024-04-20T21:38:53.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.686+0000 7f3e4e000640 1 -- 172.21.15.22:0/1056219517 learned_addr learned my addr 172.21.15.22:0/1056219517 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:53.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.686+0000 7f3e4f400640 1 --2- 172.21.15.22:0/1056219517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d2f380 0x564b00d38070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.686+0000 7f3e4f400640 1 --2- 172.21.15.22:0/1056219517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d38880 0x564b00d34880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.686+0000 7f3e4e000640 1 -- 172.21.15.22:0/1056219517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d2f380 msgr2=0x564b00d38070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.686+0000 7f3e4e000640 1 --2- 172.21.15.22:0/1056219517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d2f380 0x564b00d38070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.686+0000 7f3e4e000640 1 -- 172.21.15.22:0/1056219517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d38880 msgr2=0x564b00d34880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.686+0000 7f3e4e000640 1 --2- 172.21.15.22:0/1056219517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d38880 0x564b00d34880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.686+0000 7f3e4e000640 1 -- 172.21.15.22:0/1056219517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b00d2fad0 con 0x564b00d2fca0 2024-04-20T21:38:53.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.686+0000 7f3e4f400640 1 --2- 172.21.15.22:0/1056219517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d38880 0x564b00d34880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:53.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.686+0000 7f3e4e000640 1 --2- 172.21.15.22:0/1056219517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2fca0 0x564b00d44530 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0105f0 tx=0x7f3e3c009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9bd9599b45edee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:53.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.687+0000 7f3e3b400640 1 -- 172.21.15.22:0/1056219517 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c029030 con 0x564b00d2fca0 2024-04-20T21:38:53.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.687+0000 7f3e3b400640 1 -- 172.21.15.22:0/1056219517 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3c00fe00 con 0x564b00d2fca0 2024-04-20T21:38:53.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.687+0000 7f3e3b400640 1 -- 172.21.15.22:0/1056219517 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c010990 con 0x564b00d2fca0 2024-04-20T21:38:53.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.687+0000 7f3e5896f740 1 -- 172.21.15.22:0/1056219517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2fca0 msgr2=0x564b00d44530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.687+0000 7f3e5896f740 1 --2- 172.21.15.22:0/1056219517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2fca0 0x564b00d44530 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0105f0 tx=0x7f3e3c009ee0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.688+0000 7f3e5896f740 1 -- 172.21.15.22:0/1056219517 shutdown_connections 2024-04-20T21:38:53.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.688+0000 7f3e5896f740 1 --2- 172.21.15.22:0/1056219517 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d38880 0x564b00d34880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.688+0000 7f3e5896f740 1 --2- 172.21.15.22:0/1056219517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2fca0 0x564b00d44530 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.688+0000 7f3e5896f740 1 --2- 172.21.15.22:0/1056219517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d2f380 0x564b00d38070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.688+0000 7f3e5896f740 1 -- 172.21.15.22:0/1056219517 >> 172.21.15.22:0/1056219517 conn(0x564b00d29ab0 msgr2=0x564b00d36190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.688+0000 7f3e5896f740 1 -- 172.21.15.22:0/1056219517 shutdown_connections 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.688+0000 7f3e5896f740 1 -- 172.21.15.22:0/1056219517 wait complete. 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.688+0000 7f3e5896f740 1 -- start start 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.688+0000 7f3e5896f740 1 Processor -- start 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.689+0000 7f3e5896f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 0x564b00d32d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.689+0000 7f3e5896f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d33260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.689+0000 7f3e5896f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d38880 0x564b00dae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.689+0000 7f3e5896f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501b2ef0 con 0x564b00d2fca0 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.689+0000 7f3e5896f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501b4360 con 0x564b00d2f380 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.689+0000 7f3e5896f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501b35c0 con 0x564b00d38880 2024-04-20T21:38:53.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.689+0000 7f3e5896f740 10 client.0 _ll_register_callbacks cb 0x7f3e3c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.689+0000 7f3e4f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 0x564b00d32d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.689+0000 7f3e4e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d33260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.689+0000 7f3e4f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 0x564b00d32d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50922/0 (socket says 172.21.15.22:50922) 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.689+0000 7f3e4f400640 1 -- 172.21.15.22:0/348386531 learned_addr learned my addr 172.21.15.22:0/348386531 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.689+0000 7f3e4f400640 1 --2- 172.21.15.22:0/348386531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d38880 0x564b00dae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e4f400640 1 -- 172.21.15.22:0/348386531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d38880 msgr2=0x564b00dae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e4f400640 1 --2- 172.21.15.22:0/348386531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d38880 0x564b00dae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e4f400640 1 -- 172.21.15.22:0/348386531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 msgr2=0x564b00d33260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e4f400640 1 --2- 172.21.15.22:0/348386531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d33260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e4f400640 1 -- 172.21.15.22:0/348386531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b00dae770 con 0x564b00d2f380 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e4f400640 1 --2- 172.21.15.22:0/348386531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 0x564b00d32d20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009f10 tx=0x7f3e48013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=762aaf5e00158ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e38c00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e38c00640 1 -- 172.21.15.22:0/348386531 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48025040 con 0x564b00d2f380 2024-04-20T21:38:53.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e5896f740 1 -- 172.21.15.22:0/348386531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3c00fe00 con 0x564b00d2f380 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e38c00640 1 -- 172.21.15.22:0/348386531 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f3e4802c780 con 0x564b00d2f380 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e38c00640 1 -- 172.21.15.22:0/348386531 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3e48011610 con 0x564b00d2f380 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e5896f740 10 client.26105 fetch_fsmap learned FSMap version 20 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.690+0000 7f3e5896f740 1 -- 172.21.15.22:0/348386531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564b00dafa40 con 0x564b00d2f380 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.691+0000 7f3e38c00640 1 -- 172.21.15.22:0/348386531 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3c00fe00 con 0x564b00d2f380 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.691+0000 7f3e38c00640 1 -- 172.21.15.22:0/348386531 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f3e48005320 con 0x564b00d2f380 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.691+0000 7f3e5896f740 10 client.26105 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.691+0000 7f3e5896f740 10 client.26105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.691+0000 7f3e5896f740 20 client.26105 populate_metadata read hostname 'smithi022' 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.691+0000 7f3e5896f740 1 --2- 172.21.15.22:0/348386531 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x564b00db0020 0x564b00db2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.691+0000 7f3e5896f740 4 client.26105 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.691+0000 7f3e5896f740 1 -- 172.21.15.22:0/348386531 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564b00c9c9e0 con 0x564b00db0020 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.691+0000 7f3e4e000640 1 --2- 172.21.15.22:0/348386531 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x564b00db0020 0x564b00db2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:53.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.692+0000 7f3e4e000640 1 --2- 172.21.15.22:0/348386531 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x564b00db0020 0x564b00db2410 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b7f28fdc4bd9a58 server_cookie=180f35543cb00242 in_seq=0 out_seq=0 2024-04-20T21:38:53.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.692+0000 7f3e38c00640 10 client.26105 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.694+0000 7f3e38c00640 1 -- 172.21.15.22:0/348386531 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564b00c9c9e0 con 0x564b00db0020 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.694+0000 7f3e38c00640 10 client.26105 handle_command_reply: tid=0 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.694+0000 7f3e5896f740 1 client.26105 shutdown 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.695+0000 7f3e5896f740 1 -- 172.21.15.22:0/348386531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 msgr2=0x564b00d32d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.695+0000 7f3e5896f740 1 --2- 172.21.15.22:0/348386531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 0x564b00d32d20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009f10 tx=0x7f3e48013040 comp rx=0 tx=0).stop 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.695+0000 7f3e5896f740 1 -- 172.21.15.22:0/348386531 shutdown_connections 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.695+0000 7f3e5896f740 1 --2- 172.21.15.22:0/348386531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564b00d38880 0x564b00dae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.695+0000 7f3e5896f740 1 --2- 172.21.15.22:0/348386531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564b00d2fca0 0x564b00d33260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.695+0000 7f3e5896f740 1 --2- 172.21.15.22:0/348386531 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x564b00db0020 0x564b00db2410 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.695+0000 7f3e5896f740 1 --2- 172.21.15.22:0/348386531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564b00d2f380 0x564b00d32d20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.695+0000 7f3e5896f740 1 -- 172.21.15.22:0/348386531 >> 172.21.15.22:0/348386531 conn(0x564b00d29ab0 msgr2=0x564b00d28280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.695+0000 7f3e38c00640 0 client.26105 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:38:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.695+0000 7f3e5896f740 1 -- 172.21.15.22:0/348386531 shutdown_connections 2024-04-20T21:38:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.695+0000 7f3e5896f740 1 -- 172.21.15.22:0/348386531 wait complete. 2024-04-20T21:38:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.695+0000 7f3e5896f740 20 client.26105 trim_cache size 0 max 16384 2024-04-20T21:38:53.698 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:38:53.698 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:38:53.698 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:38:53.698 INFO:teuthology.orchestra.run.smithi022.stdout: "c66a1d41-80e3-4c32-984c-bfbbd7059f87": { 2024-04-20T21:38:53.698 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:38:53.698 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "c66a1d41-80e3-4c32-984c-bfbbd7059f87", 2024-04-20T21:38:53.698 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:38:53.698 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:38:53.699 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:38:53.699 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:38:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.696+0000 7f3e4fe00640 1 -- 172.21.15.22:0/2510904555 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3e340a3d20 msgr2=0x7f3e340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.696+0000 7f3e4fe00640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3e340a3d20 0x7f3e340a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x564b00d2ee50 tx=0x7f3e4002bc30 comp rx=0 tx=0).stop 2024-04-20T21:38:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.696+0000 7f3e4fe00640 1 -- 172.21.15.22:0/2510904555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e50102070 msgr2=0x7f3e50108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.696+0000 7f3e4fe00640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e50102070 0x7f3e50108d30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3e40006010 tx=0x7f3e4002cb30 comp rx=0 tx=0).stop 2024-04-20T21:38:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.696+0000 7f3e4fe00640 1 -- 172.21.15.22:0/2510904555 shutdown_connections 2024-04-20T21:38:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.696+0000 7f3e4fe00640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3e340a3d20 0x7f3e340a61e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.696+0000 7f3e4fe00640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e501097b0 0x7f3e501aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.696+0000 7f3e4fe00640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e50107140 0x7f3e50109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.696+0000 7f3e4fe00640 1 --2- 172.21.15.22:0/2510904555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e50102070 0x7f3e50108d30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.696+0000 7f3e4fe00640 1 -- 172.21.15.22:0/2510904555 >> 172.21.15.22:0/2510904555 conn(0x7f3e500fd930 msgr2=0x7f3e500fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.696+0000 7f3e4fe00640 1 -- 172.21.15.22:0/2510904555 shutdown_connections 2024-04-20T21:38:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:38:53.696+0000 7f3e4fe00640 1 -- 172.21.15.22:0/2510904555 wait complete. 2024-04-20T21:38:53.713 INFO:tasks.cephfs.filesystem:scrub status for tag:c66a1d41-80e3-4c32-984c-bfbbd7059f87 - {'path': '/', 'tag': 'c66a1d41-80e3-4c32-984c-bfbbd7059f87', 'options': 'recursive,force'} 2024-04-20T21:38:54.188 INFO:tasks.workunit.client.0.smithi022.stdout: 1 379989 9.73 MB/sec execute 82 sec latency 7.593 ms 2024-04-20T21:38:54.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:54 smithi122 ceph-mon[22147]: pgmap v155: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-04-20T21:38:54.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:54 smithi033 ceph-mon[21626]: pgmap v155: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-04-20T21:38:54.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:54 smithi022 ceph-mon[16843]: pgmap v155: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-04-20T21:38:55.188 INFO:tasks.workunit.client.0.smithi022.stdout: 1 381948 9.70 MB/sec execute 83 sec latency 14.979 ms 2024-04-20T21:38:55.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:55 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:38:55.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:55 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:55.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:55 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:38:55.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:55 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:55.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:55 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:38:55.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:55 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:55.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:55 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:38:55.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:55 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:55.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:55 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:38:55.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:55 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:55.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:55 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:38:55.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:55 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:56.188 INFO:tasks.workunit.client.0.smithi022.stdout: 1 383487 9.73 MB/sec execute 84 sec latency 25.340 ms 2024-04-20T21:38:56.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:56 smithi122 ceph-mon[22147]: pgmap v156: 97 pgs: 97 active+clean; 214 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-04-20T21:38:56.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:56 smithi033 ceph-mon[21626]: pgmap v156: 97 pgs: 97 active+clean; 214 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-04-20T21:38:56.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:56 smithi022 ceph-mon[16843]: pgmap v156: 97 pgs: 97 active+clean; 214 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-04-20T21:38:57.188 INFO:tasks.workunit.client.0.smithi022.stdout: 1 384753 9.67 MB/sec execute 85 sec latency 41.139 ms 2024-04-20T21:38:57.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:57 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:38:57.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:57 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:38:57.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:57 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:38:58.188 INFO:tasks.workunit.client.0.smithi022.stdout: 1 386929 9.72 MB/sec execute 86 sec latency 30.033 ms 2024-04-20T21:38:58.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:38:58 smithi122 ceph-mon[22147]: pgmap v157: 97 pgs: 97 active+clean; 214 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 162 op/s 2024-04-20T21:38:58.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:38:58 smithi033 ceph-mon[21626]: pgmap v157: 97 pgs: 97 active+clean; 214 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 162 op/s 2024-04-20T21:38:58.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:38:58 smithi022 ceph-mon[16843]: pgmap v157: 97 pgs: 97 active+clean; 214 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 162 op/s 2024-04-20T21:38:59.188 INFO:tasks.workunit.client.0.smithi022.stdout: 1 388270 9.67 MB/sec execute 87 sec latency 14.612 ms 2024-04-20T21:39:00.188 INFO:tasks.workunit.client.0.smithi022.stdout: 1 390330 9.70 MB/sec execute 88 sec latency 12.729 ms 2024-04-20T21:39:00.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:00 smithi033 ceph-mon[21626]: pgmap v158: 97 pgs: 97 active+clean; 214 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 162 op/s 2024-04-20T21:39:00.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:00 smithi022 ceph-mon[16843]: pgmap v158: 97 pgs: 97 active+clean; 214 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 162 op/s 2024-04-20T21:39:00.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:00 smithi122 ceph-mon[22147]: pgmap v158: 97 pgs: 97 active+clean; 214 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 162 op/s 2024-04-20T21:39:01.189 INFO:tasks.workunit.client.0.smithi022.stdout: 1 390513 9.61 MB/sec execute 89 sec latency 59.161 ms 2024-04-20T21:39:02.189 INFO:tasks.workunit.client.0.smithi022.stdout: 1 391458 9.53 MB/sec execute 90 sec latency 57.976 ms 2024-04-20T21:39:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:02 smithi033 ceph-mon[21626]: pgmap v159: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 236 op/s 2024-04-20T21:39:02.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:02 smithi022 ceph-mon[16843]: pgmap v159: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 236 op/s 2024-04-20T21:39:02.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:02 smithi122 ceph-mon[22147]: pgmap v159: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 236 op/s 2024-04-20T21:39:03.189 INFO:tasks.workunit.client.0.smithi022.stdout: 1 393567 9.56 MB/sec execute 91 sec latency 50.344 ms 2024-04-20T21:39:04.189 INFO:tasks.workunit.client.0.smithi022.stdout: 1 394745 9.51 MB/sec execute 92 sec latency 16.926 ms 2024-04-20T21:39:04.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:04 smithi033 ceph-mon[21626]: pgmap v160: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.4 MiB/s wr, 156 op/s 2024-04-20T21:39:04.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:04 smithi022 ceph-mon[16843]: pgmap v160: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.4 MiB/s wr, 156 op/s 2024-04-20T21:39:04.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:04 smithi122 ceph-mon[22147]: pgmap v160: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.4 MiB/s wr, 156 op/s 2024-04-20T21:39:05.189 INFO:tasks.workunit.client.0.smithi022.stdout: 1 396765 9.54 MB/sec execute 93 sec latency 14.096 ms 2024-04-20T21:39:06.189 INFO:tasks.workunit.client.0.smithi022.stdout: 1 398122 9.51 MB/sec execute 94 sec latency 24.861 ms 2024-04-20T21:39:06.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:06 smithi033 ceph-mon[21626]: pgmap v161: 97 pgs: 97 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 255 op/s 2024-04-20T21:39:06.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:06 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:39:06.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:06 smithi022 ceph-mon[16843]: pgmap v161: 97 pgs: 97 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 255 op/s 2024-04-20T21:39:06.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:06 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:39:06.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:06 smithi122 ceph-mon[22147]: pgmap v161: 97 pgs: 97 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 255 op/s 2024-04-20T21:39:06.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:06 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:39:07.189 INFO:tasks.workunit.client.0.smithi022.stdout: 1 400612 9.56 MB/sec execute 95 sec latency 31.911 ms 2024-04-20T21:39:07.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:07 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:39:07.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:07 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:39:07.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:07 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:39:07.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:07 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:39:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:07 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:39:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:07 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:39:07.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:07 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:39:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:07 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:39:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:07 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:39:08.190 INFO:tasks.workunit.client.0.smithi022.stdout: 1 401966 9.53 MB/sec execute 96 sec latency 43.096 ms 2024-04-20T21:39:08.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:08 smithi033 ceph-mon[21626]: pgmap v162: 97 pgs: 97 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-04-20T21:39:08.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:08 smithi022 ceph-mon[16843]: pgmap v162: 97 pgs: 97 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-04-20T21:39:08.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:08 smithi122 ceph-mon[22147]: pgmap v162: 97 pgs: 97 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-04-20T21:39:09.190 INFO:tasks.workunit.client.0.smithi022.stdout: 1 404652 9.60 MB/sec execute 97 sec latency 8.290 ms 2024-04-20T21:39:10.190 INFO:tasks.workunit.client.0.smithi022.stdout: 1 406340 9.56 MB/sec execute 98 sec latency 78.525 ms 2024-04-20T21:39:10.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:10 smithi033 ceph-mon[21626]: pgmap v163: 97 pgs: 97 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-04-20T21:39:10.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:10 smithi022 ceph-mon[16843]: pgmap v163: 97 pgs: 97 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-04-20T21:39:10.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:10 smithi122 ceph-mon[22147]: pgmap v163: 97 pgs: 97 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-04-20T21:39:11.190 INFO:tasks.workunit.client.0.smithi022.stdout: 1 408407 9.61 MB/sec execute 99 sec latency 11.073 ms 2024-04-20T21:39:12.190 INFO:tasks.workunit.client.0.smithi022.stdout: 1 410524 9.59 MB/sec execute 100 sec latency 33.462 ms 2024-04-20T21:39:12.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:12 smithi022 ceph-mon[16843]: pgmap v164: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 248 op/s 2024-04-20T21:39:12.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:12 smithi122 ceph-mon[22147]: pgmap v164: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 248 op/s 2024-04-20T21:39:12.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:12 smithi033 ceph-mon[21626]: pgmap v164: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 248 op/s 2024-04-20T21:39:13.190 INFO:tasks.workunit.client.0.smithi022.stdout: 1 412085 9.60 MB/sec execute 101 sec latency 46.138 ms 2024-04-20T21:39:14.190 INFO:tasks.workunit.client.0.smithi022.stdout: 1 414075 9.59 MB/sec execute 102 sec latency 19.201 ms 2024-04-20T21:39:14.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:14 smithi022 ceph-mon[16843]: pgmap v165: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 2.5 MiB/s wr, 174 op/s 2024-04-20T21:39:14.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:14 smithi122 ceph-mon[22147]: pgmap v165: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 2.5 MiB/s wr, 174 op/s 2024-04-20T21:39:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:14 smithi033 ceph-mon[21626]: pgmap v165: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 2.5 MiB/s wr, 174 op/s 2024-04-20T21:39:15.190 INFO:tasks.workunit.client.0.smithi022.stdout: 1 415699 9.60 MB/sec execute 103 sec latency 16.122 ms 2024-04-20T21:39:16.191 INFO:tasks.workunit.client.0.smithi022.stdout: 1 417611 9.59 MB/sec execute 104 sec latency 12.206 ms 2024-04-20T21:39:16.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:16 smithi022 ceph-mon[16843]: pgmap v166: 97 pgs: 97 active+clean; 236 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 256 op/s 2024-04-20T21:39:16.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:16 smithi122 ceph-mon[22147]: pgmap v166: 97 pgs: 97 active+clean; 236 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 256 op/s 2024-04-20T21:39:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:16 smithi033 ceph-mon[21626]: pgmap v166: 97 pgs: 97 active+clean; 236 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 256 op/s 2024-04-20T21:39:17.191 INFO:tasks.workunit.client.0.smithi022.stdout: 1 419272 9.60 MB/sec execute 105 sec latency 33.258 ms 2024-04-20T21:39:18.191 INFO:tasks.workunit.client.0.smithi022.stdout: 1 421501 9.63 MB/sec execute 106 sec latency 42.083 ms 2024-04-20T21:39:18.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:18 smithi022 ceph-mon[16843]: pgmap v167: 97 pgs: 97 active+clean; 236 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 158 op/s 2024-04-20T21:39:18.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:18 smithi122 ceph-mon[22147]: pgmap v167: 97 pgs: 97 active+clean; 236 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 158 op/s 2024-04-20T21:39:18.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:18 smithi033 ceph-mon[21626]: pgmap v167: 97 pgs: 97 active+clean; 236 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 158 op/s 2024-04-20T21:39:19.191 INFO:tasks.workunit.client.0.smithi022.stdout: 1 423068 9.61 MB/sec execute 107 sec latency 12.307 ms 2024-04-20T21:39:20.191 INFO:tasks.workunit.client.0.smithi022.stdout: 1 425748 9.68 MB/sec execute 108 sec latency 9.460 ms 2024-04-20T21:39:20.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:20 smithi022 ceph-mon[16843]: pgmap v168: 97 pgs: 97 active+clean; 236 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 158 op/s 2024-04-20T21:39:20.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:20 smithi122 ceph-mon[22147]: pgmap v168: 97 pgs: 97 active+clean; 236 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 158 op/s 2024-04-20T21:39:20.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:20 smithi033 ceph-mon[21626]: pgmap v168: 97 pgs: 97 active+clean; 236 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 158 op/s 2024-04-20T21:39:21.191 INFO:tasks.workunit.client.0.smithi022.stdout: 1 427313 9.65 MB/sec execute 109 sec latency 12.230 ms 2024-04-20T21:39:22.191 INFO:tasks.workunit.client.0.smithi022.stdout: 1 429602 9.69 MB/sec execute 110 sec latency 10.818 ms 2024-04-20T21:39:22.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:22 smithi022 ceph-mon[16843]: pgmap v169: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 241 op/s 2024-04-20T21:39:22.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:22 smithi122 ceph-mon[22147]: pgmap v169: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 241 op/s 2024-04-20T21:39:22.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:22 smithi033 ceph-mon[21626]: pgmap v169: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 241 op/s 2024-04-20T21:39:23.191 INFO:tasks.workunit.client.0.smithi022.stdout: 1 430815 9.65 MB/sec execute 111 sec latency 109.698 ms 2024-04-20T21:39:23.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:39:23.715 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:39:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.947+0000 7efd2c400640 1 -- 172.21.15.22:0/3307197398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd24108590 msgr2=0x7efd24108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.947+0000 7efd2c400640 1 --2- 172.21.15.22:0/3307197398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd24108590 0x7efd24108a10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd10009890 tx=0x7efd1002bb70 comp rx=0 tx=0).stop 2024-04-20T21:39:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.947+0000 7efd2c400640 1 -- 172.21.15.22:0/3307197398 shutdown_connections 2024-04-20T21:39:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.947+0000 7efd2c400640 1 --2- 172.21.15.22:0/3307197398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd24108f50 0x7efd241153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.947+0000 7efd2c400640 1 --2- 172.21.15.22:0/3307197398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd24108590 0x7efd24108a10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.947+0000 7efd2c400640 1 --2- 172.21.15.22:0/3307197398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd2410e600 0x7efd2410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.947+0000 7efd2c400640 1 -- 172.21.15.22:0/3307197398 >> 172.21.15.22:0/3307197398 conn(0x7efd240764a0 msgr2=0x7efd240788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.947+0000 7efd2c400640 1 -- 172.21.15.22:0/3307197398 shutdown_connections 2024-04-20T21:39:23.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.947+0000 7efd2c400640 1 -- 172.21.15.22:0/3307197398 wait complete. 2024-04-20T21:39:23.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.948+0000 7efd2c400640 1 Processor -- start 2024-04-20T21:39:23.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.948+0000 7efd2c400640 1 -- start start 2024-04-20T21:39:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd2c400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd24108590 0x7efd2419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd2c400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd24108f50 0x7efd2419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd2c400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd2410e600 0x7efd241a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd2c400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efd241188c0 con 0x7efd2410e600 2024-04-20T21:39:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd2c400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd24118740 con 0x7efd24108f50 2024-04-20T21:39:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd2c400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd24118a40 con 0x7efd24108590 2024-04-20T21:39:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd29a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd24108590 0x7efd2419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd29000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd24108f50 0x7efd2419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd29a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd24108590 0x7efd2419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46160/0 (socket says 172.21.15.22:46160) 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd29a00640 1 -- 172.21.15.22:0/454299258 learned_addr learned my addr 172.21.15.22:0/454299258 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd2a400640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd2410e600 0x7efd241a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd29000640 1 -- 172.21.15.22:0/454299258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd24108590 msgr2=0x7efd2419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd29000640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd24108590 0x7efd2419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd29000640 1 -- 172.21.15.22:0/454299258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd2410e600 msgr2=0x7efd241a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd29000640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd2410e600 0x7efd241a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd29000640 1 -- 172.21.15.22:0/454299258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd241a3dd0 con 0x7efd24108f50 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.949+0000 7efd2a400640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd2410e600 0x7efd241a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.950+0000 7efd29000640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd24108f50 0x7efd2419f320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efd100057c0 tx=0x7efd10004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.950+0000 7efd22000640 1 -- 172.21.15.22:0/454299258 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10041020 con 0x7efd24108f50 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.950+0000 7efd22000640 1 -- 172.21.15.22:0/454299258 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd1002c9f0 con 0x7efd24108f50 2024-04-20T21:39:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.950+0000 7efd22000640 1 -- 172.21.15.22:0/454299258 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10039040 con 0x7efd24108f50 2024-04-20T21:39:23.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.951+0000 7efd2c400640 1 -- 172.21.15.22:0/454299258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd241a4060 con 0x7efd24108f50 2024-04-20T21:39:23.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.951+0000 7efd2c400640 1 -- 172.21.15.22:0/454299258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd241ab950 con 0x7efd24108f50 2024-04-20T21:39:23.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.954+0000 7efd2f180740 1 Processor -- start 2024-04-20T21:39:23.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.955+0000 7efd2f180740 1 -- start start 2024-04-20T21:39:23.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.955+0000 7efd2f180740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1781a0 0x563fdf178580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.955+0000 7efd2f180740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 0x563fdf181160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.955+0000 7efd2f180740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1816a0 0x563fdf183a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.955+0000 7efd2f180740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563fdf0de150 con 0x563fdf1816a0 2024-04-20T21:39:23.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.955+0000 7efd2f180740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563fdf084370 con 0x563fdf1781a0 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.955+0000 7efd2f180740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563fdf0da110 con 0x563fdf178ac0 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.955+0000 7efd22000640 1 -- 172.21.15.22:0/454299258 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7efd10044020 con 0x7efd24108f50 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.955+0000 7efd2a400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1781a0 0x563fdf178580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.955+0000 7efd2a400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1781a0 0x563fdf178580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48004/0 (socket says 172.21.15.22:48004) 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.955+0000 7efd2a400640 1 -- 172.21.15.22:0/2631286776 learned_addr learned my addr 172.21.15.22:0/2631286776 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.955+0000 7efd29a00640 1 --2- 172.21.15.22:0/2631286776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 0x563fdf181160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.956+0000 7efd29000640 1 --2- 172.21.15.22:0/2631286776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1816a0 0x563fdf183a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.956+0000 7efd2a400640 1 -- 172.21.15.22:0/2631286776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 msgr2=0x563fdf181160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.956+0000 7efd2a400640 1 --2- 172.21.15.22:0/2631286776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 0x563fdf181160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.956+0000 7efd2a400640 1 -- 172.21.15.22:0/2631286776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1816a0 msgr2=0x563fdf183a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.956+0000 7efd2a400640 1 --2- 172.21.15.22:0/2631286776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1816a0 0x563fdf183a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.956+0000 7efd2a400640 1 -- 172.21.15.22:0/2631286776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fdeeba7d0 con 0x563fdf1781a0 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.956+0000 7efd29a00640 1 --2- 172.21.15.22:0/2631286776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 0x563fdf181160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.956+0000 7efd29000640 1 --2- 172.21.15.22:0/2631286776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1816a0 0x563fdf183a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:39:23.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.956+0000 7efd2a400640 1 --2- 172.21.15.22:0/2631286776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1781a0 0x563fdf178580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009fd0 tx=0x7efd1c0108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c5aa6fdf5ef14 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:23.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.956+0000 7efd16a00640 1 -- 172.21.15.22:0/2631286776 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c011760 con 0x563fdf1781a0 2024-04-20T21:39:23.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.956+0000 7efd16a00640 1 -- 172.21.15.22:0/2631286776 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd1c011de0 con 0x563fdf1781a0 2024-04-20T21:39:23.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.956+0000 7efd16a00640 1 -- 172.21.15.22:0/2631286776 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c00e3c0 con 0x563fdf1781a0 2024-04-20T21:39:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.957+0000 7efd22000640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efd0c0a3d20 0x7efd0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.957+0000 7efd2f180740 1 -- 172.21.15.22:0/2631286776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1781a0 msgr2=0x563fdf178580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.957+0000 7efd2f180740 1 --2- 172.21.15.22:0/2631286776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1781a0 0x563fdf178580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009fd0 tx=0x7efd1c0108e0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.957+0000 7efd22000640 1 -- 172.21.15.22:0/454299258 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7efd100f17f0 con 0x7efd24108f50 2024-04-20T21:39:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.957+0000 7efd29a00640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efd0c0a3d20 0x7efd0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.958+0000 7efd2f180740 1 -- 172.21.15.22:0/2631286776 shutdown_connections 2024-04-20T21:39:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.958+0000 7efd2f180740 1 --2- 172.21.15.22:0/2631286776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1816a0 0x563fdf183a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.958+0000 7efd2f180740 1 --2- 172.21.15.22:0/2631286776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 0x563fdf181160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.958+0000 7efd2f180740 1 --2- 172.21.15.22:0/2631286776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1781a0 0x563fdf178580 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.958+0000 7efd2f180740 1 -- 172.21.15.22:0/2631286776 >> 172.21.15.22:0/2631286776 conn(0x563fdf1728d0 msgr2=0x563fdf172cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.958+0000 7efd2f180740 1 -- 172.21.15.22:0/2631286776 shutdown_connections 2024-04-20T21:39:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.958+0000 7efd29a00640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efd0c0a3d20 0x7efd0c0a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x563fdf177c70 tx=0x7efd18009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:23.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.959+0000 7efd2f180740 1 -- 172.21.15.22:0/2631286776 wait complete. 2024-04-20T21:39:23.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.959+0000 7efd2f180740 1 -- start start 2024-04-20T21:39:23.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd2f180740 1 Processor -- start 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd2f180740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 0x563fdf17e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd2f180740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf178ac0 0x563fdf17ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd2f180740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1816a0 0x563fdf17f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd2f180740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563fdf184490 con 0x563fdf1816a0 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd2f180740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563fdf097740 con 0x563fdf178ac0 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd2f180740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563fdf184610 con 0x563fdf1781a0 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd2f180740 10 client.0 _ll_register_callbacks cb 0x563fdf092b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd29a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 0x563fdf17e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd29000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf178ac0 0x563fdf17ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd29a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 0x563fdf17e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46176/0 (socket says 172.21.15.22:46176) 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd29a00640 1 -- 172.21.15.22:0/3984337406 learned_addr learned my addr 172.21.15.22:0/3984337406 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.960+0000 7efd2a400640 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1816a0 0x563fdf17f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.961+0000 7efd29a00640 1 -- 172.21.15.22:0/3984337406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf178ac0 msgr2=0x563fdf17ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:23.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.961+0000 7efd29a00640 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf178ac0 0x563fdf17ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.961+0000 7efd29a00640 1 -- 172.21.15.22:0/3984337406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1816a0 msgr2=0x563fdf17f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:23.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.961+0000 7efd29a00640 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1816a0 0x563fdf17f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.961+0000 7efd29a00640 1 -- 172.21.15.22:0/3984337406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fdedf03f0 con 0x563fdf1781a0 2024-04-20T21:39:23.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.961+0000 7efd2a400640 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1816a0 0x563fdf17f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:23.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.961+0000 7efd29a00640 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 0x563fdf17e6e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd1801ea00 tx=0x7efd18004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9a7fd9ec15acaa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:23.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.961+0000 7efd0b400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:39:23.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.961+0000 7efd0b400640 1 -- 172.21.15.22:0/3984337406 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1802e060 con 0x563fdf1781a0 2024-04-20T21:39:23.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.961+0000 7efd0b400640 1 -- 172.21.15.22:0/3984337406 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7efd18018030 con 0x563fdf1781a0 2024-04-20T21:39:23.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.962+0000 7efd2f180740 1 -- 172.21.15.22:0/3984337406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd1c011de0 con 0x563fdf1781a0 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.962+0000 7efd0b400640 1 -- 172.21.15.22:0/3984337406 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7efd18005020 con 0x563fdf1781a0 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.962+0000 7efd2f180740 10 client.26140 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.962+0000 7efd2f180740 1 -- 172.21.15.22:0/3984337406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563fdedb00f0 con 0x563fdf1781a0 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.962+0000 7efd0b400640 1 -- 172.21.15.22:0/3984337406 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd1c011de0 con 0x563fdf1781a0 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.963+0000 7efd0b400640 1 -- 172.21.15.22:0/3984337406 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7efd1800f540 con 0x563fdf1781a0 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.963+0000 7efd2f180740 10 client.26140 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.963+0000 7efd2f180740 10 client.26140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.964+0000 7efd2f180740 20 client.26140 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.964+0000 7efd2f180740 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563fdf191a10 0x563fdf193e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.964+0000 7efd2f180740 4 client.26140 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.964+0000 7efd2f180740 1 -- 172.21.15.22:0/3984337406 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fdf0ebb70 con 0x563fdf191a10 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.964+0000 7efd29000640 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563fdf191a10 0x563fdf193e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.964+0000 7efd29000640 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563fdf191a10 0x563fdf193e40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a3e0ce7d68eea1 server_cookie=2eaa3f471b9852b0 in_seq=0 out_seq=0 2024-04-20T21:39:23.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.965+0000 7efd0b400640 10 client.26140 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.965+0000 7efd0b400640 1 -- 172.21.15.22:0/3984337406 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fdf0ebb70 con 0x563fdf191a10 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.965+0000 7efd0b400640 10 client.26140 handle_command_reply: tid=0 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.965+0000 7efd2f180740 1 client.26140 shutdown 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.965+0000 7efd2f180740 1 -- 172.21.15.22:0/3984337406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 msgr2=0x563fdf17e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.965+0000 7efd2f180740 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 0x563fdf17e6e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd1801ea00 tx=0x7efd18004730 comp rx=0 tx=0).stop 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.966+0000 7efd2f180740 1 -- 172.21.15.22:0/3984337406 shutdown_connections 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.966+0000 7efd2f180740 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1816a0 0x563fdf17f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.966+0000 7efd2f180740 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563fdf191a10 0x563fdf193e40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.966+0000 7efd2f180740 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf178ac0 0x563fdf17ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.966+0000 7efd2f180740 1 --2- 172.21.15.22:0/3984337406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 0x563fdf17e6e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.966+0000 7efd2f180740 1 -- 172.21.15.22:0/3984337406 >> 172.21.15.22:0/3984337406 conn(0x563fdf1728d0 msgr2=0x563fdf174620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.966+0000 7efd0b400640 0 client.26140 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:23.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.966+0000 7efd2f180740 1 -- 172.21.15.22:0/3984337406 shutdown_connections 2024-04-20T21:39:23.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.966+0000 7efd2f180740 1 -- 172.21.15.22:0/3984337406 wait complete. 2024-04-20T21:39:23.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.966+0000 7efd2f180740 20 client.26140 trim_cache size 0 max 16384 2024-04-20T21:39:23.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.989+0000 7efd2f180740 1 Processor -- start 2024-04-20T21:39:23.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.989+0000 7efd2f180740 1 -- start start 2024-04-20T21:39:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.989+0000 7efd2f180740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1781a0 0x563fdf18d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.989+0000 7efd2f180740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 0x563fdf171030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.989+0000 7efd2f180740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1816a0 0x563fdf17dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.989+0000 7efd2f180740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efd241b2320 con 0x563fdf1781a0 2024-04-20T21:39:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.989+0000 7efd2f180740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd241b4ba0 con 0x563fdf1816a0 2024-04-20T21:39:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.989+0000 7efd2f180740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd241b2990 con 0x563fdf178ac0 2024-04-20T21:39:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.990+0000 7efd29a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1781a0 0x563fdf18d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.990+0000 7efd29000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 0x563fdf171030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.990+0000 7efd29000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 0x563fdf171030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46186/0 (socket says 172.21.15.22:46186) 2024-04-20T21:39:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.990+0000 7efd29000640 1 -- 172.21.15.22:0/2276919115 learned_addr learned my addr 172.21.15.22:0/2276919115 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.990+0000 7efd29a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1781a0 0x563fdf18d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48024/0 (socket says 172.21.15.22:48024) 2024-04-20T21:39:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.990+0000 7efd2a400640 1 --2- 172.21.15.22:0/2276919115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1816a0 0x563fdf17dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.990+0000 7efd29000640 1 -- 172.21.15.22:0/2276919115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1816a0 msgr2=0x563fdf17dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.990+0000 7efd29000640 1 --2- 172.21.15.22:0/2276919115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1816a0 0x563fdf17dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.990+0000 7efd29000640 1 -- 172.21.15.22:0/2276919115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1781a0 msgr2=0x563fdf18d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.990+0000 7efd29000640 1 --2- 172.21.15.22:0/2276919115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1781a0 0x563fdf18d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.990+0000 7efd29000640 1 -- 172.21.15.22:0/2276919115 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fdf1788f0 con 0x563fdf178ac0 2024-04-20T21:39:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.990+0000 7efd29000640 1 --2- 172.21.15.22:0/2276919115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 0x563fdf171030 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efd100ef090 tx=0x7efd100a8040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f8e0e36d056a2eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.991+0000 7efd16000640 1 -- 172.21.15.22:0/2276919115 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd100c1c30 con 0x563fdf178ac0 2024-04-20T21:39:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.991+0000 7efd16000640 1 -- 172.21.15.22:0/2276919115 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd100f4e50 con 0x563fdf178ac0 2024-04-20T21:39:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.991+0000 7efd16000640 1 -- 172.21.15.22:0/2276919115 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd100b9ef0 con 0x563fdf178ac0 2024-04-20T21:39:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.992+0000 7efd2f180740 1 -- 172.21.15.22:0/2276919115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 msgr2=0x563fdf171030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.992+0000 7efd2f180740 1 --2- 172.21.15.22:0/2276919115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 0x563fdf171030 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efd100ef090 tx=0x7efd100a8040 comp rx=0 tx=0).stop 2024-04-20T21:39:23.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.992+0000 7efd2f180740 1 -- 172.21.15.22:0/2276919115 shutdown_connections 2024-04-20T21:39:23.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.992+0000 7efd2f180740 1 --2- 172.21.15.22:0/2276919115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1816a0 0x563fdf17dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.992+0000 7efd2f180740 1 --2- 172.21.15.22:0/2276919115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf178ac0 0x563fdf171030 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.992+0000 7efd2f180740 1 --2- 172.21.15.22:0/2276919115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf1781a0 0x563fdf18d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.992+0000 7efd2f180740 1 -- 172.21.15.22:0/2276919115 >> 172.21.15.22:0/2276919115 conn(0x563fdf1728d0 msgr2=0x563fdf17ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:23.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.992+0000 7efd2f180740 1 -- 172.21.15.22:0/2276919115 shutdown_connections 2024-04-20T21:39:23.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.992+0000 7efd2f180740 1 -- 172.21.15.22:0/2276919115 wait complete. 2024-04-20T21:39:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.992+0000 7efd2f180740 1 -- start start 2024-04-20T21:39:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.993+0000 7efd2f180740 1 Processor -- start 2024-04-20T21:39:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.993+0000 7efd2f180740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 0x563fdf189d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.993+0000 7efd2f180740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf178ac0 0x563fdf1850e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.993+0000 7efd2f180740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1816a0 0x563fdf185820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.993+0000 7efd2f180740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efd241b3060 con 0x563fdf178ac0 2024-04-20T21:39:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.993+0000 7efd2f180740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd241b44d0 con 0x563fdf1816a0 2024-04-20T21:39:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.993+0000 7efd2f180740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd241b3730 con 0x563fdf1781a0 2024-04-20T21:39:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.993+0000 7efd2f180740 10 client.0 _ll_register_callbacks cb 0x7efd180097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.993+0000 7efd2a400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1816a0 0x563fdf185820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.993+0000 7efd2a400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1816a0 0x563fdf185820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48032/0 (socket says 172.21.15.22:48032) 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.993+0000 7efd2a400640 1 -- 172.21.15.22:0/407906726 learned_addr learned my addr 172.21.15.22:0/407906726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.993+0000 7efd29a00640 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 0x563fdf189d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.994+0000 7efd29000640 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf178ac0 0x563fdf1850e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.994+0000 7efd2a400640 1 -- 172.21.15.22:0/407906726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 msgr2=0x563fdf189d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.994+0000 7efd2a400640 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 0x563fdf189d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.994+0000 7efd2a400640 1 -- 172.21.15.22:0/407906726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf178ac0 msgr2=0x563fdf1850e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.994+0000 7efd2a400640 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf178ac0 0x563fdf1850e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.994+0000 7efd2a400640 1 -- 172.21.15.22:0/407906726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fdf1f6f80 con 0x563fdf1816a0 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.994+0000 7efd29a00640 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 0x563fdf189d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.994+0000 7efd29000640 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf178ac0 0x563fdf1850e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.994+0000 7efd2a400640 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1816a0 0x563fdf185820 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009fa0 tx=0x7efd1c011790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=625807397926303e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.994+0000 7efd0b400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.995+0000 7efd0b400640 1 -- 172.21.15.22:0/407906726 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c011d90 con 0x563fdf1816a0 2024-04-20T21:39:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.995+0000 7efd0b400640 1 -- 172.21.15.22:0/407906726 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7efd1c005040 con 0x563fdf1816a0 2024-04-20T21:39:23.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.995+0000 7efd2f180740 1 -- 172.21.15.22:0/407906726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd100f4e50 con 0x563fdf1816a0 2024-04-20T21:39:23.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.995+0000 7efd0b400640 1 -- 172.21.15.22:0/407906726 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7efd1c033050 con 0x563fdf1816a0 2024-04-20T21:39:23.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.995+0000 7efd2f180740 10 client.26123 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:23.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.995+0000 7efd2f180740 1 -- 172.21.15.22:0/407906726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563fdf1f80d0 con 0x563fdf1816a0 2024-04-20T21:39:23.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.995+0000 7efd0b400640 1 -- 172.21.15.22:0/407906726 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd100f4e50 con 0x563fdf1816a0 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.995+0000 7efd0b400640 1 -- 172.21.15.22:0/407906726 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7efd1c02e030 con 0x563fdf1816a0 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.996+0000 7efd2f180740 10 client.26123 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.996+0000 7efd2f180740 10 client.26123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.996+0000 7efd2f180740 20 client.26123 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.996+0000 7efd2f180740 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563fdf1f8510 0x563fdf1fa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.996+0000 7efd2f180740 4 client.26123 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.996+0000 7efd2f180740 1 -- 172.21.15.22:0/407906726 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563fdf0e59e0 con 0x563fdf1f8510 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.996+0000 7efd29a00640 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563fdf1f8510 0x563fdf1fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.996+0000 7efd29a00640 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563fdf1f8510 0x563fdf1fa900 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f3a957301696ded server_cookie=d08ea9e1ca2f27bf in_seq=0 out_seq=0 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.996+0000 7efd0b400640 10 client.26123 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd0b400640 1 -- 172.21.15.22:0/407906726 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563fdf0e59e0 con 0x563fdf1f8510 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd0b400640 10 client.26123 handle_command_reply: tid=0 2024-04-20T21:39:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd2f180740 1 client.26123 shutdown 2024-04-20T21:39:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd2f180740 1 -- 172.21.15.22:0/407906726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1816a0 msgr2=0x563fdf185820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd2f180740 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1816a0 0x563fdf185820 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009fa0 tx=0x7efd1c011790 comp rx=0 tx=0).stop 2024-04-20T21:39:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd2f180740 1 -- 172.21.15.22:0/407906726 shutdown_connections 2024-04-20T21:39:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd2f180740 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563fdf1f8510 0x563fdf1fa900 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd2f180740 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fdf1816a0 0x563fdf185820 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd0b400640 0 client.26123 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd2f180740 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fdf178ac0 0x563fdf1850e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd2f180740 1 --2- 172.21.15.22:0/407906726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fdf1781a0 0x563fdf189d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd2f180740 1 -- 172.21.15.22:0/407906726 >> 172.21.15.22:0/407906726 conn(0x563fdf1728d0 msgr2=0x563fdf173bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.997+0000 7efd2f180740 1 -- 172.21.15.22:0/407906726 shutdown_connections 2024-04-20T21:39:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.998+0000 7efd2f180740 1 -- 172.21.15.22:0/407906726 wait complete. 2024-04-20T21:39:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.998+0000 7efd2f180740 20 client.26123 trim_cache size 0 max 16384 2024-04-20T21:39:24.000 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:39:24.000 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:39:24.000 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:39:24.000 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:39:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.998+0000 7efd2c400640 1 -- 172.21.15.22:0/454299258 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efd0c0a3d20 msgr2=0x7efd0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.998+0000 7efd2c400640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efd0c0a3d20 0x7efd0c0a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x563fdf177c70 tx=0x7efd18009260 comp rx=0 tx=0).stop 2024-04-20T21:39:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.998+0000 7efd2c400640 1 -- 172.21.15.22:0/454299258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd24108f50 msgr2=0x7efd2419f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.998+0000 7efd2c400640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd24108f50 0x7efd2419f320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efd100057c0 tx=0x7efd10004360 comp rx=0 tx=0).stop 2024-04-20T21:39:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.998+0000 7efd2c400640 1 -- 172.21.15.22:0/454299258 shutdown_connections 2024-04-20T21:39:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.998+0000 7efd2c400640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efd0c0a3d20 0x7efd0c0a61e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.998+0000 7efd2c400640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd2410e600 0x7efd241a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.998+0000 7efd2c400640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd24108f50 0x7efd2419f320 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.998+0000 7efd2c400640 1 --2- 172.21.15.22:0/454299258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd24108590 0x7efd2419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.998+0000 7efd2c400640 1 -- 172.21.15.22:0/454299258 >> 172.21.15.22:0/454299258 conn(0x7efd240764a0 msgr2=0x7efd24077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.999+0000 7efd2c400640 1 -- 172.21.15.22:0/454299258 shutdown_connections 2024-04-20T21:39:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:23.999+0000 7efd2c400640 1 -- 172.21.15.22:0/454299258 wait complete. 2024-04-20T21:39:24.016 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:39:24.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:24.191 INFO:tasks.workunit.client.0.smithi022.stdout: 1 433005 9.68 MB/sec execute 112 sec latency 25.589 ms 2024-04-20T21:39:24.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.228+0000 7f6ad5800640 1 -- 172.21.15.22:0/3341074998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad010e650 msgr2=0x7f6ad010ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.228+0000 7f6ad5800640 1 --2- 172.21.15.22:0/3341074998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad010e650 0x7f6ad010ea30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0009860 tx=0x7f6ac002bb70 comp rx=0 tx=0).stop 2024-04-20T21:39:24.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.228+0000 7f6ad5800640 1 -- 172.21.15.22:0/3341074998 shutdown_connections 2024-04-20T21:39:24.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.228+0000 7f6ad5800640 1 --2- 172.21.15.22:0/3341074998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad0108f50 0x7f6ad0115410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.228+0000 7f6ad5800640 1 --2- 172.21.15.22:0/3341074998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ad0108590 0x7f6ad0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.228+0000 7f6ad5800640 1 --2- 172.21.15.22:0/3341074998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad010e650 0x7f6ad010ea30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.228+0000 7f6ad5800640 1 -- 172.21.15.22:0/3341074998 >> 172.21.15.22:0/3341074998 conn(0x7f6ad0075760 msgr2=0x7f6ad0106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:24.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.228+0000 7f6ad5800640 1 -- 172.21.15.22:0/3341074998 shutdown_connections 2024-04-20T21:39:24.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.228+0000 7f6ad5800640 1 -- 172.21.15.22:0/3341074998 wait complete. 2024-04-20T21:39:24.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.229+0000 7f6ad5800640 1 Processor -- start 2024-04-20T21:39:24.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.229+0000 7f6ad5800640 1 -- start start 2024-04-20T21:39:24.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.230+0000 7f6ad5800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad0108590 0x7f6ad019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.230+0000 7f6ad5800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ad0108f50 0x7f6ad019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.230+0000 7f6ad5800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad010e650 0x7f6ad01a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.230+0000 7f6ad5800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad0118950 con 0x7f6ad0108590 2024-04-20T21:39:24.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.230+0000 7f6ad5800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad01187d0 con 0x7f6ad010e650 2024-04-20T21:39:24.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.230+0000 7f6ad5800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad0118ad0 con 0x7f6ad0108f50 2024-04-20T21:39:24.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.230+0000 7f6acf400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad010e650 0x7f6ad01a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.230+0000 7f6ace000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ad0108f50 0x7f6ad019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.230+0000 7f6acf400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad010e650 0x7f6ad01a3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48058/0 (socket says 172.21.15.22:48058) 2024-04-20T21:39:24.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.230+0000 7f6acf400640 1 -- 172.21.15.22:0/2832828811 learned_addr learned my addr 172.21.15.22:0/2832828811 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:24.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.230+0000 7f6acea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad0108590 0x7f6ad019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6acf400640 1 -- 172.21.15.22:0/2832828811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ad0108f50 msgr2=0x7f6ad019f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6acf400640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ad0108f50 0x7f6ad019f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6acf400640 1 -- 172.21.15.22:0/2832828811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad0108590 msgr2=0x7f6ad019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6acf400640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad0108590 0x7f6ad019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6acf400640 1 -- 172.21.15.22:0/2832828811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad01a3d80 con 0x7f6ad010e650 2024-04-20T21:39:24.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6ace000640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ad0108f50 0x7f6ad019f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:24.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6acea00640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad0108590 0x7f6ad019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:24.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6acf400640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad010e650 0x7f6ad01a3650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800efc0 tx=0x7f6ac800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:24.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6ac7400640 1 -- 172.21.15.22:0/2832828811 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8003970 con 0x7f6ad010e650 2024-04-20T21:39:24.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6ac7400640 1 -- 172.21.15.22:0/2832828811 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ac8004020 con 0x7f6ad010e650 2024-04-20T21:39:24.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6ac7400640 1 -- 172.21.15.22:0/2832828811 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac800b9f0 con 0x7f6ad010e650 2024-04-20T21:39:24.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6ad5800640 1 -- 172.21.15.22:0/2832828811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad01a4070 con 0x7f6ad010e650 2024-04-20T21:39:24.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.231+0000 7f6ad5800640 1 -- 172.21.15.22:0/2832828811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad01ab900 con 0x7f6ad010e650 2024-04-20T21:39:24.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.233+0000 7f6ad5800640 1 -- 172.21.15.22:0/2832828811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad00691d0 con 0x7f6ad010e650 2024-04-20T21:39:24.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.235+0000 7f6ac7400640 1 -- 172.21.15.22:0/2832828811 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f6ac801d020 con 0x7f6ad010e650 2024-04-20T21:39:24.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.236+0000 7f6ac7400640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f6ab40a3da0 0x7f6ab40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.236+0000 7f6ac7400640 1 -- 172.21.15.22:0/2832828811 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f6ac80d0670 con 0x7f6ad010e650 2024-04-20T21:39:24.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.236+0000 7f6acea00640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f6ab40a3da0 0x7f6ab40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.237+0000 7f6acea00640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f6ab40a3da0 0x7f6ab40a6260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6ac002c180 tx=0x7f6ac002e040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:24.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.241+0000 7f6ac7400640 1 -- 172.21.15.22:0/2832828811 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6ac809cc90 con 0x7f6ad010e650 2024-04-20T21:39:24.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.531+0000 7f6ad5800640 1 -- 172.21.15.22:0/2832828811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ad002d5d0 con 0x7f6ad010e650 2024-04-20T21:39:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.532+0000 7f6ac7400640 1 -- 172.21.15.22:0/2832828811 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f6ac809c590 con 0x7f6ad010e650 2024-04-20T21:39:24.531 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:39:24.531 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:39:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:39:24.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.535+0000 7f6ad5800640 1 -- 172.21.15.22:0/2832828811 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f6ab40a3da0 msgr2=0x7f6ab40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.535+0000 7f6ad5800640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f6ab40a3da0 0x7f6ab40a6260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6ac002c180 tx=0x7f6ac002e040 comp rx=0 tx=0).stop 2024-04-20T21:39:24.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.535+0000 7f6ad5800640 1 -- 172.21.15.22:0/2832828811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad010e650 msgr2=0x7f6ad01a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.535+0000 7f6ad5800640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad010e650 0x7f6ad01a3650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800efc0 tx=0x7f6ac800c800 comp rx=0 tx=0).stop 2024-04-20T21:39:24.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.535+0000 7f6ad5800640 1 -- 172.21.15.22:0/2832828811 shutdown_connections 2024-04-20T21:39:24.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.535+0000 7f6ad5800640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f6ab40a3da0 0x7f6ab40a6260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.535+0000 7f6ad5800640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad010e650 0x7f6ad01a3650 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.535+0000 7f6ad5800640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ad0108f50 0x7f6ad019f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.535+0000 7f6ad5800640 1 --2- 172.21.15.22:0/2832828811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad0108590 0x7f6ad019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.535+0000 7f6ad5800640 1 -- 172.21.15.22:0/2832828811 >> 172.21.15.22:0/2832828811 conn(0x7f6ad0075760 msgr2=0x7f6ad00767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:24.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.535+0000 7f6ad5800640 1 -- 172.21.15.22:0/2832828811 shutdown_connections 2024-04-20T21:39:24.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.535+0000 7f6ad5800640 1 -- 172.21.15.22:0/2832828811 wait complete. 2024-04-20T21:39:24.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:39:24.625 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:24 smithi022 ceph-mon[16843]: pgmap v170: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 165 op/s 2024-04-20T21:39:24.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:24 smithi122 ceph-mon[22147]: pgmap v170: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 165 op/s 2024-04-20T21:39:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.784+0000 7f0817000640 1 -- 172.21.15.22:0/2715346394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0810104c30 msgr2=0x7f0810108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.784+0000 7f0817000640 1 --2- 172.21.15.22:0/2715346394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0810104c30 0x7f0810108c70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0800009900 tx=0x7f080002bc40 comp rx=0 tx=0).stop 2024-04-20T21:39:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.784+0000 7f0817000640 1 -- 172.21.15.22:0/2715346394 shutdown_connections 2024-04-20T21:39:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.784+0000 7f0817000640 1 --2- 172.21.15.22:0/2715346394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0810109350 0x7f0810110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.784+0000 7f0817000640 1 --2- 172.21.15.22:0/2715346394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0810104c30 0x7f0810108c70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.784+0000 7f0817000640 1 --2- 172.21.15.22:0/2715346394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0810104280 0x7f0810104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.784+0000 7f0817000640 1 -- 172.21.15.22:0/2715346394 >> 172.21.15.22:0/2715346394 conn(0x7f08100ffaa0 msgr2=0x7f0810101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.784+0000 7f0817000640 1 -- 172.21.15.22:0/2715346394 shutdown_connections 2024-04-20T21:39:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.784+0000 7f0817000640 1 -- 172.21.15.22:0/2715346394 wait complete. 2024-04-20T21:39:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.786+0000 7f0817000640 1 Processor -- start 2024-04-20T21:39:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.786+0000 7f0817000640 1 -- start start 2024-04-20T21:39:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.786+0000 7f0817000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0810104280 0x7f08101a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.786+0000 7f0817000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0810109350 0x7f08101a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.786+0000 7f0817000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f081019a750 0x7f081019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.786+0000 7f0817000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0810114490 con 0x7f0810109350 2024-04-20T21:39:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.786+0000 7f0817000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0810114310 con 0x7f081019a750 2024-04-20T21:39:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.786+0000 7f0817000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0810114610 con 0x7f0810104280 2024-04-20T21:39:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.787+0000 7f0815c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0810104280 0x7f08101a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.787+0000 7f0816600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f081019a750 0x7f081019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.787+0000 7f0815c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0810104280 0x7f08101a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46254/0 (socket says 172.21.15.22:46254) 2024-04-20T21:39:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.787+0000 7f0815200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0810109350 0x7f08101a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.787+0000 7f0816600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f081019a750 0x7f081019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48078/0 (socket says 172.21.15.22:48078) 2024-04-20T21:39:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.787+0000 7f0815c00640 1 -- 172.21.15.22:0/3002785583 learned_addr learned my addr 172.21.15.22:0/3002785583 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:24.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.787+0000 7f0816600640 1 -- 172.21.15.22:0/3002785583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0810104280 msgr2=0x7f08101a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.787+0000 7f0816600640 1 --2- 172.21.15.22:0/3002785583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0810104280 0x7f08101a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.787+0000 7f0816600640 1 -- 172.21.15.22:0/3002785583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0810109350 msgr2=0x7f08101a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.787+0000 7f0816600640 1 --2- 172.21.15.22:0/3002785583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0810109350 0x7f08101a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.787+0000 7f0816600640 1 -- 172.21.15.22:0/3002785583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081019b350 con 0x7f081019a750 2024-04-20T21:39:24.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.787+0000 7f0816600640 1 --2- 172.21.15.22:0/3002785583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f081019a750 0x7f081019abd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f080800be20 tx=0x7f080800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:24.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.788+0000 7f080e000640 1 -- 172.21.15.22:0/3002785583 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808019070 con 0x7f081019a750 2024-04-20T21:39:24.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.788+0000 7f080e000640 1 -- 172.21.15.22:0/3002785583 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0808014920 con 0x7f081019a750 2024-04-20T21:39:24.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.788+0000 7f0817000640 1 -- 172.21.15.22:0/3002785583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08101a7430 con 0x7f081019a750 2024-04-20T21:39:24.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.788+0000 7f080e000640 1 -- 172.21.15.22:0/3002785583 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808012920 con 0x7f081019a750 2024-04-20T21:39:24.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.788+0000 7f0817000640 1 -- 172.21.15.22:0/3002785583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08101a7830 con 0x7f081019a750 2024-04-20T21:39:24.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.792+0000 7f081b230740 1 Processor -- start 2024-04-20T21:39:24.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.792+0000 7f080e000640 1 -- 172.21.15.22:0/3002785583 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f080801c070 con 0x7f081019a750 2024-04-20T21:39:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.792+0000 7f081b230740 1 -- start start 2024-04-20T21:39:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.792+0000 7f081b230740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8f170 0x55a0acd8f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.792+0000 7f081b230740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd8fa90 0x55a0acd98130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.792+0000 7f081b230740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 0x55a0acd9aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f081b230740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a0accf4050 con 0x55a0acd8fa90 2024-04-20T21:39:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f081b230740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a0acc9a490 con 0x55a0acd98670 2024-04-20T21:39:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f081b230740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a0accf0010 con 0x55a0acd8f170 2024-04-20T21:39:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f0815200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8f170 0x55a0acd8f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f0815c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd8fa90 0x55a0acd98130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f0815200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8f170 0x55a0acd8f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46260/0 (socket says 172.21.15.22:46260) 2024-04-20T21:39:24.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f0815c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd8fa90 0x55a0acd98130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48092/0 (socket says 172.21.15.22:48092) 2024-04-20T21:39:24.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f0815200640 1 -- 172.21.15.22:0/562698024 learned_addr learned my addr 172.21.15.22:0/562698024 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:24.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f0815200640 1 --2- 172.21.15.22:0/562698024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 0x55a0acd9aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f0815c00640 1 -- 172.21.15.22:0/562698024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8f170 msgr2=0x55a0acd8f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:24.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f0815c00640 1 --2- 172.21.15.22:0/562698024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8f170 0x55a0acd8f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f0815c00640 1 -- 172.21.15.22:0/562698024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 msgr2=0x55a0acd9aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:24.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f0815c00640 1 --2- 172.21.15.22:0/562698024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 0x55a0acd9aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f0815c00640 1 -- 172.21.15.22:0/562698024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0acca2070 con 0x55a0acd8fa90 2024-04-20T21:39:24.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.793+0000 7f0815200640 1 --2- 172.21.15.22:0/562698024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 0x55a0acd9aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:24.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.794+0000 7f0815c00640 1 --2- 172.21.15.22:0/562698024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd8fa90 0x55a0acd98130 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f080400aca0 tx=0x7f080400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae37c0a6e3d00d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:24.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.794+0000 7f07fea00640 1 -- 172.21.15.22:0/562698024 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804005220 con 0x55a0acd8fa90 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.794+0000 7f07fea00640 1 -- 172.21.15.22:0/562698024 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0804004c80 con 0x55a0acd8fa90 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.794+0000 7f07fea00640 1 -- 172.21.15.22:0/562698024 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040188e0 con 0x55a0acd8fa90 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.794+0000 7f080e000640 1 --2- 172.21.15.22:0/3002785583 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07f80a3de0 0x7f07f80a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.795+0000 7f080e000640 1 -- 172.21.15.22:0/3002785583 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f08080cfe80 con 0x7f081019a750 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.795+0000 7f0815200640 1 --2- 172.21.15.22:0/3002785583 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07f80a3de0 0x7f07f80a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.795+0000 7f081b230740 1 -- 172.21.15.22:0/562698024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd8fa90 msgr2=0x55a0acd98130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.795+0000 7f081b230740 1 --2- 172.21.15.22:0/562698024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd8fa90 0x55a0acd98130 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f080400aca0 tx=0x7f080400f790 comp rx=0 tx=0).stop 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.795+0000 7f081b230740 1 -- 172.21.15.22:0/562698024 shutdown_connections 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.795+0000 7f081b230740 1 --2- 172.21.15.22:0/562698024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 0x55a0acd9aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.795+0000 7f081b230740 1 --2- 172.21.15.22:0/562698024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd8fa90 0x55a0acd98130 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.795+0000 7f081b230740 1 --2- 172.21.15.22:0/562698024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8f170 0x55a0acd8f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.795+0000 7f081b230740 1 -- 172.21.15.22:0/562698024 >> 172.21.15.22:0/562698024 conn(0x55a0acd898a0 msgr2=0x55a0acd89c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:24.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.795+0000 7f081b230740 1 -- 172.21.15.22:0/562698024 shutdown_connections 2024-04-20T21:39:24.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.795+0000 7f0815200640 1 --2- 172.21.15.22:0/3002785583 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07f80a3de0 0x7f07f80a62a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55a0acd8aa30 tx=0x7f080002cf90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:24.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.795+0000 7f081b230740 1 -- 172.21.15.22:0/562698024 wait complete. 2024-04-20T21:39:24.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.796+0000 7f081b230740 1 -- start start 2024-04-20T21:39:24.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.796+0000 7f081b230740 1 Processor -- start 2024-04-20T21:39:24.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f081b230740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd8f170 0x55a0acd95670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f081b230740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acd95bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f081b230740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd98670 0x55a0acd960f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f081b230740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a0acd9b460 con 0x55a0acd98670 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f081b230740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a0accad860 con 0x55a0acd8f170 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f081b230740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a0acd9b5e0 con 0x55a0acd8fa90 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f081b230740 10 client.0 _ll_register_callbacks cb 0x55a0acc7bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f0815c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd8f170 0x55a0acd95670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f0815200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acd95bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f0815c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd8f170 0x55a0acd95670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48100/0 (socket says 172.21.15.22:48100) 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f0815200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acd95bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46266/0 (socket says 172.21.15.22:46266) 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f0815c00640 1 -- 172.21.15.22:0/2602359123 learned_addr learned my addr 172.21.15.22:0/2602359123 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f0816600640 1 --2- 172.21.15.22:0/2602359123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd98670 0x55a0acd960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f0815c00640 1 -- 172.21.15.22:0/2602359123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 msgr2=0x55a0acd95bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f0815c00640 1 --2- 172.21.15.22:0/2602359123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acd95bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f0815c00640 1 -- 172.21.15.22:0/2602359123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd98670 msgr2=0x55a0acd960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.797+0000 7f0815c00640 1 --2- 172.21.15.22:0/2602359123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd98670 0x55a0acd960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.798+0000 7f0815c00640 1 -- 172.21.15.22:0/2602359123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0accb1950 con 0x55a0acd8f170 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.798+0000 7f0815c00640 1 --2- 172.21.15.22:0/2602359123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd8f170 0x55a0acd95670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f08040185c0 tx=0x7f0804004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=961f45ce65c53eb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.798+0000 7f07f3400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.798+0000 7f07f3400640 1 -- 172.21.15.22:0/2602359123 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804027070 con 0x55a0acd8f170 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.798+0000 7f07f3400640 1 -- 172.21.15.22:0/2602359123 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f0804018870 con 0x55a0acd8f170 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.798+0000 7f081b230740 1 -- 172.21.15.22:0/2602359123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0804004c80 con 0x55a0acd8f170 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.799+0000 7f07f3400640 1 -- 172.21.15.22:0/2602359123 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0804033050 con 0x55a0acd8f170 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.799+0000 7f081b230740 10 client.26141 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.799+0000 7f081b230740 1 -- 172.21.15.22:0/2602359123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a0acc7f820 con 0x55a0acd8f170 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.799+0000 7f07f3400640 1 -- 172.21.15.22:0/2602359123 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0804004c80 con 0x55a0acd8f170 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.799+0000 7f07f3400640 1 -- 172.21.15.22:0/2602359123 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f08040235f0 con 0x55a0acd8f170 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.799+0000 7f081b230740 10 client.26141 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.799+0000 7f081b230740 10 client.26141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.799+0000 7f081b230740 20 client.26141 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.799+0000 7f081b230740 1 --2- 172.21.15.22:0/2602359123 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0acda85a0 0x55a0acdaa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.799+0000 7f081b230740 4 client.26141 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:24.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.799+0000 7f081b230740 1 -- 172.21.15.22:0/2602359123 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0acd01a70 con 0x55a0acda85a0 2024-04-20T21:39:24.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.800+0000 7f0815200640 1 --2- 172.21.15.22:0/2602359123 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0acda85a0 0x55a0acdaa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.800+0000 7f0815200640 1 --2- 172.21.15.22:0/2602359123 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0acda85a0 0x55a0acdaa990 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0682091a6c796e1 server_cookie=5ff03b2ca9ba8b04 in_seq=0 out_seq=0 2024-04-20T21:39:24.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.800+0000 7f07f3400640 10 client.26141 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f07f3400640 1 -- 172.21.15.22:0/2602359123 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0acd01a70 con 0x55a0acda85a0 2024-04-20T21:39:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f07f3400640 10 client.26141 handle_command_reply: tid=0 2024-04-20T21:39:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f081b230740 1 client.26141 shutdown 2024-04-20T21:39:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f081b230740 1 -- 172.21.15.22:0/2602359123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd8f170 msgr2=0x55a0acd95670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f081b230740 1 --2- 172.21.15.22:0/2602359123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd8f170 0x55a0acd95670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f08040185c0 tx=0x7f0804004970 comp rx=0 tx=0).stop 2024-04-20T21:39:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f081b230740 1 -- 172.21.15.22:0/2602359123 shutdown_connections 2024-04-20T21:39:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f081b230740 1 --2- 172.21.15.22:0/2602359123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd98670 0x55a0acd960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f081b230740 1 --2- 172.21.15.22:0/2602359123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acd95bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f081b230740 1 --2- 172.21.15.22:0/2602359123 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0acda85a0 0x55a0acdaa990 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f081b230740 1 --2- 172.21.15.22:0/2602359123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd8f170 0x55a0acd95670 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f081b230740 1 -- 172.21.15.22:0/2602359123 >> 172.21.15.22:0/2602359123 conn(0x55a0acd898a0 msgr2=0x55a0acd8b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:24.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f07f3400640 0 client.26141 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:24.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f081b230740 1 -- 172.21.15.22:0/2602359123 shutdown_connections 2024-04-20T21:39:24.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.802+0000 7f081b230740 1 -- 172.21.15.22:0/2602359123 wait complete. 2024-04-20T21:39:24.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.803+0000 7f081b230740 20 client.26141 trim_cache size 0 max 16384 2024-04-20T21:39:24.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:24 smithi033 ceph-mon[21626]: pgmap v170: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 165 op/s 2024-04-20T21:39:24.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.823+0000 7f081b230740 1 Processor -- start 2024-04-20T21:39:24.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.823+0000 7f081b230740 1 -- start start 2024-04-20T21:39:24.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.823+0000 7f081b230740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd8f170 0x55a0acd97da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.823+0000 7f081b230740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acda4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.823+0000 7f081b230740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd98670 0x55a0acd887e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.823+0000 7f081b230740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08101a9970 con 0x55a0acd98670 2024-04-20T21:39:24.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.823+0000 7f081b230740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08101ac1f0 con 0x55a0acd8f170 2024-04-20T21:39:24.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.823+0000 7f081b230740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08101a9fe0 con 0x55a0acd8fa90 2024-04-20T21:39:24.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.824+0000 7f0815200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acda4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.824+0000 7f0815200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acda4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46278/0 (socket says 172.21.15.22:46278) 2024-04-20T21:39:24.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.824+0000 7f0815200640 1 -- 172.21.15.22:0/831920367 learned_addr learned my addr 172.21.15.22:0/831920367 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:24.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.824+0000 7f0815c00640 1 --2- 172.21.15.22:0/831920367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd8f170 0x55a0acd97da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.824+0000 7f0816600640 1 --2- 172.21.15.22:0/831920367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd98670 0x55a0acd887e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.824+0000 7f0815200640 1 -- 172.21.15.22:0/831920367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd8f170 msgr2=0x55a0acd97da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.824+0000 7f0815200640 1 --2- 172.21.15.22:0/831920367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd8f170 0x55a0acd97da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.824+0000 7f0815200640 1 -- 172.21.15.22:0/831920367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd98670 msgr2=0x55a0acd887e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.824+0000 7f0815200640 1 --2- 172.21.15.22:0/831920367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd98670 0x55a0acd887e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.824+0000 7f0815200640 1 -- 172.21.15.22:0/831920367 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0aca063f0 con 0x55a0acd8fa90 2024-04-20T21:39:24.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.825+0000 7f0815200640 1 --2- 172.21.15.22:0/831920367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acda4350 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f080002c250 tx=0x7f0800004f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49e026850e8283d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.825+0000 7f07fe000640 1 -- 172.21.15.22:0/831920367 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080004e920 con 0x55a0acd8fa90 2024-04-20T21:39:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.825+0000 7f07fe000640 1 -- 172.21.15.22:0/831920367 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0800062070 con 0x55a0acd8fa90 2024-04-20T21:39:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.825+0000 7f07fe000640 1 -- 172.21.15.22:0/831920367 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08000051c0 con 0x55a0acd8fa90 2024-04-20T21:39:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.826+0000 7f081b230740 1 -- 172.21.15.22:0/831920367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 msgr2=0x55a0acda4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.826+0000 7f081b230740 1 --2- 172.21.15.22:0/831920367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acda4350 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f080002c250 tx=0x7f0800004f10 comp rx=0 tx=0).stop 2024-04-20T21:39:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.826+0000 7f081b230740 1 -- 172.21.15.22:0/831920367 shutdown_connections 2024-04-20T21:39:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.826+0000 7f081b230740 1 --2- 172.21.15.22:0/831920367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd98670 0x55a0acd887e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.826+0000 7f081b230740 1 --2- 172.21.15.22:0/831920367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acda4350 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.826+0000 7f081b230740 1 --2- 172.21.15.22:0/831920367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd8f170 0x55a0acd97da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.826+0000 7f081b230740 1 -- 172.21.15.22:0/831920367 >> 172.21.15.22:0/831920367 conn(0x55a0acd898a0 msgr2=0x55a0acd880d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.826+0000 7f081b230740 1 -- 172.21.15.22:0/831920367 shutdown_connections 2024-04-20T21:39:24.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.827+0000 7f081b230740 1 -- 172.21.15.22:0/831920367 wait complete. 2024-04-20T21:39:24.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.827+0000 7f081b230740 1 -- start start 2024-04-20T21:39:24.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.827+0000 7f081b230740 1 Processor -- start 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f081b230740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd8f170 0x55a0acda0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f081b230740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acda0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f081b230740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 0x55a0acd9bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f081b230740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08101aa6b0 con 0x55a0acd8f170 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f081b230740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08101abb20 con 0x55a0acd98670 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f081b230740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08101aad80 con 0x55a0acd8fa90 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f081b230740 10 client.0 _ll_register_callbacks cb 0x7f08040053e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f0816600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 0x55a0acd9bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f0815200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acda0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f0815c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd8f170 0x55a0acda0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f0815200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acda0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46294/0 (socket says 172.21.15.22:46294) 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f0816600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 0x55a0acd9bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48122/0 (socket says 172.21.15.22:48122) 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.828+0000 7f0815200640 1 -- 172.21.15.22:0/3084719913 learned_addr learned my addr 172.21.15.22:0/3084719913 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.829+0000 7f0815200640 1 -- 172.21.15.22:0/3084719913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 msgr2=0x55a0acd9bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:24.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.829+0000 7f0815200640 1 --2- 172.21.15.22:0/3084719913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 0x55a0acd9bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.829+0000 7f0815200640 1 -- 172.21.15.22:0/3084719913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd8f170 msgr2=0x55a0acda0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:24.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.829+0000 7f0815200640 1 --2- 172.21.15.22:0/3084719913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd8f170 0x55a0acda0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.829+0000 7f0815200640 1 -- 172.21.15.22:0/3084719913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0acad07d0 con 0x55a0acd8fa90 2024-04-20T21:39:24.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.829+0000 7f0816600640 1 --2- 172.21.15.22:0/3084719913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 0x55a0acd9bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:24.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.829+0000 7f0815200640 1 --2- 172.21.15.22:0/3084719913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acda0da0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0800036040 tx=0x7f0800009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93440ca7e428032a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:24.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.829+0000 7f07f3400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:39:24.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.829+0000 7f07f3400640 1 -- 172.21.15.22:0/3084719913 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080006b060 con 0x55a0acd8fa90 2024-04-20T21:39:24.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.829+0000 7f07f3400640 1 -- 172.21.15.22:0/3084719913 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f08000048c0 con 0x55a0acd8fa90 2024-04-20T21:39:24.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.829+0000 7f081b230740 1 -- 172.21.15.22:0/3084719913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0800062070 con 0x55a0acd8fa90 2024-04-20T21:39:24.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.830+0000 7f07f3400640 1 -- 172.21.15.22:0/3084719913 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0800004ac0 con 0x55a0acd8fa90 2024-04-20T21:39:24.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.830+0000 7f081b230740 10 client.26167 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:24.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.830+0000 7f081b230740 1 -- 172.21.15.22:0/3084719913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a0ace0e310 con 0x55a0acd8fa90 2024-04-20T21:39:24.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.830+0000 7f07f3400640 1 -- 172.21.15.22:0/3084719913 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0800062070 con 0x55a0acd8fa90 2024-04-20T21:39:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.830+0000 7f07f3400640 1 -- 172.21.15.22:0/3084719913 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f080005d110 con 0x55a0acd8fa90 2024-04-20T21:39:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.830+0000 7f081b230740 10 client.26167 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.830+0000 7f081b230740 10 client.26167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.830+0000 7f081b230740 20 client.26167 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.830+0000 7f081b230740 1 --2- 172.21.15.22:0/3084719913 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0ace0ea10 0x55a0ace10e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.830+0000 7f081b230740 4 client.26167 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:39:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.830+0000 7f081b230740 1 -- 172.21.15.22:0/3084719913 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0accfb8e0 con 0x55a0ace0ea10 2024-04-20T21:39:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.830+0000 7f0815c00640 1 --2- 172.21.15.22:0/3084719913 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0ace0ea10 0x55a0ace10e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.831+0000 7f0815c00640 1 --2- 172.21.15.22:0/3084719913 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0ace0ea10 0x55a0ace10e30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fa49dc16250008b server_cookie=f56889c96ebdbe9c in_seq=0 out_seq=0 2024-04-20T21:39:24.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.831+0000 7f07f3400640 10 client.26167 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:24.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f07f3400640 1 -- 172.21.15.22:0/3084719913 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0accfb8e0 con 0x55a0ace0ea10 2024-04-20T21:39:24.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f07f3400640 10 client.26167 handle_command_reply: tid=0 2024-04-20T21:39:24.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f081b230740 1 client.26167 shutdown 2024-04-20T21:39:24.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f081b230740 1 -- 172.21.15.22:0/3084719913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 msgr2=0x55a0acda0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f081b230740 1 --2- 172.21.15.22:0/3084719913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acda0da0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0800036040 tx=0x7f0800009230 comp rx=0 tx=0).stop 2024-04-20T21:39:24.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f081b230740 1 -- 172.21.15.22:0/3084719913 shutdown_connections 2024-04-20T21:39:24.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f081b230740 1 --2- 172.21.15.22:0/3084719913 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0ace0ea10 0x55a0ace10e30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f081b230740 1 --2- 172.21.15.22:0/3084719913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0acd98670 0x55a0acd9bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f081b230740 1 --2- 172.21.15.22:0/3084719913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0acd8fa90 0x55a0acda0da0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f081b230740 1 --2- 172.21.15.22:0/3084719913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0acd8f170 0x55a0acda0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f081b230740 1 -- 172.21.15.22:0/3084719913 >> 172.21.15.22:0/3084719913 conn(0x55a0acd898a0 msgr2=0x55a0acd95ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:24.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f07f3400640 0 client.26167 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:24.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f081b230740 1 -- 172.21.15.22:0/3084719913 shutdown_connections 2024-04-20T21:39:24.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.832+0000 7f081b230740 1 -- 172.21.15.22:0/3084719913 wait complete. 2024-04-20T21:39:24.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.833+0000 7f081b230740 20 client.26167 trim_cache size 0 max 16384 2024-04-20T21:39:24.834 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:39:24.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.833+0000 7f0817000640 1 -- 172.21.15.22:0/3002785583 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07f80a3de0 msgr2=0x7f07f80a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.833+0000 7f0817000640 1 --2- 172.21.15.22:0/3002785583 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07f80a3de0 0x7f07f80a62a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55a0acd8aa30 tx=0x7f080002cf90 comp rx=0 tx=0).stop 2024-04-20T21:39:24.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.833+0000 7f0817000640 1 -- 172.21.15.22:0/3002785583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f081019a750 msgr2=0x7f081019abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:24.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.833+0000 7f0817000640 1 --2- 172.21.15.22:0/3002785583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f081019a750 0x7f081019abd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f080800be20 tx=0x7f080800bf10 comp rx=0 tx=0).stop 2024-04-20T21:39:24.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.834+0000 7f0817000640 1 -- 172.21.15.22:0/3002785583 shutdown_connections 2024-04-20T21:39:24.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.834+0000 7f0817000640 1 --2- 172.21.15.22:0/3002785583 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07f80a3de0 0x7f07f80a62a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.834+0000 7f0817000640 1 --2- 172.21.15.22:0/3002785583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f081019a750 0x7f081019abd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.834+0000 7f0817000640 1 --2- 172.21.15.22:0/3002785583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0810109350 0x7f08101a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.834+0000 7f0817000640 1 --2- 172.21.15.22:0/3002785583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0810104280 0x7f08101a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:24.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.834+0000 7f0817000640 1 -- 172.21.15.22:0/3002785583 >> 172.21.15.22:0/3002785583 conn(0x7f08100ffaa0 msgr2=0x7f0810100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:24.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.834+0000 7f0817000640 1 -- 172.21.15.22:0/3002785583 shutdown_connections 2024-04-20T21:39:24.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:24.834+0000 7f0817000640 1 -- 172.21.15.22:0/3002785583 wait complete. 2024-04-20T21:39:25.192 INFO:tasks.workunit.client.0.smithi022.stdout: 1 434289 9.64 MB/sec execute 113 sec latency 19.323 ms 2024-04-20T21:39:25.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:25 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2832828811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:39:25.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:25 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2832828811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:39:25.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:25 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2832828811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:39:25.852 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:39:25.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:39:25.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:39:26.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.097+0000 7f7af0c00640 1 -- 172.21.15.22:0/1331898271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec108f50 msgr2=0x7f7aec1153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.097+0000 7f7af0c00640 1 --2- 172.21.15.22:0/1331898271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec108f50 0x7f7aec1153c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009ca0 tx=0x7f7ae402c100 comp rx=0 tx=0).stop 2024-04-20T21:39:26.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.097+0000 7f7af0c00640 1 -- 172.21.15.22:0/1331898271 shutdown_connections 2024-04-20T21:39:26.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.097+0000 7f7af0c00640 1 --2- 172.21.15.22:0/1331898271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec108f50 0x7f7aec1153c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.097+0000 7f7af0c00640 1 --2- 172.21.15.22:0/1331898271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec108590 0x7f7aec108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.097+0000 7f7af0c00640 1 --2- 172.21.15.22:0/1331898271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aec10e600 0x7f7aec10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.097+0000 7f7af0c00640 1 -- 172.21.15.22:0/1331898271 >> 172.21.15.22:0/1331898271 conn(0x7f7aec0764a0 msgr2=0x7f7aec0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.097+0000 7f7af0c00640 1 -- 172.21.15.22:0/1331898271 shutdown_connections 2024-04-20T21:39:26.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.097+0000 7f7af0c00640 1 -- 172.21.15.22:0/1331898271 wait complete. 2024-04-20T21:39:26.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.098+0000 7f7af0c00640 1 Processor -- start 2024-04-20T21:39:26.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.098+0000 7f7af0c00640 1 -- start start 2024-04-20T21:39:26.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.099+0000 7f7af0c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec108590 0x7f7aec1a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.099+0000 7f7af0c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec108f50 0x7f7aec1a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.099+0000 7f7af0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aec10e600 0x7f7aec19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.099+0000 7f7af0c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1188e0 con 0x7f7aec108f50 2024-04-20T21:39:26.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.099+0000 7f7af0c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec118760 con 0x7f7aec108590 2024-04-20T21:39:26.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.099+0000 7f7af0c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec118a60 con 0x7f7aec10e600 2024-04-20T21:39:26.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.099+0000 7f7ae9e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec108590 0x7f7aec1a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.099+0000 7f7ae9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec108f50 0x7f7aec1a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.099+0000 7f7ae9e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec108590 0x7f7aec1a4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48138/0 (socket says 172.21.15.22:48138) 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.099+0000 7f7ae9e00640 1 -- 172.21.15.22:0/1361310197 learned_addr learned my addr 172.21.15.22:0/1361310197 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.100+0000 7f7aea800640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aec10e600 0x7f7aec19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.100+0000 7f7ae9400640 1 -- 172.21.15.22:0/1361310197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aec10e600 msgr2=0x7f7aec19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.100+0000 7f7ae9400640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aec10e600 0x7f7aec19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.100+0000 7f7ae9400640 1 -- 172.21.15.22:0/1361310197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec108590 msgr2=0x7f7aec1a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.100+0000 7f7ae9400640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec108590 0x7f7aec1a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.100+0000 7f7ae9400640 1 -- 172.21.15.22:0/1361310197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec19f420 con 0x7f7aec108f50 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.100+0000 7f7ae9e00640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec108590 0x7f7aec1a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.100+0000 7f7aea800640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aec10e600 0x7f7aec19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.100+0000 7f7ae9400640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec108f50 0x7f7aec1a4f90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800e9f0 tx=0x7f7ad800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.101+0000 7f7ae2a00640 1 -- 172.21.15.22:0/1361310197 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad800cd80 con 0x7f7aec108f50 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.101+0000 7f7af0c00640 1 -- 172.21.15.22:0/1361310197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aec19f710 con 0x7f7aec108f50 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.101+0000 7f7ae2a00640 1 -- 172.21.15.22:0/1361310197 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad80035f0 con 0x7f7aec108f50 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.101+0000 7f7ae2a00640 1 -- 172.21.15.22:0/1361310197 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8005220 con 0x7f7aec108f50 2024-04-20T21:39:26.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.101+0000 7f7af0c00640 1 -- 172.21.15.22:0/1361310197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aec1ab9d0 con 0x7f7aec108f50 2024-04-20T21:39:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.105+0000 7f7af380d740 1 Processor -- start 2024-04-20T21:39:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.105+0000 7f7ae2a00640 1 -- 172.21.15.22:0/1361310197 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7ad80037a0 con 0x7f7aec108f50 2024-04-20T21:39:26.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.105+0000 7f7af380d740 1 -- start start 2024-04-20T21:39:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.105+0000 7f7af380d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7cd32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.105+0000 7f7af380d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7cdbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.105+0000 7f7af380d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7cde800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.105+0000 7f7af380d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cac7c37d20 con 0x55cac7cd2f10 2024-04-20T21:39:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.105+0000 7f7af380d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cac7bde3e0 con 0x55cac7cdc410 2024-04-20T21:39:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.105+0000 7f7af380d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cac7c33ce0 con 0x55cac7cd3830 2024-04-20T21:39:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.106+0000 7f7ae9400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7cde800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.106+0000 7f7ae9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7cdbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.106+0000 7f7ae9400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7cde800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48142/0 (socket says 172.21.15.22:48142) 2024-04-20T21:39:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.106+0000 7f7ae9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7cdbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46316/0 (socket says 172.21.15.22:46316) 2024-04-20T21:39:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.106+0000 7f7ae9400640 1 -- 172.21.15.22:0/2783032289 learned_addr learned my addr 172.21.15.22:0/2783032289 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.106+0000 7f7aea800640 1 --2- 172.21.15.22:0/2783032289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7cd32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.106+0000 7f7ae9400640 1 -- 172.21.15.22:0/2783032289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 msgr2=0x55cac7cdbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.106+0000 7f7ae9400640 1 --2- 172.21.15.22:0/2783032289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7cdbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.106+0000 7f7ae9400640 1 -- 172.21.15.22:0/2783032289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 msgr2=0x55cac7cd32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.106+0000 7f7ae9400640 1 --2- 172.21.15.22:0/2783032289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7cd32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.106+0000 7f7ae9400640 1 -- 172.21.15.22:0/2783032289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac7bfd980 con 0x55cac7cdc410 2024-04-20T21:39:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.107+0000 7f7ae9400640 1 --2- 172.21.15.22:0/2783032289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7cde800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7ad80ce090 tx=0x7f7ad80d4e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87926f139f2d6cbe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.107+0000 7f7ae2a00640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7ad00a3d20 0x7f7ad00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.107+0000 7f7ae2a00640 1 -- 172.21.15.22:0/1361310197 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7ad80d0a80 con 0x7f7aec108f50 2024-04-20T21:39:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.107+0000 7f7aea800640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7ad00a3d20 0x7f7ad00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.107+0000 7f7ad7000640 1 -- 172.21.15.22:0/2783032289 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80da040 con 0x55cac7cdc410 2024-04-20T21:39:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.107+0000 7f7ad7000640 1 -- 172.21.15.22:0/2783032289 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad809d0a0 con 0x55cac7cdc410 2024-04-20T21:39:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.107+0000 7f7ad7000640 1 -- 172.21.15.22:0/2783032289 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80a5610 con 0x55cac7cdc410 2024-04-20T21:39:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.108+0000 7f7aea800640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7ad00a3d20 0x7f7ad00a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4005e30 tx=0x7f7ae4005d80 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.108+0000 7f7af380d740 1 -- 172.21.15.22:0/2783032289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 msgr2=0x55cac7cde800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.108+0000 7f7af380d740 1 --2- 172.21.15.22:0/2783032289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7cde800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7ad80ce090 tx=0x7f7ad80d4e60 comp rx=0 tx=0).stop 2024-04-20T21:39:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.108+0000 7f7af380d740 1 -- 172.21.15.22:0/2783032289 shutdown_connections 2024-04-20T21:39:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.108+0000 7f7af380d740 1 --2- 172.21.15.22:0/2783032289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7cde800 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.108+0000 7f7af380d740 1 --2- 172.21.15.22:0/2783032289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7cdbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.108+0000 7f7af380d740 1 --2- 172.21.15.22:0/2783032289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7cd32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.108+0000 7f7af380d740 1 -- 172.21.15.22:0/2783032289 >> 172.21.15.22:0/2783032289 conn(0x55cac7ccd640 msgr2=0x55cac7ccda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.108+0000 7f7af380d740 1 -- 172.21.15.22:0/2783032289 shutdown_connections 2024-04-20T21:39:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.108+0000 7f7af380d740 1 -- 172.21.15.22:0/2783032289 wait complete. 2024-04-20T21:39:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.109+0000 7f7af380d740 1 -- start start 2024-04-20T21:39:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.109+0000 7f7af380d740 1 Processor -- start 2024-04-20T21:39:26.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7af380d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7cd9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7af380d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7cd9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7af380d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7cd9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7af380d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cac7cdf200 con 0x55cac7cd2f10 2024-04-20T21:39:26.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7af380d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cac7bf17a0 con 0x55cac7cdc410 2024-04-20T21:39:26.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7af380d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cac7cdf380 con 0x55cac7cd3830 2024-04-20T21:39:26.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7af380d740 10 client.0 _ll_register_callbacks cb 0x55cac7bf3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7aea800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7cd9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7aea800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7cd9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48152/0 (socket says 172.21.15.22:48152) 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7aea800640 1 -- 172.21.15.22:0/4027377733 learned_addr learned my addr 172.21.15.22:0/4027377733 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7ae9e00640 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7cd9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7ae9400640 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7cd9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7aea800640 1 -- 172.21.15.22:0/4027377733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 msgr2=0x55cac7cd9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7aea800640 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7cd9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7aea800640 1 -- 172.21.15.22:0/4027377733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 msgr2=0x55cac7cd9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7aea800640 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7cd9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.110+0000 7f7aea800640 1 -- 172.21.15.22:0/4027377733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac7a127e0 con 0x55cac7cdc410 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.111+0000 7f7aea800640 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7cd9ec0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4036000 tx=0x7f7ae402c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15483aa69ce47f1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.111+0000 7f7acb400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.111+0000 7f7acb400640 1 -- 172.21.15.22:0/4027377733 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4052020 con 0x55cac7cdc410 2024-04-20T21:39:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.111+0000 7f7acb400640 1 -- 172.21.15.22:0/4027377733 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7ae4041670 con 0x55cac7cdc410 2024-04-20T21:39:26.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.111+0000 7f7af380d740 1 -- 172.21.15.22:0/4027377733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ad809d0a0 con 0x55cac7cdc410 2024-04-20T21:39:26.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.111+0000 7f7acb400640 1 -- 172.21.15.22:0/4027377733 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7ae4039040 con 0x55cac7cdc410 2024-04-20T21:39:26.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.112+0000 7f7af380d740 10 client.26159 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:26.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.112+0000 7f7af380d740 1 -- 172.21.15.22:0/4027377733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55cac7a72f70 con 0x55cac7cdc410 2024-04-20T21:39:26.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.112+0000 7f7acb400640 1 -- 172.21.15.22:0/4027377733 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad809d0a0 con 0x55cac7cdc410 2024-04-20T21:39:26.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.112+0000 7f7acb400640 1 -- 172.21.15.22:0/4027377733 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f7ae403f040 con 0x55cac7cdc410 2024-04-20T21:39:26.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.112+0000 7f7af380d740 10 client.26159 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:26.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.112+0000 7f7af380d740 10 client.26159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.112+0000 7f7af380d740 20 client.26159 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.112+0000 7f7af380d740 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cac7cec810 0x55cac7ceec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.112+0000 7f7af380d740 4 client.26159 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.112+0000 7f7af380d740 1 -- 172.21.15.22:0/4027377733 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cac7c45740 con 0x55cac7cec810 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.112+0000 7f7ae9e00640 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cac7cec810 0x55cac7ceec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.113+0000 7f7ae9e00640 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cac7cec810 0x55cac7ceec60 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27a8f652ed0d60c server_cookie=f8f6cf26c35ca7c9 in_seq=0 out_seq=0 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.113+0000 7f7acb400640 10 client.26159 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.114+0000 7f7acb400640 1 -- 172.21.15.22:0/4027377733 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cac7c45740 con 0x55cac7cec810 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.114+0000 7f7acb400640 10 client.26159 handle_command_reply: tid=0 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.114+0000 7f7af380d740 1 client.26159 shutdown 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.115+0000 7f7af380d740 1 -- 172.21.15.22:0/4027377733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 msgr2=0x55cac7cd9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.115+0000 7f7af380d740 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7cd9ec0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4036000 tx=0x7f7ae402c100 comp rx=0 tx=0).stop 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.115+0000 7f7af380d740 1 -- 172.21.15.22:0/4027377733 shutdown_connections 2024-04-20T21:39:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.115+0000 7f7af380d740 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cac7cec810 0x55cac7ceec60 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.115+0000 7f7af380d740 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7cd9ec0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.115+0000 7f7af380d740 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7cd9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.115+0000 7f7af380d740 1 --2- 172.21.15.22:0/4027377733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7cd9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.115+0000 7f7acb400640 0 client.26159 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.115+0000 7f7af380d740 1 -- 172.21.15.22:0/4027377733 >> 172.21.15.22:0/4027377733 conn(0x55cac7ccd640 msgr2=0x55cac7ccf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.115+0000 7f7af380d740 1 -- 172.21.15.22:0/4027377733 shutdown_connections 2024-04-20T21:39:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.115+0000 7f7af380d740 1 -- 172.21.15.22:0/4027377733 wait complete. 2024-04-20T21:39:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.115+0000 7f7af380d740 20 client.26159 trim_cache size 0 max 16384 2024-04-20T21:39:26.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.136+0000 7f7af380d740 1 Processor -- start 2024-04-20T21:39:26.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.136+0000 7f7af380d740 1 -- start start 2024-04-20T21:39:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.137+0000 7f7af380d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd2f10 0x55cac7cdbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.137+0000 7f7af380d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cd3830 0x55cac7ce80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.137+0000 7f7af380d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cdc410 0x55cac7ccc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.137+0000 7f7af380d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1b2370 con 0x55cac7cdc410 2024-04-20T21:39:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.137+0000 7f7af380d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1b4bf0 con 0x55cac7cd3830 2024-04-20T21:39:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.137+0000 7f7af380d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1b29e0 con 0x55cac7cd2f10 2024-04-20T21:39:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.137+0000 7f7ae9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd2f10 0x55cac7cdbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.137+0000 7f7ae9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd2f10 0x55cac7cdbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46344/0 (socket says 172.21.15.22:46344) 2024-04-20T21:39:26.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.137+0000 7f7aea800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cdc410 0x55cac7ccc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.137+0000 7f7ae9e00640 1 -- 172.21.15.22:0/452018829 learned_addr learned my addr 172.21.15.22:0/452018829 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:26.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.137+0000 7f7ae9400640 1 --2- 172.21.15.22:0/452018829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cd3830 0x55cac7ce80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.138+0000 7f7ae9e00640 1 -- 172.21.15.22:0/452018829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cd3830 msgr2=0x55cac7ce80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.138+0000 7f7ae9e00640 1 --2- 172.21.15.22:0/452018829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cd3830 0x55cac7ce80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.138+0000 7f7ae9e00640 1 -- 172.21.15.22:0/452018829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cdc410 msgr2=0x55cac7ccc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.138+0000 7f7ae9e00640 1 --2- 172.21.15.22:0/452018829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cdc410 0x55cac7ccc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.138+0000 7f7ae9e00640 1 -- 172.21.15.22:0/452018829 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac790a170 con 0x55cac7cd2f10 2024-04-20T21:39:26.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.138+0000 7f7ae9e00640 1 --2- 172.21.15.22:0/452018829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd2f10 0x55cac7cdbb70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55cac7cd29e0 tx=0x7f7adc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e8f6dc13aa3b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.138+0000 7f7ad6600640 1 -- 172.21.15.22:0/452018829 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00f8d0 con 0x55cac7cd2f10 2024-04-20T21:39:26.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.138+0000 7f7ad6600640 1 -- 172.21.15.22:0/452018829 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7adc005020 con 0x55cac7cd2f10 2024-04-20T21:39:26.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.138+0000 7f7ad6600640 1 -- 172.21.15.22:0/452018829 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc004c30 con 0x55cac7cd2f10 2024-04-20T21:39:26.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.139+0000 7f7af380d740 1 -- 172.21.15.22:0/452018829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd2f10 msgr2=0x55cac7cdbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.139+0000 7f7af380d740 1 --2- 172.21.15.22:0/452018829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd2f10 0x55cac7cdbb70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55cac7cd29e0 tx=0x7f7adc005f90 comp rx=0 tx=0).stop 2024-04-20T21:39:26.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.139+0000 7f7af380d740 1 -- 172.21.15.22:0/452018829 shutdown_connections 2024-04-20T21:39:26.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.139+0000 7f7af380d740 1 --2- 172.21.15.22:0/452018829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cdc410 0x55cac7ccc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.139+0000 7f7af380d740 1 --2- 172.21.15.22:0/452018829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cd3830 0x55cac7ce80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.139+0000 7f7af380d740 1 --2- 172.21.15.22:0/452018829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd2f10 0x55cac7cdbb70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.139+0000 7f7af380d740 1 -- 172.21.15.22:0/452018829 >> 172.21.15.22:0/452018829 conn(0x55cac7ccd640 msgr2=0x55cac7ccbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.139+0000 7f7af380d740 1 -- 172.21.15.22:0/452018829 shutdown_connections 2024-04-20T21:39:26.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.139+0000 7f7af380d740 1 -- 172.21.15.22:0/452018829 wait complete. 2024-04-20T21:39:26.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.139+0000 7f7af380d740 1 -- start start 2024-04-20T21:39:26.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.140+0000 7f7af380d740 1 Processor -- start 2024-04-20T21:39:26.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.140+0000 7f7af380d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7ce0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.140+0000 7f7af380d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7ce0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.140+0000 7f7af380d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7d51da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.140+0000 7f7af380d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1b30b0 con 0x55cac7cd2f10 2024-04-20T21:39:26.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.140+0000 7f7af380d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1b4520 con 0x55cac7cdc410 2024-04-20T21:39:26.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.140+0000 7f7af380d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1b3780 con 0x55cac7cd3830 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.140+0000 7f7af380d740 10 client.0 _ll_register_callbacks cb 0x7f7adc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.140+0000 7f7ae9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7ce0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.140+0000 7f7aea800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7d51da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.140+0000 7f7ae9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7ce0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48186/0 (socket says 172.21.15.22:48186) 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.140+0000 7f7ae9e00640 1 -- 172.21.15.22:0/363271145 learned_addr learned my addr 172.21.15.22:0/363271145 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.141+0000 7f7ae9400640 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7ce0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.141+0000 7f7ae9e00640 1 -- 172.21.15.22:0/363271145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 msgr2=0x55cac7ce0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.141+0000 7f7ae9e00640 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7ce0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.141+0000 7f7ae9e00640 1 -- 172.21.15.22:0/363271145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 msgr2=0x55cac7d51da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.141+0000 7f7ae9e00640 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7d51da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.141+0000 7f7ae9e00640 1 -- 172.21.15.22:0/363271145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac7d522e0 con 0x55cac7cd2f10 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.141+0000 7f7aea800640 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7d51da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.141+0000 7f7ae9e00640 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7ce0770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7adc004850 tx=0x7f7adc0128f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98f1f19804c691c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.141+0000 7f7acb400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:39:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.141+0000 7f7acb400640 1 -- 172.21.15.22:0/363271145 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc012290 con 0x55cac7cd2f10 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.141+0000 7f7acb400640 1 -- 172.21.15.22:0/363271145 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7adc0049b0 con 0x55cac7cd2f10 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.141+0000 7f7af380d740 1 -- 172.21.15.22:0/363271145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7adc005020 con 0x55cac7cd2f10 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.142+0000 7f7acb400640 1 -- 172.21.15.22:0/363271145 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7adc004bb0 con 0x55cac7cd2f10 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.142+0000 7f7af380d740 10 client.16023 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.142+0000 7f7af380d740 1 -- 172.21.15.22:0/363271145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55cac7d53590 con 0x55cac7cd2f10 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.142+0000 7f7acb400640 1 -- 172.21.15.22:0/363271145 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7adc005020 con 0x55cac7cd2f10 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.142+0000 7f7acb400640 1 -- 172.21.15.22:0/363271145 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f7adc0223d0 con 0x55cac7cd2f10 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.142+0000 7f7af380d740 10 client.16023 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.142+0000 7f7af380d740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.142+0000 7f7af380d740 20 client.16023 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.142+0000 7f7af380d740 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cac7d53bf0 0x55cac7d55fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.142+0000 7f7af380d740 4 client.16023 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.142+0000 7f7af380d740 1 -- 172.21.15.22:0/363271145 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cac7c3f5b0 con 0x55cac7d53bf0 2024-04-20T21:39:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.143+0000 7f7ae9400640 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cac7d53bf0 0x55cac7d55fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.143+0000 7f7ae9400640 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cac7d53bf0 0x55cac7d55fe0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59375daa9ef0f68b server_cookie=a1555f24525f9f15 in_seq=0 out_seq=0 2024-04-20T21:39:26.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.143+0000 7f7acb400640 10 client.16023 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.144+0000 7f7acb400640 1 -- 172.21.15.22:0/363271145 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cac7c3f5b0 con 0x55cac7d53bf0 2024-04-20T21:39:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.144+0000 7f7acb400640 10 client.16023 handle_command_reply: tid=0 2024-04-20T21:39:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.144+0000 7f7af380d740 1 client.16023 shutdown 2024-04-20T21:39:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.145+0000 7f7af380d740 1 -- 172.21.15.22:0/363271145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 msgr2=0x55cac7ce0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.145+0000 7f7af380d740 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7ce0770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7adc004850 tx=0x7f7adc0128f0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.145+0000 7f7af380d740 1 -- 172.21.15.22:0/363271145 shutdown_connections 2024-04-20T21:39:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.145+0000 7f7af380d740 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cac7d53bf0 0x55cac7d55fe0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.145+0000 7f7af380d740 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cac7cdc410 0x55cac7d51da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.145+0000 7f7af380d740 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cac7cd3830 0x55cac7ce0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.145+0000 7f7af380d740 1 --2- 172.21.15.22:0/363271145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cac7cd2f10 0x55cac7ce0770 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.145+0000 7f7acb400640 0 client.16023 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.145+0000 7f7af380d740 1 -- 172.21.15.22:0/363271145 >> 172.21.15.22:0/363271145 conn(0x55cac7ccd640 msgr2=0x55cac7cd9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.145+0000 7f7af380d740 1 -- 172.21.15.22:0/363271145 shutdown_connections 2024-04-20T21:39:26.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.145+0000 7f7af380d740 1 -- 172.21.15.22:0/363271145 wait complete. 2024-04-20T21:39:26.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.145+0000 7f7af380d740 20 client.16023 trim_cache size 0 max 16384 2024-04-20T21:39:26.147 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:39:26.147 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:39:26.147 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "3dc133fb-4e50-43a2-a396-c5bd143d4b04", 2024-04-20T21:39:26.147 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:39:26.147 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:39:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.146+0000 7f7af0c00640 1 -- 172.21.15.22:0/1361310197 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7ad00a3d20 msgr2=0x7f7ad00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.146+0000 7f7af0c00640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7ad00a3d20 0x7f7ad00a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4005e30 tx=0x7f7ae4005d80 comp rx=0 tx=0).stop 2024-04-20T21:39:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.146+0000 7f7af0c00640 1 -- 172.21.15.22:0/1361310197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec108f50 msgr2=0x7f7aec1a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.146+0000 7f7af0c00640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec108f50 0x7f7aec1a4f90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800e9f0 tx=0x7f7ad800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.146+0000 7f7af0c00640 1 -- 172.21.15.22:0/1361310197 shutdown_connections 2024-04-20T21:39:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.146+0000 7f7af0c00640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f7ad00a3d20 0x7f7ad00a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.146+0000 7f7af0c00640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aec10e600 0x7f7aec19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.146+0000 7f7af0c00640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aec108f50 0x7f7aec1a4f90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.146+0000 7f7af0c00640 1 --2- 172.21.15.22:0/1361310197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec108590 0x7f7aec1a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.146+0000 7f7af0c00640 1 -- 172.21.15.22:0/1361310197 >> 172.21.15.22:0/1361310197 conn(0x7f7aec0764a0 msgr2=0x7f7aec077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.147+0000 7f7af0c00640 1 -- 172.21.15.22:0/1361310197 shutdown_connections 2024-04-20T21:39:26.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.147+0000 7f7af0c00640 1 -- 172.21.15.22:0/1361310197 wait complete. 2024-04-20T21:39:26.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:39:26.167 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:39:26.192 INFO:tasks.workunit.client.0.smithi022.stdout: 1 436524 9.68 MB/sec execute 114 sec latency 13.776 ms 2024-04-20T21:39:26.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.372+0000 7fd38d600640 1 -- 172.21.15.22:0/1990878937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108ea0 msgr2=0x7fd3881152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.372+0000 7fd38d600640 1 --2- 172.21.15.22:0/1990878937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108ea0 0x7fd3881152c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd380009be0 tx=0x7fd38002bf80 comp rx=0 tx=0).stop 2024-04-20T21:39:26.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.372+0000 7fd38d600640 1 -- 172.21.15.22:0/1990878937 shutdown_connections 2024-04-20T21:39:26.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.372+0000 7fd38d600640 1 --2- 172.21.15.22:0/1990878937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108ea0 0x7fd3881152c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.372+0000 7fd38d600640 1 --2- 172.21.15.22:0/1990878937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd388108500 0x7fd388108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.372+0000 7fd38d600640 1 --2- 172.21.15.22:0/1990878937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd38810d3d0 0x7fd38810d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.372+0000 7fd38d600640 1 -- 172.21.15.22:0/1990878937 >> 172.21.15.22:0/1990878937 conn(0x7fd388077d80 msgr2=0x7fd388106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.372+0000 7fd38d600640 1 -- 172.21.15.22:0/1990878937 shutdown_connections 2024-04-20T21:39:26.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.372+0000 7fd38d600640 1 -- 172.21.15.22:0/1990878937 wait complete. 2024-04-20T21:39:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.374+0000 7fd38d600640 1 Processor -- start 2024-04-20T21:39:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.374+0000 7fd38d600640 1 -- start start 2024-04-20T21:39:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.374+0000 7fd38d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108500 0x7fd388082560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.374+0000 7fd38d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd388108ea0 0x7fd38807b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.374+0000 7fd38d600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd38810d3d0 0x7fd38807bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.374+0000 7fd38d600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881187f0 con 0x7fd388108ea0 2024-04-20T21:39:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.374+0000 7fd38d600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd388118670 con 0x7fd38810d3d0 2024-04-20T21:39:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.374+0000 7fd38d600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd388118970 con 0x7fd388108500 2024-04-20T21:39:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.375+0000 7fd387e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108500 0x7fd388082560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.375+0000 7fd387e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108500 0x7fd388082560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46360/0 (socket says 172.21.15.22:46360) 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.375+0000 7fd387e00640 1 -- 172.21.15.22:0/235274717 learned_addr learned my addr 172.21.15.22:0/235274717 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.375+0000 7fd38cc00640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd38810d3d0 0x7fd38807bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.375+0000 7fd387400640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd388108ea0 0x7fd38807b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.375+0000 7fd387e00640 1 -- 172.21.15.22:0/235274717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd38810d3d0 msgr2=0x7fd38807bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.375+0000 7fd387e00640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd38810d3d0 0x7fd38807bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.375+0000 7fd387e00640 1 -- 172.21.15.22:0/235274717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd388108ea0 msgr2=0x7fd38807b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.375+0000 7fd387e00640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd388108ea0 0x7fd38807b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.375+0000 7fd387e00640 1 -- 172.21.15.22:0/235274717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38807c0e0 con 0x7fd388108500 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.376+0000 7fd387e00640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108500 0x7fd388082560 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd37800e9c0 tx=0x7fd37800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.376+0000 7fd384c00640 1 -- 172.21.15.22:0/235274717 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37800cde0 con 0x7fd388108500 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.376+0000 7fd384c00640 1 -- 172.21.15.22:0/235274717 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3780035f0 con 0x7fd388108500 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.376+0000 7fd384c00640 1 -- 172.21.15.22:0/235274717 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378010690 con 0x7fd388108500 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.376+0000 7fd38d600640 1 -- 172.21.15.22:0/235274717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38807c3d0 con 0x7fd388108500 2024-04-20T21:39:26.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.376+0000 7fd38d600640 1 -- 172.21.15.22:0/235274717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3880803b0 con 0x7fd388108500 2024-04-20T21:39:26.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.379+0000 7fd384c00640 1 -- 172.21.15.22:0/235274717 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd378004020 con 0x7fd388108500 2024-04-20T21:39:26.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.379+0000 7fd3915fd740 1 Processor -- start 2024-04-20T21:39:26.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.380+0000 7fd3915fd740 1 -- start start 2024-04-20T21:39:26.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.380+0000 7fd3915fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7380 0x55d74b1f7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.380+0000 7fd3915fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7ca0 0x55d74b200340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.380+0000 7fd3915fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b202c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.380+0000 7fd3915fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d74b15d150 con 0x55d74b200880 2024-04-20T21:39:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.380+0000 7fd3915fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d74b103370 con 0x55d74b1f7ca0 2024-04-20T21:39:26.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.380+0000 7fd3915fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d74b159110 con 0x55d74b1f7380 2024-04-20T21:39:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.380+0000 7fd387e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7ca0 0x55d74b200340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.381+0000 7fd38cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7380 0x55d74b1f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.381+0000 7fd387400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b202c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.381+0000 7fd38cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7380 0x55d74b1f7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46366/0 (socket says 172.21.15.22:46366) 2024-04-20T21:39:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.381+0000 7fd38cc00640 1 -- 172.21.15.22:0/890069896 learned_addr learned my addr 172.21.15.22:0/890069896 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.381+0000 7fd384c00640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd36c0a3da0 0x7fd36c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.381+0000 7fd38cc00640 1 -- 172.21.15.22:0/890069896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7ca0 msgr2=0x55d74b200340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.381+0000 7fd38cc00640 1 --2- 172.21.15.22:0/890069896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7ca0 0x55d74b200340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.381+0000 7fd38cc00640 1 -- 172.21.15.22:0/890069896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 msgr2=0x55d74b202c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.381+0000 7fd38cc00640 1 --2- 172.21.15.22:0/890069896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b202c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.381+0000 7fd38cc00640 1 -- 172.21.15.22:0/890069896 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74af397d0 con 0x55d74b1f7380 2024-04-20T21:39:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.381+0000 7fd384c00640 1 -- 172.21.15.22:0/235274717 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd3780d0a70 con 0x7fd388108500 2024-04-20T21:39:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.382+0000 7fd38cc00640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd36c0a3da0 0x7fd36c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.382+0000 7fd38cc00640 1 --2- 172.21.15.22:0/890069896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7380 0x55d74b1f7760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd380009be0 tx=0x7fd380033470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=118378b956c825bd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.382+0000 7fd37d200640 1 -- 172.21.15.22:0/890069896 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380043040 con 0x55d74b1f7380 2024-04-20T21:39:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.382+0000 7fd37d200640 1 -- 172.21.15.22:0/890069896 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd380003e30 con 0x55d74b1f7380 2024-04-20T21:39:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.382+0000 7fd37d200640 1 -- 172.21.15.22:0/890069896 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380033780 con 0x55d74b1f7380 2024-04-20T21:39:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.383+0000 7fd3915fd740 1 -- 172.21.15.22:0/890069896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7380 msgr2=0x55d74b1f7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.383+0000 7fd3915fd740 1 --2- 172.21.15.22:0/890069896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7380 0x55d74b1f7760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd380009be0 tx=0x7fd380033470 comp rx=0 tx=0).stop 2024-04-20T21:39:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.383+0000 7fd38cc00640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd36c0a3da0 0x7fd36c0a6260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd380048000 tx=0x7fd3800339e0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.383+0000 7fd3915fd740 1 -- 172.21.15.22:0/890069896 shutdown_connections 2024-04-20T21:39:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.383+0000 7fd3915fd740 1 --2- 172.21.15.22:0/890069896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b202c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.383+0000 7fd3915fd740 1 --2- 172.21.15.22:0/890069896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7ca0 0x55d74b200340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.383+0000 7fd3915fd740 1 --2- 172.21.15.22:0/890069896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7380 0x55d74b1f7760 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.383+0000 7fd3915fd740 1 -- 172.21.15.22:0/890069896 >> 172.21.15.22:0/890069896 conn(0x55d74b1f1ab0 msgr2=0x55d74b1f1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.383+0000 7fd3915fd740 1 -- 172.21.15.22:0/890069896 shutdown_connections 2024-04-20T21:39:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.383+0000 7fd3915fd740 1 -- 172.21.15.22:0/890069896 wait complete. 2024-04-20T21:39:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.384+0000 7fd3915fd740 1 -- start start 2024-04-20T21:39:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.384+0000 7fd3915fd740 1 Processor -- start 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.384+0000 7fd3915fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b1fd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd3915fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b1f7ca0 0x55d74b1fdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd3915fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b200880 0x55d74b1fe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd3915fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d74b203670 con 0x55d74b1f7ca0 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd3915fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d74b116740 con 0x55d74b1f7380 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd3915fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d74b2037f0 con 0x55d74b200880 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd3915fd740 10 client.0 _ll_register_callbacks cb 0x55d74b111b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd387e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b1fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd387e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b1fd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48202/0 (socket says 172.21.15.22:48202) 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd387e00640 1 -- 172.21.15.22:0/2119838019 learned_addr learned my addr 172.21.15.22:0/2119838019 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd387400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b1f7ca0 0x55d74b1fdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd38cc00640 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b200880 0x55d74b1fe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd387e00640 1 -- 172.21.15.22:0/2119838019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b200880 msgr2=0x55d74b1fe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd387e00640 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b200880 0x55d74b1fe290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd387e00640 1 -- 172.21.15.22:0/2119838019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b1f7ca0 msgr2=0x55d74b1fdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd387e00640 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b1f7ca0 0x55d74b1fdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd387e00640 1 -- 172.21.15.22:0/2119838019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74ae6f3f0 con 0x55d74b1f7380 2024-04-20T21:39:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.385+0000 7fd387400640 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b1f7ca0 0x55d74b1fdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.386+0000 7fd387e00640 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b1fd810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd3780ce1a0 tx=0x7fd37809c590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f023f345d3792f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.386+0000 7fd371600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:39:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.386+0000 7fd371600640 1 -- 172.21.15.22:0/2119838019 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3780d0e60 con 0x55d74b1f7380 2024-04-20T21:39:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.386+0000 7fd371600640 1 -- 172.21.15.22:0/2119838019 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd3780ce3c0 con 0x55d74b1f7380 2024-04-20T21:39:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.386+0000 7fd3915fd740 1 -- 172.21.15.22:0/2119838019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd380003e30 con 0x55d74b1f7380 2024-04-20T21:39:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.386+0000 7fd371600640 1 -- 172.21.15.22:0/2119838019 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd3780ce5c0 con 0x55d74b1f7380 2024-04-20T21:39:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.386+0000 7fd3915fd740 10 client.26177 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.386+0000 7fd3915fd740 1 -- 172.21.15.22:0/2119838019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d74ae2f0f0 con 0x55d74b1f7380 2024-04-20T21:39:26.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.386+0000 7fd371600640 1 -- 172.21.15.22:0/2119838019 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd380003e30 con 0x55d74b1f7380 2024-04-20T21:39:26.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.387+0000 7fd371600640 1 -- 172.21.15.22:0/2119838019 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fd3780870b0 con 0x55d74b1f7380 2024-04-20T21:39:26.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.387+0000 7fd3915fd740 10 client.26177 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:26.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.387+0000 7fd3915fd740 10 client.26177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:26.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.387+0000 7fd3915fd740 20 client.26177 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:26.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.387+0000 7fd3915fd740 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d74b210bc0 0x55d74b212ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.387+0000 7fd3915fd740 4 client.26177 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:26.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.387+0000 7fd3915fd740 1 -- 172.21.15.22:0/2119838019 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d74b16ab70 con 0x55d74b210bc0 2024-04-20T21:39:26.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.387+0000 7fd387400640 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d74b210bc0 0x55d74b212ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.388+0000 7fd387400640 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d74b210bc0 0x55d74b212ff0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34054fbac15675b7 server_cookie=5ccc122e404c72ba in_seq=0 out_seq=0 2024-04-20T21:39:26.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.388+0000 7fd371600640 10 client.26177 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:26.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.388+0000 7fd371600640 1 -- 172.21.15.22:0/2119838019 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d74b16ab70 con 0x55d74b210bc0 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.388+0000 7fd371600640 10 client.26177 handle_command_reply: tid=0 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd3915fd740 1 client.26177 shutdown 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd3915fd740 1 -- 172.21.15.22:0/2119838019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 msgr2=0x55d74b1fd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd3915fd740 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b1fd810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd3780ce1a0 tx=0x7fd37809c590 comp rx=0 tx=0).stop 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd3915fd740 1 -- 172.21.15.22:0/2119838019 shutdown_connections 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd3915fd740 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b200880 0x55d74b1fe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd3915fd740 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b1f7ca0 0x55d74b1fdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd3915fd740 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d74b210bc0 0x55d74b212ff0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd3915fd740 1 --2- 172.21.15.22:0/2119838019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b1fd810 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd3915fd740 1 -- 172.21.15.22:0/2119838019 >> 172.21.15.22:0/2119838019 conn(0x55d74b1f1ab0 msgr2=0x55d74b1f3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd371600640 0 client.26177 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd3915fd740 1 -- 172.21.15.22:0/2119838019 shutdown_connections 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd3915fd740 1 -- 172.21.15.22:0/2119838019 wait complete. 2024-04-20T21:39:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.389+0000 7fd3915fd740 20 client.26177 trim_cache size 0 max 16384 2024-04-20T21:39:26.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd3915fd740 1 Processor -- start 2024-04-20T21:39:26.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd3915fd740 1 -- start start 2024-04-20T21:39:26.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd3915fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b1fff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd3915fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7ca0 0x55d74b1f3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd3915fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b1fe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd3915fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881ba960 con 0x55d74b200880 2024-04-20T21:39:26.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd3915fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881bd1e0 con 0x55d74b1f7380 2024-04-20T21:39:26.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd3915fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881bafd0 con 0x55d74b1f7ca0 2024-04-20T21:39:26.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd387e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b1fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd387e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b1fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48216/0 (socket says 172.21.15.22:48216) 2024-04-20T21:39:26.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd387e00640 1 -- 172.21.15.22:0/3169593548 learned_addr learned my addr 172.21.15.22:0/3169593548 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:26.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd387400640 1 --2- 172.21.15.22:0/3169593548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7ca0 0x55d74b1f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.412+0000 7fd38cc00640 1 --2- 172.21.15.22:0/3169593548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b1fe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.413+0000 7fd387e00640 1 -- 172.21.15.22:0/3169593548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7ca0 msgr2=0x55d74b1f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.413+0000 7fd387e00640 1 --2- 172.21.15.22:0/3169593548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7ca0 0x55d74b1f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.413+0000 7fd387e00640 1 -- 172.21.15.22:0/3169593548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 msgr2=0x55d74b1fe0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.413+0000 7fd387e00640 1 --2- 172.21.15.22:0/3169593548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b1fe0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.413+0000 7fd387e00640 1 -- 172.21.15.22:0/3169593548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74b1f18e0 con 0x55d74b1f7380 2024-04-20T21:39:26.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.413+0000 7fd387400640 1 --2- 172.21.15.22:0/3169593548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7ca0 0x55d74b1f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:26.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.413+0000 7fd38cc00640 1 --2- 172.21.15.22:0/3169593548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b1fe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:26.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.413+0000 7fd387e00640 1 --2- 172.21.15.22:0/3169593548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b1fff40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd37809c920 tx=0x7fd37809c600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c8f22a4001b54f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.413+0000 7fd373e00640 1 -- 172.21.15.22:0/3169593548 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3780a5600 con 0x55d74b1f7380 2024-04-20T21:39:26.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.413+0000 7fd373e00640 1 -- 172.21.15.22:0/3169593548 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3780a0d40 con 0x55d74b1f7380 2024-04-20T21:39:26.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.413+0000 7fd373e00640 1 -- 172.21.15.22:0/3169593548 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3780ce3c0 con 0x55d74b1f7380 2024-04-20T21:39:26.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.414+0000 7fd3915fd740 1 -- 172.21.15.22:0/3169593548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 msgr2=0x55d74b1fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.414+0000 7fd3915fd740 1 --2- 172.21.15.22:0/3169593548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b1fff40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd37809c920 tx=0x7fd37809c600 comp rx=0 tx=0).stop 2024-04-20T21:39:26.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.414+0000 7fd3915fd740 1 -- 172.21.15.22:0/3169593548 shutdown_connections 2024-04-20T21:39:26.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.414+0000 7fd3915fd740 1 --2- 172.21.15.22:0/3169593548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b1fe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.414+0000 7fd3915fd740 1 --2- 172.21.15.22:0/3169593548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7ca0 0x55d74b1f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.414+0000 7fd3915fd740 1 --2- 172.21.15.22:0/3169593548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b1fff40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.414+0000 7fd3915fd740 1 -- 172.21.15.22:0/3169593548 >> 172.21.15.22:0/3169593548 conn(0x55d74b1f1ab0 msgr2=0x55d74b1f0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.414+0000 7fd3915fd740 1 -- 172.21.15.22:0/3169593548 shutdown_connections 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.414+0000 7fd3915fd740 1 -- 172.21.15.22:0/3169593548 wait complete. 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.415+0000 7fd3915fd740 1 -- start start 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.415+0000 7fd3915fd740 1 Processor -- start 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd3915fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b204d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd3915fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7ca0 0x55d74b205240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd3915fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b270170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd3915fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881bb6a0 con 0x55d74b200880 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd3915fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881bcb10 con 0x55d74b1f7380 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd3915fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881bbd70 con 0x55d74b1f7ca0 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd3915fd740 10 client.0 _ll_register_callbacks cb 0x7fd3780095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd387e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b204d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd387e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b204d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48232/0 (socket says 172.21.15.22:48232) 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd387e00640 1 -- 172.21.15.22:0/1198349818 learned_addr learned my addr 172.21.15.22:0/1198349818 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd387400640 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7ca0 0x55d74b205240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd38cc00640 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b270170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd387e00640 1 -- 172.21.15.22:0/1198349818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7ca0 msgr2=0x55d74b205240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd387e00640 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7ca0 0x55d74b205240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd387e00640 1 -- 172.21.15.22:0/1198349818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 msgr2=0x55d74b270170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:26.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd387e00640 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b270170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.416+0000 7fd387e00640 1 -- 172.21.15.22:0/1198349818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74b202c70 con 0x55d74b1f7380 2024-04-20T21:39:26.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.417+0000 7fd387e00640 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b204d00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd378099360 tx=0x7fd37809c590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f900146d0491c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.417+0000 7fd371600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:39:26.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.417+0000 7fd371600640 1 -- 172.21.15.22:0/1198349818 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3780ce5d0 con 0x55d74b1f7380 2024-04-20T21:39:26.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.417+0000 7fd371600640 1 -- 172.21.15.22:0/1198349818 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd37809bde0 con 0x55d74b1f7380 2024-04-20T21:39:26.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.417+0000 7fd3915fd740 1 -- 172.21.15.22:0/1198349818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3780a0d40 con 0x55d74b1f7380 2024-04-20T21:39:26.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.417+0000 7fd371600640 1 -- 172.21.15.22:0/1198349818 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd3780a5600 con 0x55d74b1f7380 2024-04-20T21:39:26.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.417+0000 7fd3915fd740 10 client.26183 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:26.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.417+0000 7fd3915fd740 1 -- 172.21.15.22:0/1198349818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d74b277990 con 0x55d74b1f7380 2024-04-20T21:39:26.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.418+0000 7fd371600640 1 -- 172.21.15.22:0/1198349818 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3780a0d40 con 0x55d74b1f7380 2024-04-20T21:39:26.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.418+0000 7fd371600640 1 -- 172.21.15.22:0/1198349818 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fd3780a1ed0 con 0x55d74b1f7380 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.418+0000 7fd3915fd740 10 client.26183 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.418+0000 7fd3915fd740 10 client.26183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.418+0000 7fd3915fd740 20 client.26183 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.418+0000 7fd3915fd740 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d74b277d20 0x55d74b27a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.418+0000 7fd3915fd740 4 client.26183 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.418+0000 7fd3915fd740 1 -- 172.21.15.22:0/1198349818 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d74b1649e0 con 0x55d74b277d20 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.418+0000 7fd387400640 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d74b277d20 0x55d74b27a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.419+0000 7fd387400640 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d74b277d20 0x55d74b27a110 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d1c90a8087e43f server_cookie=bbfdb7cd2c6817ed in_seq=0 out_seq=0 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.419+0000 7fd371600640 10 client.26183 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.419+0000 7fd371600640 1 -- 172.21.15.22:0/1198349818 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d74b1649e0 con 0x55d74b277d20 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.419+0000 7fd371600640 10 client.26183 handle_command_reply: tid=0 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.419+0000 7fd3915fd740 1 client.26183 shutdown 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.420+0000 7fd3915fd740 1 -- 172.21.15.22:0/1198349818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 msgr2=0x55d74b204d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.420+0000 7fd3915fd740 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b204d00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd378099360 tx=0x7fd37809c590 comp rx=0 tx=0).stop 2024-04-20T21:39:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.420+0000 7fd3915fd740 1 -- 172.21.15.22:0/1198349818 shutdown_connections 2024-04-20T21:39:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.420+0000 7fd3915fd740 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d74b200880 0x55d74b270170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.420+0000 7fd3915fd740 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d74b1f7ca0 0x55d74b205240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.420+0000 7fd3915fd740 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d74b277d20 0x55d74b27a110 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.420+0000 7fd3915fd740 1 --2- 172.21.15.22:0/1198349818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d74b1f7380 0x55d74b204d00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.420+0000 7fd3915fd740 1 -- 172.21.15.22:0/1198349818 >> 172.21.15.22:0/1198349818 conn(0x55d74b1f1ab0 msgr2=0x55d74b1f35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.420+0000 7fd371600640 0 client.26183 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.420+0000 7fd3915fd740 1 -- 172.21.15.22:0/1198349818 shutdown_connections 2024-04-20T21:39:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.420+0000 7fd3915fd740 1 -- 172.21.15.22:0/1198349818 wait complete. 2024-04-20T21:39:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.420+0000 7fd3915fd740 20 client.26183 trim_cache size 0 max 16384 2024-04-20T21:39:26.422 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:39:26.423 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:39:26.423 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:39:26.423 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:39:26.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.421+0000 7fd38d600640 1 -- 172.21.15.22:0/235274717 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd36c0a3da0 msgr2=0x7fd36c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.421+0000 7fd38d600640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd36c0a3da0 0x7fd36c0a6260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd380048000 tx=0x7fd3800339e0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.421+0000 7fd38d600640 1 -- 172.21.15.22:0/235274717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108500 msgr2=0x7fd388082560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.421+0000 7fd38d600640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108500 0x7fd388082560 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd37800e9c0 tx=0x7fd37800ee80 comp rx=0 tx=0).stop 2024-04-20T21:39:26.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.421+0000 7fd38d600640 1 -- 172.21.15.22:0/235274717 shutdown_connections 2024-04-20T21:39:26.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.421+0000 7fd38d600640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd36c0a3da0 0x7fd36c0a6260 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.421+0000 7fd38d600640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd38810d3d0 0x7fd38807bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.421+0000 7fd38d600640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd388108ea0 0x7fd38807b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.421+0000 7fd38d600640 1 --2- 172.21.15.22:0/235274717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd388108500 0x7fd388082560 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.421+0000 7fd38d600640 1 -- 172.21.15.22:0/235274717 >> 172.21.15.22:0/235274717 conn(0x7fd388077d80 msgr2=0x7fd3880783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.421+0000 7fd38d600640 1 -- 172.21.15.22:0/235274717 shutdown_connections 2024-04-20T21:39:26.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.421+0000 7fd38d600640 1 -- 172.21.15.22:0/235274717 wait complete. 2024-04-20T21:39:26.440 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:39:26.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:26.553 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:26 smithi022 ceph-mon[16843]: pgmap v171: 97 pgs: 97 active+clean; 251 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 247 op/s 2024-04-20T21:39:26.553 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:26 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:39:26.553 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:26 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:26.553 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:26 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:39:26.554 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:26 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:26.554 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:26 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:39:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.694+0000 7fba92600640 1 -- 172.21.15.22:0/3642111594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba8c1021a0 msgr2=0x7fba8c102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.694+0000 7fba92600640 1 --2- 172.21.15.22:0/3642111594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba8c1021a0 0x7fba8c102580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009860 tx=0x7fba7c02bb70 comp rx=0 tx=0).stop 2024-04-20T21:39:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.694+0000 7fba92600640 1 -- 172.21.15.22:0/3642111594 shutdown_connections 2024-04-20T21:39:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.694+0000 7fba92600640 1 --2- 172.21.15.22:0/3642111594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba8c107270 0x7fba8c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.694+0000 7fba92600640 1 --2- 172.21.15.22:0/3642111594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba8c102b50 0x7fba8c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.694+0000 7fba92600640 1 --2- 172.21.15.22:0/3642111594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba8c1021a0 0x7fba8c102580 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.694+0000 7fba92600640 1 -- 172.21.15.22:0/3642111594 >> 172.21.15.22:0/3642111594 conn(0x7fba8c079320 msgr2=0x7fba8c079730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.694+0000 7fba92600640 1 -- 172.21.15.22:0/3642111594 shutdown_connections 2024-04-20T21:39:26.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.695+0000 7fba92600640 1 -- 172.21.15.22:0/3642111594 wait complete. 2024-04-20T21:39:26.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.696+0000 7fba92600640 1 Processor -- start 2024-04-20T21:39:26.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.696+0000 7fba92600640 1 -- start start 2024-04-20T21:39:26.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.696+0000 7fba92600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba8c1021a0 0x7fba8c1a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.696+0000 7fba92600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba8c107270 0x7fba8c1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.696+0000 7fba92600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba8c19a8d0 0x7fba8c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba92600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c112300 con 0x7fba8c107270 2024-04-20T21:39:26.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba92600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c112180 con 0x7fba8c19a8d0 2024-04-20T21:39:26.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba92600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c112480 con 0x7fba8c1021a0 2024-04-20T21:39:26.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba8b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba8c1021a0 0x7fba8c1a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba8b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba8c1021a0 0x7fba8c1a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46412/0 (socket says 172.21.15.22:46412) 2024-04-20T21:39:26.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba8b400640 1 -- 172.21.15.22:0/2931209727 learned_addr learned my addr 172.21.15.22:0/2931209727 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:26.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba8aa00640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba8c107270 0x7fba8c1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba8be00640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba8c19a8d0 0x7fba8c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba8b400640 1 -- 172.21.15.22:0/2931209727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba8c19a8d0 msgr2=0x7fba8c19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba8b400640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba8c19a8d0 0x7fba8c19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba8b400640 1 -- 172.21.15.22:0/2931209727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba8c107270 msgr2=0x7fba8c1a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba8b400640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba8c107270 0x7fba8c1a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba8b400640 1 -- 172.21.15.22:0/2931209727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c19b3f0 con 0x7fba8c1021a0 2024-04-20T21:39:26.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.697+0000 7fba8be00640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba8c19a8d0 0x7fba8c19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:39:26.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.698+0000 7fba8b400640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba8c1021a0 0x7fba8c1a0720 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba7c037040 tx=0x7fba7c0045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.698+0000 7fba83e00640 1 -- 172.21.15.22:0/2931209727 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c041020 con 0x7fba8c1021a0 2024-04-20T21:39:26.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.698+0000 7fba83e00640 1 -- 172.21.15.22:0/2931209727 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba7c02c9f0 con 0x7fba8c1021a0 2024-04-20T21:39:26.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.698+0000 7fba92600640 1 -- 172.21.15.22:0/2931209727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8c19b680 con 0x7fba8c1021a0 2024-04-20T21:39:26.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.698+0000 7fba83e00640 1 -- 172.21.15.22:0/2931209727 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c039040 con 0x7fba8c1021a0 2024-04-20T21:39:26.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.698+0000 7fba92600640 1 -- 172.21.15.22:0/2931209727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8c1a7870 con 0x7fba8c1021a0 2024-04-20T21:39:26.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.699+0000 7fba92600640 1 -- 172.21.15.22:0/2931209727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba8c0691d0 con 0x7fba8c1021a0 2024-04-20T21:39:26.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.702+0000 7fba83e00640 1 -- 172.21.15.22:0/2931209727 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fba7c044020 con 0x7fba8c1021a0 2024-04-20T21:39:26.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.703+0000 7fba83e00640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fba700a3d20 0x7fba700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:26.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.704+0000 7fba83e00640 1 -- 172.21.15.22:0/2931209727 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fba7c0f20b0 con 0x7fba8c1021a0 2024-04-20T21:39:26.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.704+0000 7fba8aa00640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fba700a3d20 0x7fba700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:26.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.704+0000 7fba8aa00640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fba700a3d20 0x7fba700a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0761f0 tx=0x7fba78009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:26.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.707+0000 7fba83e00640 1 -- 172.21.15.22:0/2931209727 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fba7c04b020 con 0x7fba8c1021a0 2024-04-20T21:39:26.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:26 smithi122 ceph-mon[22147]: pgmap v171: 97 pgs: 97 active+clean; 251 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 247 op/s 2024-04-20T21:39:26.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:26 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:39:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:26 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:26 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:39:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:26 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:26.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:26 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:39:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:26 smithi033 ceph-mon[21626]: pgmap v171: 97 pgs: 97 active+clean; 251 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 247 op/s 2024-04-20T21:39:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:26 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:39:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:26 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:26 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:39:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:26 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:26 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:39:26.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.981+0000 7fba92600640 1 -- 172.21.15.22:0/2931209727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba8c19be00 con 0x7fba8c1021a0 2024-04-20T21:39:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.983+0000 7fba83e00640 1 -- 172.21.15.22:0/2931209727 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7fba7c0be7e0 con 0x7fba8c1021a0 2024-04-20T21:39:26.982 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:39:26.982 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:39:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:39:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.986+0000 7fba92600640 1 -- 172.21.15.22:0/2931209727 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fba700a3d20 msgr2=0x7fba700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.986+0000 7fba92600640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fba700a3d20 0x7fba700a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0761f0 tx=0x7fba78009260 comp rx=0 tx=0).stop 2024-04-20T21:39:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.986+0000 7fba92600640 1 -- 172.21.15.22:0/2931209727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba8c1021a0 msgr2=0x7fba8c1a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.986+0000 7fba92600640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba8c1021a0 0x7fba8c1a0720 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba7c037040 tx=0x7fba7c0045b0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.986+0000 7fba92600640 1 -- 172.21.15.22:0/2931209727 shutdown_connections 2024-04-20T21:39:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.986+0000 7fba92600640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fba700a3d20 0x7fba700a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.986+0000 7fba92600640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba8c19a8d0 0x7fba8c19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.986+0000 7fba92600640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba8c107270 0x7fba8c1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.986+0000 7fba92600640 1 --2- 172.21.15.22:0/2931209727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba8c1021a0 0x7fba8c1a0720 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:26.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.986+0000 7fba92600640 1 -- 172.21.15.22:0/2931209727 >> 172.21.15.22:0/2931209727 conn(0x7fba8c079320 msgr2=0x7fba8c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:26.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.986+0000 7fba92600640 1 -- 172.21.15.22:0/2931209727 shutdown_connections 2024-04-20T21:39:26.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:26.986+0000 7fba92600640 1 -- 172.21.15.22:0/2931209727 wait complete. 2024-04-20T21:39:27.007 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:39:27.192 INFO:tasks.workunit.client.0.smithi022.stdout: 1 438291 9.65 MB/sec execute 115 sec latency 31.968 ms 2024-04-20T21:39:27.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.227+0000 7f487f200640 1 -- 172.21.15.22:0/1014583825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4880102fc0 msgr2=0x7f48801033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:27.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.227+0000 7f487f200640 1 --2- 172.21.15.22:0/1014583825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4880102fc0 0x7f48801033a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4870009900 tx=0x7f487002bcd0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.227+0000 7f487f200640 1 -- 172.21.15.22:0/1014583825 shutdown_connections 2024-04-20T21:39:27.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.227+0000 7f487f200640 1 --2- 172.21.15.22:0/1014583825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488010c3e0 0x7f488010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.227+0000 7f487f200640 1 --2- 172.21.15.22:0/1014583825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48801038e0 0x7f488010bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.227+0000 7f487f200640 1 --2- 172.21.15.22:0/1014583825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4880102fc0 0x7f48801033a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.227+0000 7f487f200640 1 -- 172.21.15.22:0/1014583825 >> 172.21.15.22:0/1014583825 conn(0x7f48800fc7e0 msgr2=0x7f48800fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:27.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.227+0000 7f487f200640 1 -- 172.21.15.22:0/1014583825 shutdown_connections 2024-04-20T21:39:27.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.227+0000 7f487f200640 1 -- 172.21.15.22:0/1014583825 wait complete. 2024-04-20T21:39:27.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.228+0000 7f487f200640 1 Processor -- start 2024-04-20T21:39:27.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.229+0000 7f487f200640 1 -- start start 2024-04-20T21:39:27.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.229+0000 7f487f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4880102fc0 0x7f48801a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.229+0000 7f487f200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48801038e0 0x7f48801a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.229+0000 7f487f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488010c3e0 0x7f488019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.229+0000 7f487f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f48801121f0 con 0x7f4880102fc0 2024-04-20T21:39:27.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.229+0000 7f487f200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4880112070 con 0x7f48801038e0 2024-04-20T21:39:27.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.229+0000 7f487f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4880112370 con 0x7f488010c3e0 2024-04-20T21:39:27.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f487d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48801038e0 0x7f48801a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f487e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488010c3e0 0x7f488019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f487e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488010c3e0 0x7f488019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46430/0 (socket says 172.21.15.22:46430) 2024-04-20T21:39:27.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f487e800640 1 -- 172.21.15.22:0/488313926 learned_addr learned my addr 172.21.15.22:0/488313926 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:27.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f487de00640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4880102fc0 0x7f48801a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f487e800640 1 -- 172.21.15.22:0/488313926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48801038e0 msgr2=0x7f48801a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:27.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f487e800640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48801038e0 0x7f48801a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f487e800640 1 -- 172.21.15.22:0/488313926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4880102fc0 msgr2=0x7f48801a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:27.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f487e800640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4880102fc0 0x7f48801a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f487e800640 1 -- 172.21.15.22:0/488313926 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488019afb0 con 0x7f488010c3e0 2024-04-20T21:39:27.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f487de00640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4880102fc0 0x7f48801a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:27.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f487e800640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488010c3e0 0x7f488019a6f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f487800e970 tx=0x7f487800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:27.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.230+0000 7f4876a00640 1 -- 172.21.15.22:0/488313926 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487800ccc0 con 0x7f488010c3e0 2024-04-20T21:39:27.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.231+0000 7f487f200640 1 -- 172.21.15.22:0/488313926 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488019b2a0 con 0x7f488010c3e0 2024-04-20T21:39:27.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.231+0000 7f4876a00640 1 -- 172.21.15.22:0/488313926 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487800ce60 con 0x7f488010c3e0 2024-04-20T21:39:27.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.231+0000 7f4876a00640 1 -- 172.21.15.22:0/488313926 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878010490 con 0x7f488010c3e0 2024-04-20T21:39:27.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.231+0000 7f487f200640 1 -- 172.21.15.22:0/488313926 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48801a75b0 con 0x7f488010c3e0 2024-04-20T21:39:27.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.234+0000 7f4876a00640 1 -- 172.21.15.22:0/488313926 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4878005000 con 0x7f488010c3e0 2024-04-20T21:39:27.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.234+0000 7f48875f8740 1 Processor -- start 2024-04-20T21:39:27.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.234+0000 7f48875f8740 1 -- start start 2024-04-20T21:39:27.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.234+0000 7f48875f8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd27550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.234+0000 7f48875f8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd30130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.234+0000 7f48875f8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd30670 0x55be8bd32a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f48875f8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55be8bc8c050 con 0x55be8bd27a90 2024-04-20T21:39:27.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f48875f8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55be8bc32490 con 0x55be8bd27170 2024-04-20T21:39:27.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f48875f8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55be8bc88010 con 0x55be8bd30670 2024-04-20T21:39:27.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f487d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd27550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f487d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd27550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48302/0 (socket says 172.21.15.22:48302) 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f487de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd30130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f487d400640 1 -- 172.21.15.22:0/184557730 learned_addr learned my addr 172.21.15.22:0/184557730 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f487d400640 1 --2- 172.21.15.22:0/184557730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd30670 0x55be8bd32a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f487d400640 1 -- 172.21.15.22:0/184557730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd30670 msgr2=0x55be8bd32a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f487d400640 1 --2- 172.21.15.22:0/184557730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd30670 0x55be8bd32a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f487d400640 1 -- 172.21.15.22:0/184557730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 msgr2=0x55be8bd30130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f487d400640 1 --2- 172.21.15.22:0/184557730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd30130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.235+0000 7f487d400640 1 -- 172.21.15.22:0/184557730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be8bc3a070 con 0x55be8bd27170 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.236+0000 7f487d400640 1 --2- 172.21.15.22:0/184557730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd27550 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4868011b80 tx=0x7f4868004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ac85b0a44810da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.236+0000 7f4876a00640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f48600a3d20 0x7f48600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.236+0000 7f486f000640 1 -- 172.21.15.22:0/184557730 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486800fc50 con 0x55be8bd27170 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.236+0000 7f486f000640 1 -- 172.21.15.22:0/184557730 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486800fdf0 con 0x55be8bd27170 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.236+0000 7f486f000640 1 -- 172.21.15.22:0/184557730 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486800e7d0 con 0x55be8bd27170 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.236+0000 7f4876a00640 1 -- 172.21.15.22:0/488313926 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f48780d0d10 con 0x7f488010c3e0 2024-04-20T21:39:27.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.236+0000 7f487de00640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f48600a3d20 0x7f48600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.237+0000 7f48875f8740 1 -- 172.21.15.22:0/184557730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 msgr2=0x55be8bd27550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:27.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.237+0000 7f48875f8740 1 --2- 172.21.15.22:0/184557730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd27550 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4868011b80 tx=0x7f4868004ab0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.237+0000 7f48875f8740 1 -- 172.21.15.22:0/184557730 shutdown_connections 2024-04-20T21:39:27.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.237+0000 7f48875f8740 1 --2- 172.21.15.22:0/184557730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd30670 0x55be8bd32a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.237+0000 7f48875f8740 1 --2- 172.21.15.22:0/184557730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd30130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.237+0000 7f48875f8740 1 --2- 172.21.15.22:0/184557730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd27550 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.237+0000 7f48875f8740 1 -- 172.21.15.22:0/184557730 >> 172.21.15.22:0/184557730 conn(0x55be8bd218a0 msgr2=0x55be8bd21c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.237+0000 7f487de00640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f48600a3d20 0x7f48600a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4870006010 tx=0x7f4870005b50 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.237+0000 7f48875f8740 1 -- 172.21.15.22:0/184557730 shutdown_connections 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.237+0000 7f48875f8740 1 -- 172.21.15.22:0/184557730 wait complete. 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.238+0000 7f48875f8740 1 -- start start 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.238+0000 7f48875f8740 1 Processor -- start 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.238+0000 7f48875f8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd2d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.238+0000 7f48875f8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd2dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.238+0000 7f48875f8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd30670 0x55be8bd2e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.238+0000 7f48875f8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55be8bd33460 con 0x55be8bd27a90 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.238+0000 7f48875f8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55be8bc45860 con 0x55be8bd27170 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.238+0000 7f48875f8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55be8bd335e0 con 0x55be8bd30670 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f48875f8740 10 client.0 _ll_register_callbacks cb 0x55be8bc13fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f487de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd2d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f487de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd2d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48306/0 (socket says 172.21.15.22:48306) 2024-04-20T21:39:27.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f487de00640 1 -- 172.21.15.22:0/563085978 learned_addr learned my addr 172.21.15.22:0/563085978 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f487e800640 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd30670 0x55be8bd2e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f487de00640 1 -- 172.21.15.22:0/563085978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd30670 msgr2=0x55be8bd2e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f487de00640 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd30670 0x55be8bd2e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f487d400640 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd2dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f487de00640 1 -- 172.21.15.22:0/563085978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 msgr2=0x55be8bd2dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f487de00640 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd2dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f487de00640 1 -- 172.21.15.22:0/563085978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be8bc49950 con 0x55be8bd27170 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f487d400640 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd2dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f487de00640 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd2d6e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f487002cba0 tx=0x7f487002bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed8b3227e4eabd5c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.239+0000 7f4867400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.240+0000 7f4867400640 1 -- 172.21.15.22:0/563085978 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870044070 con 0x55be8bd27170 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.240+0000 7f4867400640 1 -- 172.21.15.22:0/563085978 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f4870005450 con 0x55be8bd27170 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.240+0000 7f48875f8740 1 -- 172.21.15.22:0/563085978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f486800fdf0 con 0x55be8bd27170 2024-04-20T21:39:27.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.240+0000 7f4867400640 1 -- 172.21.15.22:0/563085978 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4870059050 con 0x55be8bd27170 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.240+0000 7f48875f8740 10 client.26198 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.240+0000 7f48875f8740 1 -- 172.21.15.22:0/563085978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55be8bc17820 con 0x55be8bd27170 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.240+0000 7f4867400640 1 -- 172.21.15.22:0/563085978 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486800fdf0 con 0x55be8bd27170 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.240+0000 7f4867400640 1 -- 172.21.15.22:0/563085978 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f4870003830 con 0x55be8bd27170 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.241+0000 7f48875f8740 10 client.26198 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.241+0000 7f48875f8740 10 client.26198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.241+0000 7f48875f8740 20 client.26198 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.241+0000 7f48875f8740 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55be8bd40650 0x55be8bd42b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.241+0000 7f48875f8740 4 client.26198 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.241+0000 7f48875f8740 1 -- 172.21.15.22:0/563085978 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be8bc99a70 con 0x55be8bd40650 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.241+0000 7f487d400640 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55be8bd40650 0x55be8bd42b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.241+0000 7f487d400640 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55be8bd40650 0x55be8bd42b10 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1a43317bfd3104 server_cookie=56601f0927ac3472 in_seq=0 out_seq=0 2024-04-20T21:39:27.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.241+0000 7f4867400640 10 client.26198 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:27.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.242+0000 7f4867400640 1 -- 172.21.15.22:0/563085978 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be8bc99a70 con 0x55be8bd40650 2024-04-20T21:39:27.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.242+0000 7f4867400640 10 client.26198 handle_command_reply: tid=0 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.242+0000 7f48875f8740 1 client.26198 shutdown 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.242+0000 7f48875f8740 1 -- 172.21.15.22:0/563085978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 msgr2=0x55be8bd2d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.242+0000 7f48875f8740 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd2d6e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f487002cba0 tx=0x7f487002bcd0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.243+0000 7f48875f8740 1 -- 172.21.15.22:0/563085978 shutdown_connections 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.243+0000 7f48875f8740 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd30670 0x55be8bd2e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.243+0000 7f48875f8740 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd2dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.243+0000 7f48875f8740 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55be8bd40650 0x55be8bd42b10 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.243+0000 7f48875f8740 1 --2- 172.21.15.22:0/563085978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd2d6e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.243+0000 7f48875f8740 1 -- 172.21.15.22:0/563085978 >> 172.21.15.22:0/563085978 conn(0x55be8bd218a0 msgr2=0x55be8bd235f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.243+0000 7f4867400640 0 client.26198 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.243+0000 7f48875f8740 1 -- 172.21.15.22:0/563085978 shutdown_connections 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.243+0000 7f48875f8740 1 -- 172.21.15.22:0/563085978 wait complete. 2024-04-20T21:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.243+0000 7f48875f8740 20 client.26198 trim_cache size 0 max 16384 2024-04-20T21:39:27.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.263+0000 7f48875f8740 1 Processor -- start 2024-04-20T21:39:27.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.263+0000 7f48875f8740 1 -- start start 2024-04-20T21:39:27.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.263+0000 7f48875f8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd27170 0x55be8bd2fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.263+0000 7f48875f8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd3c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.263+0000 7f48875f8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd30670 0x55be8bd233e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.263+0000 7f48875f8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f48801add70 con 0x55be8bd27a90 2024-04-20T21:39:27.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.264+0000 7f48875f8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b05f0 con 0x55be8bd30670 2024-04-20T21:39:27.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.264+0000 7f48875f8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f48801ae3e0 con 0x55be8bd27170 2024-04-20T21:39:27.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.264+0000 7f487e800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd30670 0x55be8bd233e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.264+0000 7f487e800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd30670 0x55be8bd233e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48308/0 (socket says 172.21.15.22:48308) 2024-04-20T21:39:27.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.264+0000 7f487e800640 1 -- 172.21.15.22:0/636865187 learned_addr learned my addr 172.21.15.22:0/636865187 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:27.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.264+0000 7f487de00640 1 --2- 172.21.15.22:0/636865187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd27170 0x55be8bd2fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.264+0000 7f487d400640 1 --2- 172.21.15.22:0/636865187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd3c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.264+0000 7f487e800640 1 -- 172.21.15.22:0/636865187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd27170 msgr2=0x55be8bd2fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.264+0000 7f487e800640 1 --2- 172.21.15.22:0/636865187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd27170 0x55be8bd2fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.264+0000 7f487e800640 1 -- 172.21.15.22:0/636865187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 msgr2=0x55be8bd3c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.264+0000 7f487e800640 1 --2- 172.21.15.22:0/636865187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd3c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.264+0000 7f487e800640 1 -- 172.21.15.22:0/636865187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be8b99e3f0 con 0x55be8bd30670 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.265+0000 7f487e800640 1 --2- 172.21.15.22:0/636865187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd30670 0x55be8bd233e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f487800d010 tx=0x7f487809c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24ebb7924b136f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.265+0000 7f486e600640 1 -- 172.21.15.22:0/636865187 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878087040 con 0x55be8bd30670 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.265+0000 7f486e600640 1 -- 172.21.15.22:0/636865187 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48780a58a0 con 0x55be8bd30670 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.265+0000 7f486e600640 1 -- 172.21.15.22:0/636865187 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48780052f0 con 0x55be8bd30670 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.265+0000 7f48875f8740 1 -- 172.21.15.22:0/636865187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd30670 msgr2=0x55be8bd233e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.265+0000 7f48875f8740 1 --2- 172.21.15.22:0/636865187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd30670 0x55be8bd233e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f487800d010 tx=0x7f487809c650 comp rx=0 tx=0).stop 2024-04-20T21:39:27.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.265+0000 7f48875f8740 1 -- 172.21.15.22:0/636865187 shutdown_connections 2024-04-20T21:39:27.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.265+0000 7f48875f8740 1 --2- 172.21.15.22:0/636865187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd30670 0x55be8bd233e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.265+0000 7f48875f8740 1 --2- 172.21.15.22:0/636865187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd27a90 0x55be8bd3c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.265+0000 7f48875f8740 1 --2- 172.21.15.22:0/636865187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd27170 0x55be8bd2fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.265+0000 7f48875f8740 1 -- 172.21.15.22:0/636865187 >> 172.21.15.22:0/636865187 conn(0x55be8bd218a0 msgr2=0x55be8bd22cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:27.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.266+0000 7f48875f8740 1 -- 172.21.15.22:0/636865187 shutdown_connections 2024-04-20T21:39:27.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.266+0000 7f48875f8740 1 -- 172.21.15.22:0/636865187 wait complete. 2024-04-20T21:39:27.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.266+0000 7f48875f8740 1 -- start start 2024-04-20T21:39:27.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.266+0000 7f48875f8740 1 Processor -- start 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f48875f8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd34230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f48875f8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd27a90 0x55be8bd34770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f48875f8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd30670 0x55be8bd34cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f48875f8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f48801aeab0 con 0x55be8bd30670 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f48875f8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48801aff20 con 0x55be8bd27170 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f48875f8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f48801af180 con 0x55be8bd27a90 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f48875f8740 10 client.0 _ll_register_callbacks cb 0x7f487002c790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f487e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd30670 0x55be8bd34cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f487de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd34230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f487e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd30670 0x55be8bd34cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48308/0 (socket says 172.21.15.22:48308) 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f487e800640 1 -- 172.21.15.22:0/562387032 learned_addr learned my addr 172.21.15.22:0/562387032 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f487de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd34230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48316/0 (socket says 172.21.15.22:48316) 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.267+0000 7f487d400640 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd27a90 0x55be8bd34770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.268+0000 7f487d400640 1 -- 172.21.15.22:0/562387032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 msgr2=0x55be8bd34230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.268+0000 7f487d400640 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd34230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.268+0000 7f487d400640 1 -- 172.21.15.22:0/562387032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd30670 msgr2=0x55be8bd34cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.268+0000 7f487d400640 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd30670 0x55be8bd34cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.268+0000 7f487d400640 1 -- 172.21.15.22:0/562387032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be8ba687d0 con 0x55be8bd27a90 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.268+0000 7f487de00640 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd34230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.268+0000 7f487d400640 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd27a90 0x55be8bd34770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f486800e4b0 tx=0x7f4868004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7338f4d973c3d1ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.268+0000 7f4867400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.268+0000 7f4867400640 1 -- 172.21.15.22:0/562387032 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48680114d0 con 0x55be8bd27a90 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.268+0000 7f48875f8740 1 -- 172.21.15.22:0/562387032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48780a58a0 con 0x55be8bd27a90 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.268+0000 7f4867400640 1 -- 172.21.15.22:0/562387032 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f486800fc20 con 0x55be8bd27a90 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.269+0000 7f4867400640 1 -- 172.21.15.22:0/562387032 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4868009e10 con 0x55be8bd27a90 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.269+0000 7f48875f8740 10 client.26227 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.269+0000 7f48875f8740 1 -- 172.21.15.22:0/562387032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55be8bda6d50 con 0x55be8bd27a90 2024-04-20T21:39:27.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.269+0000 7f4867400640 1 -- 172.21.15.22:0/562387032 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48780a58a0 con 0x55be8bd27a90 2024-04-20T21:39:27.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.269+0000 7f4867400640 1 -- 172.21.15.22:0/562387032 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f4868027070 con 0x55be8bd27a90 2024-04-20T21:39:27.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.269+0000 7f48875f8740 10 client.26227 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:27.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.269+0000 7f48875f8740 10 client.26227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:27.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.269+0000 7f48875f8740 20 client.26227 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:27.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.269+0000 7f48875f8740 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55be8bda72e0 0x55be8bda9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:27.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.269+0000 7f48875f8740 4 client.26227 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:39:27.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.269+0000 7f48875f8740 1 -- 172.21.15.22:0/562387032 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be8bc938e0 con 0x55be8bda72e0 2024-04-20T21:39:27.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.270+0000 7f487de00640 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55be8bda72e0 0x55be8bda9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:27.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.270+0000 7f487de00640 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55be8bda72e0 0x55be8bda9700 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8531d0127de1f104 server_cookie=4ed294160aa767cf in_seq=0 out_seq=0 2024-04-20T21:39:27.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.270+0000 7f4867400640 10 client.26227 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:27.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.270+0000 7f4867400640 1 -- 172.21.15.22:0/562387032 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be8bc938e0 con 0x55be8bda72e0 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.270+0000 7f4867400640 10 client.26227 handle_command_reply: tid=0 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f48875f8740 1 client.26227 shutdown 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f48875f8740 1 -- 172.21.15.22:0/562387032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd27a90 msgr2=0x55be8bd34770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f48875f8740 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd27a90 0x55be8bd34770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f486800e4b0 tx=0x7f4868004a40 comp rx=0 tx=0).stop 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f48875f8740 1 -- 172.21.15.22:0/562387032 shutdown_connections 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f48875f8740 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be8bd30670 0x55be8bd34cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f48875f8740 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be8bd27a90 0x55be8bd34770 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f48875f8740 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55be8bda72e0 0x55be8bda9700 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f48875f8740 1 --2- 172.21.15.22:0/562387032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be8bd27170 0x55be8bd34230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f4867400640 0 client.26227 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f48875f8740 1 -- 172.21.15.22:0/562387032 >> 172.21.15.22:0/562387032 conn(0x55be8bd218a0 msgr2=0x55be8bd23560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f48875f8740 1 -- 172.21.15.22:0/562387032 shutdown_connections 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f48875f8740 1 -- 172.21.15.22:0/562387032 wait complete. 2024-04-20T21:39:27.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.271+0000 7f48875f8740 20 client.26227 trim_cache size 0 max 16384 2024-04-20T21:39:27.274 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:39:27.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.272+0000 7f487f200640 1 -- 172.21.15.22:0/488313926 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f48600a3d20 msgr2=0x7f48600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:27.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.272+0000 7f487f200640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f48600a3d20 0x7f48600a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4870006010 tx=0x7f4870005b50 comp rx=0 tx=0).stop 2024-04-20T21:39:27.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.272+0000 7f487f200640 1 -- 172.21.15.22:0/488313926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488010c3e0 msgr2=0x7f488019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:27.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.272+0000 7f487f200640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488010c3e0 0x7f488019a6f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f487800e970 tx=0x7f487800ed70 comp rx=0 tx=0).stop 2024-04-20T21:39:27.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.272+0000 7f487f200640 1 -- 172.21.15.22:0/488313926 shutdown_connections 2024-04-20T21:39:27.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.272+0000 7f487f200640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f48600a3d20 0x7f48600a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.272+0000 7f487f200640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f488010c3e0 0x7f488019a6f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.272+0000 7f487f200640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48801038e0 0x7f48801a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.272+0000 7f487f200640 1 --2- 172.21.15.22:0/488313926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4880102fc0 0x7f48801a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:27.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.272+0000 7f487f200640 1 -- 172.21.15.22:0/488313926 >> 172.21.15.22:0/488313926 conn(0x7f48800fc7e0 msgr2=0x7f4880100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:27.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.272+0000 7f487f200640 1 -- 172.21.15.22:0/488313926 shutdown_connections 2024-04-20T21:39:27.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:27.272+0000 7f487f200640 1 -- 172.21.15.22:0/488313926 wait complete. 2024-04-20T21:39:27.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:27 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2931209727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:39:27.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:27 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2931209727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:39:27.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:27 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2931209727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:39:28.192 INFO:tasks.workunit.client.0.smithi022.stdout: 1 440247 9.68 MB/sec execute 116 sec latency 76.434 ms 2024-04-20T21:39:28.291 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:39:28.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:39:28.291 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:39:28.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.552+0000 7f05cb200640 1 -- 172.21.15.22:0/3297191947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05cc109340 msgr2=0x7f05cc110f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.552+0000 7f05cb200640 1 --2- 172.21.15.22:0/3297191947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05cc109340 0x7f05cc110f10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009ca0 tx=0x7f05c402c100 comp rx=0 tx=0).stop 2024-04-20T21:39:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.552+0000 7f05cb200640 1 -- 172.21.15.22:0/3297191947 shutdown_connections 2024-04-20T21:39:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.552+0000 7f05cb200640 1 --2- 172.21.15.22:0/3297191947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05cc109340 0x7f05cc110f10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.552+0000 7f05cb200640 1 --2- 172.21.15.22:0/3297191947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05cc104c20 0x7f05cc108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.552+0000 7f05cb200640 1 --2- 172.21.15.22:0/3297191947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05cc104270 0x7f05cc104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.552+0000 7f05cb200640 1 -- 172.21.15.22:0/3297191947 >> 172.21.15.22:0/3297191947 conn(0x7f05cc0ffa70 msgr2=0x7f05cc101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.552+0000 7f05cb200640 1 -- 172.21.15.22:0/3297191947 shutdown_connections 2024-04-20T21:39:28.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.552+0000 7f05cb200640 1 -- 172.21.15.22:0/3297191947 wait complete. 2024-04-20T21:39:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.553+0000 7f05cb200640 1 Processor -- start 2024-04-20T21:39:28.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.554+0000 7f05cb200640 1 -- start start 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.554+0000 7f05cb200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05cc104270 0x7f05cc1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.554+0000 7f05cb200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05cc109340 0x7f05cc1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.554+0000 7f05cb200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05cc19a680 0x7f05cc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.554+0000 7f05cb200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1143e0 con 0x7f05cc109340 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.554+0000 7f05cb200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc114260 con 0x7f05cc104270 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.554+0000 7f05cb200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc114560 con 0x7f05cc19a680 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.554+0000 7f05c9e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05cc104270 0x7f05cc1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.554+0000 7f05c9e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05cc104270 0x7f05cc1a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48332/0 (socket says 172.21.15.22:48332) 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.554+0000 7f05c9e00640 1 -- 172.21.15.22:0/4119242982 learned_addr learned my addr 172.21.15.22:0/4119242982 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.554+0000 7f05c3e00640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05cc109340 0x7f05cc1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.554+0000 7f05ca800640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05cc19a680 0x7f05cc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.555+0000 7f05c9e00640 1 -- 172.21.15.22:0/4119242982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05cc19a680 msgr2=0x7f05cc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.555+0000 7f05c9e00640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05cc19a680 0x7f05cc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.555+0000 7f05c9e00640 1 -- 172.21.15.22:0/4119242982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05cc109340 msgr2=0x7f05cc1a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.555+0000 7f05c9e00640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05cc109340 0x7f05cc1a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.555+0000 7f05c9e00640 1 -- 172.21.15.22:0/4119242982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05cc19b280 con 0x7f05cc104270 2024-04-20T21:39:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.555+0000 7f05c9e00640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05cc104270 0x7f05cc1a05c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00e9f0 tx=0x7f05bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.555+0000 7f05c2a00640 1 -- 172.21.15.22:0/4119242982 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc00cd80 con 0x7f05cc104270 2024-04-20T21:39:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.555+0000 7f05c2a00640 1 -- 172.21.15.22:0/4119242982 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05bc010910 con 0x7f05cc104270 2024-04-20T21:39:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.555+0000 7f05c2a00640 1 -- 172.21.15.22:0/4119242982 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc018990 con 0x7f05cc104270 2024-04-20T21:39:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.555+0000 7f05cb200640 1 -- 172.21.15.22:0/4119242982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05cc1a7360 con 0x7f05cc104270 2024-04-20T21:39:28.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.555+0000 7f05cb200640 1 -- 172.21.15.22:0/4119242982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05cc1a7710 con 0x7f05cc104270 2024-04-20T21:39:28.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.560+0000 7f05d3faf740 1 Processor -- start 2024-04-20T21:39:28.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.560+0000 7f05c2a00640 1 -- 172.21.15.22:0/4119242982 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f05bc010ac0 con 0x7f05cc104270 2024-04-20T21:39:28.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.560+0000 7f05d3faf740 1 -- start start 2024-04-20T21:39:28.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.560+0000 7f05d3faf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bafb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05d3faf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 0x56408bb040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05d3faf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05d3faf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56408ba5fd20 con 0x56408bb045f0 2024-04-20T21:39:28.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05d3faf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56408ba063e0 con 0x56408bafb0f0 2024-04-20T21:39:28.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05d3faf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56408ba5bce0 con 0x56408bafba10 2024-04-20T21:39:28.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05ca800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bafb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05c9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 0x56408bb040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05ca800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bafb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48342/0 (socket says 172.21.15.22:48342) 2024-04-20T21:39:28.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05ca800640 1 -- 172.21.15.22:0/2124128779 learned_addr learned my addr 172.21.15.22:0/2124128779 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:28.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05c3e00640 1 --2- 172.21.15.22:0/2124128779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05ca800640 1 -- 172.21.15.22:0/2124128779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 msgr2=0x56408bb040b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05ca800640 1 --2- 172.21.15.22:0/2124128779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 0x56408bb040b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05ca800640 1 -- 172.21.15.22:0/2124128779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 msgr2=0x56408bb069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05ca800640 1 --2- 172.21.15.22:0/2124128779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05ca800640 1 -- 172.21.15.22:0/2124128779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56408ba25980 con 0x56408bafb0f0 2024-04-20T21:39:28.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.561+0000 7f05c9e00640 1 --2- 172.21.15.22:0/2124128779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 0x56408bb040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:28.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.562+0000 7f05ca800640 1 --2- 172.21.15.22:0/2124128779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bafb4d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f05c402c710 tx=0x7f05c4005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4610704daa396d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.562+0000 7f05bb400640 1 -- 172.21.15.22:0/2124128779 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4043070 con 0x56408bafb0f0 2024-04-20T21:39:28.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.562+0000 7f05c2a00640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f05b00a3d20 0x7f05b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.562+0000 7f05bb400640 1 -- 172.21.15.22:0/2124128779 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c4032c90 con 0x56408bafb0f0 2024-04-20T21:39:28.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.562+0000 7f05bb400640 1 -- 172.21.15.22:0/2124128779 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4039040 con 0x56408bafb0f0 2024-04-20T21:39:28.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.562+0000 7f05c2a00640 1 -- 172.21.15.22:0/4119242982 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f05bc0d0ba0 con 0x7f05cc104270 2024-04-20T21:39:28.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.563+0000 7f05c9e00640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f05b00a3d20 0x7f05b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.563+0000 7f05d3faf740 1 -- 172.21.15.22:0/2124128779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 msgr2=0x56408bafb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.563+0000 7f05d3faf740 1 --2- 172.21.15.22:0/2124128779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bafb4d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f05c402c710 tx=0x7f05c4005f80 comp rx=0 tx=0).stop 2024-04-20T21:39:28.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.563+0000 7f05d3faf740 1 -- 172.21.15.22:0/2124128779 shutdown_connections 2024-04-20T21:39:28.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.563+0000 7f05d3faf740 1 --2- 172.21.15.22:0/2124128779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.563+0000 7f05d3faf740 1 --2- 172.21.15.22:0/2124128779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 0x56408bb040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.563+0000 7f05d3faf740 1 --2- 172.21.15.22:0/2124128779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bafb4d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.563+0000 7f05d3faf740 1 -- 172.21.15.22:0/2124128779 >> 172.21.15.22:0/2124128779 conn(0x56408baf5820 msgr2=0x56408baf5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.564+0000 7f05d3faf740 1 -- 172.21.15.22:0/2124128779 shutdown_connections 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.564+0000 7f05c9e00640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f05b00a3d20 0x7f05b00a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x56408bafabc0 tx=0x7f05bc0d4650 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.564+0000 7f05d3faf740 1 -- 172.21.15.22:0/2124128779 wait complete. 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.565+0000 7f05d3faf740 1 -- start start 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.565+0000 7f05d3faf740 1 Processor -- start 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.566+0000 7f05d3faf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafb0f0 0x56408bb016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.566+0000 7f05d3faf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafba10 0x56408bb01bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.566+0000 7f05d3faf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb02130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.566+0000 7f05d3faf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56408bb07380 con 0x56408bb045f0 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.566+0000 7f05d3faf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56408ba197a0 con 0x56408bafba10 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.566+0000 7f05d3faf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56408bb07500 con 0x56408bafb0f0 2024-04-20T21:39:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.566+0000 7f05d3faf740 10 client.0 _ll_register_callbacks cb 0x56408ba1bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:28.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.566+0000 7f05ca800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafb0f0 0x56408bb016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.566+0000 7f05ca800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafb0f0 0x56408bb016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46518/0 (socket says 172.21.15.22:46518) 2024-04-20T21:39:28.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.566+0000 7f05ca800640 1 -- 172.21.15.22:0/3511295798 learned_addr learned my addr 172.21.15.22:0/3511295798 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:28.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.566+0000 7f05c3e00640 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafba10 0x56408bb01bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.566+0000 7f05ca800640 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb02130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.567+0000 7f05c3e00640 1 -- 172.21.15.22:0/3511295798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafb0f0 msgr2=0x56408bb016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.567+0000 7f05c3e00640 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafb0f0 0x56408bb016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.567+0000 7f05c3e00640 1 -- 172.21.15.22:0/3511295798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 msgr2=0x56408bb02130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.567+0000 7f05c3e00640 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb02130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.567+0000 7f05c3e00640 1 -- 172.21.15.22:0/3511295798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408b83a7e0 con 0x56408bafba10 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.567+0000 7f05c3e00640 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafba10 0x56408bb01bf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f05cc19bed0 tx=0x7f05b400bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccb50678cea72c72 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.567+0000 7f05afe00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.567+0000 7f05afe00640 1 -- 172.21.15.22:0/3511295798 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4005020 con 0x56408bafba10 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.567+0000 7f05afe00640 1 -- 172.21.15.22:0/3511295798 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f05b40051c0 con 0x56408bafba10 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.567+0000 7f05d3faf740 1 -- 172.21.15.22:0/3511295798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05c4032c90 con 0x56408bafba10 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.568+0000 7f05afe00640 1 -- 172.21.15.22:0/3511295798 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f05b40053c0 con 0x56408bafba10 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.568+0000 7f05d3faf740 10 client.26216 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.568+0000 7f05d3faf740 1 -- 172.21.15.22:0/3511295798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56408b89af70 con 0x56408bafba10 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.568+0000 7f05afe00640 1 -- 172.21.15.22:0/3511295798 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c4032c90 con 0x56408bafba10 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.568+0000 7f05afe00640 1 -- 172.21.15.22:0/3511295798 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f05b4019030 con 0x56408bafba10 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.568+0000 7f05d3faf740 10 client.26216 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.568+0000 7f05d3faf740 10 client.26216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.568+0000 7f05d3faf740 20 client.26216 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.568+0000 7f05d3faf740 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56408bb1d260 0x56408bb1f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.568+0000 7f05d3faf740 4 client.26216 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.568+0000 7f05d3faf740 1 -- 172.21.15.22:0/3511295798 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56408ba6d740 con 0x56408bb1d260 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.569+0000 7f05ca800640 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56408bb1d260 0x56408bb1f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.569+0000 7f05ca800640 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56408bb1d260 0x56408bb1f650 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eafd118d6d263e94 server_cookie=24b6b2e99a5ddcf4 in_seq=0 out_seq=0 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.569+0000 7f05afe00640 10 client.26216 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.570+0000 7f05afe00640 1 -- 172.21.15.22:0/3511295798 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56408ba6d740 con 0x56408bb1d260 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.570+0000 7f05afe00640 10 client.26216 handle_command_reply: tid=0 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.570+0000 7f05d3faf740 1 client.26216 shutdown 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.570+0000 7f05d3faf740 1 -- 172.21.15.22:0/3511295798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafba10 msgr2=0x56408bb01bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.570+0000 7f05d3faf740 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafba10 0x56408bb01bf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f05cc19bed0 tx=0x7f05b400bda0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.570+0000 7f05d3faf740 1 -- 172.21.15.22:0/3511295798 shutdown_connections 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.570+0000 7f05d3faf740 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb02130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.570+0000 7f05d3faf740 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56408bb1d260 0x56408bb1f650 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.570+0000 7f05d3faf740 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafba10 0x56408bb01bf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.570+0000 7f05d3faf740 1 --2- 172.21.15.22:0/3511295798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafb0f0 0x56408bb016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.570+0000 7f05d3faf740 1 -- 172.21.15.22:0/3511295798 >> 172.21.15.22:0/3511295798 conn(0x56408baf5820 msgr2=0x56408baf7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.570+0000 7f05afe00640 0 client.26216 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.571+0000 7f05d3faf740 1 -- 172.21.15.22:0/3511295798 shutdown_connections 2024-04-20T21:39:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.571+0000 7f05d3faf740 1 -- 172.21.15.22:0/3511295798 wait complete. 2024-04-20T21:39:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.571+0000 7f05d3faf740 20 client.26216 trim_cache size 0 max 16384 2024-04-20T21:39:28.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.590+0000 7f05d3faf740 1 Processor -- start 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.590+0000 7f05d3faf740 1 -- start start 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05d3faf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bb03de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05d3faf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bafba10 0x56408bb18ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05d3faf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bb045f0 0x56408bb24ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05d3faf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1a99d0 con 0x56408bafba10 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05d3faf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1ac290 con 0x56408bafb0f0 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05d3faf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1aa080 con 0x56408bb045f0 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05c3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bafba10 0x56408bb18ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05c3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bafba10 0x56408bb18ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48354/0 (socket says 172.21.15.22:48354) 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05c3e00640 1 -- 172.21.15.22:0/3608140084 learned_addr learned my addr 172.21.15.22:0/3608140084 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05ca800640 1 --2- 172.21.15.22:0/3608140084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bb045f0 0x56408bb24ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05ca800640 1 --2- 172.21.15.22:0/3608140084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bb03de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05c3e00640 1 -- 172.21.15.22:0/3608140084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bb045f0 msgr2=0x56408bb24ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05c3e00640 1 --2- 172.21.15.22:0/3608140084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bb045f0 0x56408bb24ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05c3e00640 1 -- 172.21.15.22:0/3608140084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 msgr2=0x56408bb03de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05c3e00640 1 --2- 172.21.15.22:0/3608140084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bb03de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.591+0000 7f05c3e00640 1 -- 172.21.15.22:0/3608140084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56408b732170 con 0x56408bafba10 2024-04-20T21:39:28.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.592+0000 7f05c3e00640 1 --2- 172.21.15.22:0/3608140084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bafba10 0x56408bb18ad0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f05b400de40 tx=0x7f05b4004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2684cb3916619449 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.592+0000 7f05baa00640 1 -- 172.21.15.22:0/3608140084 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4029040 con 0x56408bafba10 2024-04-20T21:39:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.592+0000 7f05baa00640 1 -- 172.21.15.22:0/3608140084 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b4003db0 con 0x56408bafba10 2024-04-20T21:39:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.592+0000 7f05baa00640 1 -- 172.21.15.22:0/3608140084 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b40053c0 con 0x56408bafba10 2024-04-20T21:39:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.593+0000 7f05d3faf740 1 -- 172.21.15.22:0/3608140084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bafba10 msgr2=0x56408bb18ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.593+0000 7f05d3faf740 1 --2- 172.21.15.22:0/3608140084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bafba10 0x56408bb18ad0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f05b400de40 tx=0x7f05b4004260 comp rx=0 tx=0).stop 2024-04-20T21:39:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.593+0000 7f05d3faf740 1 -- 172.21.15.22:0/3608140084 shutdown_connections 2024-04-20T21:39:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.593+0000 7f05d3faf740 1 --2- 172.21.15.22:0/3608140084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bb045f0 0x56408bb24ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.593+0000 7f05d3faf740 1 --2- 172.21.15.22:0/3608140084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bafba10 0x56408bb18ad0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.593+0000 7f05d3faf740 1 --2- 172.21.15.22:0/3608140084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bb03de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.593+0000 7f05d3faf740 1 -- 172.21.15.22:0/3608140084 >> 172.21.15.22:0/3608140084 conn(0x56408baf5820 msgr2=0x56408baf4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.593+0000 7f05d3faf740 1 -- 172.21.15.22:0/3608140084 shutdown_connections 2024-04-20T21:39:28.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.593+0000 7f05d3faf740 1 -- 172.21.15.22:0/3608140084 wait complete. 2024-04-20T21:39:28.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.594+0000 7f05d3faf740 1 -- start start 2024-04-20T21:39:28.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.594+0000 7f05d3faf740 1 Processor -- start 2024-04-20T21:39:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05d3faf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bb11130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05d3faf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 0x56408bb11670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05d3faf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb82750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05d3faf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1aa750 con 0x56408bb045f0 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05d3faf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1abbc0 con 0x56408bafb0f0 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05d3faf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1aae20 con 0x56408bafba10 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05d3faf740 10 client.0 _ll_register_callbacks cb 0x7f05b4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05c3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 0x56408bb11670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05c3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 0x56408bb11670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46536/0 (socket says 172.21.15.22:46536) 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05c3e00640 1 -- 172.21.15.22:0/824581565 learned_addr learned my addr 172.21.15.22:0/824581565 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05ca800640 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bb11130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05ca800640 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb82750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05c3e00640 1 -- 172.21.15.22:0/824581565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 msgr2=0x56408bb11130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05c3e00640 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bb11130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05c3e00640 1 -- 172.21.15.22:0/824581565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 msgr2=0x56408bb82750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05c3e00640 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb82750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05c3e00640 1 -- 172.21.15.22:0/824581565 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408bb82c90 con 0x56408bafba10 2024-04-20T21:39:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.595+0000 7f05ca800640 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb82750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:39:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.596+0000 7f05c3e00640 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 0x56408bb11670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f05b400def0 tx=0x7f05b4003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=909945d03b09fdfd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.596+0000 7f05afe00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:39:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.596+0000 7f05afe00640 1 -- 172.21.15.22:0/824581565 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b40053c0 con 0x56408bafba10 2024-04-20T21:39:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.596+0000 7f05afe00640 1 -- 172.21.15.22:0/824581565 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f05b400e350 con 0x56408bafba10 2024-04-20T21:39:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.596+0000 7f05d3faf740 1 -- 172.21.15.22:0/824581565 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b4003db0 con 0x56408bafba10 2024-04-20T21:39:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.597+0000 7f05afe00640 1 -- 172.21.15.22:0/824581565 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f05b4005850 con 0x56408bafba10 2024-04-20T21:39:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.597+0000 7f05d3faf740 10 client.26245 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.597+0000 7f05d3faf740 1 -- 172.21.15.22:0/824581565 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56408bb83f80 con 0x56408bafba10 2024-04-20T21:39:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.597+0000 7f05afe00640 1 -- 172.21.15.22:0/824581565 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b4003db0 con 0x56408bafba10 2024-04-20T21:39:28.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.597+0000 7f05afe00640 1 -- 172.21.15.22:0/824581565 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f05b4019030 con 0x56408bafba10 2024-04-20T21:39:28.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.597+0000 7f05d3faf740 10 client.26245 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:28.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.597+0000 7f05d3faf740 10 client.26245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:28.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.597+0000 7f05d3faf740 20 client.26245 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:28.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.597+0000 7f05d3faf740 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56408bb84560 0x56408bb86980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.597+0000 7f05d3faf740 4 client.26245 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:39:28.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.597+0000 7f05d3faf740 1 -- 172.21.15.22:0/824581565 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56408ba675b0 con 0x56408bb84560 2024-04-20T21:39:28.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.598+0000 7f05ca800640 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56408bb84560 0x56408bb86980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.598+0000 7f05ca800640 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56408bb84560 0x56408bb86980 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=269f117b0feda978 server_cookie=d0726bb74e356fd9 in_seq=0 out_seq=0 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.598+0000 7f05afe00640 10 client.26245 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.599+0000 7f05afe00640 1 -- 172.21.15.22:0/824581565 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56408ba675b0 con 0x56408bb84560 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.599+0000 7f05afe00640 10 client.26245 handle_command_reply: tid=0 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.599+0000 7f05d3faf740 1 client.26245 shutdown 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.600+0000 7f05d3faf740 1 -- 172.21.15.22:0/824581565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 msgr2=0x56408bb11670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.600+0000 7f05d3faf740 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 0x56408bb11670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f05b400def0 tx=0x7f05b4003f60 comp rx=0 tx=0).stop 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.600+0000 7f05d3faf740 1 -- 172.21.15.22:0/824581565 shutdown_connections 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.600+0000 7f05d3faf740 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56408bb045f0 0x56408bb82750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.600+0000 7f05d3faf740 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56408bafba10 0x56408bb11670 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.600+0000 7f05d3faf740 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56408bb84560 0x56408bb86980 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.600+0000 7f05d3faf740 1 --2- 172.21.15.22:0/824581565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56408bafb0f0 0x56408bb11130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.600+0000 7f05d3faf740 1 -- 172.21.15.22:0/824581565 >> 172.21.15.22:0/824581565 conn(0x56408baf5820 msgr2=0x56408bb01670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.600+0000 7f05afe00640 0 client.26245 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:28.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.600+0000 7f05d3faf740 1 -- 172.21.15.22:0/824581565 shutdown_connections 2024-04-20T21:39:28.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.600+0000 7f05d3faf740 1 -- 172.21.15.22:0/824581565 wait complete. 2024-04-20T21:39:28.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.601+0000 7f05d3faf740 20 client.26245 trim_cache size 0 max 16384 2024-04-20T21:39:28.602 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:39:28.602 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:39:28.602 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "bdcfc32a-41f6-41e2-af9c-9baa03996104", 2024-04-20T21:39:28.602 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:39:28.603 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:39:28.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.601+0000 7f05cb200640 1 -- 172.21.15.22:0/4119242982 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f05b00a3d20 msgr2=0x7f05b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.601+0000 7f05cb200640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f05b00a3d20 0x7f05b00a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x56408bafabc0 tx=0x7f05bc0d4650 comp rx=0 tx=0).stop 2024-04-20T21:39:28.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.601+0000 7f05cb200640 1 -- 172.21.15.22:0/4119242982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05cc104270 msgr2=0x7f05cc1a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.601+0000 7f05cb200640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05cc104270 0x7f05cc1a05c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00e9f0 tx=0x7f05bc00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.601+0000 7f05cb200640 1 -- 172.21.15.22:0/4119242982 shutdown_connections 2024-04-20T21:39:28.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.601+0000 7f05cb200640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f05b00a3d20 0x7f05b00a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.601+0000 7f05cb200640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05cc19a680 0x7f05cc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.601+0000 7f05cb200640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05cc109340 0x7f05cc1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.601+0000 7f05cb200640 1 --2- 172.21.15.22:0/4119242982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05cc104270 0x7f05cc1a05c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.601+0000 7f05cb200640 1 -- 172.21.15.22:0/4119242982 >> 172.21.15.22:0/4119242982 conn(0x7f05cc0ffa70 msgr2=0x7f05cc100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.602+0000 7f05cb200640 1 -- 172.21.15.22:0/4119242982 shutdown_connections 2024-04-20T21:39:28.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.602+0000 7f05cb200640 1 -- 172.21.15.22:0/4119242982 wait complete. 2024-04-20T21:39:28.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:39:28.621 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:39:28.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:28 smithi022 ceph-mon[16843]: pgmap v172: 97 pgs: 97 active+clean; 251 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 164 op/s 2024-04-20T21:39:28.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:28 smithi122 ceph-mon[22147]: pgmap v172: 97 pgs: 97 active+clean; 251 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 164 op/s 2024-04-20T21:39:28.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:28 smithi033 ceph-mon[21626]: pgmap v172: 97 pgs: 97 active+clean; 251 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 164 op/s 2024-04-20T21:39:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.820+0000 7fbf44200640 1 -- 172.21.15.22:0/1665076379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c1021a0 msgr2=0x7fbf3c102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.820+0000 7fbf44200640 1 --2- 172.21.15.22:0/1665076379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c1021a0 0x7fbf3c102580 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009900 tx=0x7fbf3002bc50 comp rx=0 tx=0).stop 2024-04-20T21:39:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.820+0000 7fbf44200640 1 -- 172.21.15.22:0/1665076379 shutdown_connections 2024-04-20T21:39:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.820+0000 7fbf44200640 1 --2- 172.21.15.22:0/1665076379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf3c107270 0x7fbf3c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.820+0000 7fbf44200640 1 --2- 172.21.15.22:0/1665076379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf3c102b50 0x7fbf3c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.820+0000 7fbf44200640 1 --2- 172.21.15.22:0/1665076379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c1021a0 0x7fbf3c102580 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.820+0000 7fbf44200640 1 -- 172.21.15.22:0/1665076379 >> 172.21.15.22:0/1665076379 conn(0x7fbf3c0793b0 msgr2=0x7fbf3c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.820+0000 7fbf44200640 1 -- 172.21.15.22:0/1665076379 shutdown_connections 2024-04-20T21:39:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.820+0000 7fbf44200640 1 -- 172.21.15.22:0/1665076379 wait complete. 2024-04-20T21:39:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.821+0000 7fbf44200640 1 Processor -- start 2024-04-20T21:39:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf44200640 1 -- start start 2024-04-20T21:39:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf44200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c1021a0 0x7fbf3c1a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf44200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf3c107270 0x7fbf3c1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf44200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf3c19a8b0 0x7fbf3c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf44200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c112350 con 0x7fbf3c19a8b0 2024-04-20T21:39:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf44200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1121d0 con 0x7fbf3c1021a0 2024-04-20T21:39:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf44200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1124d0 con 0x7fbf3c107270 2024-04-20T21:39:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf41800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c1021a0 0x7fbf3c1a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf40e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf3c107270 0x7fbf3c1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf41800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c1021a0 0x7fbf3c1a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48380/0 (socket says 172.21.15.22:48380) 2024-04-20T21:39:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf42200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf3c19a8b0 0x7fbf3c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf41800640 1 -- 172.21.15.22:0/644250088 learned_addr learned my addr 172.21.15.22:0/644250088 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.822+0000 7fbf40e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf3c107270 0x7fbf3c1a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46550/0 (socket says 172.21.15.22:46550) 2024-04-20T21:39:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.823+0000 7fbf40e00640 1 -- 172.21.15.22:0/644250088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c1021a0 msgr2=0x7fbf3c1a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.823+0000 7fbf40e00640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c1021a0 0x7fbf3c1a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.823+0000 7fbf40e00640 1 -- 172.21.15.22:0/644250088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf3c19a8b0 msgr2=0x7fbf3c19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.823+0000 7fbf40e00640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf3c19a8b0 0x7fbf3c19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.823+0000 7fbf40e00640 1 -- 172.21.15.22:0/644250088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c19b3d0 con 0x7fbf3c107270 2024-04-20T21:39:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.823+0000 7fbf42200640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf3c19a8b0 0x7fbf3c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.823+0000 7fbf40e00640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf3c107270 0x7fbf3c1a0c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800e9c0 tx=0x7fbf2800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.824+0000 7fbf3a000640 1 -- 172.21.15.22:0/644250088 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2800ccc0 con 0x7fbf3c107270 2024-04-20T21:39:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.824+0000 7fbf44200640 1 -- 172.21.15.22:0/644250088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3c19b6c0 con 0x7fbf3c107270 2024-04-20T21:39:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.824+0000 7fbf3a000640 1 -- 172.21.15.22:0/644250088 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf28010910 con 0x7fbf3c107270 2024-04-20T21:39:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.824+0000 7fbf3a000640 1 -- 172.21.15.22:0/644250088 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf28018880 con 0x7fbf3c107270 2024-04-20T21:39:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.824+0000 7fbf44200640 1 -- 172.21.15.22:0/644250088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3c1a7960 con 0x7fbf3c107270 2024-04-20T21:39:28.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.827+0000 7fbf46f8d740 1 Processor -- start 2024-04-20T21:39:28.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.827+0000 7fbf46f8d740 1 -- start start 2024-04-20T21:39:28.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.827+0000 7fbf3a000640 1 -- 172.21.15.22:0/644250088 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbf28010ac0 con 0x7fbf3c107270 2024-04-20T21:39:28.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.828+0000 7fbf46f8d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b0714580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.828+0000 7fbf46f8d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b0714ac0 0x5636b071d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.828+0000 7fbf46f8d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 0x5636b071fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.828+0000 7fbf46f8d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5636b067a150 con 0x5636b071d6a0 2024-04-20T21:39:28.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.828+0000 7fbf46f8d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636b0620370 con 0x5636b0714ac0 2024-04-20T21:39:28.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.828+0000 7fbf46f8d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5636b0676110 con 0x5636b07141a0 2024-04-20T21:39:28.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.828+0000 7fbf42200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b0714580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.828+0000 7fbf40e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 0x5636b071fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.828+0000 7fbf41800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b0714ac0 0x5636b071d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.828+0000 7fbf42200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b0714580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46564/0 (socket says 172.21.15.22:46564) 2024-04-20T21:39:28.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.828+0000 7fbf42200640 1 -- 172.21.15.22:0/4130539992 learned_addr learned my addr 172.21.15.22:0/4130539992 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:28.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.829+0000 7fbf41800640 1 -- 172.21.15.22:0/4130539992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 msgr2=0x5636b0714580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.829+0000 7fbf41800640 1 --2- 172.21.15.22:0/4130539992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b0714580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.829+0000 7fbf41800640 1 -- 172.21.15.22:0/4130539992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 msgr2=0x5636b071fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.829+0000 7fbf41800640 1 --2- 172.21.15.22:0/4130539992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 0x5636b071fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.829+0000 7fbf41800640 1 -- 172.21.15.22:0/4130539992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b04567d0 con 0x5636b0714ac0 2024-04-20T21:39:28.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.829+0000 7fbf40e00640 1 --2- 172.21.15.22:0/4130539992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 0x5636b071fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:28.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.829+0000 7fbf41800640 1 --2- 172.21.15.22:0/4130539992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b0714ac0 0x5636b071d160 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c19b9f0 tx=0x7fbf300095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=868751c8a1f992fe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.829+0000 7fbf2ea00640 1 -- 172.21.15.22:0/4130539992 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30042070 con 0x5636b0714ac0 2024-04-20T21:39:28.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.829+0000 7fbf2ea00640 1 -- 172.21.15.22:0/4130539992 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf30004240 con 0x5636b0714ac0 2024-04-20T21:39:28.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.830+0000 7fbf2ea00640 1 -- 172.21.15.22:0/4130539992 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30004ee0 con 0x5636b0714ac0 2024-04-20T21:39:28.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.830+0000 7fbf3a000640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbf240a3d20 0x7fbf240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.830+0000 7fbf3a000640 1 -- 172.21.15.22:0/644250088 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fbf280d0bb0 con 0x7fbf3c107270 2024-04-20T21:39:28.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.830+0000 7fbf42200640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbf240a3d20 0x7fbf240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.830+0000 7fbf46f8d740 1 -- 172.21.15.22:0/4130539992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b0714ac0 msgr2=0x5636b071d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.830+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/4130539992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b0714ac0 0x5636b071d160 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c19b9f0 tx=0x7fbf300095e0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.830+0000 7fbf46f8d740 1 -- 172.21.15.22:0/4130539992 shutdown_connections 2024-04-20T21:39:28.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.830+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/4130539992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 0x5636b071fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.830+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/4130539992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b0714ac0 0x5636b071d160 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.830+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/4130539992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b0714580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.830+0000 7fbf46f8d740 1 -- 172.21.15.22:0/4130539992 >> 172.21.15.22:0/4130539992 conn(0x5636b070e8d0 msgr2=0x5636b070ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.831+0000 7fbf46f8d740 1 -- 172.21.15.22:0/4130539992 shutdown_connections 2024-04-20T21:39:28.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.831+0000 7fbf46f8d740 1 -- 172.21.15.22:0/4130539992 wait complete. 2024-04-20T21:39:28.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.831+0000 7fbf42200640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbf240a3d20 0x7fbf240a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf34011950 tx=0x7fbf340096a0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.831+0000 7fbf46f8d740 1 -- start start 2024-04-20T21:39:28.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.832+0000 7fbf46f8d740 1 Processor -- start 2024-04-20T21:39:28.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.832+0000 7fbf46f8d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b071a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.832+0000 7fbf46f8d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b0714ac0 0x5636b071ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.832+0000 7fbf46f8d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 0x5636b071b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.832+0000 7fbf46f8d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5636b0720430 con 0x5636b071d6a0 2024-04-20T21:39:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.832+0000 7fbf46f8d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636b0633740 con 0x5636b0714ac0 2024-04-20T21:39:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.832+0000 7fbf46f8d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5636b07205b0 con 0x5636b07141a0 2024-04-20T21:39:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.832+0000 7fbf46f8d740 10 client.0 _ll_register_callbacks cb 0x5636b062eb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.832+0000 7fbf41800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b071a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.832+0000 7fbf41800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b071a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46572/0 (socket says 172.21.15.22:46572) 2024-04-20T21:39:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.833+0000 7fbf41800640 1 -- 172.21.15.22:0/2111876919 learned_addr learned my addr 172.21.15.22:0/2111876919 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.833+0000 7fbf40e00640 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b0714ac0 0x5636b071ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.833+0000 7fbf40e00640 1 -- 172.21.15.22:0/2111876919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 msgr2=0x5636b071a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.833+0000 7fbf40e00640 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b071a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.833+0000 7fbf40e00640 1 -- 172.21.15.22:0/2111876919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 msgr2=0x5636b071b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.833+0000 7fbf40e00640 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 0x5636b071b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.833+0000 7fbf40e00640 1 -- 172.21.15.22:0/2111876919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b038c3f0 con 0x5636b0714ac0 2024-04-20T21:39:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.833+0000 7fbf41800640 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b071a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.833+0000 7fbf40e00640 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b0714ac0 0x5636b071ac30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbf280ce280 tx=0x7fbf28098ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=755c8046c2632a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.833+0000 7fbf23400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:39:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.834+0000 7fbf23400640 1 -- 172.21.15.22:0/2111876919 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf280ce700 con 0x5636b0714ac0 2024-04-20T21:39:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.834+0000 7fbf23400640 1 -- 172.21.15.22:0/2111876919 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fbf2809bfd0 con 0x5636b0714ac0 2024-04-20T21:39:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.834+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2111876919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf30004240 con 0x5636b0714ac0 2024-04-20T21:39:28.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.834+0000 7fbf23400640 1 -- 172.21.15.22:0/2111876919 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbf2809cb80 con 0x5636b0714ac0 2024-04-20T21:39:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.834+0000 7fbf46f8d740 10 client.26234 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.834+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2111876919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5636b034c0f0 con 0x5636b0714ac0 2024-04-20T21:39:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.834+0000 7fbf23400640 1 -- 172.21.15.22:0/2111876919 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf30004240 con 0x5636b0714ac0 2024-04-20T21:39:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.834+0000 7fbf23400640 1 -- 172.21.15.22:0/2111876919 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fbf2809ba40 con 0x5636b0714ac0 2024-04-20T21:39:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.834+0000 7fbf46f8d740 10 client.26234 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.834+0000 7fbf46f8d740 10 client.26234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.834+0000 7fbf46f8d740 20 client.26234 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.834+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5636b072d9d0 0x5636b072fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.835+0000 7fbf46f8d740 4 client.26234 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.835+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2111876919 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636b0687b70 con 0x5636b072d9d0 2024-04-20T21:39:28.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.835+0000 7fbf41800640 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5636b072d9d0 0x5636b072fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.835+0000 7fbf41800640 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5636b072d9d0 0x5636b072fe40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed9c6d863b470099 server_cookie=a2a066a316e5c81f in_seq=0 out_seq=0 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.835+0000 7fbf23400640 10 client.26234 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.836+0000 7fbf23400640 1 -- 172.21.15.22:0/2111876919 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636b0687b70 con 0x5636b072d9d0 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.836+0000 7fbf23400640 10 client.26234 handle_command_reply: tid=0 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.836+0000 7fbf46f8d740 1 client.26234 shutdown 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.836+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2111876919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b0714ac0 msgr2=0x5636b071ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.836+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b0714ac0 0x5636b071ac30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbf280ce280 tx=0x7fbf28098ee0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.837+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2111876919 shutdown_connections 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.837+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 0x5636b071b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.837+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5636b072d9d0 0x5636b072fe40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.837+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b0714ac0 0x5636b071ac30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.837+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2111876919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b071a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.837+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2111876919 >> 172.21.15.22:0/2111876919 conn(0x5636b070e8d0 msgr2=0x5636b0710620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.837+0000 7fbf23400640 0 client.26234 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:28.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.837+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2111876919 shutdown_connections 2024-04-20T21:39:28.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.837+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2111876919 wait complete. 2024-04-20T21:39:28.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.837+0000 7fbf46f8d740 20 client.26234 trim_cache size 0 max 16384 2024-04-20T21:39:28.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf46f8d740 1 Processor -- start 2024-04-20T21:39:28.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf46f8d740 1 -- start start 2024-04-20T21:39:28.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf46f8d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b071ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf46f8d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b0714ac0 0x5636b07292f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf46f8d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b071d6a0 0x5636b07196d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf46f8d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1adf00 con 0x5636b0714ac0 2024-04-20T21:39:28.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf46f8d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1b0780 con 0x5636b071d6a0 2024-04-20T21:39:28.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf46f8d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1ae570 con 0x5636b07141a0 2024-04-20T21:39:28.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf41800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b071ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf40e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b0714ac0 0x5636b07292f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf41800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b071ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:46578/0 (socket says 172.21.15.22:46578) 2024-04-20T21:39:28.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf41800640 1 -- 172.21.15.22:0/2804696218 learned_addr learned my addr 172.21.15.22:0/2804696218 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:28.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.857+0000 7fbf42200640 1 --2- 172.21.15.22:0/2804696218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b071d6a0 0x5636b07196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.858+0000 7fbf41800640 1 -- 172.21.15.22:0/2804696218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b071d6a0 msgr2=0x5636b07196d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.858+0000 7fbf41800640 1 --2- 172.21.15.22:0/2804696218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b071d6a0 0x5636b07196d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.858+0000 7fbf41800640 1 -- 172.21.15.22:0/2804696218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b0714ac0 msgr2=0x5636b07292f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.858+0000 7fbf41800640 1 --2- 172.21.15.22:0/2804696218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b0714ac0 0x5636b07292f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.858+0000 7fbf41800640 1 -- 172.21.15.22:0/2804696218 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b070e400 con 0x5636b07141a0 2024-04-20T21:39:28.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.858+0000 7fbf40e00640 1 --2- 172.21.15.22:0/2804696218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b0714ac0 0x5636b07292f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:28.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.858+0000 7fbf41800640 1 --2- 172.21.15.22:0/2804696218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b071ce20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbf30036040 tx=0x7fbf30000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eca5168ee24a496 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.858+0000 7fbf2e000640 1 -- 172.21.15.22:0/2804696218 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30042040 con 0x5636b07141a0 2024-04-20T21:39:28.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.858+0000 7fbf2e000640 1 -- 172.21.15.22:0/2804696218 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf30004800 con 0x5636b07141a0 2024-04-20T21:39:28.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.858+0000 7fbf2e000640 1 -- 172.21.15.22:0/2804696218 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30005590 con 0x5636b07141a0 2024-04-20T21:39:28.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.859+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2804696218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 msgr2=0x5636b071ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.859+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2804696218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b071ce20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbf30036040 tx=0x7fbf30000bc0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.859+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2804696218 shutdown_connections 2024-04-20T21:39:28.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.859+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2804696218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b071d6a0 0x5636b07196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.859+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2804696218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b0714ac0 0x5636b07292f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.859+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2804696218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b07141a0 0x5636b071ce20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.859+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2804696218 >> 172.21.15.22:0/2804696218 conn(0x5636b070e8d0 msgr2=0x5636b0718fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.859+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2804696218 shutdown_connections 2024-04-20T21:39:28.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.859+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2804696218 wait complete. 2024-04-20T21:39:28.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.860+0000 7fbf46f8d740 1 -- start start 2024-04-20T21:39:28.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.860+0000 7fbf46f8d740 1 Processor -- start 2024-04-20T21:39:28.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.860+0000 7fbf46f8d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b07141a0 0x5636b0725cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.860+0000 7fbf46f8d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b0714ac0 0x5636b0717cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.860+0000 7fbf46f8d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 0x5636b07182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.860+0000 7fbf46f8d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1aec40 con 0x5636b071d6a0 2024-04-20T21:39:28.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.860+0000 7fbf46f8d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1b00b0 con 0x5636b07141a0 2024-04-20T21:39:28.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.860+0000 7fbf46f8d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1af310 con 0x5636b0714ac0 2024-04-20T21:39:28.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf46f8d740 10 client.0 _ll_register_callbacks cb 0x7fbf280d4350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:28.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf41800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b07141a0 0x5636b0725cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf41800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b07141a0 0x5636b0725cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:48404/0 (socket says 172.21.15.22:48404) 2024-04-20T21:39:28.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf41800640 1 -- 172.21.15.22:0/2834556708 learned_addr learned my addr 172.21.15.22:0/2834556708 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:28.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf40e00640 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b0714ac0 0x5636b0717cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf42200640 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 0x5636b07182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf40e00640 1 -- 172.21.15.22:0/2834556708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b07141a0 msgr2=0x5636b0725cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf40e00640 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b07141a0 0x5636b0725cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf40e00640 1 -- 172.21.15.22:0/2834556708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 msgr2=0x5636b07182d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:28.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf40e00640 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 0x5636b07182d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf40e00640 1 -- 172.21.15.22:0/2834556708 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b0716b10 con 0x5636b0714ac0 2024-04-20T21:39:28.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf41800640 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b07141a0 0x5636b0725cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:28.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf40e00640 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b0714ac0 0x5636b0717cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c6a0 tx=0x7fbf28009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aee556a3e9a8d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:28.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf23400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:39:28.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf23400640 1 -- 172.21.15.22:0/2834556708 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2809c6c0 con 0x5636b0714ac0 2024-04-20T21:39:28.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf23400640 1 -- 172.21.15.22:0/2834556708 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fbf2809bfd0 con 0x5636b0714ac0 2024-04-20T21:39:28.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.861+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2834556708 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf30004800 con 0x5636b0714ac0 2024-04-20T21:39:28.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.862+0000 7fbf23400640 1 -- 172.21.15.22:0/2834556708 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbf280a0e80 con 0x5636b0714ac0 2024-04-20T21:39:28.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.862+0000 7fbf46f8d740 10 client.26263 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:28.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.862+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2834556708 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5636b07933f0 con 0x5636b0714ac0 2024-04-20T21:39:28.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.862+0000 7fbf23400640 1 -- 172.21.15.22:0/2834556708 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf30004800 con 0x5636b0714ac0 2024-04-20T21:39:28.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.862+0000 7fbf23400640 1 -- 172.21.15.22:0/2834556708 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fbf28018a20 con 0x5636b0714ac0 2024-04-20T21:39:28.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.862+0000 7fbf46f8d740 10 client.26263 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:28.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.862+0000 7fbf46f8d740 10 client.26263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:28.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.862+0000 7fbf46f8d740 20 client.26263 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:28.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.862+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5636b0793b70 0x5636b0795f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:28.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.862+0000 7fbf46f8d740 4 client.26263 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:39:28.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.862+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2834556708 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636b06819e0 con 0x5636b0793b70 2024-04-20T21:39:28.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.863+0000 7fbf41800640 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5636b0793b70 0x5636b0795f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:28.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.863+0000 7fbf41800640 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5636b0793b70 0x5636b0795f90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e0033c76692997 server_cookie=1b31e17a62cd3a75 in_seq=0 out_seq=0 2024-04-20T21:39:28.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.863+0000 7fbf23400640 10 client.26263 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:28.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.864+0000 7fbf23400640 1 -- 172.21.15.22:0/2834556708 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5636b06819e0 con 0x5636b0793b70 2024-04-20T21:39:28.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.864+0000 7fbf23400640 10 client.26263 handle_command_reply: tid=0 2024-04-20T21:39:28.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.864+0000 7fbf46f8d740 1 client.26263 shutdown 2024-04-20T21:39:28.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.864+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2834556708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b0714ac0 msgr2=0x5636b0717cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.864+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b0714ac0 0x5636b0717cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c6a0 tx=0x7fbf28009700 comp rx=0 tx=0).stop 2024-04-20T21:39:28.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.864+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2834556708 shutdown_connections 2024-04-20T21:39:28.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.864+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636b071d6a0 0x5636b07182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.864+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636b0714ac0 0x5636b0717cc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.864+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5636b0793b70 0x5636b0795f90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.864+0000 7fbf46f8d740 1 --2- 172.21.15.22:0/2834556708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b07141a0 0x5636b0725cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.864+0000 7fbf23400640 0 client.26263 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:28.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.864+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2834556708 >> 172.21.15.22:0/2834556708 conn(0x5636b070e8d0 msgr2=0x5636b071a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.865+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2834556708 shutdown_connections 2024-04-20T21:39:28.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.865+0000 7fbf46f8d740 1 -- 172.21.15.22:0/2834556708 wait complete. 2024-04-20T21:39:28.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.865+0000 7fbf46f8d740 20 client.26263 trim_cache size 0 max 16384 2024-04-20T21:39:28.870 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:39:28.870 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:39:28.870 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:39:28.870 INFO:teuthology.orchestra.run.smithi022.stdout: "bdcfc32a-41f6-41e2-af9c-9baa03996104": { 2024-04-20T21:39:28.870 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:39:28.871 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "bdcfc32a-41f6-41e2-af9c-9baa03996104", 2024-04-20T21:39:28.871 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:39:28.871 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:39:28.871 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:39:28.871 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:39:28.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.865+0000 7fbf44200640 1 -- 172.21.15.22:0/644250088 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbf240a3d20 msgr2=0x7fbf240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.865+0000 7fbf44200640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbf240a3d20 0x7fbf240a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf34011950 tx=0x7fbf340096a0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.865+0000 7fbf44200640 1 -- 172.21.15.22:0/644250088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf3c107270 msgr2=0x7fbf3c1a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:28.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.865+0000 7fbf44200640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf3c107270 0x7fbf3c1a0c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800e9c0 tx=0x7fbf2800ee80 comp rx=0 tx=0).stop 2024-04-20T21:39:28.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.865+0000 7fbf44200640 1 -- 172.21.15.22:0/644250088 shutdown_connections 2024-04-20T21:39:28.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.865+0000 7fbf44200640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbf240a3d20 0x7fbf240a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.866+0000 7fbf44200640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf3c19a8b0 0x7fbf3c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.866+0000 7fbf44200640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf3c107270 0x7fbf3c1a0c40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.866+0000 7fbf44200640 1 --2- 172.21.15.22:0/644250088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c1021a0 0x7fbf3c1a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:28.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.866+0000 7fbf44200640 1 -- 172.21.15.22:0/644250088 >> 172.21.15.22:0/644250088 conn(0x7fbf3c0793b0 msgr2=0x7fbf3c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:28.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.866+0000 7fbf44200640 1 -- 172.21.15.22:0/644250088 shutdown_connections 2024-04-20T21:39:28.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:28.866+0000 7fbf44200640 1 -- 172.21.15.22:0/644250088 wait complete. 2024-04-20T21:39:28.882 INFO:tasks.cephfs.filesystem:scrub status for tag:bdcfc32a-41f6-41e2-af9c-9baa03996104 - {'path': '/', 'tag': 'bdcfc32a-41f6-41e2-af9c-9baa03996104', 'options': 'recursive,force'} 2024-04-20T21:39:29.192 INFO:tasks.workunit.client.0.smithi022.stdout: 1 442298 9.66 MB/sec execute 117 sec latency 52.374 ms 2024-04-20T21:39:29.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:29 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:39:29.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:29 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:29.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:29 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:39:29.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:29 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:29.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:29 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:39:29.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:29 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:29.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:29 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:39:29.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:29 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:29.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:29 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:39:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:29 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:29 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:39:29.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:29 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:30.192 INFO:tasks.workunit.client.0.smithi022.stdout: 1 443952 9.68 MB/sec execute 118 sec latency 10.106 ms 2024-04-20T21:39:30.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:30 smithi122 ceph-mon[22147]: pgmap v173: 97 pgs: 97 active+clean; 251 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 164 op/s 2024-04-20T21:39:30.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:30 smithi033 ceph-mon[21626]: pgmap v173: 97 pgs: 97 active+clean; 251 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 164 op/s 2024-04-20T21:39:30.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:30 smithi022 ceph-mon[16843]: pgmap v173: 97 pgs: 97 active+clean; 251 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 164 op/s 2024-04-20T21:39:31.192 INFO:tasks.workunit.client.0.smithi022.stdout: 1 445890 9.66 MB/sec execute 119 sec latency 16.488 ms 2024-04-20T21:39:31.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:31 smithi122 ceph-mon[22147]: pgmap v174: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 244 op/s 2024-04-20T21:39:31.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:31 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:39:31.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:31 smithi033 ceph-mon[21626]: pgmap v174: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 244 op/s 2024-04-20T21:39:31.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:31 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:39:31.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:31 smithi022 ceph-mon[16843]: pgmap v174: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 244 op/s 2024-04-20T21:39:31.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:31 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:39:32.193 INFO:tasks.workunit.client.0.smithi022.stdout: 1 447484 9.68 MB/sec execute 120 sec latency 20.730 ms 2024-04-20T21:39:33.193 INFO:tasks.workunit.client.0.smithi022.stdout: 1 449312 9.65 MB/sec execute 121 sec latency 33.449 ms 2024-04-20T21:39:34.193 INFO:tasks.workunit.client.0.smithi022.stdout: 1 450898 9.67 MB/sec execute 122 sec latency 26.629 ms 2024-04-20T21:39:34.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:34 smithi033 ceph-mon[21626]: pgmap v175: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 2.7 MiB/s wr, 161 op/s 2024-04-20T21:39:34.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:34 smithi022 ceph-mon[16843]: pgmap v175: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 2.7 MiB/s wr, 161 op/s 2024-04-20T21:39:34.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:34 smithi122 ceph-mon[22147]: pgmap v175: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 2.7 MiB/s wr, 161 op/s 2024-04-20T21:39:35.193 INFO:tasks.workunit.client.0.smithi022.stdout: 1 452199 9.64 MB/sec execute 123 sec latency 24.303 ms 2024-04-20T21:39:36.193 INFO:tasks.workunit.client.0.smithi022.stdout: 1 454237 9.67 MB/sec execute 124 sec latency 18.256 ms 2024-04-20T21:39:36.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:36 smithi022 ceph-mon[16843]: pgmap v176: 97 pgs: 97 active+clean; 263 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 230 op/s 2024-04-20T21:39:36.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:36 smithi122 ceph-mon[22147]: pgmap v176: 97 pgs: 97 active+clean; 263 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 230 op/s 2024-04-20T21:39:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:36 smithi033 ceph-mon[21626]: pgmap v176: 97 pgs: 97 active+clean; 263 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 230 op/s 2024-04-20T21:39:37.193 INFO:tasks.workunit.client.0.smithi022.stdout: 1 455462 9.64 MB/sec execute 125 sec latency 18.532 ms 2024-04-20T21:39:38.193 INFO:tasks.workunit.client.0.smithi022.stdout: 1 457010 9.62 MB/sec execute 126 sec latency 75.087 ms 2024-04-20T21:39:38.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:38 smithi022 ceph-mon[16843]: pgmap v177: 97 pgs: 97 active+clean; 263 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-20T21:39:38.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:38 smithi122 ceph-mon[22147]: pgmap v177: 97 pgs: 97 active+clean; 263 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-20T21:39:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:38 smithi033 ceph-mon[21626]: pgmap v177: 97 pgs: 97 active+clean; 263 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-20T21:39:39.193 INFO:tasks.workunit.client.0.smithi022.stdout: 1 458307 9.60 MB/sec execute 127 sec latency 42.102 ms 2024-04-20T21:39:40.194 INFO:tasks.workunit.client.0.smithi022.stdout: 1 459159 9.69 MB/sec execute 128 sec latency 70.707 ms 2024-04-20T21:39:40.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:40 smithi022 ceph-mon[16843]: pgmap v178: 97 pgs: 97 active+clean; 263 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-20T21:39:40.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:40 smithi122 ceph-mon[22147]: pgmap v178: 97 pgs: 97 active+clean; 263 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-20T21:39:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:40 smithi033 ceph-mon[21626]: pgmap v178: 97 pgs: 97 active+clean; 263 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-20T21:39:41.193 INFO:tasks.workunit.client.0.smithi022.stdout: 1 460796 9.67 MB/sec execute 129 sec latency 54.065 ms 2024-04-20T21:39:42.194 INFO:tasks.workunit.client.0.smithi022.stdout: 1 462626 9.69 MB/sec execute 130 sec latency 16.993 ms 2024-04-20T21:39:42.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:42 smithi022 ceph-mon[16843]: pgmap v179: 97 pgs: 97 active+clean; 268 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-04-20T21:39:42.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:42 smithi122 ceph-mon[22147]: pgmap v179: 97 pgs: 97 active+clean; 268 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-04-20T21:39:42.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:42 smithi033 ceph-mon[21626]: pgmap v179: 97 pgs: 97 active+clean; 268 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-04-20T21:39:43.194 INFO:tasks.workunit.client.0.smithi022.stdout: 1 464311 9.66 MB/sec execute 131 sec latency 63.153 ms 2024-04-20T21:39:44.194 INFO:tasks.workunit.client.0.smithi022.stdout: 1 466045 9.68 MB/sec execute 132 sec latency 17.819 ms 2024-04-20T21:39:44.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:44 smithi122 ceph-mon[22147]: pgmap v180: 97 pgs: 97 active+clean; 268 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 2.1 MiB/s wr, 130 op/s 2024-04-20T21:39:44.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:44 smithi022 ceph-mon[16843]: pgmap v180: 97 pgs: 97 active+clean; 268 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 2.1 MiB/s wr, 130 op/s 2024-04-20T21:39:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:44 smithi033 ceph-mon[21626]: pgmap v180: 97 pgs: 97 active+clean; 268 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 2.1 MiB/s wr, 130 op/s 2024-04-20T21:39:45.194 INFO:tasks.workunit.client.0.smithi022.stdout: 1 467364 9.65 MB/sec execute 133 sec latency 21.632 ms 2024-04-20T21:39:46.194 INFO:tasks.workunit.client.0.smithi022.stdout: 1 469235 9.68 MB/sec execute 134 sec latency 17.146 ms 2024-04-20T21:39:46.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:46 smithi022 ceph-mon[16843]: pgmap v181: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 212 op/s 2024-04-20T21:39:46.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:46 smithi122 ceph-mon[22147]: pgmap v181: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 212 op/s 2024-04-20T21:39:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:46 smithi033 ceph-mon[21626]: pgmap v181: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 212 op/s 2024-04-20T21:39:47.194 INFO:tasks.workunit.client.0.smithi022.stdout: 1 470445 9.65 MB/sec execute 135 sec latency 25.165 ms 2024-04-20T21:39:48.194 INFO:tasks.workunit.client.0.smithi022.stdout: 1 472001 9.63 MB/sec execute 136 sec latency 45.084 ms 2024-04-20T21:39:48.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:48 smithi022 ceph-mon[16843]: pgmap v182: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 2.7 MiB/s wr, 142 op/s 2024-04-20T21:39:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:48 smithi122 ceph-mon[22147]: pgmap v182: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 2.7 MiB/s wr, 142 op/s 2024-04-20T21:39:48.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:48 smithi033 ceph-mon[21626]: pgmap v182: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 2.7 MiB/s wr, 142 op/s 2024-04-20T21:39:49.195 INFO:tasks.workunit.client.0.smithi022.stdout: 1 473366 9.61 MB/sec execute 137 sec latency 32.479 ms 2024-04-20T21:39:50.195 INFO:tasks.workunit.client.0.smithi022.stdout: 1 474993 9.59 MB/sec execute 138 sec latency 28.867 ms 2024-04-20T21:39:50.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:50 smithi022 ceph-mon[16843]: pgmap v183: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 2.7 MiB/s wr, 142 op/s 2024-04-20T21:39:50.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:50 smithi122 ceph-mon[22147]: pgmap v183: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 2.7 MiB/s wr, 142 op/s 2024-04-20T21:39:50.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:50 smithi033 ceph-mon[21626]: pgmap v183: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 2.7 MiB/s wr, 142 op/s 2024-04-20T21:39:51.195 INFO:tasks.workunit.client.0.smithi022.stdout: 1 476555 9.61 MB/sec execute 139 sec latency 11.948 ms 2024-04-20T21:39:52.195 INFO:tasks.workunit.client.0.smithi022.stdout: 1 477853 9.58 MB/sec execute 140 sec latency 24.445 ms 2024-04-20T21:39:52.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:52 smithi022 ceph-mon[16843]: pgmap v184: 97 pgs: 97 active+clean; 277 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-04-20T21:39:52.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:52 smithi122 ceph-mon[22147]: pgmap v184: 97 pgs: 97 active+clean; 277 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-04-20T21:39:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:52 smithi033 ceph-mon[21626]: pgmap v184: 97 pgs: 97 active+clean; 277 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-04-20T21:39:53.195 INFO:tasks.workunit.client.0.smithi022.stdout: 1 479914 9.60 MB/sec execute 141 sec latency 16.437 ms 2024-04-20T21:39:54.195 INFO:tasks.workunit.client.0.smithi022.stdout: 1 481106 9.57 MB/sec execute 142 sec latency 33.770 ms 2024-04-20T21:39:54.354 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:54 smithi033 ceph-mon[21626]: pgmap v185: 97 pgs: 97 active+clean; 277 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-04-20T21:39:54.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:54 smithi022 ceph-mon[16843]: pgmap v185: 97 pgs: 97 active+clean; 277 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-04-20T21:39:54.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:54 smithi122 ceph-mon[22147]: pgmap v185: 97 pgs: 97 active+clean; 277 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 2.6 MiB/s wr, 150 op/s 2024-04-20T21:39:55.195 INFO:tasks.workunit.client.0.smithi022.stdout: 1 483026 9.57 MB/sec execute 143 sec latency 78.672 ms 2024-04-20T21:39:56.195 INFO:tasks.workunit.client.0.smithi022.stdout: 1 484389 9.55 MB/sec execute 144 sec latency 24.667 ms 2024-04-20T21:39:56.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:56 smithi022 ceph-mon[16843]: pgmap v186: 97 pgs: 97 active+clean; 281 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 219 op/s 2024-04-20T21:39:56.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:56 smithi122 ceph-mon[22147]: pgmap v186: 97 pgs: 97 active+clean; 281 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 219 op/s 2024-04-20T21:39:56.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:56 smithi033 ceph-mon[21626]: pgmap v186: 97 pgs: 97 active+clean; 281 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 219 op/s 2024-04-20T21:39:57.196 INFO:tasks.workunit.client.0.smithi022.stdout: 1 486412 9.57 MB/sec execute 145 sec latency 20.121 ms 2024-04-20T21:39:58.196 INFO:tasks.workunit.client.0.smithi022.stdout: 1 487874 9.55 MB/sec execute 146 sec latency 20.467 ms 2024-04-20T21:39:58.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:39:58 smithi022 ceph-mon[16843]: pgmap v187: 97 pgs: 97 active+clean; 281 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 2.3 MiB/s wr, 138 op/s 2024-04-20T21:39:58.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:39:58 smithi122 ceph-mon[22147]: pgmap v187: 97 pgs: 97 active+clean; 281 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 2.3 MiB/s wr, 138 op/s 2024-04-20T21:39:58.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:39:58 smithi033 ceph-mon[21626]: pgmap v187: 97 pgs: 97 active+clean; 281 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 2.3 MiB/s wr, 138 op/s 2024-04-20T21:39:58.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:39:58.884 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:39:59.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.115+0000 7f284ec00640 1 -- 172.21.15.22:0/1266743398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2848101550 msgr2=0x7f284810e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.115+0000 7f284ec00640 1 --2- 172.21.15.22:0/1266743398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2848101550 0x7f284810e210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2834009900 tx=0x7f283402bc40 comp rx=0 tx=0).stop 2024-04-20T21:39:59.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.115+0000 7f284ec00640 1 -- 172.21.15.22:0/1266743398 shutdown_connections 2024-04-20T21:39:59.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.115+0000 7f284ec00640 1 --2- 172.21.15.22:0/1266743398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284810e750 0x7f2848110b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.115+0000 7f284ec00640 1 --2- 172.21.15.22:0/1266743398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2848101550 0x7f284810e210 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.115+0000 7f284ec00640 1 --2- 172.21.15.22:0/1266743398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2848100ba0 0x7f2848100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.115+0000 7f284ec00640 1 -- 172.21.15.22:0/1266743398 >> 172.21.15.22:0/1266743398 conn(0x7f28480fc7e0 msgr2=0x7f28480fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:59.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.115+0000 7f284ec00640 1 -- 172.21.15.22:0/1266743398 shutdown_connections 2024-04-20T21:39:59.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.115+0000 7f284ec00640 1 -- 172.21.15.22:0/1266743398 wait complete. 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.116+0000 7f284ec00640 1 Processor -- start 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.116+0000 7f284ec00640 1 -- start start 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.117+0000 7f284ec00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2848100ba0 0x7f28481a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.117+0000 7f284ec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2848101550 0x7f28481a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.117+0000 7f284ec00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284810e750 0x7f284819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.117+0000 7f284ec00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2848114440 con 0x7f2848100ba0 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.117+0000 7f284ec00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f28481142c0 con 0x7f284810e750 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.117+0000 7f284ec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f28481145c0 con 0x7f2848101550 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.117+0000 7f284d800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2848100ba0 0x7f28481a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.117+0000 7f284d800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2848100ba0 0x7f28481a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53974/0 (socket says 172.21.15.22:53974) 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.117+0000 7f284d800640 1 -- 172.21.15.22:0/3974110222 learned_addr learned my addr 172.21.15.22:0/3974110222 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.117+0000 7f284ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2848101550 0x7f28481a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.117+0000 7f284e200640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284810e750 0x7f284819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.118+0000 7f284d800640 1 -- 172.21.15.22:0/3974110222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2848101550 msgr2=0x7f28481a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.118+0000 7f284d800640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2848101550 0x7f28481a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.118+0000 7f284d800640 1 -- 172.21.15.22:0/3974110222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284810e750 msgr2=0x7f284819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.118+0000 7f284d800640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284810e750 0x7f284819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.118+0000 7f284d800640 1 -- 172.21.15.22:0/3974110222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284819afa0 con 0x7f2848100ba0 2024-04-20T21:39:59.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.118+0000 7f284e200640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284810e750 0x7f284819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:59.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.118+0000 7f284d800640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2848100ba0 0x7f28481a0620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f283800ebc0 tx=0x7f283800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:59.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.118+0000 7f2846000640 1 -- 172.21.15.22:0/3974110222 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838010630 con 0x7f2848100ba0 2024-04-20T21:39:59.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.118+0000 7f284ec00640 1 -- 172.21.15.22:0/3974110222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284819b290 con 0x7f2848100ba0 2024-04-20T21:39:59.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.118+0000 7f2846000640 1 -- 172.21.15.22:0/3974110222 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2838010cb0 con 0x7f2848100ba0 2024-04-20T21:39:59.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.118+0000 7f2846000640 1 -- 172.21.15.22:0/3974110222 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838018bd0 con 0x7f2848100ba0 2024-04-20T21:39:59.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.118+0000 7f284ec00640 1 -- 172.21.15.22:0/3974110222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28481a75a0 con 0x7f2848100ba0 2024-04-20T21:39:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.123+0000 7f2852fef740 1 Processor -- start 2024-04-20T21:39:59.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.123+0000 7f2852fef740 1 -- start start 2024-04-20T21:39:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.123+0000 7f2852fef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 0x55eaf3696760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.123+0000 7f2852fef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3696ca0 0x55eaf369f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.123+0000 7f2852fef740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf369f880 0x55eaf36a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.123+0000 7f2852fef740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55eaf35fc150 con 0x55eaf369f880 2024-04-20T21:39:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.123+0000 7f2852fef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eaf35a2370 con 0x55eaf3696380 2024-04-20T21:39:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.123+0000 7f2852fef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eaf35f8110 con 0x55eaf3696ca0 2024-04-20T21:39:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.124+0000 7f284e200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 0x55eaf3696760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.124+0000 7f284e200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 0x55eaf3696760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39008/0 (socket says 172.21.15.22:39008) 2024-04-20T21:39:59.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.124+0000 7f284e200640 1 -- 172.21.15.22:0/3994259311 learned_addr learned my addr 172.21.15.22:0/3994259311 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:59.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.124+0000 7f284ce00640 1 --2- 172.21.15.22:0/3994259311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf369f880 0x55eaf36a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.124+0000 7f2846000640 1 -- 172.21.15.22:0/3974110222 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f28380107d0 con 0x7f2848100ba0 2024-04-20T21:39:59.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.124+0000 7f284e200640 1 -- 172.21.15.22:0/3994259311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3696ca0 msgr2=0x55eaf369f340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:39:59.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.124+0000 7f284e200640 1 --2- 172.21.15.22:0/3994259311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3696ca0 0x55eaf369f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.124+0000 7f284e200640 1 -- 172.21.15.22:0/3994259311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf369f880 msgr2=0x55eaf36a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:59.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.124+0000 7f284e200640 1 --2- 172.21.15.22:0/3994259311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf369f880 0x55eaf36a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.124+0000 7f284e200640 1 -- 172.21.15.22:0/3994259311 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaf33d87d0 con 0x55eaf3696380 2024-04-20T21:39:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.125+0000 7f284e200640 1 --2- 172.21.15.22:0/3994259311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 0x55eaf3696760 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2840009be0 tx=0x7f28400117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54c3471aedd6bc59 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.125+0000 7f283ea00640 1 -- 172.21.15.22:0/3994259311 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840012450 con 0x55eaf3696380 2024-04-20T21:39:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.125+0000 7f283ea00640 1 -- 172.21.15.22:0/3994259311 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2840012ad0 con 0x55eaf3696380 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.125+0000 7f283ea00640 1 -- 172.21.15.22:0/3994259311 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284000fbf0 con 0x55eaf3696380 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.126+0000 7f2846000640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f282c0a3d20 0x7f282c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.126+0000 7f2852fef740 1 -- 172.21.15.22:0/3994259311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 msgr2=0x55eaf3696760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.126+0000 7f2852fef740 1 --2- 172.21.15.22:0/3994259311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 0x55eaf3696760 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2840009be0 tx=0x7f28400117b0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.126+0000 7f2846000640 1 -- 172.21.15.22:0/3974110222 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f28380d0ec0 con 0x7f2848100ba0 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.126+0000 7f2852fef740 1 -- 172.21.15.22:0/3994259311 shutdown_connections 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.126+0000 7f2852fef740 1 --2- 172.21.15.22:0/3994259311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf369f880 0x55eaf36a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.126+0000 7f2852fef740 1 --2- 172.21.15.22:0/3994259311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3696ca0 0x55eaf369f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.126+0000 7f2852fef740 1 --2- 172.21.15.22:0/3994259311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 0x55eaf3696760 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.126+0000 7f2852fef740 1 -- 172.21.15.22:0/3994259311 >> 172.21.15.22:0/3994259311 conn(0x55eaf3690ab0 msgr2=0x55eaf3690e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.126+0000 7f284d800640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f282c0a3d20 0x7f282c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.126+0000 7f2852fef740 1 -- 172.21.15.22:0/3994259311 shutdown_connections 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.126+0000 7f2852fef740 1 -- 172.21.15.22:0/3994259311 wait complete. 2024-04-20T21:39:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.127+0000 7f2852fef740 1 -- start start 2024-04-20T21:39:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.127+0000 7f284d800640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f282c0a3d20 0x7f282c0a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55eaf3695e50 tx=0x7f2838018670 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.127+0000 7f2852fef740 1 Processor -- start 2024-04-20T21:39:59.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.127+0000 7f2852fef740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696380 0x55eaf369c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.127+0000 7f2852fef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696ca0 0x55eaf369ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.127+0000 7f2852fef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf369f880 0x55eaf369d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.127+0000 7f2852fef740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55eaf36a2670 con 0x55eaf3696380 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.127+0000 7f2852fef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eaf35b5740 con 0x55eaf3696ca0 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.127+0000 7f2852fef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eaf36a27f0 con 0x55eaf369f880 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.127+0000 7f2852fef740 10 client.0 _ll_register_callbacks cb 0x55eaf35b0b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f284ce00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696ca0 0x55eaf369ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f284e200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696380 0x55eaf369c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f284ce00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696ca0 0x55eaf369ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39024/0 (socket says 172.21.15.22:39024) 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f284ce00640 1 -- 172.21.15.22:0/2854927405 learned_addr learned my addr 172.21.15.22:0/2854927405 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f284e200640 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf369f880 0x55eaf369d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f284ce00640 1 -- 172.21.15.22:0/2854927405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf369f880 msgr2=0x55eaf369d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f284ce00640 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf369f880 0x55eaf369d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f284ce00640 1 -- 172.21.15.22:0/2854927405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696380 msgr2=0x55eaf369c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f284ce00640 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696380 0x55eaf369c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f284ce00640 1 -- 172.21.15.22:0/2854927405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaf330e3f0 con 0x55eaf3696ca0 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f284e200640 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696380 0x55eaf369c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f284ce00640 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696ca0 0x55eaf369ce30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2834009900 tx=0x7f28340045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35632b792ded49e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.128+0000 7f2833400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.129+0000 7f2833400640 1 -- 172.21.15.22:0/2854927405 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283402cca0 con 0x55eaf3696ca0 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.129+0000 7f2833400640 1 -- 172.21.15.22:0/2854927405 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f28340046c0 con 0x55eaf3696ca0 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.129+0000 7f2852fef740 1 -- 172.21.15.22:0/2854927405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2840012ad0 con 0x55eaf3696ca0 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.129+0000 7f2833400640 1 -- 172.21.15.22:0/2854927405 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f28340048c0 con 0x55eaf3696ca0 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.129+0000 7f2852fef740 10 client.26252 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.129+0000 7f2852fef740 1 -- 172.21.15.22:0/2854927405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55eaf32ce0f0 con 0x55eaf3696ca0 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.129+0000 7f2833400640 1 -- 172.21.15.22:0/2854927405 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2840012ad0 con 0x55eaf3696ca0 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.130+0000 7f2833400640 1 -- 172.21.15.22:0/2854927405 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f2834046070 con 0x55eaf3696ca0 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.130+0000 7f2852fef740 10 client.26252 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.130+0000 7f2852fef740 10 client.26252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:59.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.130+0000 7f2852fef740 20 client.26252 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:59.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.130+0000 7f2852fef740 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55eaf36afba0 0x55eaf36b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.130+0000 7f2852fef740 4 client.26252 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:59.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.130+0000 7f2852fef740 1 -- 172.21.15.22:0/2854927405 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaf3609b70 con 0x55eaf36afba0 2024-04-20T21:39:59.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.130+0000 7f284e200640 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55eaf36afba0 0x55eaf36b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.131+0000 7f284e200640 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55eaf36afba0 0x55eaf36b2000 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4315da8e9329641a server_cookie=744034657560d554 in_seq=0 out_seq=0 2024-04-20T21:39:59.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.131+0000 7f2833400640 10 client.26252 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:59.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2833400640 1 -- 172.21.15.22:0/2854927405 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eaf3609b70 con 0x55eaf36afba0 2024-04-20T21:39:59.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2833400640 10 client.26252 handle_command_reply: tid=0 2024-04-20T21:39:59.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2852fef740 1 client.26252 shutdown 2024-04-20T21:39:59.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2852fef740 1 -- 172.21.15.22:0/2854927405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696ca0 msgr2=0x55eaf369ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2852fef740 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696ca0 0x55eaf369ce30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2834009900 tx=0x7f28340045f0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2852fef740 1 -- 172.21.15.22:0/2854927405 shutdown_connections 2024-04-20T21:39:59.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2852fef740 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf369f880 0x55eaf369d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2852fef740 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55eaf36afba0 0x55eaf36b2000 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2852fef740 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696ca0 0x55eaf369ce30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2852fef740 1 --2- 172.21.15.22:0/2854927405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696380 0x55eaf369c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2852fef740 1 -- 172.21.15.22:0/2854927405 >> 172.21.15.22:0/2854927405 conn(0x55eaf3690ab0 msgr2=0x55eaf3692800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:59.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2833400640 0 client.26252 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:59.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2852fef740 1 -- 172.21.15.22:0/2854927405 shutdown_connections 2024-04-20T21:39:59.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2852fef740 1 -- 172.21.15.22:0/2854927405 wait complete. 2024-04-20T21:39:59.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.132+0000 7f2852fef740 20 client.26252 trim_cache size 0 max 16384 2024-04-20T21:39:59.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.152+0000 7f2852fef740 1 Processor -- start 2024-04-20T21:39:59.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.152+0000 7f2852fef740 1 -- start start 2024-04-20T21:39:59.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.152+0000 7f2852fef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3690ab0 0x55eaf369f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.152+0000 7f2852fef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 0x55eaf36ab530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.152+0000 7f2852fef740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696ca0 0x55eaf369d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.152+0000 7f2852fef740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28481add70 con 0x55eaf3696ca0 2024-04-20T21:39:59.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.152+0000 7f2852fef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f28481b05f0 con 0x55eaf3696380 2024-04-20T21:39:59.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.152+0000 7f2852fef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f28481ae3e0 con 0x55eaf3690ab0 2024-04-20T21:39:59.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.153+0000 7f284e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3690ab0 0x55eaf369f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.153+0000 7f284e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3690ab0 0x55eaf369f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42010/0 (socket says 172.21.15.22:42010) 2024-04-20T21:39:59.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.153+0000 7f284e200640 1 -- 172.21.15.22:0/2549588096 learned_addr learned my addr 172.21.15.22:0/2549588096 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:59.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.153+0000 7f284ce00640 1 --2- 172.21.15.22:0/2549588096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 0x55eaf36ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.153+0000 7f284e200640 1 --2- 172.21.15.22:0/2549588096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696ca0 0x55eaf369d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.153+0000 7f284ce00640 1 -- 172.21.15.22:0/2549588096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3690ab0 msgr2=0x55eaf369f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:59.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.153+0000 7f284ce00640 1 --2- 172.21.15.22:0/2549588096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3690ab0 0x55eaf369f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.153+0000 7f284ce00640 1 -- 172.21.15.22:0/2549588096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696ca0 msgr2=0x55eaf369d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:59.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.153+0000 7f284ce00640 1 --2- 172.21.15.22:0/2549588096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696ca0 0x55eaf369d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.153+0000 7f284ce00640 1 -- 172.21.15.22:0/2549588096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaf36a1fe0 con 0x55eaf3696380 2024-04-20T21:39:59.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.153+0000 7f284ce00640 1 --2- 172.21.15.22:0/2549588096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 0x55eaf36ab530 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2834001eb0 tx=0x7f283402bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d39358900eecb323 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:59.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.154+0000 7f283e000640 1 -- 172.21.15.22:0/2549588096 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834000e60 con 0x55eaf3696380 2024-04-20T21:39:59.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.154+0000 7f283e000640 1 -- 172.21.15.22:0/2549588096 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2834003d60 con 0x55eaf3696380 2024-04-20T21:39:59.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.154+0000 7f283e000640 1 -- 172.21.15.22:0/2549588096 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28340051e0 con 0x55eaf3696380 2024-04-20T21:39:59.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.154+0000 7f2852fef740 1 -- 172.21.15.22:0/2549588096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 msgr2=0x55eaf36ab530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.154+0000 7f2852fef740 1 --2- 172.21.15.22:0/2549588096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 0x55eaf36ab530 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2834001eb0 tx=0x7f283402bc40 comp rx=0 tx=0).stop 2024-04-20T21:39:59.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.154+0000 7f2852fef740 1 -- 172.21.15.22:0/2549588096 shutdown_connections 2024-04-20T21:39:59.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.154+0000 7f2852fef740 1 --2- 172.21.15.22:0/2549588096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696ca0 0x55eaf369d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.154+0000 7f2852fef740 1 --2- 172.21.15.22:0/2549588096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3696380 0x55eaf36ab530 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.154+0000 7f2852fef740 1 --2- 172.21.15.22:0/2549588096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3690ab0 0x55eaf369f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.154+0000 7f2852fef740 1 -- 172.21.15.22:0/2549588096 >> 172.21.15.22:0/2549588096 conn(0x55eaf3697470 msgr2=0x55eaf3697ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:59.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.155+0000 7f2852fef740 1 -- 172.21.15.22:0/2549588096 shutdown_connections 2024-04-20T21:39:59.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.155+0000 7f2852fef740 1 -- 172.21.15.22:0/2549588096 wait complete. 2024-04-20T21:39:59.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.155+0000 7f2852fef740 1 -- start start 2024-04-20T21:39:59.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.155+0000 7f2852fef740 1 Processor -- start 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f2852fef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3690ab0 0x55eaf3699830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f2852fef740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696380 0x55eaf3699d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f2852fef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3696ca0 0x55eaf369a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f2852fef740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28481aeab0 con 0x55eaf3696380 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f2852fef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f28481aff20 con 0x55eaf3690ab0 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f2852fef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f28481af180 con 0x55eaf3696ca0 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f2852fef740 10 client.0 _ll_register_callbacks cb 0x7f2834009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f284e200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3690ab0 0x55eaf3699830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f284e200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3690ab0 0x55eaf3699830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39048/0 (socket says 172.21.15.22:39048) 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f284e200640 1 -- 172.21.15.22:0/400823719 learned_addr learned my addr 172.21.15.22:0/400823719 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f284ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696380 0x55eaf3699d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f284e200640 1 --2- 172.21.15.22:0/400823719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3696ca0 0x55eaf369a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f284e200640 1 -- 172.21.15.22:0/400823719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3696ca0 msgr2=0x55eaf369a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f284e200640 1 --2- 172.21.15.22:0/400823719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3696ca0 0x55eaf369a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f284e200640 1 -- 172.21.15.22:0/400823719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696380 msgr2=0x55eaf3699d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f284e200640 1 --2- 172.21.15.22:0/400823719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696380 0x55eaf3699d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.156+0000 7f284e200640 1 -- 172.21.15.22:0/400823719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaf37153e0 con 0x55eaf3690ab0 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.157+0000 7f284e200640 1 --2- 172.21.15.22:0/400823719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3690ab0 0x55eaf3699830 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2840009bb0 tx=0x7f28400117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6751d174bbd6db66 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.157+0000 7f2833400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.157+0000 7f2833400640 1 -- 172.21.15.22:0/400823719 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840005040 con 0x55eaf3690ab0 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.157+0000 7f2833400640 1 -- 172.21.15.22:0/400823719 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f28400051e0 con 0x55eaf3690ab0 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.157+0000 7f2852fef740 1 -- 172.21.15.22:0/400823719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2834003d60 con 0x55eaf3690ab0 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.157+0000 7f2833400640 1 -- 172.21.15.22:0/400823719 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f284003e050 con 0x55eaf3690ab0 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.157+0000 7f2852fef740 10 client.26258 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.157+0000 7f2852fef740 1 -- 172.21.15.22:0/400823719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55eaf3716730 con 0x55eaf3690ab0 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.158+0000 7f2833400640 1 -- 172.21.15.22:0/400823719 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2834003d60 con 0x55eaf3690ab0 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.158+0000 7f2833400640 1 -- 172.21.15.22:0/400823719 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f2840026040 con 0x55eaf3690ab0 2024-04-20T21:39:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.158+0000 7f2852fef740 10 client.26258 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.158+0000 7f2852fef740 10 client.26258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.158+0000 7f2852fef740 20 client.26258 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.158+0000 7f2852fef740 1 --2- 172.21.15.22:0/400823719 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55eaf369f880 0x55eaf3718d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.158+0000 7f2852fef740 4 client.26258 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.158+0000 7f2852fef740 1 -- 172.21.15.22:0/400823719 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eaf36039e0 con 0x55eaf369f880 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.158+0000 7f284ce00640 1 --2- 172.21.15.22:0/400823719 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55eaf369f880 0x55eaf3718d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f284ce00640 1 --2- 172.21.15.22:0/400823719 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55eaf369f880 0x55eaf3718d20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3539b67a6b1028ce server_cookie=bbd144376068ec63 in_seq=0 out_seq=0 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2833400640 10 client.26258 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2833400640 1 -- 172.21.15.22:0/400823719 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55eaf36039e0 con 0x55eaf369f880 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2833400640 10 client.26258 handle_command_reply: tid=0 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2852fef740 1 client.26258 shutdown 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2852fef740 1 -- 172.21.15.22:0/400823719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3690ab0 msgr2=0x55eaf3699830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2852fef740 1 --2- 172.21.15.22:0/400823719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3690ab0 0x55eaf3699830 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2840009bb0 tx=0x7f28400117b0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2852fef740 1 -- 172.21.15.22:0/400823719 shutdown_connections 2024-04-20T21:39:59.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2852fef740 1 --2- 172.21.15.22:0/400823719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eaf3696ca0 0x55eaf369a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2852fef740 1 --2- 172.21.15.22:0/400823719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaf3696380 0x55eaf3699d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2852fef740 1 --2- 172.21.15.22:0/400823719 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55eaf369f880 0x55eaf3718d20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2852fef740 1 --2- 172.21.15.22:0/400823719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf3690ab0 0x55eaf3699830 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2833400640 0 client.26258 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:59.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2852fef740 1 -- 172.21.15.22:0/400823719 >> 172.21.15.22:0/400823719 conn(0x55eaf3697470 msgr2=0x55eaf369d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:59.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2852fef740 1 -- 172.21.15.22:0/400823719 shutdown_connections 2024-04-20T21:39:59.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2852fef740 1 -- 172.21.15.22:0/400823719 wait complete. 2024-04-20T21:39:59.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.159+0000 7f2852fef740 20 client.26258 trim_cache size 0 max 16384 2024-04-20T21:39:59.162 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:39:59.162 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:39:59.163 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:39:59.163 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:39:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.160+0000 7f284ec00640 1 -- 172.21.15.22:0/3974110222 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f282c0a3d20 msgr2=0x7f282c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.160+0000 7f284ec00640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f282c0a3d20 0x7f282c0a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55eaf3695e50 tx=0x7f2838018670 comp rx=0 tx=0).stop 2024-04-20T21:39:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.160+0000 7f284ec00640 1 -- 172.21.15.22:0/3974110222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2848100ba0 msgr2=0x7f28481a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.160+0000 7f284ec00640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2848100ba0 0x7f28481a0620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f283800ebc0 tx=0x7f283800c6a0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.160+0000 7f284ec00640 1 -- 172.21.15.22:0/3974110222 shutdown_connections 2024-04-20T21:39:59.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.160+0000 7f284ec00640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f282c0a3d20 0x7f282c0a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.160+0000 7f284ec00640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284810e750 0x7f284819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.160+0000 7f284ec00640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2848101550 0x7f28481a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.160+0000 7f284ec00640 1 --2- 172.21.15.22:0/3974110222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2848100ba0 0x7f28481a0620 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.160+0000 7f284ec00640 1 -- 172.21.15.22:0/3974110222 >> 172.21.15.22:0/3974110222 conn(0x7f28480fc7e0 msgr2=0x7f28480febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:59.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.160+0000 7f284ec00640 1 -- 172.21.15.22:0/3974110222 shutdown_connections 2024-04-20T21:39:59.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.160+0000 7f284ec00640 1 -- 172.21.15.22:0/3974110222 wait complete. 2024-04-20T21:39:59.176 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:39:59.176 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:59.196 INFO:tasks.workunit.client.0.smithi022.stdout: 1 489523 9.55 MB/sec execute 147 sec latency 161.864 ms 2024-04-20T21:39:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.389+0000 7fefcb000640 1 -- 172.21.15.22:0/1925004934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefc4102b60 msgr2=0x7fefc4106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.389+0000 7fefcb000640 1 --2- 172.21.15.22:0/1925004934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefc4102b60 0x7fefc4106ba0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009900 tx=0x7fefb002bc40 comp rx=0 tx=0).stop 2024-04-20T21:39:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.389+0000 7fefcb000640 1 -- 172.21.15.22:0/1925004934 shutdown_connections 2024-04-20T21:39:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.389+0000 7fefcb000640 1 --2- 172.21.15.22:0/1925004934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc4107280 0x7fefc410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.389+0000 7fefcb000640 1 --2- 172.21.15.22:0/1925004934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefc4102b60 0x7fefc4106ba0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.389+0000 7fefcb000640 1 --2- 172.21.15.22:0/1925004934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefc41021b0 0x7fefc4102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.389+0000 7fefcb000640 1 -- 172.21.15.22:0/1925004934 >> 172.21.15.22:0/1925004934 conn(0x7fefc4079360 msgr2=0x7fefc4079770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.389+0000 7fefcb000640 1 -- 172.21.15.22:0/1925004934 shutdown_connections 2024-04-20T21:39:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.389+0000 7fefcb000640 1 -- 172.21.15.22:0/1925004934 wait complete. 2024-04-20T21:39:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.390+0000 7fefcb000640 1 Processor -- start 2024-04-20T21:39:59.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.390+0000 7fefcb000640 1 -- start start 2024-04-20T21:39:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.391+0000 7fefcb000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc41021b0 0x7fefc41a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.391+0000 7fefcb000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefc4107280 0x7fefc41a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.391+0000 7fefcb000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefc419a920 0x7fefc419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.391+0000 7fefcb000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fefc40750f0 con 0x7fefc41021b0 2024-04-20T21:39:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.391+0000 7fefcb000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4074f70 con 0x7fefc4107280 2024-04-20T21:39:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.391+0000 7fefcb000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4075270 con 0x7fefc419a920 2024-04-20T21:39:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.391+0000 7fefc3400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefc4107280 0x7fefc41a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.391+0000 7fefc3400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefc4107280 0x7fefc41a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39072/0 (socket says 172.21.15.22:39072) 2024-04-20T21:39:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.391+0000 7fefc3400640 1 -- 172.21.15.22:0/3969552099 learned_addr learned my addr 172.21.15.22:0/3969552099 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.391+0000 7fefc9000640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefc419a920 0x7fefc419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.391+0000 7fefc3e00640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc41021b0 0x7fefc41a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.392+0000 7fefc9000640 1 -- 172.21.15.22:0/3969552099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefc4107280 msgr2=0x7fefc41a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.392+0000 7fefc9000640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefc4107280 0x7fefc41a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.392+0000 7fefc9000640 1 -- 172.21.15.22:0/3969552099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc41021b0 msgr2=0x7fefc41a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.392+0000 7fefc9000640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc41021b0 0x7fefc41a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.392+0000 7fefc9000640 1 -- 172.21.15.22:0/3969552099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc419b4d0 con 0x7fefc419a920 2024-04-20T21:39:59.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.392+0000 7fefc3e00640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc41021b0 0x7fefc41a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:59.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.392+0000 7fefc9000640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefc419a920 0x7fefc419add0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00da10 tx=0x7fefbc00ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:59.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.393+0000 7fefc0c00640 1 -- 172.21.15.22:0/3969552099 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc00bd70 con 0x7fefc419a920 2024-04-20T21:39:59.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.393+0000 7fefc0c00640 1 -- 172.21.15.22:0/3969552099 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefbc003e50 con 0x7fefc419a920 2024-04-20T21:39:59.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.393+0000 7fefc0c00640 1 -- 172.21.15.22:0/3969552099 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0106a0 con 0x7fefc419a920 2024-04-20T21:39:59.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.393+0000 7fefcb000640 1 -- 172.21.15.22:0/3969552099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc41a7610 con 0x7fefc419a920 2024-04-20T21:39:59.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.393+0000 7fefcb000640 1 -- 172.21.15.22:0/3969552099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc41a79c0 con 0x7fefc419a920 2024-04-20T21:39:59.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.394+0000 7fefcb000640 1 -- 172.21.15.22:0/3969552099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc4068d80 con 0x7fefc419a920 2024-04-20T21:39:59.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.396+0000 7fefc0c00640 1 -- 172.21.15.22:0/3969552099 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fefbc003970 con 0x7fefc419a920 2024-04-20T21:39:59.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.398+0000 7fefc0c00640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fefa80a3da0 0x7fefa80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.398+0000 7fefc0c00640 1 -- 172.21.15.22:0/3969552099 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fefbc0d0b60 con 0x7fefc419a920 2024-04-20T21:39:59.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.398+0000 7fefc3e00640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fefa80a3da0 0x7fefa80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.399+0000 7fefc3e00640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fefa80a3da0 0x7fefa80a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009c20 tx=0x7fefb4005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:59.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.402+0000 7fefc0c00640 1 -- 172.21.15.22:0/3969552099 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fefbc09d180 con 0x7fefc419a920 2024-04-20T21:39:59.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.688+0000 7fefcb000640 1 -- 172.21.15.22:0/3969552099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefc4102bc0 con 0x7fefc419a920 2024-04-20T21:39:59.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.689+0000 7fefc0c00640 1 -- 172.21.15.22:0/3969552099 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7fefbc0bd0c0 con 0x7fefc419a920 2024-04-20T21:39:59.688 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:39:59.688 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:39:59.688 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.691+0000 7fefcb000640 1 -- 172.21.15.22:0/3969552099 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fefa80a3da0 msgr2=0x7fefa80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.691+0000 7fefcb000640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fefa80a3da0 0x7fefa80a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009c20 tx=0x7fefb4005f90 comp rx=0 tx=0).stop 2024-04-20T21:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.691+0000 7fefcb000640 1 -- 172.21.15.22:0/3969552099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefc419a920 msgr2=0x7fefc419add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.691+0000 7fefcb000640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefc419a920 0x7fefc419add0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00da10 tx=0x7fefbc00ded0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.692+0000 7fefcb000640 1 -- 172.21.15.22:0/3969552099 shutdown_connections 2024-04-20T21:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.692+0000 7fefcb000640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fefa80a3da0 0x7fefa80a6260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.692+0000 7fefcb000640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefc419a920 0x7fefc419add0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.692+0000 7fefcb000640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefc4107280 0x7fefc41a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.692+0000 7fefcb000640 1 --2- 172.21.15.22:0/3969552099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc41021b0 0x7fefc41a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.692+0000 7fefcb000640 1 -- 172.21.15.22:0/3969552099 >> 172.21.15.22:0/3969552099 conn(0x7fefc4079360 msgr2=0x7fefc40ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.692+0000 7fefcb000640 1 -- 172.21.15.22:0/3969552099 shutdown_connections 2024-04-20T21:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.692+0000 7fefcb000640 1 -- 172.21.15.22:0/3969552099 wait complete. 2024-04-20T21:39:59.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:39:59.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.977+0000 7faa7f200640 1 -- 172.21.15.22:0/2859604447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80106f90 msgr2=0x7faa80107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.977+0000 7faa7f200640 1 --2- 172.21.15.22:0/2859604447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80106f90 0x7faa80107370 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faa70009900 tx=0x7faa7002bc30 comp rx=0 tx=0).stop 2024-04-20T21:39:59.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.977+0000 7faa7f200640 1 -- 172.21.15.22:0/2859604447 shutdown_connections 2024-04-20T21:39:59.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.977+0000 7faa7f200640 1 --2- 172.21.15.22:0/2859604447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa801010f0 0x7faa8010ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.977+0000 7faa7f200640 1 --2- 172.21.15.22:0/2859604447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa80100730 0x7faa80100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.977+0000 7faa7f200640 1 --2- 172.21.15.22:0/2859604447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80106f90 0x7faa80107370 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.977+0000 7faa7f200640 1 -- 172.21.15.22:0/2859604447 >> 172.21.15.22:0/2859604447 conn(0x7faa80076080 msgr2=0x7faa80076490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:59.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.977+0000 7faa7f200640 1 -- 172.21.15.22:0/2859604447 shutdown_connections 2024-04-20T21:39:59.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.977+0000 7faa7f200640 1 -- 172.21.15.22:0/2859604447 wait complete. 2024-04-20T21:39:59.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.979+0000 7faa7f200640 1 Processor -- start 2024-04-20T21:39:59.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.979+0000 7faa7f200640 1 -- start start 2024-04-20T21:39:59.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.979+0000 7faa7f200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80100730 0x7faa801a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.979+0000 7faa7f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa801010f0 0x7faa801a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.979+0000 7faa7f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa80106f90 0x7faa8019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.979+0000 7faa7f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faa801121b0 con 0x7faa80106f90 2024-04-20T21:39:59.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.979+0000 7faa7f200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa80112030 con 0x7faa80100730 2024-04-20T21:39:59.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa80112330 con 0x7faa801010f0 2024-04-20T21:39:59.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa801010f0 0x7faa801a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80100730 0x7faa801a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa801010f0 0x7faa801a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42052/0 (socket says 172.21.15.22:42052) 2024-04-20T21:39:59.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7d400640 1 -- 172.21.15.22:0/459832686 learned_addr learned my addr 172.21.15.22:0/459832686 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:59.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7e800640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa80106f90 0x7faa8019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7d400640 1 -- 172.21.15.22:0/459832686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80100730 msgr2=0x7faa801a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7d400640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80100730 0x7faa801a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7d400640 1 -- 172.21.15.22:0/459832686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa80106f90 msgr2=0x7faa8019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7d400640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa80106f90 0x7faa8019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7d400640 1 -- 172.21.15.22:0/459832686 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8019af60 con 0x7faa801010f0 2024-04-20T21:39:59.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7de00640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80100730 0x7faa801a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:59.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.980+0000 7faa7e800640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa80106f90 0x7faa8019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:59.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.981+0000 7faa7d400640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa801010f0 0x7faa801a0b20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e9f0 tx=0x7faa6c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:59.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.981+0000 7faa76a00640 1 -- 172.21.15.22:0/459832686 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00cd80 con 0x7faa801010f0 2024-04-20T21:39:59.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.981+0000 7faa76a00640 1 -- 172.21.15.22:0/459832686 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa6c010910 con 0x7faa801010f0 2024-04-20T21:39:59.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.981+0000 7faa7f200640 1 -- 172.21.15.22:0/459832686 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8019b250 con 0x7faa801010f0 2024-04-20T21:39:59.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.981+0000 7faa76a00640 1 -- 172.21.15.22:0/459832686 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c018990 con 0x7faa801010f0 2024-04-20T21:39:59.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.981+0000 7faa7f200640 1 -- 172.21.15.22:0/459832686 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa801a7560 con 0x7faa801010f0 2024-04-20T21:39:59.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.984+0000 7faa8798d740 1 Processor -- start 2024-04-20T21:39:59.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.984+0000 7faa76a00640 1 -- 172.21.15.22:0/459832686 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7faa6c010ac0 con 0x7faa801010f0 2024-04-20T21:39:59.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.984+0000 7faa8798d740 1 -- start start 2024-04-20T21:39:59.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.985+0000 7faa8798d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 0x5628ea48e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.985+0000 7faa8798d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 0x5628ea497130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.985+0000 7faa8798d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea497670 0x5628ea499a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.985+0000 7faa8798d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5628ea3f3050 con 0x5628ea48e170 2024-04-20T21:39:59.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.985+0000 7faa8798d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5628ea399490 con 0x5628ea48ea90 2024-04-20T21:39:59.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.985+0000 7faa8798d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5628ea3ef010 con 0x5628ea497670 2024-04-20T21:39:59.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.985+0000 7faa7de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 0x5628ea497130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.985+0000 7faa7de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 0x5628ea497130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39098/0 (socket says 172.21.15.22:39098) 2024-04-20T21:39:59.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.985+0000 7faa7de00640 1 -- 172.21.15.22:0/1309681921 learned_addr learned my addr 172.21.15.22:0/1309681921 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:59.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.985+0000 7faa7e800640 1 --2- 172.21.15.22:0/1309681921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 0x5628ea48e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.985+0000 7faa7d400640 1 --2- 172.21.15.22:0/1309681921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea497670 0x5628ea499a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.986+0000 7faa7de00640 1 -- 172.21.15.22:0/1309681921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea497670 msgr2=0x5628ea499a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:59.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.986+0000 7faa7de00640 1 --2- 172.21.15.22:0/1309681921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea497670 0x5628ea499a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.986+0000 7faa7de00640 1 -- 172.21.15.22:0/1309681921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 msgr2=0x5628ea48e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:59.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.986+0000 7faa7de00640 1 --2- 172.21.15.22:0/1309681921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 0x5628ea48e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.986+0000 7faa7de00640 1 -- 172.21.15.22:0/1309681921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628ea3a1070 con 0x5628ea48ea90 2024-04-20T21:39:59.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.986+0000 7faa7d400640 1 --2- 172.21.15.22:0/1309681921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea497670 0x5628ea499a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:59.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.986+0000 7faa7de00640 1 --2- 172.21.15.22:0/1309681921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 0x5628ea497130 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faa70009900 tx=0x7faa70005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4fbaf31fb6af778 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:59.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.986+0000 7faa6b000640 1 -- 172.21.15.22:0/1309681921 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70042070 con 0x5628ea48ea90 2024-04-20T21:39:59.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.986+0000 7faa76a00640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7faa640a3da0 0x7faa640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.986+0000 7faa6b000640 1 -- 172.21.15.22:0/1309681921 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa70004a70 con 0x5628ea48ea90 2024-04-20T21:39:59.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.986+0000 7faa6b000640 1 -- 172.21.15.22:0/1309681921 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700052a0 con 0x5628ea48ea90 2024-04-20T21:39:59.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.987+0000 7faa76a00640 1 -- 172.21.15.22:0/459832686 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7faa6c0d0bb0 con 0x7faa801010f0 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.987+0000 7faa7e800640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7faa640a3da0 0x7faa640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.987+0000 7faa8798d740 1 -- 172.21.15.22:0/1309681921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 msgr2=0x5628ea497130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.987+0000 7faa8798d740 1 --2- 172.21.15.22:0/1309681921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 0x5628ea497130 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faa70009900 tx=0x7faa70005e60 comp rx=0 tx=0).stop 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.987+0000 7faa8798d740 1 -- 172.21.15.22:0/1309681921 shutdown_connections 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.987+0000 7faa8798d740 1 --2- 172.21.15.22:0/1309681921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea497670 0x5628ea499a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.987+0000 7faa8798d740 1 --2- 172.21.15.22:0/1309681921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 0x5628ea497130 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.987+0000 7faa8798d740 1 --2- 172.21.15.22:0/1309681921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 0x5628ea48e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.987+0000 7faa8798d740 1 -- 172.21.15.22:0/1309681921 >> 172.21.15.22:0/1309681921 conn(0x5628ea4888a0 msgr2=0x5628ea488c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.988+0000 7faa8798d740 1 -- 172.21.15.22:0/1309681921 shutdown_connections 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.988+0000 7faa8798d740 1 -- 172.21.15.22:0/1309681921 wait complete. 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.988+0000 7faa7e800640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7faa640a3da0 0x7faa640a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faa78005fd0 tx=0x7faa78009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.988+0000 7faa8798d740 1 -- start start 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.988+0000 7faa8798d740 1 Processor -- start 2024-04-20T21:39:59.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa8798d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 0x5628ea4947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa8798d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 0x5628ea494ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa8798d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea497670 0x5628ea495220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa8798d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5628ea49a400 con 0x5628ea48e170 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa8798d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5628ea3ac860 con 0x5628ea48ea90 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa8798d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5628ea49a580 con 0x5628ea497670 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa8798d740 10 client.0 _ll_register_callbacks cb 0x5628ea37afe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa7d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 0x5628ea494ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa7d400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 0x5628ea494ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39104/0 (socket says 172.21.15.22:39104) 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa7d400640 1 -- 172.21.15.22:0/2913240070 learned_addr learned my addr 172.21.15.22:0/2913240070 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa7de00640 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 0x5628ea4947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa7e800640 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea497670 0x5628ea495220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa7d400640 1 -- 172.21.15.22:0/2913240070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea497670 msgr2=0x5628ea495220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa7d400640 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea497670 0x5628ea495220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa7d400640 1 -- 172.21.15.22:0/2913240070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 msgr2=0x5628ea4947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:59.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa7d400640 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 0x5628ea4947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.989+0000 7faa7d400640 1 -- 172.21.15.22:0/2913240070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628ea3b0950 con 0x5628ea48ea90 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.990+0000 7faa7d400640 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 0x5628ea494ce0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0ce2e0 tx=0x7faa6c098ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71e443a0ba9afb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.990+0000 7faa5f400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.990+0000 7faa5f400640 1 -- 172.21.15.22:0/2913240070 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c0ce700 con 0x5628ea48ea90 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.990+0000 7faa8798d740 1 -- 172.21.15.22:0/2913240070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa70004a70 con 0x5628ea48ea90 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.990+0000 7faa5f400640 1 -- 172.21.15.22:0/2913240070 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7faa6c09bf20 con 0x5628ea48ea90 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.990+0000 7faa5f400640 1 -- 172.21.15.22:0/2913240070 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7faa6c09d1d0 con 0x5628ea48ea90 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.990+0000 7faa8798d740 10 client.26276 fetch_fsmap learned FSMap version 20 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.990+0000 7faa8798d740 1 -- 172.21.15.22:0/2913240070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5628ea37e820 con 0x5628ea48ea90 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.990+0000 7faa5f400640 1 -- 172.21.15.22:0/2913240070 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa70004a70 con 0x5628ea48ea90 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.991+0000 7faa5f400640 1 -- 172.21.15.22:0/2913240070 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7faa6c018bc0 con 0x5628ea48ea90 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.991+0000 7faa8798d740 10 client.26276 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.991+0000 7faa8798d740 10 client.26276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:59.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.991+0000 7faa8798d740 20 client.26276 populate_metadata read hostname 'smithi022' 2024-04-20T21:39:59.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.991+0000 7faa8798d740 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5628ea4a7710 0x5628ea4a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:59.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.991+0000 7faa8798d740 4 client.26276 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:59.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.991+0000 7faa8798d740 1 -- 172.21.15.22:0/2913240070 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628ea400a70 con 0x5628ea4a7710 2024-04-20T21:39:59.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.991+0000 7faa7de00640 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5628ea4a7710 0x5628ea4a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:59.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.992+0000 7faa7de00640 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5628ea4a7710 0x5628ea4a9b00 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c2a87c347eea58 server_cookie=72adc66912c4d33a in_seq=0 out_seq=0 2024-04-20T21:39:59.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.992+0000 7faa5f400640 10 client.26276 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:59.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.993+0000 7faa5f400640 1 -- 172.21.15.22:0/2913240070 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5628ea400a70 con 0x5628ea4a7710 2024-04-20T21:39:59.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.993+0000 7faa5f400640 10 client.26276 handle_command_reply: tid=0 2024-04-20T21:39:59.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.993+0000 7faa8798d740 1 client.26276 shutdown 2024-04-20T21:39:59.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.993+0000 7faa8798d740 1 -- 172.21.15.22:0/2913240070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 msgr2=0x5628ea494ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:59.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.993+0000 7faa8798d740 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 0x5628ea494ce0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0ce2e0 tx=0x7faa6c098ea0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.993+0000 7faa8798d740 1 -- 172.21.15.22:0/2913240070 shutdown_connections 2024-04-20T21:39:59.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.993+0000 7faa8798d740 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea497670 0x5628ea495220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.993+0000 7faa8798d740 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5628ea4a7710 0x5628ea4a9b00 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.993+0000 7faa8798d740 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea48ea90 0x5628ea494ce0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.993+0000 7faa8798d740 1 --2- 172.21.15.22:0/2913240070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 0x5628ea4947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:59.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.993+0000 7faa8798d740 1 -- 172.21.15.22:0/2913240070 >> 172.21.15.22:0/2913240070 conn(0x5628ea4888a0 msgr2=0x5628ea48a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:59.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.993+0000 7faa5f400640 0 client.26276 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:39:59.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.994+0000 7faa8798d740 1 -- 172.21.15.22:0/2913240070 shutdown_connections 2024-04-20T21:39:59.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.994+0000 7faa8798d740 1 -- 172.21.15.22:0/2913240070 wait complete. 2024-04-20T21:39:59.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:39:59.994+0000 7faa8798d740 20 client.26276 trim_cache size 0 max 16384 2024-04-20T21:40:00.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.009+0000 7faa8798d740 1 Processor -- start 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa8798d740 1 -- start start 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa8798d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea48e170 0x5628ea496ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa8798d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48ea90 0x5628ea4a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa8798d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 0x5628ea4877e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa8798d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faa801add10 con 0x5628ea48ea90 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa8798d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa801b0590 con 0x5628ea497670 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa8798d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa801ae380 con 0x5628ea48e170 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa7e800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 0x5628ea4877e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa7de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea48e170 0x5628ea496ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa7e800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 0x5628ea4877e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39108/0 (socket says 172.21.15.22:39108) 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa7e800640 1 -- 172.21.15.22:0/981432651 learned_addr learned my addr 172.21.15.22:0/981432651 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa7de00640 1 --2- 172.21.15.22:0/981432651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea48e170 0x5628ea496ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42072/0 (socket says 172.21.15.22:42072) 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.010+0000 7faa7d400640 1 --2- 172.21.15.22:0/981432651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48ea90 0x5628ea4a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.011+0000 7faa7de00640 1 -- 172.21.15.22:0/981432651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 msgr2=0x5628ea4877e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:00.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.011+0000 7faa7de00640 1 --2- 172.21.15.22:0/981432651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 0x5628ea4877e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.011+0000 7faa7de00640 1 -- 172.21.15.22:0/981432651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48ea90 msgr2=0x5628ea4a32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.011+0000 7faa7de00640 1 --2- 172.21.15.22:0/981432651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48ea90 0x5628ea4a32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.011+0000 7faa7de00640 1 -- 172.21.15.22:0/981432651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628ea1053f0 con 0x5628ea48e170 2024-04-20T21:40:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.011+0000 7faa7de00640 1 --2- 172.21.15.22:0/981432651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea48e170 0x5628ea496ed0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faa70004650 tx=0x7faa7002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb8ec23da4ad6209 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.011+0000 7faa6a600640 1 -- 172.21.15.22:0/981432651 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70042040 con 0x5628ea48e170 2024-04-20T21:40:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.011+0000 7faa6a600640 1 -- 172.21.15.22:0/981432651 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa70004a70 con 0x5628ea48e170 2024-04-20T21:40:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.011+0000 7faa6a600640 1 -- 172.21.15.22:0/981432651 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700346a0 con 0x5628ea48e170 2024-04-20T21:40:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.012+0000 7faa8798d740 1 -- 172.21.15.22:0/981432651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea48e170 msgr2=0x5628ea496ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.012+0000 7faa8798d740 1 --2- 172.21.15.22:0/981432651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea48e170 0x5628ea496ed0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faa70004650 tx=0x7faa7002bc30 comp rx=0 tx=0).stop 2024-04-20T21:40:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.012+0000 7faa8798d740 1 -- 172.21.15.22:0/981432651 shutdown_connections 2024-04-20T21:40:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.012+0000 7faa8798d740 1 --2- 172.21.15.22:0/981432651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 0x5628ea4877e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.012+0000 7faa8798d740 1 --2- 172.21.15.22:0/981432651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48ea90 0x5628ea4a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.012+0000 7faa8798d740 1 --2- 172.21.15.22:0/981432651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea48e170 0x5628ea496ed0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.012+0000 7faa8798d740 1 -- 172.21.15.22:0/981432651 >> 172.21.15.22:0/981432651 conn(0x5628ea4888a0 msgr2=0x5628ea4870d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.014+0000 7faa8798d740 1 -- 172.21.15.22:0/981432651 shutdown_connections 2024-04-20T21:40:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.014+0000 7faa8798d740 1 -- 172.21.15.22:0/981432651 wait complete. 2024-04-20T21:40:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.014+0000 7faa8798d740 1 -- start start 2024-04-20T21:40:00.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.014+0000 7faa8798d740 1 Processor -- start 2024-04-20T21:40:00.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa8798d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 0x5628ea49f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:00.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa8798d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea48ea90 0x5628ea49fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:00.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa8798d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 0x5628ea49af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:00.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa8798d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faa801aea50 con 0x5628ea48e170 2024-04-20T21:40:00.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa8798d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa801afec0 con 0x5628ea497670 2024-04-20T21:40:00.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa8798d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa801af120 con 0x5628ea48ea90 2024-04-20T21:40:00.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa8798d740 10 client.0 _ll_register_callbacks cb 0x7faa6c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa7e800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 0x5628ea49af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa7de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 0x5628ea49f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa7e800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 0x5628ea49af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39120/0 (socket says 172.21.15.22:39120) 2024-04-20T21:40:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa7e800640 1 -- 172.21.15.22:0/1393097161 learned_addr learned my addr 172.21.15.22:0/1393097161 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa7d400640 1 --2- 172.21.15.22:0/1393097161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea48ea90 0x5628ea49fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa7e800640 1 -- 172.21.15.22:0/1393097161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea48ea90 msgr2=0x5628ea49fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa7e800640 1 --2- 172.21.15.22:0/1393097161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea48ea90 0x5628ea49fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa7e800640 1 -- 172.21.15.22:0/1393097161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 msgr2=0x5628ea49f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:00.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa7e800640 1 --2- 172.21.15.22:0/1393097161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 0x5628ea49f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.015+0000 7faa7e800640 1 -- 172.21.15.22:0/1393097161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628ea1cf7d0 con 0x5628ea497670 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.016+0000 7faa7e800640 1 --2- 172.21.15.22:0/1393097161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 0x5628ea49af10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faa78010be0 tx=0x7faa78004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72945936888a6b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.016+0000 7faa5f400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.016+0000 7faa5f400640 1 -- 172.21.15.22:0/1393097161 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7802e060 con 0x5628ea497670 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.016+0000 7faa5f400640 1 -- 172.21.15.22:0/1393097161 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7faa7801ec30 con 0x5628ea497670 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.016+0000 7faa8798d740 1 -- 172.21.15.22:0/1393097161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa70004a70 con 0x5628ea497670 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.016+0000 7faa5f400640 1 -- 172.21.15.22:0/1393097161 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7faa78032430 con 0x5628ea497670 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.016+0000 7faa8798d740 10 client.26282 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.016+0000 7faa8798d740 1 -- 172.21.15.22:0/1393097161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5628ea50d730 con 0x5628ea497670 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.017+0000 7faa5f400640 1 -- 172.21.15.22:0/1393097161 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa70004a70 con 0x5628ea497670 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.017+0000 7faa5f400640 1 -- 172.21.15.22:0/1393097161 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7faa78028070 con 0x5628ea497670 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.017+0000 7faa8798d740 10 client.26282 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:00.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.017+0000 7faa8798d740 10 client.26282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:00.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.017+0000 7faa8798d740 20 client.26282 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:00.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.017+0000 7faa8798d740 1 --2- 172.21.15.22:0/1393097161 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5628ea50dee0 0x5628ea510300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:00.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.017+0000 7faa8798d740 4 client.26282 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:40:00.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.017+0000 7faa8798d740 1 -- 172.21.15.22:0/1393097161 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628ea3fa8e0 con 0x5628ea50dee0 2024-04-20T21:40:00.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.017+0000 7faa7de00640 1 --2- 172.21.15.22:0/1393097161 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5628ea50dee0 0x5628ea510300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:00.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.018+0000 7faa7de00640 1 --2- 172.21.15.22:0/1393097161 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5628ea50dee0 0x5628ea510300 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae1aaa21f8c7539b server_cookie=7ac1f76bbdaf8f62 in_seq=0 out_seq=0 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.018+0000 7faa5f400640 10 client.26282 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa5f400640 1 -- 172.21.15.22:0/1393097161 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628ea3fa8e0 con 0x5628ea50dee0 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa5f400640 10 client.26282 handle_command_reply: tid=0 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa8798d740 1 client.26282 shutdown 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa8798d740 1 -- 172.21.15.22:0/1393097161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 msgr2=0x5628ea49af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa8798d740 1 --2- 172.21.15.22:0/1393097161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 0x5628ea49af10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faa78010be0 tx=0x7faa78004a40 comp rx=0 tx=0).stop 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa8798d740 1 -- 172.21.15.22:0/1393097161 shutdown_connections 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa8798d740 1 --2- 172.21.15.22:0/1393097161 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5628ea50dee0 0x5628ea510300 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa8798d740 1 --2- 172.21.15.22:0/1393097161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5628ea497670 0x5628ea49af10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa8798d740 1 --2- 172.21.15.22:0/1393097161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ea48ea90 0x5628ea49fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa8798d740 1 --2- 172.21.15.22:0/1393097161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628ea48e170 0x5628ea49f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa5f400640 0 client.26282 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa8798d740 1 -- 172.21.15.22:0/1393097161 >> 172.21.15.22:0/1393097161 conn(0x5628ea4888a0 msgr2=0x5628ea48a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:00.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa8798d740 1 -- 172.21.15.22:0/1393097161 shutdown_connections 2024-04-20T21:40:00.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa8798d740 1 -- 172.21.15.22:0/1393097161 wait complete. 2024-04-20T21:40:00.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.020+0000 7faa8798d740 20 client.26282 trim_cache size 0 max 16384 2024-04-20T21:40:00.022 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:40:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.021+0000 7faa7f200640 1 -- 172.21.15.22:0/459832686 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7faa640a3da0 msgr2=0x7faa640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.021+0000 7faa7f200640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7faa640a3da0 0x7faa640a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faa78005fd0 tx=0x7faa78009500 comp rx=0 tx=0).stop 2024-04-20T21:40:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.021+0000 7faa7f200640 1 -- 172.21.15.22:0/459832686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa801010f0 msgr2=0x7faa801a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.021+0000 7faa7f200640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa801010f0 0x7faa801a0b20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e9f0 tx=0x7faa6c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.021+0000 7faa7f200640 1 -- 172.21.15.22:0/459832686 shutdown_connections 2024-04-20T21:40:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.021+0000 7faa7f200640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7faa640a3da0 0x7faa640a6260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.021+0000 7faa7f200640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa80106f90 0x7faa8019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.021+0000 7faa7f200640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa801010f0 0x7faa801a0b20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.021+0000 7faa7f200640 1 --2- 172.21.15.22:0/459832686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80100730 0x7faa801a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.021+0000 7faa7f200640 1 -- 172.21.15.22:0/459832686 >> 172.21.15.22:0/459832686 conn(0x7faa80076080 msgr2=0x7faa80101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.021+0000 7faa7f200640 1 -- 172.21.15.22:0/459832686 shutdown_connections 2024-04-20T21:40:00.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:00.021+0000 7faa7f200640 1 -- 172.21.15.22:0/459832686 wait complete. 2024-04-20T21:40:00.196 INFO:tasks.workunit.client.0.smithi022.stdout: 1 491109 9.54 MB/sec execute 148 sec latency 22.510 ms 2024-04-20T21:40:00.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:00 smithi022 ceph-mon[16843]: pgmap v188: 97 pgs: 97 active+clean; 281 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 2.3 MiB/s wr, 138 op/s 2024-04-20T21:40:00.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:00 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3969552099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:40:00.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:00 smithi022 ceph-mon[16843]: overall HEALTH_OK 2024-04-20T21:40:00.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:00 smithi122 ceph-mon[22147]: pgmap v188: 97 pgs: 97 active+clean; 281 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 2.3 MiB/s wr, 138 op/s 2024-04-20T21:40:00.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:00 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3969552099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:40:00.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:00 smithi122 ceph-mon[22147]: overall HEALTH_OK 2024-04-20T21:40:00.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:00 smithi033 ceph-mon[21626]: pgmap v188: 97 pgs: 97 active+clean; 281 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 2.3 MiB/s wr, 138 op/s 2024-04-20T21:40:00.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:00 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3969552099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:40:00.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:00 smithi033 ceph-mon[21626]: overall HEALTH_OK 2024-04-20T21:40:01.040 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:40:01.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:40:01.041 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:40:01.196 INFO:tasks.workunit.client.0.smithi022.stdout: 1 491733 9.51 MB/sec execute 149 sec latency 66.270 ms 2024-04-20T21:40:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.273+0000 7fa453c00640 1 -- 172.21.15.22:0/736693462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c104dc0 msgr2=0x7fa44c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.273+0000 7fa453c00640 1 --2- 172.21.15.22:0/736693462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c104dc0 0x7fa44c108e00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa434009900 tx=0x7fa43402bc40 comp rx=0 tx=0).stop 2024-04-20T21:40:01.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.273+0000 7fa453c00640 1 -- 172.21.15.22:0/736693462 shutdown_connections 2024-04-20T21:40:01.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.273+0000 7fa453c00640 1 --2- 172.21.15.22:0/736693462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c1094e0 0x7fa44c1110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.273+0000 7fa453c00640 1 --2- 172.21.15.22:0/736693462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c104dc0 0x7fa44c108e00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.273+0000 7fa453c00640 1 --2- 172.21.15.22:0/736693462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa44c104410 0x7fa44c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.273+0000 7fa453c00640 1 -- 172.21.15.22:0/736693462 >> 172.21.15.22:0/736693462 conn(0x7fa44c0ffcf0 msgr2=0x7fa44c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.273+0000 7fa453c00640 1 -- 172.21.15.22:0/736693462 shutdown_connections 2024-04-20T21:40:01.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.273+0000 7fa453c00640 1 -- 172.21.15.22:0/736693462 wait complete. 2024-04-20T21:40:01.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.274+0000 7fa453c00640 1 Processor -- start 2024-04-20T21:40:01.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.274+0000 7fa453c00640 1 -- start start 2024-04-20T21:40:01.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.275+0000 7fa453c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa44c104410 0x7fa44c1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.275+0000 7fa453c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c1094e0 0x7fa44c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.275+0000 7fa453c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c19a9a0 0x7fa44c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.275+0000 7fa453c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c1145c0 con 0x7fa44c19a9a0 2024-04-20T21:40:01.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.275+0000 7fa453c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c114440 con 0x7fa44c1094e0 2024-04-20T21:40:01.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.275+0000 7fa453c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c114740 con 0x7fa44c104410 2024-04-20T21:40:01.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.275+0000 7fa451200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa44c104410 0x7fa44c1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.275+0000 7fa451200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa44c104410 0x7fa44c1a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42104/0 (socket says 172.21.15.22:42104) 2024-04-20T21:40:01.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.275+0000 7fa451200640 1 -- 172.21.15.22:0/2485564679 learned_addr learned my addr 172.21.15.22:0/2485564679 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:01.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.275+0000 7fa451c00640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c19a9a0 0x7fa44c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.275+0000 7fa44be00640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c1094e0 0x7fa44c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.276+0000 7fa451200640 1 -- 172.21.15.22:0/2485564679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c1094e0 msgr2=0x7fa44c1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.276+0000 7fa451200640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c1094e0 0x7fa44c1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.276+0000 7fa451200640 1 -- 172.21.15.22:0/2485564679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c19a9a0 msgr2=0x7fa44c19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.276+0000 7fa451200640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c19a9a0 0x7fa44c19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.276+0000 7fa451200640 1 -- 172.21.15.22:0/2485564679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c19b4c0 con 0x7fa44c104410 2024-04-20T21:40:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.276+0000 7fa44be00640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c1094e0 0x7fa44c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.276+0000 7fa451c00640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c19a9a0 0x7fa44c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.277+0000 7fa451200640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa44c104410 0x7fa44c1a07a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa44400eb90 tx=0x7fa44400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.277+0000 7fa449600640 1 -- 172.21.15.22:0/2485564679 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444004d20 con 0x7fa44c104410 2024-04-20T21:40:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.277+0000 7fa449600640 1 -- 172.21.15.22:0/2485564679 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa44400ee90 con 0x7fa44c104410 2024-04-20T21:40:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.277+0000 7fa453c00640 1 -- 172.21.15.22:0/2485564679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44c19b7b0 con 0x7fa44c104410 2024-04-20T21:40:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.277+0000 7fa449600640 1 -- 172.21.15.22:0/2485564679 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444010830 con 0x7fa44c104410 2024-04-20T21:40:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.277+0000 7fa453c00640 1 -- 172.21.15.22:0/2485564679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa44c1a79b0 con 0x7fa44c104410 2024-04-20T21:40:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.280+0000 7fa456983740 1 Processor -- start 2024-04-20T21:40:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.281+0000 7fa449600640 1 -- 172.21.15.22:0/2485564679 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa444005020 con 0x7fa44c104410 2024-04-20T21:40:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.281+0000 7fa456983740 1 -- start start 2024-04-20T21:40:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.281+0000 7fa456983740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 0x556dc15732f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.281+0000 7fa456983740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1573830 0x556dc157bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.281+0000 7fa456983740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc157c410 0x556dc157e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.281+0000 7fa456983740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556dc14d7d20 con 0x556dc157c410 2024-04-20T21:40:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.281+0000 7fa456983740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556dc147e3e0 con 0x556dc1572f10 2024-04-20T21:40:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.281+0000 7fa456983740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556dc14d3ce0 con 0x556dc1573830 2024-04-20T21:40:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.281+0000 7fa451c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 0x556dc15732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.281+0000 7fa451c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 0x556dc15732f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39138/0 (socket says 172.21.15.22:39138) 2024-04-20T21:40:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.281+0000 7fa451c00640 1 -- 172.21.15.22:0/2586349108 learned_addr learned my addr 172.21.15.22:0/2586349108 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:01.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.281+0000 7fa44be00640 1 --2- 172.21.15.22:0/2586349108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc157c410 0x556dc157e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.282+0000 7fa451200640 1 --2- 172.21.15.22:0/2586349108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1573830 0x556dc157bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.282+0000 7fa451c00640 1 -- 172.21.15.22:0/2586349108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1573830 msgr2=0x556dc157bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.282+0000 7fa451c00640 1 --2- 172.21.15.22:0/2586349108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1573830 0x556dc157bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.282+0000 7fa451c00640 1 -- 172.21.15.22:0/2586349108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc157c410 msgr2=0x556dc157e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.282+0000 7fa451c00640 1 --2- 172.21.15.22:0/2586349108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc157c410 0x556dc157e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.282+0000 7fa451c00640 1 -- 172.21.15.22:0/2586349108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dc149d980 con 0x556dc1572f10 2024-04-20T21:40:01.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.282+0000 7fa451200640 1 --2- 172.21.15.22:0/2586349108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1573830 0x556dc157bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.282+0000 7fa451c00640 1 --2- 172.21.15.22:0/2586349108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 0x556dc15732f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00dc70 tx=0x7fa43c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7382e377de3de76b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.282+0000 7fa442000640 1 -- 172.21.15.22:0/2586349108 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c015070 con 0x556dc1572f10 2024-04-20T21:40:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.282+0000 7fa449600640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa4300a3da0 0x7fa4300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.282+0000 7fa442000640 1 -- 172.21.15.22:0/2586349108 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa43c0035f0 con 0x556dc1572f10 2024-04-20T21:40:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.282+0000 7fa442000640 1 -- 172.21.15.22:0/2586349108 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c0116a0 con 0x556dc1572f10 2024-04-20T21:40:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.283+0000 7fa449600640 1 -- 172.21.15.22:0/2485564679 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fa4440d0fa0 con 0x7fa44c104410 2024-04-20T21:40:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.283+0000 7fa451200640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa4300a3da0 0x7fa4300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.283+0000 7fa456983740 1 -- 172.21.15.22:0/2586349108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 msgr2=0x556dc15732f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.283+0000 7fa456983740 1 --2- 172.21.15.22:0/2586349108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 0x556dc15732f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00dc70 tx=0x7fa43c00f730 comp rx=0 tx=0).stop 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.284+0000 7fa456983740 1 -- 172.21.15.22:0/2586349108 shutdown_connections 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.284+0000 7fa456983740 1 --2- 172.21.15.22:0/2586349108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc157c410 0x556dc157e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.284+0000 7fa456983740 1 --2- 172.21.15.22:0/2586349108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1573830 0x556dc157bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.284+0000 7fa456983740 1 --2- 172.21.15.22:0/2586349108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 0x556dc15732f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.284+0000 7fa456983740 1 -- 172.21.15.22:0/2586349108 >> 172.21.15.22:0/2586349108 conn(0x556dc156d640 msgr2=0x556dc156da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.284+0000 7fa451200640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa4300a3da0 0x7fa4300a6260 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x556dc15729e0 tx=0x7fa44400cf90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.284+0000 7fa456983740 1 -- 172.21.15.22:0/2586349108 shutdown_connections 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.284+0000 7fa456983740 1 -- 172.21.15.22:0/2586349108 wait complete. 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.284+0000 7fa456983740 1 -- start start 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.284+0000 7fa456983740 1 Processor -- start 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.285+0000 7fa456983740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 0x556dc15794b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.285+0000 7fa456983740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc1573830 0x556dc15799f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.285+0000 7fa456983740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc157c410 0x556dc1579f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.285+0000 7fa456983740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556dc157f200 con 0x556dc1573830 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.285+0000 7fa456983740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556dc14917a0 con 0x556dc1572f10 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.285+0000 7fa456983740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556dc157f380 con 0x556dc157c410 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.285+0000 7fa456983740 10 client.0 _ll_register_callbacks cb 0x556dc1493b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.285+0000 7fa451c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 0x556dc15794b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.285+0000 7fa451c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 0x556dc15794b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39142/0 (socket says 172.21.15.22:39142) 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.285+0000 7fa451c00640 1 -- 172.21.15.22:0/1080449643 learned_addr learned my addr 172.21.15.22:0/1080449643 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.285+0000 7fa44be00640 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc1573830 0x556dc15799f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.285+0000 7fa451c00640 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc157c410 0x556dc1579f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.286+0000 7fa451c00640 1 -- 172.21.15.22:0/1080449643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc157c410 msgr2=0x556dc1579f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.286+0000 7fa451c00640 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc157c410 0x556dc1579f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.286+0000 7fa451c00640 1 -- 172.21.15.22:0/1080449643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc1573830 msgr2=0x556dc15799f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.286+0000 7fa451c00640 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc1573830 0x556dc15799f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.286+0000 7fa451c00640 1 -- 172.21.15.22:0/1080449643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dc12b27e0 con 0x556dc1572f10 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.286+0000 7fa451c00640 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 0x556dc15794b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00dc40 tx=0x7fa43c0056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821da08ac5ba9f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.286+0000 7fa43aa00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:40:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.286+0000 7fa43aa00640 1 -- 172.21.15.22:0/1080449643 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c015070 con 0x556dc1572f10 2024-04-20T21:40:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.286+0000 7fa43aa00640 1 -- 172.21.15.22:0/1080449643 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fa43c004200 con 0x556dc1572f10 2024-04-20T21:40:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.286+0000 7fa456983740 1 -- 172.21.15.22:0/1080449643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa43c0035f0 con 0x556dc1572f10 2024-04-20T21:40:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.287+0000 7fa43aa00640 1 -- 172.21.15.22:0/1080449643 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa43c02f050 con 0x556dc1572f10 2024-04-20T21:40:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.287+0000 7fa456983740 10 client.26294 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.287+0000 7fa456983740 1 -- 172.21.15.22:0/1080449643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556dc1312f70 con 0x556dc1572f10 2024-04-20T21:40:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.287+0000 7fa43aa00640 1 -- 172.21.15.22:0/1080449643 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa43c0035f0 con 0x556dc1572f10 2024-04-20T21:40:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.287+0000 7fa43aa00640 1 -- 172.21.15.22:0/1080449643 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fa43c027020 con 0x556dc1572f10 2024-04-20T21:40:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.287+0000 7fa456983740 10 client.26294 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.287+0000 7fa456983740 10 client.26294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.287+0000 7fa456983740 20 client.26294 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:01.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.287+0000 7fa456983740 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556dc158c790 0x556dc158eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.287+0000 7fa456983740 4 client.26294 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.287+0000 7fa456983740 1 -- 172.21.15.22:0/1080449643 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dc14e5740 con 0x556dc158c790 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.287+0000 7fa44be00640 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556dc158c790 0x556dc158eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.288+0000 7fa44be00640 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556dc158c790 0x556dc158eb80 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b16fa2876a85cd4 server_cookie=4d681b5026132b1f in_seq=0 out_seq=0 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.288+0000 7fa43aa00640 10 client.26294 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.289+0000 7fa43aa00640 1 -- 172.21.15.22:0/1080449643 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556dc14e5740 con 0x556dc158c790 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.289+0000 7fa43aa00640 10 client.26294 handle_command_reply: tid=0 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.289+0000 7fa456983740 1 client.26294 shutdown 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.289+0000 7fa456983740 1 -- 172.21.15.22:0/1080449643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 msgr2=0x556dc15794b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.289+0000 7fa456983740 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 0x556dc15794b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00dc40 tx=0x7fa43c0056d0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.290+0000 7fa456983740 1 -- 172.21.15.22:0/1080449643 shutdown_connections 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.290+0000 7fa456983740 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc157c410 0x556dc1579f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.290+0000 7fa456983740 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc1573830 0x556dc15799f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.290+0000 7fa456983740 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556dc158c790 0x556dc158eb80 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.290+0000 7fa456983740 1 --2- 172.21.15.22:0/1080449643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1572f10 0x556dc15794b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.290+0000 7fa43aa00640 0 client.26294 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:01.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.290+0000 7fa456983740 1 -- 172.21.15.22:0/1080449643 >> 172.21.15.22:0/1080449643 conn(0x556dc156d640 msgr2=0x556dc156f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.291+0000 7fa456983740 1 -- 172.21.15.22:0/1080449643 shutdown_connections 2024-04-20T21:40:01.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.291+0000 7fa456983740 1 -- 172.21.15.22:0/1080449643 wait complete. 2024-04-20T21:40:01.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.291+0000 7fa456983740 20 client.26294 trim_cache size 0 max 16384 2024-04-20T21:40:01.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.314+0000 7fa456983740 1 Processor -- start 2024-04-20T21:40:01.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.314+0000 7fa456983740 1 -- start start 2024-04-20T21:40:01.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.314+0000 7fa456983740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 0x556dc157bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.314+0000 7fa456983740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc1573830 0x556dc15880c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.314+0000 7fa456983740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc157c410 0x556dc156c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.314+0000 7fa456983740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c1adff0 con 0x556dc1573830 2024-04-20T21:40:01.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.314+0000 7fa456983740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c1b0870 con 0x556dc157c410 2024-04-20T21:40:01.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.314+0000 7fa456983740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c1ae660 con 0x556dc1572f10 2024-04-20T21:40:01.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa451c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 0x556dc157bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa451c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 0x556dc157bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42116/0 (socket says 172.21.15.22:42116) 2024-04-20T21:40:01.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa451c00640 1 -- 172.21.15.22:0/125564247 learned_addr learned my addr 172.21.15.22:0/125564247 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:01.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa451c00640 1 --2- 172.21.15.22:0/125564247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc157c410 0x556dc156c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa44be00640 1 --2- 172.21.15.22:0/125564247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc1573830 0x556dc15880c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa451c00640 1 -- 172.21.15.22:0/125564247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc157c410 msgr2=0x556dc156c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa451c00640 1 --2- 172.21.15.22:0/125564247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc157c410 0x556dc156c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa451c00640 1 -- 172.21.15.22:0/125564247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc1573830 msgr2=0x556dc15880c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa451c00640 1 --2- 172.21.15.22:0/125564247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc1573830 0x556dc15880c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa451c00640 1 -- 172.21.15.22:0/125564247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dc11aa170 con 0x556dc1572f10 2024-04-20T21:40:01.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa44be00640 1 --2- 172.21.15.22:0/125564247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc1573830 0x556dc15880c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:40:01.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa451c00640 1 --2- 172.21.15.22:0/125564247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 0x556dc157bbe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa43c019720 tx=0x7fa43c00d350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a673324d900cead4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.315+0000 7fa441600640 1 -- 172.21.15.22:0/125564247 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c015040 con 0x556dc1572f10 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.316+0000 7fa441600640 1 -- 172.21.15.22:0/125564247 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa43c01e020 con 0x556dc1572f10 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.316+0000 7fa441600640 1 -- 172.21.15.22:0/125564247 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c004180 con 0x556dc1572f10 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.316+0000 7fa456983740 1 -- 172.21.15.22:0/125564247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 msgr2=0x556dc157bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.316+0000 7fa456983740 1 --2- 172.21.15.22:0/125564247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 0x556dc157bbe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa43c019720 tx=0x7fa43c00d350 comp rx=0 tx=0).stop 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.316+0000 7fa456983740 1 -- 172.21.15.22:0/125564247 shutdown_connections 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.316+0000 7fa456983740 1 --2- 172.21.15.22:0/125564247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc157c410 0x556dc156c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.316+0000 7fa456983740 1 --2- 172.21.15.22:0/125564247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc1573830 0x556dc15880c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.316+0000 7fa456983740 1 --2- 172.21.15.22:0/125564247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 0x556dc157bbe0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.316+0000 7fa456983740 1 -- 172.21.15.22:0/125564247 >> 172.21.15.22:0/125564247 conn(0x556dc156d640 msgr2=0x556dc156bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.317+0000 7fa456983740 1 -- 172.21.15.22:0/125564247 shutdown_connections 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.317+0000 7fa456983740 1 -- 172.21.15.22:0/125564247 wait complete. 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.317+0000 7fa456983740 1 -- start start 2024-04-20T21:40:01.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.317+0000 7fa456983740 1 Processor -- start 2024-04-20T21:40:01.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa456983740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 0x556dc1580670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa456983740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1573830 0x556dc1580bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa456983740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc157c410 0x556dc15f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa456983740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c1aed30 con 0x556dc157c410 2024-04-20T21:40:01.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa456983740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c1b01a0 con 0x556dc1573830 2024-04-20T21:40:01.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa456983740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c1af400 con 0x556dc1572f10 2024-04-20T21:40:01.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa456983740 10 client.0 _ll_register_callbacks cb 0x7fa4440093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:01.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa451c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 0x556dc1580670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa44be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1573830 0x556dc1580bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa451c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc157c410 0x556dc15f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa44be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1573830 0x556dc1580bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39156/0 (socket says 172.21.15.22:39156) 2024-04-20T21:40:01.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa44be00640 1 -- 172.21.15.22:0/3243052141 learned_addr learned my addr 172.21.15.22:0/3243052141 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:01.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa44be00640 1 -- 172.21.15.22:0/3243052141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 msgr2=0x556dc1580670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa44be00640 1 --2- 172.21.15.22:0/3243052141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 0x556dc1580670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa44be00640 1 -- 172.21.15.22:0/3243052141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc157c410 msgr2=0x556dc15f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa44be00640 1 --2- 172.21.15.22:0/3243052141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc157c410 0x556dc15f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa44be00640 1 -- 172.21.15.22:0/3243052141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dc15f1f00 con 0x556dc1573830 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.318+0000 7fa451c00640 1 --2- 172.21.15.22:0/3243052141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 0x556dc1580670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.319+0000 7fa44be00640 1 --2- 172.21.15.22:0/3243052141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1573830 0x556dc1580bb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa43402e010 tx=0x7fa434005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cbb7d4f7f53afab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.319+0000 7fa43aa00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.319+0000 7fa43aa00640 1 -- 172.21.15.22:0/3243052141 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43404b070 con 0x556dc1573830 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.319+0000 7fa43aa00640 1 -- 172.21.15.22:0/3243052141 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fa434047dc0 con 0x556dc1573830 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.319+0000 7fa456983740 1 -- 172.21.15.22:0/3243052141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa43c01e020 con 0x556dc1573830 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.319+0000 7fa43aa00640 1 -- 172.21.15.22:0/3243052141 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa434057050 con 0x556dc1573830 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.319+0000 7fa456983740 10 client.26300 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.319+0000 7fa456983740 1 -- 172.21.15.22:0/3243052141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556dc15f3170 con 0x556dc1573830 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.320+0000 7fa43aa00640 1 -- 172.21.15.22:0/3243052141 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa43c01e020 con 0x556dc1573830 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.320+0000 7fa43aa00640 1 -- 172.21.15.22:0/3243052141 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fa434005040 con 0x556dc1573830 2024-04-20T21:40:01.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.320+0000 7fa456983740 10 client.26300 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.320+0000 7fa456983740 10 client.26300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.320+0000 7fa456983740 20 client.26300 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.320+0000 7fa456983740 1 --2- 172.21.15.22:0/3243052141 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556dc15f38c0 0x556dc15f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.320+0000 7fa456983740 4 client.26300 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.320+0000 7fa456983740 1 -- 172.21.15.22:0/3243052141 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556dc14df5b0 con 0x556dc15f38c0 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.320+0000 7fa451c00640 1 --2- 172.21.15.22:0/3243052141 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556dc15f38c0 0x556dc15f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.321+0000 7fa451c00640 1 --2- 172.21.15.22:0/3243052141 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556dc15f38c0 0x556dc15f5ce0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f7874277c46a8c server_cookie=a654dcf6819cba8f in_seq=0 out_seq=0 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.321+0000 7fa43aa00640 10 client.26300 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.321+0000 7fa43aa00640 1 -- 172.21.15.22:0/3243052141 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556dc14df5b0 con 0x556dc15f38c0 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.321+0000 7fa43aa00640 10 client.26300 handle_command_reply: tid=0 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.321+0000 7fa456983740 1 client.26300 shutdown 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.322+0000 7fa456983740 1 -- 172.21.15.22:0/3243052141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1573830 msgr2=0x556dc1580bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.322+0000 7fa456983740 1 --2- 172.21.15.22:0/3243052141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1573830 0x556dc1580bb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa43402e010 tx=0x7fa434005b50 comp rx=0 tx=0).stop 2024-04-20T21:40:01.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.322+0000 7fa456983740 1 -- 172.21.15.22:0/3243052141 shutdown_connections 2024-04-20T21:40:01.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.322+0000 7fa456983740 1 --2- 172.21.15.22:0/3243052141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556dc157c410 0x556dc15f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.322+0000 7fa456983740 1 --2- 172.21.15.22:0/3243052141 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556dc15f38c0 0x556dc15f5ce0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.322+0000 7fa456983740 1 --2- 172.21.15.22:0/3243052141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dc1573830 0x556dc1580bb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.322+0000 7fa456983740 1 --2- 172.21.15.22:0/3243052141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556dc1572f10 0x556dc1580670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.322+0000 7fa43aa00640 0 client.26300 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.322+0000 7fa456983740 1 -- 172.21.15.22:0/3243052141 >> 172.21.15.22:0/3243052141 conn(0x556dc156d640 msgr2=0x556dc1579d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.322+0000 7fa456983740 1 -- 172.21.15.22:0/3243052141 shutdown_connections 2024-04-20T21:40:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.322+0000 7fa456983740 1 -- 172.21.15.22:0/3243052141 wait complete. 2024-04-20T21:40:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.322+0000 7fa456983740 20 client.26300 trim_cache size 0 max 16384 2024-04-20T21:40:01.324 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:40:01.324 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:40:01.324 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "a3274662-08f0-4a1d-aeff-507b6ef15904", 2024-04-20T21:40:01.324 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:40:01.325 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:40:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.323+0000 7fa453c00640 1 -- 172.21.15.22:0/2485564679 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa4300a3da0 msgr2=0x7fa4300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.323+0000 7fa453c00640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa4300a3da0 0x7fa4300a6260 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x556dc15729e0 tx=0x7fa44400cf90 comp rx=0 tx=0).stop 2024-04-20T21:40:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.323+0000 7fa453c00640 1 -- 172.21.15.22:0/2485564679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa44c104410 msgr2=0x7fa44c1a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.323+0000 7fa453c00640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa44c104410 0x7fa44c1a07a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa44400eb90 tx=0x7fa44400c6a0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.323+0000 7fa453c00640 1 -- 172.21.15.22:0/2485564679 shutdown_connections 2024-04-20T21:40:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.323+0000 7fa453c00640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa4300a3da0 0x7fa4300a6260 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.323+0000 7fa453c00640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c19a9a0 0x7fa44c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.323+0000 7fa453c00640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c1094e0 0x7fa44c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.323+0000 7fa453c00640 1 --2- 172.21.15.22:0/2485564679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa44c104410 0x7fa44c1a07a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.323+0000 7fa453c00640 1 -- 172.21.15.22:0/2485564679 >> 172.21.15.22:0/2485564679 conn(0x7fa44c0ffcf0 msgr2=0x7fa44c1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.324+0000 7fa453c00640 1 -- 172.21.15.22:0/2485564679 shutdown_connections 2024-04-20T21:40:01.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.324+0000 7fa453c00640 1 -- 172.21.15.22:0/2485564679 wait complete. 2024-04-20T21:40:01.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:40:01.343 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:40:01.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.561+0000 7f5cf4c00640 1 -- 172.21.15.22:0/801968817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf0104dc0 msgr2=0x7f5cf0108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.561+0000 7f5cf4c00640 1 --2- 172.21.15.22:0/801968817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf0104dc0 0x7f5cf0108e00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009900 tx=0x7f5cd802bc30 comp rx=0 tx=0).stop 2024-04-20T21:40:01.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.562+0000 7f5cf4c00640 1 -- 172.21.15.22:0/801968817 shutdown_connections 2024-04-20T21:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.562+0000 7f5cf4c00640 1 --2- 172.21.15.22:0/801968817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf01094e0 0x7f5cf01110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.562+0000 7f5cf4c00640 1 --2- 172.21.15.22:0/801968817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf0104dc0 0x7f5cf0108e00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.562+0000 7f5cf4c00640 1 --2- 172.21.15.22:0/801968817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cf0104410 0x7f5cf01047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.562+0000 7f5cf4c00640 1 -- 172.21.15.22:0/801968817 >> 172.21.15.22:0/801968817 conn(0x7f5cf00ffcf0 msgr2=0x7f5cf0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.562+0000 7f5cf4c00640 1 -- 172.21.15.22:0/801968817 shutdown_connections 2024-04-20T21:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.562+0000 7f5cf4c00640 1 -- 172.21.15.22:0/801968817 wait complete. 2024-04-20T21:40:01.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.564+0000 7f5cf4c00640 1 Processor -- start 2024-04-20T21:40:01.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.564+0000 7f5cf4c00640 1 -- start start 2024-04-20T21:40:01.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cf4c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf0104410 0x7f5cf01a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cf4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cf01094e0 0x7f5cf01a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cf4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf019a9d0 0x7f5cf019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cf4c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf0114570 con 0x7f5cf01094e0 2024-04-20T21:40:01.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cf4c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01143f0 con 0x7f5cf0104410 2024-04-20T21:40:01.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cf4c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01146f0 con 0x7f5cf019a9d0 2024-04-20T21:40:01.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cede00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf0104410 0x7f5cf01a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5ce5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cf01094e0 0x7f5cf01a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cede00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf0104410 0x7f5cf01a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39174/0 (socket says 172.21.15.22:39174) 2024-04-20T21:40:01.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cede00640 1 -- 172.21.15.22:0/3300221631 learned_addr learned my addr 172.21.15.22:0/3300221631 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:01.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cee800640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf019a9d0 0x7f5cf019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cede00640 1 -- 172.21.15.22:0/3300221631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf019a9d0 msgr2=0x7f5cf019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cede00640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf019a9d0 0x7f5cf019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cede00640 1 -- 172.21.15.22:0/3300221631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cf01094e0 msgr2=0x7f5cf01a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cede00640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cf01094e0 0x7f5cf01a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.565+0000 7f5cede00640 1 -- 172.21.15.22:0/3300221631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf019b4f0 con 0x7f5cf0104410 2024-04-20T21:40:01.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.566+0000 7f5cede00640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf0104410 0x7f5cf01a07d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000ebc0 tx=0x7f5ce000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.566+0000 7f5ce7400640 1 -- 172.21.15.22:0/3300221631 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0010630 con 0x7f5cf0104410 2024-04-20T21:40:01.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.566+0000 7f5ce7400640 1 -- 172.21.15.22:0/3300221631 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce0010cb0 con 0x7f5cf0104410 2024-04-20T21:40:01.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.566+0000 7f5cf4c00640 1 -- 172.21.15.22:0/3300221631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf019b7e0 con 0x7f5cf0104410 2024-04-20T21:40:01.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.566+0000 7f5ce7400640 1 -- 172.21.15.22:0/3300221631 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0018bd0 con 0x7f5cf0104410 2024-04-20T21:40:01.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.566+0000 7f5cf4c00640 1 -- 172.21.15.22:0/3300221631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf01a7a30 con 0x7f5cf0104410 2024-04-20T21:40:01.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.570+0000 7f5ce7400640 1 -- 172.21.15.22:0/3300221631 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5ce00173d0 con 0x7f5cf0104410 2024-04-20T21:40:01.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.571+0000 7f5cf797f740 1 Processor -- start 2024-04-20T21:40:01.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.571+0000 7f5cf797f740 1 -- start start 2024-04-20T21:40:01.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.571+0000 7f5cf797f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9adfe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.571+0000 7f5cf797f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9adfeac0 0x560b9ae07160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.571+0000 7f5cf797f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9ae076a0 0x560b9ae09a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.571+0000 7f5cf797f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560b9ad64150 con 0x560b9adfeac0 2024-04-20T21:40:01.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.571+0000 7f5cf797f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b9ad0a370 con 0x560b9adfe1a0 2024-04-20T21:40:01.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.571+0000 7f5cf797f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560b9ad60110 con 0x560b9ae076a0 2024-04-20T21:40:01.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5cee800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9adfe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5cee800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9adfe580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39182/0 (socket says 172.21.15.22:39182) 2024-04-20T21:40:01.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5cee800640 1 -- 172.21.15.22:0/3985161869 learned_addr learned my addr 172.21.15.22:0/3985161869 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:01.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5cede00640 1 --2- 172.21.15.22:0/3985161869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9adfeac0 0x560b9ae07160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5ce5400640 1 --2- 172.21.15.22:0/3985161869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9ae076a0 0x560b9ae09a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5cee800640 1 -- 172.21.15.22:0/3985161869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9ae076a0 msgr2=0x560b9ae09a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5cee800640 1 --2- 172.21.15.22:0/3985161869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9ae076a0 0x560b9ae09a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5cee800640 1 -- 172.21.15.22:0/3985161869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9adfeac0 msgr2=0x560b9ae07160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5cee800640 1 --2- 172.21.15.22:0/3985161869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9adfeac0 0x560b9ae07160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5cee800640 1 -- 172.21.15.22:0/3985161869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9ab407d0 con 0x560b9adfe1a0 2024-04-20T21:40:01.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5ce5400640 1 --2- 172.21.15.22:0/3985161869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9ae076a0 0x560b9ae09a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5cede00640 1 --2- 172.21.15.22:0/3985161869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9adfeac0 0x560b9ae07160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5ce7400640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5cd40a3da0 0x7f5cd40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.572+0000 7f5cee800640 1 --2- 172.21.15.22:0/3985161869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9adfe580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009c20 tx=0x7f5ce8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70333ffe289134c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.573+0000 7f5ce7400640 1 -- 172.21.15.22:0/3300221631 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5ce00d0d90 con 0x7f5cf0104410 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.573+0000 7f5cede00640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5cd40a3da0 0x7f5cd40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.573+0000 7f5cdf000640 1 -- 172.21.15.22:0/3985161869 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce801e070 con 0x560b9adfe1a0 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.573+0000 7f5cdf000640 1 -- 172.21.15.22:0/3985161869 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce8019b10 con 0x560b9adfe1a0 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.573+0000 7f5cdf000640 1 -- 172.21.15.22:0/3985161869 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800eae0 con 0x560b9adfe1a0 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.574+0000 7f5cf797f740 1 -- 172.21.15.22:0/3985161869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 msgr2=0x560b9adfe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.574+0000 7f5cede00640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5cd40a3da0 0x7f5cd40a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x560b9adfdc70 tx=0x7f5ce000cf90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.574+0000 7f5cf797f740 1 --2- 172.21.15.22:0/3985161869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9adfe580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009c20 tx=0x7f5ce8011810 comp rx=0 tx=0).stop 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.574+0000 7f5cf797f740 1 -- 172.21.15.22:0/3985161869 shutdown_connections 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.574+0000 7f5cf797f740 1 --2- 172.21.15.22:0/3985161869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9ae076a0 0x560b9ae09a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.574+0000 7f5cf797f740 1 --2- 172.21.15.22:0/3985161869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9adfeac0 0x560b9ae07160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.574+0000 7f5cf797f740 1 --2- 172.21.15.22:0/3985161869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9adfe580 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.574+0000 7f5cf797f740 1 -- 172.21.15.22:0/3985161869 >> 172.21.15.22:0/3985161869 conn(0x560b9adf88d0 msgr2=0x560b9adf8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.575+0000 7f5cf797f740 1 -- 172.21.15.22:0/3985161869 shutdown_connections 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.575+0000 7f5cf797f740 1 -- 172.21.15.22:0/3985161869 wait complete. 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.575+0000 7f5cf797f740 1 -- start start 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.575+0000 7f5cf797f740 1 Processor -- start 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.576+0000 7f5cf797f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9ae04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.576+0000 7f5cf797f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 0x560b9ae04ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.576+0000 7f5cf797f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.576+0000 7f5cf797f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560b9ae0a490 con 0x560b9ae076a0 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.576+0000 7f5cf797f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b9ad1d740 con 0x560b9adfe1a0 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.576+0000 7f5cf797f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560b9ae0a610 con 0x560b9adfeac0 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.576+0000 7f5cf797f740 10 client.0 _ll_register_callbacks cb 0x560b9ad18b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.576+0000 7f5ce5400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 0x560b9ae04ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.576+0000 7f5cee800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9ae04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.576+0000 7f5ce5400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 0x560b9ae04ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42156/0 (socket says 172.21.15.22:42156) 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.576+0000 7f5ce5400640 1 -- 172.21.15.22:0/1483966963 learned_addr learned my addr 172.21.15.22:0/1483966963 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.576+0000 7f5cee800640 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.577+0000 7f5ce5400640 1 -- 172.21.15.22:0/1483966963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 msgr2=0x560b9ae04760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.577+0000 7f5ce5400640 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9ae04760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.577+0000 7f5ce5400640 1 -- 172.21.15.22:0/1483966963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 msgr2=0x560b9ae051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.577+0000 7f5ce5400640 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.577+0000 7f5ce5400640 1 -- 172.21.15.22:0/1483966963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9aa763f0 con 0x560b9adfeac0 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.577+0000 7f5cee800640 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.577+0000 7f5ce5400640 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 0x560b9ae04ca0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5cf006b6d0 tx=0x7f5cd802ce10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c058f39c9e62d687 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.577+0000 7f5cd3400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.578+0000 7f5cd3400640 1 -- 172.21.15.22:0/1483966963 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8004240 con 0x560b9adfeac0 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.578+0000 7f5cd3400640 1 -- 172.21.15.22:0/1483966963 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5cd80043e0 con 0x560b9adfeac0 2024-04-20T21:40:01.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.578+0000 7f5cf797f740 1 -- 172.21.15.22:0/1483966963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce8019b10 con 0x560b9adfeac0 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.578+0000 7f5cd3400640 1 -- 172.21.15.22:0/1483966963 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5cd80046c0 con 0x560b9adfeac0 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.578+0000 7f5cf797f740 10 client.26332 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.578+0000 7f5cf797f740 1 -- 172.21.15.22:0/1483966963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560b9aa360f0 con 0x560b9adfeac0 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.578+0000 7f5cd3400640 1 -- 172.21.15.22:0/1483966963 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce8019b10 con 0x560b9adfeac0 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.579+0000 7f5cd3400640 1 -- 172.21.15.22:0/1483966963 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f5cd80035f0 con 0x560b9adfeac0 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.579+0000 7f5cf797f740 10 client.26332 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.579+0000 7f5cf797f740 10 client.26332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.579+0000 7f5cf797f740 20 client.26332 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.579+0000 7f5cf797f740 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560b9ae17b00 0x560b9ae19f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.579+0000 7f5cf797f740 4 client.26332 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.579+0000 7f5cf797f740 1 -- 172.21.15.22:0/1483966963 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b9ad71b70 con 0x560b9ae17b00 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.579+0000 7f5cee800640 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560b9ae17b00 0x560b9ae19f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.580+0000 7f5cee800640 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560b9ae17b00 0x560b9ae19f20 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b7386e1cd900f6 server_cookie=2727720200bea60a in_seq=0 out_seq=0 2024-04-20T21:40:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.580+0000 7f5cd3400640 10 client.26332 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:01.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cd3400640 1 -- 172.21.15.22:0/1483966963 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b9ad71b70 con 0x560b9ae17b00 2024-04-20T21:40:01.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cd3400640 10 client.26332 handle_command_reply: tid=0 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cf797f740 1 client.26332 shutdown 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cf797f740 1 -- 172.21.15.22:0/1483966963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 msgr2=0x560b9ae04ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cf797f740 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 0x560b9ae04ca0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5cf006b6d0 tx=0x7f5cd802ce10 comp rx=0 tx=0).stop 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cf797f740 1 -- 172.21.15.22:0/1483966963 shutdown_connections 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cf797f740 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cf797f740 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 0x560b9ae04ca0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cf797f740 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560b9ae17b00 0x560b9ae19f20 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cf797f740 1 --2- 172.21.15.22:0/1483966963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9ae04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cd3400640 0 client.26332 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cf797f740 1 -- 172.21.15.22:0/1483966963 >> 172.21.15.22:0/1483966963 conn(0x560b9adf88d0 msgr2=0x560b9adfa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cf797f740 1 -- 172.21.15.22:0/1483966963 shutdown_connections 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.581+0000 7f5cf797f740 1 -- 172.21.15.22:0/1483966963 wait complete. 2024-04-20T21:40:01.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.582+0000 7f5cf797f740 20 client.26332 trim_cache size 0 max 16384 2024-04-20T21:40:01.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.602+0000 7f5cf797f740 1 Processor -- start 2024-04-20T21:40:01.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.602+0000 7f5cf797f740 1 -- start start 2024-04-20T21:40:01.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.603+0000 7f5cf797f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9ae06e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.603+0000 7f5cf797f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 0x560b9ae13350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.603+0000 7f5cf797f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.603+0000 7f5cf797f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01ae090 con 0x560b9ae076a0 2024-04-20T21:40:01.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.603+0000 7f5cf797f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01b0910 con 0x560b9adfe1a0 2024-04-20T21:40:01.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.603+0000 7f5cf797f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01ae700 con 0x560b9adfeac0 2024-04-20T21:40:01.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.603+0000 7f5cee800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9ae06e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.603+0000 7f5ce5400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 0x560b9ae13350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.603+0000 7f5ce5400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 0x560b9ae13350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42164/0 (socket says 172.21.15.22:42164) 2024-04-20T21:40:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.603+0000 7f5ce5400640 1 -- 172.21.15.22:0/2013446282 learned_addr learned my addr 172.21.15.22:0/2013446282 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.603+0000 7f5cee800640 1 --2- 172.21.15.22:0/2013446282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.604+0000 7f5ce5400640 1 -- 172.21.15.22:0/2013446282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 msgr2=0x560b9ae06e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.604+0000 7f5ce5400640 1 --2- 172.21.15.22:0/2013446282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9ae06e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.604+0000 7f5ce5400640 1 -- 172.21.15.22:0/2013446282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 msgr2=0x560b9ae036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.604+0000 7f5ce5400640 1 --2- 172.21.15.22:0/2013446282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.604+0000 7f5ce5400640 1 -- 172.21.15.22:0/2013446282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9ae09a90 con 0x560b9adfeac0 2024-04-20T21:40:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.604+0000 7f5ce5400640 1 --2- 172.21.15.22:0/2013446282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 0x560b9ae13350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80096c0 tx=0x7f5cd8005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cedc9f60b157d812 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.604+0000 7f5cde600640 1 -- 172.21.15.22:0/2013446282 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8004860 con 0x560b9adfeac0 2024-04-20T21:40:01.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.604+0000 7f5cde600640 1 -- 172.21.15.22:0/2013446282 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd80053a0 con 0x560b9adfeac0 2024-04-20T21:40:01.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.604+0000 7f5cde600640 1 -- 172.21.15.22:0/2013446282 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8003e90 con 0x560b9adfeac0 2024-04-20T21:40:01.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.605+0000 7f5cf797f740 1 -- 172.21.15.22:0/2013446282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 msgr2=0x560b9ae13350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.605+0000 7f5cf797f740 1 --2- 172.21.15.22:0/2013446282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 0x560b9ae13350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80096c0 tx=0x7f5cd8005850 comp rx=0 tx=0).stop 2024-04-20T21:40:01.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.605+0000 7f5cf797f740 1 -- 172.21.15.22:0/2013446282 shutdown_connections 2024-04-20T21:40:01.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.605+0000 7f5cf797f740 1 --2- 172.21.15.22:0/2013446282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.605+0000 7f5cf797f740 1 --2- 172.21.15.22:0/2013446282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfeac0 0x560b9ae13350 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.605+0000 7f5cf797f740 1 --2- 172.21.15.22:0/2013446282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfe1a0 0x560b9ae06e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.605+0000 7f5cf797f740 1 -- 172.21.15.22:0/2013446282 >> 172.21.15.22:0/2013446282 conn(0x560b9adf88d0 msgr2=0x560b9ae04fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.605+0000 7f5cf797f740 1 -- 172.21.15.22:0/2013446282 shutdown_connections 2024-04-20T21:40:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.605+0000 7f5cf797f740 1 -- 172.21.15.22:0/2013446282 wait complete. 2024-04-20T21:40:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.606+0000 7f5cf797f740 1 -- start start 2024-04-20T21:40:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.606+0000 7f5cf797f740 1 Processor -- start 2024-04-20T21:40:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.606+0000 7f5cf797f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfe1a0 0x560b9ae0b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.606+0000 7f5cf797f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfeac0 0x560b9ae0b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.606+0000 7f5cf797f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae0be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.606+0000 7f5cf797f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01aedd0 con 0x560b9ae076a0 2024-04-20T21:40:01.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.606+0000 7f5cf797f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01b0240 con 0x560b9adfeac0 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.606+0000 7f5cf797f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01af4a0 con 0x560b9adfe1a0 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.606+0000 7f5cf797f740 10 client.0 _ll_register_callbacks cb 0x7f5ce8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5ce5400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfeac0 0x560b9ae0b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5ce5400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfeac0 0x560b9ae0b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:39210/0 (socket says 172.21.15.22:39210) 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5ce5400640 1 -- 172.21.15.22:0/2360878667 learned_addr learned my addr 172.21.15.22:0/2360878667 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5cee800640 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfe1a0 0x560b9ae0b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5cee800640 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae0be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5ce5400640 1 -- 172.21.15.22:0/2360878667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfe1a0 msgr2=0x560b9ae0b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5ce5400640 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfe1a0 0x560b9ae0b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5ce5400640 1 -- 172.21.15.22:0/2360878667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 msgr2=0x560b9ae0be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5ce5400640 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae0be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5ce5400640 1 -- 172.21.15.22:0/2360878667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9ae7d210 con 0x560b9adfeac0 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5cee800640 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae0be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5ce5400640 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfeac0 0x560b9ae0b940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5cd802c170 tx=0x7f5cd8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc54d0d657a06e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:01.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.607+0000 7f5cd3400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.608+0000 7f5cd3400640 1 -- 172.21.15.22:0/2360878667 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8005550 con 0x560b9adfeac0 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.608+0000 7f5cd3400640 1 -- 172.21.15.22:0/2360878667 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5cd8004ec0 con 0x560b9adfeac0 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.608+0000 7f5cf797f740 1 -- 172.21.15.22:0/2360878667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd80053a0 con 0x560b9adfeac0 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.608+0000 7f5cd3400640 1 -- 172.21.15.22:0/2360878667 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5cd8067050 con 0x560b9adfeac0 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.608+0000 7f5cf797f740 10 client.26318 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.608+0000 7f5cf797f740 1 -- 172.21.15.22:0/2360878667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560b9ae7e4e0 con 0x560b9adfeac0 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.608+0000 7f5cd3400640 1 -- 172.21.15.22:0/2360878667 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd80053a0 con 0x560b9adfeac0 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.608+0000 7f5cd3400640 1 -- 172.21.15.22:0/2360878667 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f5cd8060060 con 0x560b9adfeac0 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.609+0000 7f5cf797f740 10 client.26318 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.609+0000 7f5cf797f740 10 client.26318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.609+0000 7f5cf797f740 20 client.26318 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.609+0000 7f5cf797f740 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560b9ae7e8c0 0x560b9ae80cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.609+0000 7f5cf797f740 4 client.26318 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:40:01.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.609+0000 7f5cf797f740 1 -- 172.21.15.22:0/2360878667 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b9ad6b9e0 con 0x560b9ae7e8c0 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.609+0000 7f5cee800640 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560b9ae7e8c0 0x560b9ae80cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.609+0000 7f5cee800640 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560b9ae7e8c0 0x560b9ae80cb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28495ddd303de581 server_cookie=6ec46d4e88b87e0e in_seq=0 out_seq=0 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.609+0000 7f5cd3400640 10 client.26318 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cd3400640 1 -- 172.21.15.22:0/2360878667 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560b9ad6b9e0 con 0x560b9ae7e8c0 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cd3400640 10 client.26318 handle_command_reply: tid=0 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cf797f740 1 client.26318 shutdown 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cf797f740 1 -- 172.21.15.22:0/2360878667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfeac0 msgr2=0x560b9ae0b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cf797f740 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfeac0 0x560b9ae0b940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5cd802c170 tx=0x7f5cd8005fd0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cf797f740 1 -- 172.21.15.22:0/2360878667 shutdown_connections 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cf797f740 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b9ae076a0 0x560b9ae0be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cf797f740 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x560b9ae7e8c0 0x560b9ae80cb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cf797f740 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b9adfeac0 0x560b9ae0b940 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cf797f740 1 --2- 172.21.15.22:0/2360878667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9adfe1a0 0x560b9ae0b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cd3400640 0 client.26318 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:01.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cf797f740 1 -- 172.21.15.22:0/2360878667 >> 172.21.15.22:0/2360878667 conn(0x560b9adf88d0 msgr2=0x560b9adf7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cf797f740 1 -- 172.21.15.22:0/2360878667 shutdown_connections 2024-04-20T21:40:01.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cf797f740 1 -- 172.21.15.22:0/2360878667 wait complete. 2024-04-20T21:40:01.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.610+0000 7f5cf797f740 20 client.26318 trim_cache size 0 max 16384 2024-04-20T21:40:01.614 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:40:01.614 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:40:01.614 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:40:01.614 INFO:teuthology.orchestra.run.smithi022.stdout: "a3274662-08f0-4a1d-aeff-507b6ef15904": { 2024-04-20T21:40:01.614 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:40:01.615 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "a3274662-08f0-4a1d-aeff-507b6ef15904", 2024-04-20T21:40:01.615 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:40:01.615 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:40:01.615 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:40:01.615 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:40:01.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.611+0000 7f5cf4c00640 1 -- 172.21.15.22:0/3300221631 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5cd40a3da0 msgr2=0x7f5cd40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.611+0000 7f5cf4c00640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5cd40a3da0 0x7f5cd40a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x560b9adfdc70 tx=0x7f5ce000cf90 comp rx=0 tx=0).stop 2024-04-20T21:40:01.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.611+0000 7f5cf4c00640 1 -- 172.21.15.22:0/3300221631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf0104410 msgr2=0x7f5cf01a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:01.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.611+0000 7f5cf4c00640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf0104410 0x7f5cf01a07d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000ebc0 tx=0x7f5ce000c6a0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.611+0000 7f5cf4c00640 1 -- 172.21.15.22:0/3300221631 shutdown_connections 2024-04-20T21:40:01.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.611+0000 7f5cf4c00640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5cd40a3da0 0x7f5cd40a6260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.611+0000 7f5cf4c00640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf019a9d0 0x7f5cf019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.611+0000 7f5cf4c00640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cf01094e0 0x7f5cf01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.611+0000 7f5cf4c00640 1 --2- 172.21.15.22:0/3300221631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf0104410 0x7f5cf01a07d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:01.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.611+0000 7f5cf4c00640 1 -- 172.21.15.22:0/3300221631 >> 172.21.15.22:0/3300221631 conn(0x7f5cf00ffcf0 msgr2=0x7f5cf0101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:01.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.611+0000 7f5cf4c00640 1 -- 172.21.15.22:0/3300221631 shutdown_connections 2024-04-20T21:40:01.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:01.611+0000 7f5cf4c00640 1 -- 172.21.15.22:0/3300221631 wait complete. 2024-04-20T21:40:01.629 INFO:tasks.cephfs.filesystem:scrub status for tag:a3274662-08f0-4a1d-aeff-507b6ef15904 - {'path': '/', 'tag': 'a3274662-08f0-4a1d-aeff-507b6ef15904', 'options': 'recursive,force'} 2024-04-20T21:40:01.692 INFO:tasks.workunit.client.0.smithi022.stdout:[492572] write failed on handle 12978 (Resource temporarily unavailable) 2024-04-20T21:40:01.692 INFO:tasks.workunit.client.0.smithi022.stdout:Child failed with status 1 2024-04-20T21:40:01.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:40:01.695 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-20T21:40:01.695 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-20T21:40:02.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:02 smithi022 ceph-mon[16843]: pgmap v189: 97 pgs: 97 active+clean; 287 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 212 op/s 2024-04-20T21:40:02.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:02 smithi122 ceph-mon[22147]: pgmap v189: 97 pgs: 97 active+clean; 287 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 212 op/s 2024-04-20T21:40:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:02 smithi033 ceph-mon[21626]: pgmap v189: 97 pgs: 97 active+clean; 287 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 212 op/s 2024-04-20T21:40:02.829 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi022 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-20T21:40:03.066 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7e9ec60b3de24c11955d7d0e6bf852e1 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi022 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-20T21:40:03.069 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-20T21:40:03.079 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-20T21:40:03.080 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T21:40:03.080 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-04-20T21:40:03.080 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-20T21:40:03.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-20T21:40:03.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:03 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:40:03.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:03 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:03.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:03 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:40:03.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:03 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:03.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:03 smithi022 ceph-mon[16843]: osdmap e94: 12 total, 12 up, 12 in 2024-04-20T21:40:03.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:03 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:40:03.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:03 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:03.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:03 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:40:03.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:03 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:03.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:03 smithi122 ceph-mon[22147]: osdmap e94: 12 total, 12 up, 12 in 2024-04-20T21:40:03.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:03 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:40:03.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:03 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:03.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:03 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:40:03.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:03 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:03.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:03 smithi033 ceph-mon[21626]: osdmap e94: 12 total, 12 up, 12 in 2024-04-20T21:40:04.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:04 smithi022 ceph-mon[16843]: pgmap v191: 97 pgs: 97 active+clean; 287 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 172 op/s 2024-04-20T21:40:04.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:04 smithi122 ceph-mon[22147]: pgmap v191: 97 pgs: 97 active+clean; 287 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 172 op/s 2024-04-20T21:40:04.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:04 smithi033 ceph-mon[21626]: pgmap v191: 97 pgs: 97 active+clean; 287 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 172 op/s 2024-04-20T21:40:06.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:06 smithi022 ceph-mon[16843]: pgmap v192: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-20T21:40:06.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:06 smithi122 ceph-mon[22147]: pgmap v192: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-20T21:40:06.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:06 smithi033 ceph-mon[21626]: pgmap v192: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-20T21:40:07.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:07 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:40:07.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:07 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:40:07.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:07 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:40:08.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:08 smithi122 ceph-mon[22147]: pgmap v193: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-20T21:40:08.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:08 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:40:08.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:08 smithi033 ceph-mon[21626]: pgmap v193: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-20T21:40:08.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:08 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:40:08.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:08 smithi022 ceph-mon[16843]: pgmap v193: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-20T21:40:08.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:08 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:40:09.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:09 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:40:09.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:09 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:40:09.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:09 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:40:09.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:09 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:40:09.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:09 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:40:09.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:09 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:40:09.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:09 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:40:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:09 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:40:09.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:09 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:40:10.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:10 smithi122 ceph-mon[22147]: pgmap v194: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-20T21:40:10.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:10 smithi122 ceph-mon[22147]: osdmap e95: 12 total, 12 up, 12 in 2024-04-20T21:40:10.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:10 smithi033 ceph-mon[21626]: pgmap v194: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-20T21:40:10.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:10 smithi033 ceph-mon[21626]: osdmap e95: 12 total, 12 up, 12 in 2024-04-20T21:40:10.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:10 smithi022 ceph-mon[16843]: pgmap v194: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-20T21:40:10.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:10 smithi022 ceph-mon[16843]: osdmap e95: 12 total, 12 up, 12 in 2024-04-20T21:40:12.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:12 smithi122 ceph-mon[22147]: pgmap v196: 97 pgs: 97 active+clean; 295 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 2.0 MiB/s wr, 116 op/s 2024-04-20T21:40:12.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:12 smithi033 ceph-mon[21626]: pgmap v196: 97 pgs: 97 active+clean; 295 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 2.0 MiB/s wr, 116 op/s 2024-04-20T21:40:12.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:12 smithi022 ceph-mon[16843]: pgmap v196: 97 pgs: 97 active+clean; 295 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 2.0 MiB/s wr, 116 op/s 2024-04-20T21:40:14.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:14 smithi122 ceph-mon[22147]: pgmap v197: 97 pgs: 97 active+clean; 295 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.8 MiB/s wr, 103 op/s 2024-04-20T21:40:14.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:14 smithi033 ceph-mon[21626]: pgmap v197: 97 pgs: 97 active+clean; 295 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.8 MiB/s wr, 103 op/s 2024-04-20T21:40:14.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:14 smithi022 ceph-mon[16843]: pgmap v197: 97 pgs: 97 active+clean; 295 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.8 MiB/s wr, 103 op/s 2024-04-20T21:40:16.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:16 smithi122 ceph-mon[22147]: pgmap v198: 97 pgs: 97 active+clean; 312 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.1 MiB/s wr, 58 op/s 2024-04-20T21:40:16.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:16 smithi033 ceph-mon[21626]: pgmap v198: 97 pgs: 97 active+clean; 312 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.1 MiB/s wr, 58 op/s 2024-04-20T21:40:16.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:16 smithi022 ceph-mon[16843]: pgmap v198: 97 pgs: 97 active+clean; 312 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.1 MiB/s wr, 58 op/s 2024-04-20T21:40:18.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:18 smithi122 ceph-mon[22147]: pgmap v199: 97 pgs: 97 active+clean; 312 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.1 MiB/s wr, 58 op/s 2024-04-20T21:40:18.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:18 smithi033 ceph-mon[21626]: pgmap v199: 97 pgs: 97 active+clean; 312 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.1 MiB/s wr, 58 op/s 2024-04-20T21:40:18.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:18 smithi022 ceph-mon[16843]: pgmap v199: 97 pgs: 97 active+clean; 312 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.1 MiB/s wr, 58 op/s 2024-04-20T21:40:20.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:20 smithi122 ceph-mon[22147]: pgmap v200: 97 pgs: 97 active+clean; 312 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.1 MiB/s wr, 58 op/s 2024-04-20T21:40:20.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:20 smithi033 ceph-mon[21626]: pgmap v200: 97 pgs: 97 active+clean; 312 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.1 MiB/s wr, 58 op/s 2024-04-20T21:40:20.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:20 smithi022 ceph-mon[16843]: pgmap v200: 97 pgs: 97 active+clean; 312 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.1 MiB/s wr, 58 op/s 2024-04-20T21:40:22.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:22 smithi122 ceph-mon[22147]: pgmap v201: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 3.3 MiB/s wr, 71 op/s 2024-04-20T21:40:22.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:22 smithi033 ceph-mon[21626]: pgmap v201: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 3.3 MiB/s wr, 71 op/s 2024-04-20T21:40:22.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:22 smithi022 ceph-mon[16843]: pgmap v201: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 3.3 MiB/s wr, 71 op/s 2024-04-20T21:40:24.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:24 smithi122 ceph-mon[22147]: pgmap v202: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 2.7 MiB/s wr, 47 op/s 2024-04-20T21:40:24.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:24 smithi033 ceph-mon[21626]: pgmap v202: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 2.7 MiB/s wr, 47 op/s 2024-04-20T21:40:24.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:24 smithi022 ceph-mon[16843]: pgmap v202: 97 pgs: 97 active+clean; 310 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 2.7 MiB/s wr, 47 op/s 2024-04-20T21:40:26.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:26 smithi122 ceph-mon[22147]: pgmap v203: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 3.2 MiB/s wr, 66 op/s 2024-04-20T21:40:26.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:26 smithi033 ceph-mon[21626]: pgmap v203: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 3.2 MiB/s wr, 66 op/s 2024-04-20T21:40:26.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:26 smithi022 ceph-mon[16843]: pgmap v203: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 3.2 MiB/s wr, 66 op/s 2024-04-20T21:40:28.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:28 smithi122 ceph-mon[22147]: pgmap v204: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.1 MiB/s wr, 41 op/s 2024-04-20T21:40:28.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:28 smithi033 ceph-mon[21626]: pgmap v204: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.1 MiB/s wr, 41 op/s 2024-04-20T21:40:28.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:28 smithi022 ceph-mon[16843]: pgmap v204: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.1 MiB/s wr, 41 op/s 2024-04-20T21:40:30.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:30 smithi122 ceph-mon[22147]: pgmap v205: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.1 MiB/s wr, 41 op/s 2024-04-20T21:40:30.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:30 smithi033 ceph-mon[21626]: pgmap v205: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.1 MiB/s wr, 41 op/s 2024-04-20T21:40:30.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:30 smithi022 ceph-mon[16843]: pgmap v205: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.1 MiB/s wr, 41 op/s 2024-04-20T21:40:31.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:40:31.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:40:31.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.844+0000 7f0766400640 1 -- 172.21.15.22:0/561401931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0760108590 msgr2=0x7f0760108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:31.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.844+0000 7f0766400640 1 --2- 172.21.15.22:0/561401931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0760108590 0x7f0760108a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f074c009930 tx=0x7f074c02bc20 comp rx=0 tx=0).stop 2024-04-20T21:40:31.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.844+0000 7f0766400640 1 -- 172.21.15.22:0/561401931 shutdown_connections 2024-04-20T21:40:31.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.844+0000 7f0766400640 1 --2- 172.21.15.22:0/561401931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0760108f50 0x7f07601153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.844+0000 7f0766400640 1 --2- 172.21.15.22:0/561401931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0760108590 0x7f0760108a10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.844+0000 7f0766400640 1 --2- 172.21.15.22:0/561401931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f076010e600 0x7f076010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.844+0000 7f0766400640 1 -- 172.21.15.22:0/561401931 >> 172.21.15.22:0/561401931 conn(0x7f07600764a0 msgr2=0x7f07600788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:31.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.844+0000 7f0766400640 1 -- 172.21.15.22:0/561401931 shutdown_connections 2024-04-20T21:40:31.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.844+0000 7f0766400640 1 -- 172.21.15.22:0/561401931 wait complete. 2024-04-20T21:40:31.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.845+0000 7f0766400640 1 Processor -- start 2024-04-20T21:40:31.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.845+0000 7f0766400640 1 -- start start 2024-04-20T21:40:31.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.846+0000 7f0766400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0760108590 0x7f076019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.846+0000 7f0766400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0760108f50 0x7f076019f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.846+0000 7f0766400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f076010e600 0x7f07601a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.846+0000 7f0766400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0760118910 con 0x7f0760108590 2024-04-20T21:40:31.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.846+0000 7f0766400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0760118790 con 0x7f076010e600 2024-04-20T21:40:31.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.846+0000 7f0766400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0760118a90 con 0x7f0760108f50 2024-04-20T21:40:31.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.846+0000 7f075f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0760108590 0x7f076019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.846+0000 7f075ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0760108f50 0x7f076019f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.846+0000 7f075f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0760108590 0x7f076019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53074/0 (socket says 172.21.15.22:53074) 2024-04-20T21:40:31.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.846+0000 7f075f400640 1 -- 172.21.15.22:0/330113160 learned_addr learned my addr 172.21.15.22:0/330113160 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:31.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.846+0000 7f075ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0760108f50 0x7f076019f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42422/0 (socket says 172.21.15.22:42422) 2024-04-20T21:40:31.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.846+0000 7f075fe00640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f076010e600 0x7f07601a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.847+0000 7f075ea00640 1 -- 172.21.15.22:0/330113160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f076010e600 msgr2=0x7f07601a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:31.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.847+0000 7f075ea00640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f076010e600 0x7f07601a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.847+0000 7f075ea00640 1 -- 172.21.15.22:0/330113160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0760108590 msgr2=0x7f076019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:31.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.847+0000 7f075ea00640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0760108590 0x7f076019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.847+0000 7f075ea00640 1 -- 172.21.15.22:0/330113160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07601a3e00 con 0x7f0760108f50 2024-04-20T21:40:31.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.847+0000 7f075f400640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0760108590 0x7f076019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:31.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.847+0000 7f075ea00640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0760108f50 0x7f076019f350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f074c036040 tx=0x7f074c004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:31.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.847+0000 7f0757e00640 1 -- 172.21.15.22:0/330113160 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c043020 con 0x7f0760108f50 2024-04-20T21:40:31.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.847+0000 7f0757e00640 1 -- 172.21.15.22:0/330113160 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074c005000 con 0x7f0760108f50 2024-04-20T21:40:31.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.847+0000 7f0757e00640 1 -- 172.21.15.22:0/330113160 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c0039e0 con 0x7f0760108f50 2024-04-20T21:40:31.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.847+0000 7f0766400640 1 -- 172.21.15.22:0/330113160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07601a4090 con 0x7f0760108f50 2024-04-20T21:40:31.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.848+0000 7f0766400640 1 -- 172.21.15.22:0/330113160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07601ab980 con 0x7f0760108f50 2024-04-20T21:40:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.852+0000 7f076902a740 1 Processor -- start 2024-04-20T21:40:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.852+0000 7f076902a740 1 -- start start 2024-04-20T21:40:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f076902a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbdfb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f076902a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe04160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f076902a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbe046a0 0x55e8bbe06a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f0757e00640 1 -- 172.21.15.22:0/330113160 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f074c046020 con 0x7f0760108f50 2024-04-20T21:40:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f076902a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e8bbd61150 con 0x55e8bbe046a0 2024-04-20T21:40:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f076902a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e8bbd07370 con 0x55e8bbdfbac0 2024-04-20T21:40:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f076902a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e8bbd5d110 con 0x55e8bbdfb1a0 2024-04-20T21:40:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f075fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbdfb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f075f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe04160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f075fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbdfb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42432/0 (socket says 172.21.15.22:42432) 2024-04-20T21:40:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f075fe00640 1 -- 172.21.15.22:0/738157927 learned_addr learned my addr 172.21.15.22:0/738157927 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f075ea00640 1 --2- 172.21.15.22:0/738157927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbe046a0 0x55e8bbe06a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f075f400640 1 -- 172.21.15.22:0/738157927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 msgr2=0x55e8bbdfb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f075f400640 1 --2- 172.21.15.22:0/738157927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbdfb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f075f400640 1 -- 172.21.15.22:0/738157927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbe046a0 msgr2=0x55e8bbe06a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f075f400640 1 --2- 172.21.15.22:0/738157927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbe046a0 0x55e8bbe06a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.853+0000 7f075f400640 1 -- 172.21.15.22:0/738157927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8bbb3d7d0 con 0x55e8bbdfbac0 2024-04-20T21:40:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.854+0000 7f075f400640 1 --2- 172.21.15.22:0/738157927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe04160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f07500114f0 tx=0x7f07500119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=136ae4014f930281 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.854+0000 7f0754c00640 1 -- 172.21.15.22:0/738157927 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075001e070 con 0x55e8bbdfbac0 2024-04-20T21:40:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.854+0000 7f0754c00640 1 -- 172.21.15.22:0/738157927 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07500035f0 con 0x55e8bbdfbac0 2024-04-20T21:40:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.854+0000 7f0754c00640 1 -- 172.21.15.22:0/738157927 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075000f820 con 0x55e8bbdfbac0 2024-04-20T21:40:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.855+0000 7f0757e00640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07440a3d20 0x7f07440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.855+0000 7f0757e00640 1 -- 172.21.15.22:0/330113160 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f074c0f25b0 con 0x7f0760108f50 2024-04-20T21:40:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.855+0000 7f075fe00640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07440a3d20 0x7f07440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.855+0000 7f076902a740 1 -- 172.21.15.22:0/738157927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 msgr2=0x55e8bbe04160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.855+0000 7f076902a740 1 --2- 172.21.15.22:0/738157927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe04160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f07500114f0 tx=0x7f07500119b0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.855+0000 7f076902a740 1 -- 172.21.15.22:0/738157927 shutdown_connections 2024-04-20T21:40:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.855+0000 7f076902a740 1 --2- 172.21.15.22:0/738157927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbe046a0 0x55e8bbe06a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.855+0000 7f076902a740 1 --2- 172.21.15.22:0/738157927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe04160 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.855+0000 7f076902a740 1 --2- 172.21.15.22:0/738157927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbdfb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.855+0000 7f076902a740 1 -- 172.21.15.22:0/738157927 >> 172.21.15.22:0/738157927 conn(0x55e8bbdf58d0 msgr2=0x55e8bbdf5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:31.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.855+0000 7f076902a740 1 -- 172.21.15.22:0/738157927 shutdown_connections 2024-04-20T21:40:31.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.855+0000 7f076902a740 1 -- 172.21.15.22:0/738157927 wait complete. 2024-04-20T21:40:31.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.856+0000 7f075fe00640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07440a3d20 0x7f07440a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0758012680 tx=0x7f0758009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:31.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.856+0000 7f076902a740 1 -- start start 2024-04-20T21:40:31.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.856+0000 7f076902a740 1 Processor -- start 2024-04-20T21:40:31.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f076902a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe01670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f076902a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe01bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f076902a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbe046a0 0x55e8bbe020f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f076902a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e8bbe07490 con 0x55e8bbdfb1a0 2024-04-20T21:40:31.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f076902a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e8bbd1a740 con 0x55e8bbdfbac0 2024-04-20T21:40:31.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f076902a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e8bbe07610 con 0x55e8bbe046a0 2024-04-20T21:40:31.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f076902a740 10 client.0 _ll_register_callbacks cb 0x55e8bbd15b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:31.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f075ea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe01bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f075fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbe046a0 0x55e8bbe020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f075ea00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe01bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:44092/0 (socket says 172.21.15.22:44092) 2024-04-20T21:40:31.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f075ea00640 1 -- 172.21.15.22:0/3614597895 learned_addr learned my addr 172.21.15.22:0/3614597895 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:31.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f075f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe01670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f075ea00640 1 -- 172.21.15.22:0/3614597895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbe046a0 msgr2=0x55e8bbe020f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:31.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f075ea00640 1 --2- 172.21.15.22:0/3614597895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbe046a0 0x55e8bbe020f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f075ea00640 1 -- 172.21.15.22:0/3614597895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbdfb1a0 msgr2=0x55e8bbe01670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:31.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.857+0000 7f075ea00640 1 --2- 172.21.15.22:0/3614597895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe01670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.858+0000 7f075ea00640 1 -- 172.21.15.22:0/3614597895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8bba733f0 con 0x55e8bbdfbac0 2024-04-20T21:40:31.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.858+0000 7f075ea00640 1 --2- 172.21.15.22:0/3614597895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe01bb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f074c02c810 tx=0x7f074c004f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6b244af08b136c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:31.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.858+0000 7f0749600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:40:31.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.858+0000 7f0749600640 1 -- 172.21.15.22:0/3614597895 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c0be380 con 0x55e8bbdfbac0 2024-04-20T21:40:31.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.858+0000 7f0749600640 1 -- 172.21.15.22:0/3614597895 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f074c0efed0 con 0x55e8bbdfbac0 2024-04-20T21:40:31.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.858+0000 7f076902a740 1 -- 172.21.15.22:0/3614597895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07500035f0 con 0x55e8bbdfbac0 2024-04-20T21:40:31.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.858+0000 7f0749600640 1 -- 172.21.15.22:0/3614597895 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f074c0bdf10 con 0x55e8bbdfbac0 2024-04-20T21:40:31.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.858+0000 7f076902a740 10 client.26330 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:31.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.858+0000 7f076902a740 1 -- 172.21.15.22:0/3614597895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e8bba330f0 con 0x55e8bbdfbac0 2024-04-20T21:40:31.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.859+0000 7f0749600640 1 -- 172.21.15.22:0/3614597895 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07500035f0 con 0x55e8bbdfbac0 2024-04-20T21:40:31.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.859+0000 7f0749600640 1 -- 172.21.15.22:0/3614597895 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f074c0c6d90 con 0x55e8bbdfbac0 2024-04-20T21:40:31.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.859+0000 7f076902a740 10 client.26330 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:31.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.859+0000 7f076902a740 10 client.26330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:31.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.859+0000 7f076902a740 20 client.26330 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:31.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.859+0000 7f076902a740 1 --2- 172.21.15.22:0/3614597895 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e8bbe149b0 0x55e8bbe16e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.859+0000 7f076902a740 4 client.26330 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:31.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.859+0000 7f076902a740 1 -- 172.21.15.22:0/3614597895 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8bbd6eb70 con 0x55e8bbe149b0 2024-04-20T21:40:31.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.859+0000 7f075f400640 1 --2- 172.21.15.22:0/3614597895 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e8bbe149b0 0x55e8bbe16e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.860+0000 7f075f400640 1 --2- 172.21.15.22:0/3614597895 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e8bbe149b0 0x55e8bbe16e30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f50adbe5e3ae0ff9 server_cookie=368c5a0d16a76a02 in_seq=0 out_seq=0 2024-04-20T21:40:31.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.860+0000 7f0749600640 10 client.26330 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:31.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f0749600640 1 -- 172.21.15.22:0/3614597895 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e8bbd6eb70 con 0x55e8bbe149b0 2024-04-20T21:40:31.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f0749600640 10 client.26330 handle_command_reply: tid=0 2024-04-20T21:40:31.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f076902a740 1 client.26330 shutdown 2024-04-20T21:40:31.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f076902a740 1 -- 172.21.15.22:0/3614597895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 msgr2=0x55e8bbe01bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:31.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f076902a740 1 --2- 172.21.15.22:0/3614597895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe01bb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f074c02c810 tx=0x7f074c004f80 comp rx=0 tx=0).stop 2024-04-20T21:40:31.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f076902a740 1 -- 172.21.15.22:0/3614597895 shutdown_connections 2024-04-20T21:40:31.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f076902a740 1 --2- 172.21.15.22:0/3614597895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbe046a0 0x55e8bbe020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f076902a740 1 --2- 172.21.15.22:0/3614597895 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e8bbe149b0 0x55e8bbe16e30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f076902a740 1 --2- 172.21.15.22:0/3614597895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe01bb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f076902a740 1 --2- 172.21.15.22:0/3614597895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe01670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f0749600640 0 client.26330 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:31.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f076902a740 1 -- 172.21.15.22:0/3614597895 >> 172.21.15.22:0/3614597895 conn(0x55e8bbdf58d0 msgr2=0x55e8bbdf7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:31.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f076902a740 1 -- 172.21.15.22:0/3614597895 shutdown_connections 2024-04-20T21:40:31.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.861+0000 7f076902a740 1 -- 172.21.15.22:0/3614597895 wait complete. 2024-04-20T21:40:31.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.862+0000 7f076902a740 20 client.26330 trim_cache size 0 max 16384 2024-04-20T21:40:31.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.881+0000 7f076902a740 1 Processor -- start 2024-04-20T21:40:31.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.881+0000 7f076902a740 1 -- start start 2024-04-20T21:40:31.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.881+0000 7f076902a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe03da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.881+0000 7f076902a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe005b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.881+0000 7f076902a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbe046a0 0x55e8bbe10350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.881+0000 7f076902a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f07601b2320 con 0x55e8bbdfb1a0 2024-04-20T21:40:31.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.881+0000 7f076902a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07601b4b80 con 0x55e8bbe046a0 2024-04-20T21:40:31.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.881+0000 7f076902a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07601b2970 con 0x55e8bbdfbac0 2024-04-20T21:40:31.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.882+0000 7f075f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe03da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.882+0000 7f075ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.882+0000 7f075ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe005b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42448/0 (socket says 172.21.15.22:42448) 2024-04-20T21:40:31.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.882+0000 7f075ea00640 1 -- 172.21.15.22:0/26030317 learned_addr learned my addr 172.21.15.22:0/26030317 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:31.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.882+0000 7f075fe00640 1 --2- 172.21.15.22:0/26030317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbe046a0 0x55e8bbe10350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.882+0000 7f075ea00640 1 -- 172.21.15.22:0/26030317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbe046a0 msgr2=0x55e8bbe10350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:31.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.882+0000 7f075ea00640 1 --2- 172.21.15.22:0/26030317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbe046a0 0x55e8bbe10350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.882+0000 7f075ea00640 1 -- 172.21.15.22:0/26030317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbdfb1a0 msgr2=0x55e8bbe03da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:31.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.882+0000 7f075ea00640 1 --2- 172.21.15.22:0/26030317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe03da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.882+0000 7f075ea00640 1 -- 172.21.15.22:0/26030317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8bbe07370 con 0x55e8bbdfbac0 2024-04-20T21:40:31.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.882+0000 7f075f400640 1 --2- 172.21.15.22:0/26030317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe03da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:31.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.882+0000 7f075ea00640 1 --2- 172.21.15.22:0/26030317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe005b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f074c0baf80 tx=0x7f074c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e6954db90e030fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:31.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.883+0000 7f074be00640 1 -- 172.21.15.22:0/26030317 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c0be0f0 con 0x55e8bbdfbac0 2024-04-20T21:40:31.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.883+0000 7f074be00640 1 -- 172.21.15.22:0/26030317 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074c0becb0 con 0x55e8bbdfbac0 2024-04-20T21:40:31.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.883+0000 7f074be00640 1 -- 172.21.15.22:0/26030317 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c0f00a0 con 0x55e8bbdfbac0 2024-04-20T21:40:31.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.883+0000 7f076902a740 1 -- 172.21.15.22:0/26030317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfbac0 msgr2=0x55e8bbe005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:31.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.883+0000 7f076902a740 1 --2- 172.21.15.22:0/26030317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe005b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f074c0baf80 tx=0x7f074c005bd0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.883+0000 7f076902a740 1 -- 172.21.15.22:0/26030317 shutdown_connections 2024-04-20T21:40:31.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.883+0000 7f076902a740 1 --2- 172.21.15.22:0/26030317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbe046a0 0x55e8bbe10350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.883+0000 7f076902a740 1 --2- 172.21.15.22:0/26030317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe005b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.883+0000 7f076902a740 1 --2- 172.21.15.22:0/26030317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe03da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.883+0000 7f076902a740 1 -- 172.21.15.22:0/26030317 >> 172.21.15.22:0/26030317 conn(0x55e8bbdf58d0 msgr2=0x55e8bbdf48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:31.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.884+0000 7f076902a740 1 -- 172.21.15.22:0/26030317 shutdown_connections 2024-04-20T21:40:31.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.884+0000 7f076902a740 1 -- 172.21.15.22:0/26030317 wait complete. 2024-04-20T21:40:31.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.884+0000 7f076902a740 1 -- start start 2024-04-20T21:40:31.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.884+0000 7f076902a740 1 Processor -- start 2024-04-20T21:40:31.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f076902a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe0c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f076902a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe0cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f076902a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbe046a0 0x55e8bbe089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f076902a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f07601b3040 con 0x55e8bbe046a0 2024-04-20T21:40:31.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f076902a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07601b44b0 con 0x55e8bbdfbac0 2024-04-20T21:40:31.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f076902a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07601b3710 con 0x55e8bbdfb1a0 2024-04-20T21:40:31.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f076902a740 10 client.0 _ll_register_callbacks cb 0x7f074c0f5b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:31.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f075f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe0c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f075f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe0c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42462/0 (socket says 172.21.15.22:42462) 2024-04-20T21:40:31.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f075f400640 1 -- 172.21.15.22:0/2664500358 learned_addr learned my addr 172.21.15.22:0/2664500358 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:31.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f075fe00640 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbe046a0 0x55e8bbe089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f075ea00640 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe0cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f075f400640 1 -- 172.21.15.22:0/2664500358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 msgr2=0x55e8bbe0cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:31.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f075f400640 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe0cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f075f400640 1 -- 172.21.15.22:0/2664500358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbe046a0 msgr2=0x55e8bbe089f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:31.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f075f400640 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbe046a0 0x55e8bbe089f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.885+0000 7f075f400640 1 -- 172.21.15.22:0/2664500358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8bbe09180 con 0x55e8bbdfb1a0 2024-04-20T21:40:31.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.886+0000 7f075ea00640 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe0cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:31.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.886+0000 7f075f400640 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe0c570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0750011eb0 tx=0x7f0750009c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88bcd1fe260c0539 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:31.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.886+0000 7f0749600640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:40:31.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.886+0000 7f0749600640 1 -- 172.21.15.22:0/2664500358 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075002e020 con 0x55e8bbdfb1a0 2024-04-20T21:40:31.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.886+0000 7f0749600640 1 -- 172.21.15.22:0/2664500358 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f0750011040 con 0x55e8bbdfb1a0 2024-04-20T21:40:31.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.886+0000 7f076902a740 1 -- 172.21.15.22:0/2664500358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074c0becb0 con 0x55e8bbdfb1a0 2024-04-20T21:40:31.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.886+0000 7f0749600640 1 -- 172.21.15.22:0/2664500358 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0750039050 con 0x55e8bbdfb1a0 2024-04-20T21:40:31.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.887+0000 7f076902a740 10 client.26356 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:31.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.887+0000 7f076902a740 1 -- 172.21.15.22:0/2664500358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e8bbe7ba10 con 0x55e8bbdfb1a0 2024-04-20T21:40:31.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.887+0000 7f0749600640 1 -- 172.21.15.22:0/2664500358 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074c0becb0 con 0x55e8bbdfb1a0 2024-04-20T21:40:31.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.887+0000 7f0749600640 1 -- 172.21.15.22:0/2664500358 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f0750003680 con 0x55e8bbdfb1a0 2024-04-20T21:40:31.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.887+0000 7f076902a740 10 client.26356 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:31.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.887+0000 7f076902a740 10 client.26356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:31.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.887+0000 7f076902a740 20 client.26356 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:31.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.887+0000 7f076902a740 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e8bbe7c020 0x55e8bbe7e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:31.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.887+0000 7f076902a740 4 client.26356 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:40:31.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.887+0000 7f076902a740 1 -- 172.21.15.22:0/2664500358 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e8bbd689e0 con 0x55e8bbe7c020 2024-04-20T21:40:31.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.888+0000 7f075ea00640 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e8bbe7c020 0x55e8bbe7e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:31.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.888+0000 7f075ea00640 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e8bbe7c020 0x55e8bbe7e440 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bceb298cf23ff8bb server_cookie=52b347b825873e9d in_seq=0 out_seq=0 2024-04-20T21:40:31.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.888+0000 7f0749600640 10 client.26356 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:31.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.888+0000 7f0749600640 1 -- 172.21.15.22:0/2664500358 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e8bbd689e0 con 0x55e8bbe7c020 2024-04-20T21:40:31.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.888+0000 7f0749600640 10 client.26356 handle_command_reply: tid=0 2024-04-20T21:40:31.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.888+0000 7f076902a740 1 client.26356 shutdown 2024-04-20T21:40:31.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.888+0000 7f076902a740 1 -- 172.21.15.22:0/2664500358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 msgr2=0x55e8bbe0c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:31.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.888+0000 7f076902a740 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe0c570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0750011eb0 tx=0x7f0750009c10 comp rx=0 tx=0).stop 2024-04-20T21:40:31.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f076902a740 1 -- 172.21.15.22:0/2664500358 shutdown_connections 2024-04-20T21:40:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f076902a740 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8bbe046a0 0x55e8bbe089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f076902a740 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e8bbe7c020 0x55e8bbe7e440 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f076902a740 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e8bbdfbac0 0x55e8bbe0cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f0749600640 0 client.26356 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f076902a740 1 --2- 172.21.15.22:0/2664500358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8bbdfb1a0 0x55e8bbe0c570 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f076902a740 1 -- 172.21.15.22:0/2664500358 >> 172.21.15.22:0/2664500358 conn(0x55e8bbdf58d0 msgr2=0x55e8bbdf4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f076902a740 1 -- 172.21.15.22:0/2664500358 shutdown_connections 2024-04-20T21:40:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f076902a740 1 -- 172.21.15.22:0/2664500358 wait complete. 2024-04-20T21:40:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f076902a740 20 client.26356 trim_cache size 0 max 16384 2024-04-20T21:40:31.895 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:40:31.895 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:40:31.895 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:40:31.895 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:40:31.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f0766400640 1 -- 172.21.15.22:0/330113160 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07440a3d20 msgr2=0x7f07440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:31.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f0766400640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07440a3d20 0x7f07440a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0758012680 tx=0x7f0758009500 comp rx=0 tx=0).stop 2024-04-20T21:40:31.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f0766400640 1 -- 172.21.15.22:0/330113160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0760108f50 msgr2=0x7f076019f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:31.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.889+0000 7f0766400640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0760108f50 0x7f076019f350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f074c036040 tx=0x7f074c004250 comp rx=0 tx=0).stop 2024-04-20T21:40:31.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.890+0000 7f0766400640 1 -- 172.21.15.22:0/330113160 shutdown_connections 2024-04-20T21:40:31.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.890+0000 7f0766400640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07440a3d20 0x7f07440a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.890+0000 7f0766400640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f076010e600 0x7f07601a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.890+0000 7f0766400640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0760108f50 0x7f076019f350 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.890+0000 7f0766400640 1 --2- 172.21.15.22:0/330113160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0760108590 0x7f076019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:31.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.890+0000 7f0766400640 1 -- 172.21.15.22:0/330113160 >> 172.21.15.22:0/330113160 conn(0x7f07600764a0 msgr2=0x7f0760077d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:31.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.890+0000 7f0766400640 1 -- 172.21.15.22:0/330113160 shutdown_connections 2024-04-20T21:40:31.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:31.890+0000 7f0766400640 1 -- 172.21.15.22:0/330113160 wait complete. 2024-04-20T21:40:31.906 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:40:31.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:40:32.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.115+0000 7f011b200640 1 -- 172.21.15.22:0/4161655832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c107100 msgr2=0x7f011c10eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.115+0000 7f011b200640 1 --2- 172.21.15.22:0/4161655832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c107100 0x7f011c10eca0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0114009860 tx=0x7f011402bb70 comp rx=0 tx=0).stop 2024-04-20T21:40:32.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.115+0000 7f011b200640 1 -- 172.21.15.22:0/4161655832 shutdown_connections 2024-04-20T21:40:32.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.115+0000 7f011b200640 1 --2- 172.21.15.22:0/4161655832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c107100 0x7f011c10eca0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.115+0000 7f011b200640 1 --2- 172.21.15.22:0/4161655832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011c069d10 0x7f011c1069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.115+0000 7f011b200640 1 --2- 172.21.15.22:0/4161655832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c069360 0x7f011c069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.115+0000 7f011b200640 1 -- 172.21.15.22:0/4161655832 >> 172.21.15.22:0/4161655832 conn(0x7f011c0767a0 msgr2=0x7f011c076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:32.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.115+0000 7f011b200640 1 -- 172.21.15.22:0/4161655832 shutdown_connections 2024-04-20T21:40:32.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.115+0000 7f011b200640 1 -- 172.21.15.22:0/4161655832 wait complete. 2024-04-20T21:40:32.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.116+0000 7f011b200640 1 Processor -- start 2024-04-20T21:40:32.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.117+0000 7f011b200640 1 -- start start 2024-04-20T21:40:32.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.117+0000 7f011b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011c069360 0x7f011c1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.117+0000 7f011b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c107100 0x7f011c1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.117+0000 7f011b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c19a650 0x7f011c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.117+0000 7f011b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f011c112170 con 0x7f011c069360 2024-04-20T21:40:32.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.117+0000 7f011b200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f011c111ff0 con 0x7f011c19a650 2024-04-20T21:40:32.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.117+0000 7f011b200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1122f0 con 0x7f011c107100 2024-04-20T21:40:32.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.118+0000 7f0119400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c107100 0x7f011c1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.118+0000 7f0119400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c107100 0x7f011c1a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42482/0 (socket says 172.21.15.22:42482) 2024-04-20T21:40:32.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.118+0000 7f0119400640 1 -- 172.21.15.22:0/3776041824 learned_addr learned my addr 172.21.15.22:0/3776041824 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:32.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.118+0000 7f0119e00640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011c069360 0x7f011c1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.118+0000 7f011a800640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c19a650 0x7f011c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.118+0000 7f0119400640 1 -- 172.21.15.22:0/3776041824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c19a650 msgr2=0x7f011c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.118+0000 7f0119400640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c19a650 0x7f011c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.118+0000 7f0119400640 1 -- 172.21.15.22:0/3776041824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011c069360 msgr2=0x7f011c1a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.118+0000 7f0119400640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011c069360 0x7f011c1a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.118+0000 7f0119400640 1 -- 172.21.15.22:0/3776041824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c19b250 con 0x7f011c107100 2024-04-20T21:40:32.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.118+0000 7f0119400640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c107100 0x7f011c1a0ad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f010400c470 tx=0x7f010400c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:32.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.119+0000 7f0112a00640 1 -- 172.21.15.22:0/3776041824 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104003970 con 0x7f011c107100 2024-04-20T21:40:32.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.119+0000 7f0112a00640 1 -- 172.21.15.22:0/3776041824 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010400b6f0 con 0x7f011c107100 2024-04-20T21:40:32.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.119+0000 7f0112a00640 1 -- 172.21.15.22:0/3776041824 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104004480 con 0x7f011c107100 2024-04-20T21:40:32.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.119+0000 7f011b200640 1 -- 172.21.15.22:0/3776041824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011c1a7330 con 0x7f011c107100 2024-04-20T21:40:32.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.119+0000 7f011b200640 1 -- 172.21.15.22:0/3776041824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011c1a76e0 con 0x7f011c107100 2024-04-20T21:40:32.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.120+0000 7f011b200640 1 -- 172.21.15.22:0/3776041824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011c1034b0 con 0x7f011c107100 2024-04-20T21:40:32.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.122+0000 7f0112a00640 1 -- 172.21.15.22:0/3776041824 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f010401d020 con 0x7f011c107100 2024-04-20T21:40:32.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.123+0000 7f0112a00640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f00fc0a3da0 0x7f00fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.124+0000 7f0112a00640 1 -- 172.21.15.22:0/3776041824 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f01040d0bb0 con 0x7f011c107100 2024-04-20T21:40:32.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.124+0000 7f0119e00640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f00fc0a3da0 0x7f00fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.124+0000 7f0119e00640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f00fc0a3da0 0x7f00fc0a6260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f010c009bd0 tx=0x7f010c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:32.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.126+0000 7f0112a00640 1 -- 172.21.15.22:0/3776041824 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f010409d1a0 con 0x7f011c107100 2024-04-20T21:40:32.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.378+0000 7f011b200640 1 -- 172.21.15.22:0/3776041824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f011c068ca0 con 0x7f011c107100 2024-04-20T21:40:32.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.379+0000 7f0112a00640 1 -- 172.21.15.22:0/3776041824 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f01040bd0e0 con 0x7f011c107100 2024-04-20T21:40:32.379 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:40:32.379 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:40:32.379 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:40:32.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.382+0000 7f011b200640 1 -- 172.21.15.22:0/3776041824 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f00fc0a3da0 msgr2=0x7f00fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.382+0000 7f011b200640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f00fc0a3da0 0x7f00fc0a6260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f010c009bd0 tx=0x7f010c005f90 comp rx=0 tx=0).stop 2024-04-20T21:40:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.382+0000 7f011b200640 1 -- 172.21.15.22:0/3776041824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c107100 msgr2=0x7f011c1a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.382+0000 7f011b200640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c107100 0x7f011c1a0ad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f010400c470 tx=0x7f010400c930 comp rx=0 tx=0).stop 2024-04-20T21:40:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.382+0000 7f011b200640 1 -- 172.21.15.22:0/3776041824 shutdown_connections 2024-04-20T21:40:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.382+0000 7f011b200640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f00fc0a3da0 0x7f00fc0a6260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.382+0000 7f011b200640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f011c19a650 0x7f011c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.382+0000 7f011b200640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f011c107100 0x7f011c1a0ad0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.382+0000 7f011b200640 1 --2- 172.21.15.22:0/3776041824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f011c069360 0x7f011c1a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.382+0000 7f011b200640 1 -- 172.21.15.22:0/3776041824 >> 172.21.15.22:0/3776041824 conn(0x7f011c0767a0 msgr2=0x7f011c10d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.382+0000 7f011b200640 1 -- 172.21.15.22:0/3776041824 shutdown_connections 2024-04-20T21:40:32.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.382+0000 7f011b200640 1 -- 172.21.15.22:0/3776041824 wait complete. 2024-04-20T21:40:32.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:40:32.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:32 smithi122 ceph-mon[22147]: pgmap v206: 97 pgs: 97 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 1.8 MiB/s wr, 63 op/s 2024-04-20T21:40:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:32 smithi033 ceph-mon[21626]: pgmap v206: 97 pgs: 97 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 1.8 MiB/s wr, 63 op/s 2024-04-20T21:40:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.619+0000 7f6441200640 1 -- 172.21.15.22:0/2130535664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c104dc0 msgr2=0x7f643c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.619+0000 7f6441200640 1 --2- 172.21.15.22:0/2130535664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c104dc0 0x7f643c108e00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6424009900 tx=0x7f642402bc30 comp rx=0 tx=0).stop 2024-04-20T21:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.619+0000 7f6441200640 1 -- 172.21.15.22:0/2130535664 shutdown_connections 2024-04-20T21:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.619+0000 7f6441200640 1 --2- 172.21.15.22:0/2130535664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c1094e0 0x7f643c1110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.619+0000 7f6441200640 1 --2- 172.21.15.22:0/2130535664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c104dc0 0x7f643c108e00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.619+0000 7f6441200640 1 --2- 172.21.15.22:0/2130535664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f643c104410 0x7f643c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.619+0000 7f6441200640 1 -- 172.21.15.22:0/2130535664 >> 172.21.15.22:0/2130535664 conn(0x7f643c0ffcf0 msgr2=0x7f643c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.619+0000 7f6441200640 1 -- 172.21.15.22:0/2130535664 shutdown_connections 2024-04-20T21:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.619+0000 7f6441200640 1 -- 172.21.15.22:0/2130535664 wait complete. 2024-04-20T21:40:32.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.620+0000 7f6441200640 1 Processor -- start 2024-04-20T21:40:32.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.621+0000 7f6441200640 1 -- start start 2024-04-20T21:40:32.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.621+0000 7f6441200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c104410 0x7f643c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.621+0000 7f6441200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f643c1094e0 0x7f643c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.621+0000 7f6441200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c19a950 0x7f643c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.621+0000 7f6441200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f643c114550 con 0x7f643c104410 2024-04-20T21:40:32.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.621+0000 7f6441200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f643c1143d0 con 0x7f643c19a950 2024-04-20T21:40:32.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.621+0000 7f6441200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f643c1146d0 con 0x7f643c1094e0 2024-04-20T21:40:32.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.621+0000 7f6439e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c104410 0x7f643c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.621+0000 7f6439e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c104410 0x7f643c1a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40648/0 (socket says 172.21.15.22:40648) 2024-04-20T21:40:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.621+0000 7f6439e00640 1 -- 172.21.15.22:0/619926643 learned_addr learned my addr 172.21.15.22:0/619926643 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.622+0000 7f643a800640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c19a950 0x7f643c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.622+0000 7f6433e00640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f643c1094e0 0x7f643c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.622+0000 7f6439e00640 1 -- 172.21.15.22:0/619926643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f643c1094e0 msgr2=0x7f643c1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.622+0000 7f6439e00640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f643c1094e0 0x7f643c1a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.622+0000 7f6439e00640 1 -- 172.21.15.22:0/619926643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c19a950 msgr2=0x7f643c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.622+0000 7f6439e00640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c19a950 0x7f643c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.622+0000 7f6439e00640 1 -- 172.21.15.22:0/619926643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c19b470 con 0x7f643c104410 2024-04-20T21:40:32.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.622+0000 7f6439e00640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c104410 0x7f643c1a0750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f642c00e9f0 tx=0x7f642c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:32.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.623+0000 7f6432a00640 1 -- 172.21.15.22:0/619926643 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c00cd80 con 0x7f643c104410 2024-04-20T21:40:32.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.623+0000 7f6432a00640 1 -- 172.21.15.22:0/619926643 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f642c010910 con 0x7f643c104410 2024-04-20T21:40:32.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.623+0000 7f6432a00640 1 -- 172.21.15.22:0/619926643 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c018990 con 0x7f643c104410 2024-04-20T21:40:32.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.623+0000 7f6441200640 1 -- 172.21.15.22:0/619926643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c19b760 con 0x7f643c104410 2024-04-20T21:40:32.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.623+0000 7f6441200640 1 -- 172.21.15.22:0/619926643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c1a7960 con 0x7f643c104410 2024-04-20T21:40:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.626+0000 7f6443fc6740 1 Processor -- start 2024-04-20T21:40:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.627+0000 7f6432a00640 1 -- 172.21.15.22:0/619926643 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f642c010ac0 con 0x7f643c104410 2024-04-20T21:40:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.627+0000 7f6443fc6740 1 -- start start 2024-04-20T21:40:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.627+0000 7f6443fc6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf0fb0 0x561defbf1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.627+0000 7f6443fc6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 0x561defbf9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.627+0000 7f6443fc6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbfa4b0 0x561defbfc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.627+0000 7f6443fc6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561defb56050 con 0x561defbfa4b0 2024-04-20T21:40:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.627+0000 7f6443fc6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561defafc490 con 0x561defbf0fb0 2024-04-20T21:40:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.627+0000 7f6443fc6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561defb52010 con 0x561defbf18d0 2024-04-20T21:40:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.627+0000 7f643a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf0fb0 0x561defbf1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.627+0000 7f643a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf0fb0 0x561defbf1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:32942/0 (socket says 172.21.15.22:32942) 2024-04-20T21:40:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.627+0000 7f643a800640 1 -- 172.21.15.22:0/2609548396 learned_addr learned my addr 172.21.15.22:0/2609548396 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.627+0000 7f6433e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbfa4b0 0x561defbfc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.628+0000 7f6439e00640 1 --2- 172.21.15.22:0/2609548396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 0x561defbf9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.628+0000 7f643a800640 1 -- 172.21.15.22:0/2609548396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 msgr2=0x561defbf9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.628+0000 7f643a800640 1 --2- 172.21.15.22:0/2609548396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 0x561defbf9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.628+0000 7f643a800640 1 -- 172.21.15.22:0/2609548396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbfa4b0 msgr2=0x561defbfc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.628+0000 7f643a800640 1 --2- 172.21.15.22:0/2609548396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbfa4b0 0x561defbfc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.628+0000 7f643a800640 1 -- 172.21.15.22:0/2609548396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561defb04070 con 0x561defbf0fb0 2024-04-20T21:40:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.628+0000 7f6439e00640 1 --2- 172.21.15.22:0/2609548396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 0x561defbf9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.628+0000 7f643a800640 1 --2- 172.21.15.22:0/2609548396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf0fb0 0x561defbf1390 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6434009f90 tx=0x7f6434010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7295c61643aa83c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.628+0000 7f6432a00640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f64200a3d20 0x7f64200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.629+0000 7f642b400640 1 -- 172.21.15.22:0/2609548396 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434011700 con 0x561defbf0fb0 2024-04-20T21:40:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.629+0000 7f642b400640 1 -- 172.21.15.22:0/2609548396 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6434011d80 con 0x561defbf0fb0 2024-04-20T21:40:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.629+0000 7f642b400640 1 -- 172.21.15.22:0/2609548396 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434018380 con 0x561defbf0fb0 2024-04-20T21:40:32.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.629+0000 7f6432a00640 1 -- 172.21.15.22:0/619926643 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f642c0d0be0 con 0x7f643c104410 2024-04-20T21:40:32.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.629+0000 7f6439e00640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f64200a3d20 0x7f64200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.629+0000 7f6443fc6740 1 -- 172.21.15.22:0/2609548396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf0fb0 msgr2=0x561defbf1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.629+0000 7f6439e00640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f64200a3d20 0x7f64200a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x561defbf0a80 tx=0x7f642c000bc0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:32.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.629+0000 7f6443fc6740 1 --2- 172.21.15.22:0/2609548396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf0fb0 0x561defbf1390 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6434009f90 tx=0x7f6434010860 comp rx=0 tx=0).stop 2024-04-20T21:40:32.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.630+0000 7f6443fc6740 1 -- 172.21.15.22:0/2609548396 shutdown_connections 2024-04-20T21:40:32.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.630+0000 7f6443fc6740 1 --2- 172.21.15.22:0/2609548396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbfa4b0 0x561defbfc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.630+0000 7f6443fc6740 1 --2- 172.21.15.22:0/2609548396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 0x561defbf9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.630+0000 7f6443fc6740 1 --2- 172.21.15.22:0/2609548396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf0fb0 0x561defbf1390 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.630+0000 7f6443fc6740 1 -- 172.21.15.22:0/2609548396 >> 172.21.15.22:0/2609548396 conn(0x561defbeb6e0 msgr2=0x561defbebac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:32.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.630+0000 7f6443fc6740 1 -- 172.21.15.22:0/2609548396 shutdown_connections 2024-04-20T21:40:32.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.630+0000 7f6443fc6740 1 -- 172.21.15.22:0/2609548396 wait complete. 2024-04-20T21:40:32.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.630+0000 7f6443fc6740 1 -- start start 2024-04-20T21:40:32.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.630+0000 7f6443fc6740 1 Processor -- start 2024-04-20T21:40:32.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.631+0000 7f6443fc6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf0fb0 0x561defbf74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.631+0000 7f6443fc6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf18d0 0x561defbf7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.631+0000 7f6443fc6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbfa4b0 0x561defbf7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.631+0000 7f6443fc6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561defbfd2a0 con 0x561defbf0fb0 2024-04-20T21:40:32.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.631+0000 7f6443fc6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561defb0f860 con 0x561defbf18d0 2024-04-20T21:40:32.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.631+0000 7f6443fc6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561defbfd420 con 0x561defbfa4b0 2024-04-20T21:40:32.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.631+0000 7f6443fc6740 10 client.0 _ll_register_callbacks cb 0x561defaddfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:32.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.631+0000 7f643a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf0fb0 0x561defbf74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.631+0000 7f643a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf0fb0 0x561defbf74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40670/0 (socket says 172.21.15.22:40670) 2024-04-20T21:40:32.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.631+0000 7f643a800640 1 -- 172.21.15.22:0/68507987 learned_addr learned my addr 172.21.15.22:0/68507987 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:32.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.631+0000 7f6433e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf18d0 0x561defbf7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.631+0000 7f643a800640 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbfa4b0 0x561defbf7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.632+0000 7f6433e00640 1 -- 172.21.15.22:0/68507987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbfa4b0 msgr2=0x561defbf7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:32.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.632+0000 7f6433e00640 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbfa4b0 0x561defbf7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.632+0000 7f6433e00640 1 -- 172.21.15.22:0/68507987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf0fb0 msgr2=0x561defbf74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:32.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.632+0000 7f6433e00640 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf0fb0 0x561defbf74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.632+0000 7f6433e00640 1 -- 172.21.15.22:0/68507987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561defb13950 con 0x561defbf18d0 2024-04-20T21:40:32.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.632+0000 7f643a800640 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf0fb0 0x561defbf74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:32.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.632+0000 7f6433e00640 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf18d0 0x561defbf7a30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6424036040 tx=0x7f6424009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=444e8f16f1ef6e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.632+0000 7f641fe00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:40:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.632+0000 7f641fe00640 1 -- 172.21.15.22:0/68507987 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424004300 con 0x561defbf18d0 2024-04-20T21:40:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.632+0000 7f6443fc6740 1 -- 172.21.15.22:0/68507987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6434011d80 con 0x561defbf18d0 2024-04-20T21:40:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.633+0000 7f641fe00640 1 -- 172.21.15.22:0/68507987 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f64240046c0 con 0x561defbf18d0 2024-04-20T21:40:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.633+0000 7f641fe00640 1 -- 172.21.15.22:0/68507987 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f64240044a0 con 0x561defbf18d0 2024-04-20T21:40:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.633+0000 7f6443fc6740 10 client.26354 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.633+0000 7f6443fc6740 1 -- 172.21.15.22:0/68507987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561defae1820 con 0x561defbf18d0 2024-04-20T21:40:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.633+0000 7f641fe00640 1 -- 172.21.15.22:0/68507987 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6434011d80 con 0x561defbf18d0 2024-04-20T21:40:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.633+0000 7f641fe00640 1 -- 172.21.15.22:0/68507987 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f64240035f0 con 0x561defbf18d0 2024-04-20T21:40:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.633+0000 7f6443fc6740 10 client.26354 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.633+0000 7f6443fc6740 10 client.26354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.633+0000 7f6443fc6740 20 client.26354 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.633+0000 7f6443fc6740 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561defc0a460 0x561defc0c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.634+0000 7f6443fc6740 4 client.26354 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.634+0000 7f6443fc6740 1 -- 172.21.15.22:0/68507987 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561defb63a70 con 0x561defc0a460 2024-04-20T21:40:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.634+0000 7f643a800640 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561defc0a460 0x561defc0c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.634+0000 7f643a800640 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561defc0a460 0x561defc0c850 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=960f3bcd2c03542 server_cookie=529be67127fdcaff in_seq=0 out_seq=0 2024-04-20T21:40:32.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.634+0000 7f641fe00640 10 client.26354 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:32.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.635+0000 7f641fe00640 1 -- 172.21.15.22:0/68507987 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561defb63a70 con 0x561defc0a460 2024-04-20T21:40:32.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.635+0000 7f641fe00640 10 client.26354 handle_command_reply: tid=0 2024-04-20T21:40:32.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.635+0000 7f6443fc6740 1 client.26354 shutdown 2024-04-20T21:40:32.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.635+0000 7f6443fc6740 1 -- 172.21.15.22:0/68507987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf18d0 msgr2=0x561defbf7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.635+0000 7f6443fc6740 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf18d0 0x561defbf7a30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6424036040 tx=0x7f6424009580 comp rx=0 tx=0).stop 2024-04-20T21:40:32.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.636+0000 7f6443fc6740 1 -- 172.21.15.22:0/68507987 shutdown_connections 2024-04-20T21:40:32.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.636+0000 7f6443fc6740 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbfa4b0 0x561defbf7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.636+0000 7f6443fc6740 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561defc0a460 0x561defc0c850 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.636+0000 7f6443fc6740 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf18d0 0x561defbf7a30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.636+0000 7f6443fc6740 1 --2- 172.21.15.22:0/68507987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf0fb0 0x561defbf74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.636+0000 7f6443fc6740 1 -- 172.21.15.22:0/68507987 >> 172.21.15.22:0/68507987 conn(0x561defbeb6e0 msgr2=0x561defbed430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:32.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.636+0000 7f641fe00640 0 client.26354 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:32.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.636+0000 7f6443fc6740 1 -- 172.21.15.22:0/68507987 shutdown_connections 2024-04-20T21:40:32.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.636+0000 7f6443fc6740 1 -- 172.21.15.22:0/68507987 wait complete. 2024-04-20T21:40:32.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.636+0000 7f6443fc6740 20 client.26354 trim_cache size 0 max 16384 2024-04-20T21:40:32.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.651+0000 7f6443fc6740 1 Processor -- start 2024-04-20T21:40:32.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.651+0000 7f6443fc6740 1 -- start start 2024-04-20T21:40:32.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.651+0000 7f6443fc6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf0fb0 0x561defbf9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.651+0000 7f6443fc6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf18d0 0x561defc06160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.651+0000 7f6443fc6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbfa4b0 0x561defc12120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.651+0000 7f6443fc6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f643c1ae0d0 con 0x561defbf18d0 2024-04-20T21:40:32.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.651+0000 7f6443fc6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f643c1b0930 con 0x561defbfa4b0 2024-04-20T21:40:32.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.651+0000 7f6443fc6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f643c1ae720 con 0x561defbf0fb0 2024-04-20T21:40:32.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.652+0000 7f643a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf0fb0 0x561defbf9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.652+0000 7f643a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf0fb0 0x561defbf9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:34118/0 (socket says 172.21.15.22:34118) 2024-04-20T21:40:32.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.652+0000 7f643a800640 1 -- 172.21.15.22:0/73046498 learned_addr learned my addr 172.21.15.22:0/73046498 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:32.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.652+0000 7f6433e00640 1 --2- 172.21.15.22:0/73046498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf18d0 0x561defc06160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.652+0000 7f643a800640 1 --2- 172.21.15.22:0/73046498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbfa4b0 0x561defc12120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.652+0000 7f6433e00640 1 -- 172.21.15.22:0/73046498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf0fb0 msgr2=0x561defbf9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:32.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.652+0000 7f6433e00640 1 --2- 172.21.15.22:0/73046498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf0fb0 0x561defbf9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.652+0000 7f6433e00640 1 -- 172.21.15.22:0/73046498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbfa4b0 msgr2=0x561defc12120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:32.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.652+0000 7f6433e00640 1 --2- 172.21.15.22:0/73046498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbfa4b0 0x561defc12120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.652+0000 7f6433e00640 1 -- 172.21.15.22:0/73046498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561def8683f0 con 0x561defbf18d0 2024-04-20T21:40:32.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.652+0000 7f6433e00640 1 --2- 172.21.15.22:0/73046498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf18d0 0x561defc06160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6424003b10 tx=0x7f6424000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c5ed4391a0cd22 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:32.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.653+0000 7f642aa00640 1 -- 172.21.15.22:0/73046498 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424054070 con 0x561defbf18d0 2024-04-20T21:40:32.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.653+0000 7f642aa00640 1 -- 172.21.15.22:0/73046498 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6424005550 con 0x561defbf18d0 2024-04-20T21:40:32.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.653+0000 7f642aa00640 1 -- 172.21.15.22:0/73046498 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64240046c0 con 0x561defbf18d0 2024-04-20T21:40:32.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.653+0000 7f6443fc6740 1 -- 172.21.15.22:0/73046498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf18d0 msgr2=0x561defc06160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.653+0000 7f6443fc6740 1 --2- 172.21.15.22:0/73046498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf18d0 0x561defc06160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6424003b10 tx=0x7f6424000bc0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.653+0000 7f6443fc6740 1 -- 172.21.15.22:0/73046498 shutdown_connections 2024-04-20T21:40:32.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.653+0000 7f6443fc6740 1 --2- 172.21.15.22:0/73046498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbfa4b0 0x561defc12120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.653+0000 7f6443fc6740 1 --2- 172.21.15.22:0/73046498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbf18d0 0x561defc06160 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.653+0000 7f6443fc6740 1 --2- 172.21.15.22:0/73046498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf0fb0 0x561defbf9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.653+0000 7f6443fc6740 1 -- 172.21.15.22:0/73046498 >> 172.21.15.22:0/73046498 conn(0x561defbeb6e0 msgr2=0x561defbf7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:32.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.653+0000 7f6443fc6740 1 -- 172.21.15.22:0/73046498 shutdown_connections 2024-04-20T21:40:32.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.653+0000 7f6443fc6740 1 -- 172.21.15.22:0/73046498 wait complete. 2024-04-20T21:40:32.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.654+0000 7f6443fc6740 1 -- start start 2024-04-20T21:40:32.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.654+0000 7f6443fc6740 1 Processor -- start 2024-04-20T21:40:32.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.654+0000 7f6443fc6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf0fb0 0x561defbfe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.654+0000 7f6443fc6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 0x561defbfe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.654+0000 7f6443fc6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbfa4b0 0x561defbfedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.654+0000 7f6443fc6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f643c1aedf0 con 0x561defbfa4b0 2024-04-20T21:40:32.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.654+0000 7f6443fc6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f643c1b0260 con 0x561defbf0fb0 2024-04-20T21:40:32.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.654+0000 7f6443fc6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f643c1af4c0 con 0x561defbf18d0 2024-04-20T21:40:32.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.655+0000 7f6443fc6740 10 client.0 _ll_register_callbacks cb 0x7f642402cd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:32.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.655+0000 7f6433e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 0x561defbfe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.655+0000 7f643a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf0fb0 0x561defbfe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.655+0000 7f6433e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 0x561defbfe870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:34128/0 (socket says 172.21.15.22:34128) 2024-04-20T21:40:32.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.655+0000 7f6433e00640 1 -- 172.21.15.22:0/3026505200 learned_addr learned my addr 172.21.15.22:0/3026505200 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:32.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.655+0000 7f643a800640 1 --2- 172.21.15.22:0/3026505200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbfa4b0 0x561defbfedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.655+0000 7f6433e00640 1 -- 172.21.15.22:0/3026505200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf0fb0 msgr2=0x561defbfe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:32.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.655+0000 7f6433e00640 1 --2- 172.21.15.22:0/3026505200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf0fb0 0x561defbfe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.655+0000 7f6433e00640 1 -- 172.21.15.22:0/3026505200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbfa4b0 msgr2=0x561defbfedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:32.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.655+0000 7f6433e00640 1 --2- 172.21.15.22:0/3026505200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbfa4b0 0x561defbfedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.655+0000 7f6433e00640 1 -- 172.21.15.22:0/3026505200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561def9327d0 con 0x561defbf18d0 2024-04-20T21:40:32.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.656+0000 7f6433e00640 1 --2- 172.21.15.22:0/3026505200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 0x561defbfe870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6424005c40 tx=0x7f6424005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd362d5d6ae2c4cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:32.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.656+0000 7f641fe00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:40:32.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.656+0000 7f641fe00640 1 -- 172.21.15.22:0/3026505200 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424054070 con 0x561defbf18d0 2024-04-20T21:40:32.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.656+0000 7f641fe00640 1 -- 172.21.15.22:0/3026505200 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f6424048670 con 0x561defbf18d0 2024-04-20T21:40:32.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.656+0000 7f6443fc6740 1 -- 172.21.15.22:0/3026505200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6424005550 con 0x561defbf18d0 2024-04-20T21:40:32.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.656+0000 7f641fe00640 1 -- 172.21.15.22:0/3026505200 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6424003da0 con 0x561defbf18d0 2024-04-20T21:40:32.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.656+0000 7f6443fc6740 10 client.26380 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:32.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.656+0000 7f6443fc6740 1 -- 172.21.15.22:0/3026505200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561defc70d40 con 0x561defbf18d0 2024-04-20T21:40:32.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.657+0000 7f641fe00640 1 -- 172.21.15.22:0/3026505200 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6424005550 con 0x561defbf18d0 2024-04-20T21:40:32.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.657+0000 7f641fe00640 1 -- 172.21.15.22:0/3026505200 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f6424060840 con 0x561defbf18d0 2024-04-20T21:40:32.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:32 smithi022 ceph-mon[16843]: pgmap v206: 97 pgs: 97 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 1.8 MiB/s wr, 63 op/s 2024-04-20T21:40:32.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.657+0000 7f6443fc6740 10 client.26380 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:32.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.657+0000 7f6443fc6740 10 client.26380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:32.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.657+0000 7f6443fc6740 20 client.26380 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:32.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.657+0000 7f6443fc6740 1 --2- 172.21.15.22:0/3026505200 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561defc713f0 0x561defc73810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:32.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.657+0000 7f6443fc6740 4 client.26380 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:40:32.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.657+0000 7f6443fc6740 1 -- 172.21.15.22:0/3026505200 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561defb5d8e0 con 0x561defc713f0 2024-04-20T21:40:32.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.658+0000 7f643a800640 1 --2- 172.21.15.22:0/3026505200 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561defc713f0 0x561defc73810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:32.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.658+0000 7f643a800640 1 --2- 172.21.15.22:0/3026505200 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561defc713f0 0x561defc73810 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ade4d0145aac69e server_cookie=63802019d378022d in_seq=0 out_seq=0 2024-04-20T21:40:32.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.658+0000 7f641fe00640 10 client.26380 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:32.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.658+0000 7f641fe00640 1 -- 172.21.15.22:0/3026505200 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561defb5d8e0 con 0x561defc713f0 2024-04-20T21:40:32.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.658+0000 7f641fe00640 10 client.26380 handle_command_reply: tid=0 2024-04-20T21:40:32.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.658+0000 7f6443fc6740 1 client.26380 shutdown 2024-04-20T21:40:32.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.658+0000 7f6443fc6740 1 -- 172.21.15.22:0/3026505200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 msgr2=0x561defbfe870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.658+0000 7f6443fc6740 1 --2- 172.21.15.22:0/3026505200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 0x561defbfe870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6424005c40 tx=0x7f6424005c90 comp rx=0 tx=0).stop 2024-04-20T21:40:32.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.659+0000 7f6443fc6740 1 -- 172.21.15.22:0/3026505200 shutdown_connections 2024-04-20T21:40:32.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.659+0000 7f6443fc6740 1 --2- 172.21.15.22:0/3026505200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561defbfa4b0 0x561defbfedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.659+0000 7f6443fc6740 1 --2- 172.21.15.22:0/3026505200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561defbf18d0 0x561defbfe870 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.659+0000 7f6443fc6740 1 --2- 172.21.15.22:0/3026505200 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x561defc713f0 0x561defc73810 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.659+0000 7f6443fc6740 1 --2- 172.21.15.22:0/3026505200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561defbf0fb0 0x561defbfe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.659+0000 7f641fe00640 0 client.26380 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:32.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.659+0000 7f6443fc6740 1 -- 172.21.15.22:0/3026505200 >> 172.21.15.22:0/3026505200 conn(0x561defbeb6e0 msgr2=0x561defbec9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:32.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.659+0000 7f6443fc6740 1 -- 172.21.15.22:0/3026505200 shutdown_connections 2024-04-20T21:40:32.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.659+0000 7f6443fc6740 1 -- 172.21.15.22:0/3026505200 wait complete. 2024-04-20T21:40:32.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.659+0000 7f6443fc6740 20 client.26380 trim_cache size 0 max 16384 2024-04-20T21:40:32.665 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:40:32.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.659+0000 7f6441200640 1 -- 172.21.15.22:0/619926643 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f64200a3d20 msgr2=0x7f64200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.659+0000 7f6441200640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f64200a3d20 0x7f64200a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x561defbf0a80 tx=0x7f642c000bc0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.660+0000 7f6441200640 1 -- 172.21.15.22:0/619926643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c104410 msgr2=0x7f643c1a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:32.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.660+0000 7f6441200640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c104410 0x7f643c1a0750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f642c00e9f0 tx=0x7f642c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.660+0000 7f6441200640 1 -- 172.21.15.22:0/619926643 shutdown_connections 2024-04-20T21:40:32.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.660+0000 7f6441200640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f64200a3d20 0x7f64200a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.660+0000 7f6441200640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c19a950 0x7f643c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.660+0000 7f6441200640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f643c1094e0 0x7f643c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.660+0000 7f6441200640 1 --2- 172.21.15.22:0/619926643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f643c104410 0x7f643c1a0750 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:32.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.660+0000 7f6441200640 1 -- 172.21.15.22:0/619926643 >> 172.21.15.22:0/619926643 conn(0x7f643c0ffcf0 msgr2=0x7f643c101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:32.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.660+0000 7f6441200640 1 -- 172.21.15.22:0/619926643 shutdown_connections 2024-04-20T21:40:32.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:32.660+0000 7f6441200640 1 -- 172.21.15.22:0/619926643 wait complete. 2024-04-20T21:40:33.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:33 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3776041824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:40:33.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:33 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3776041824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:40:33.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:33 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3776041824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:40:33.677 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:40:33.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:40:33.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:40:33.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.901+0000 7f01c6c00640 1 -- 172.21.15.22:0/3359384060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01c0102a30 msgr2=0x7f01c0106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:33.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.901+0000 7f01c6c00640 1 --2- 172.21.15.22:0/3359384060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01c0102a30 0x7f01c0106a70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f01ac009900 tx=0x7f01ac02bc40 comp rx=0 tx=0).stop 2024-04-20T21:40:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.901+0000 7f01c6c00640 1 -- 172.21.15.22:0/3359384060 shutdown_connections 2024-04-20T21:40:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.901+0000 7f01c6c00640 1 --2- 172.21.15.22:0/3359384060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c0107150 0x7f01c010ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.901+0000 7f01c6c00640 1 --2- 172.21.15.22:0/3359384060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01c0102a30 0x7f01c0106a70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.901+0000 7f01c6c00640 1 --2- 172.21.15.22:0/3359384060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c0102080 0x7f01c0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.901+0000 7f01c6c00640 1 -- 172.21.15.22:0/3359384060 >> 172.21.15.22:0/3359384060 conn(0x7f01c00fd900 msgr2=0x7f01c00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.901+0000 7f01c6c00640 1 -- 172.21.15.22:0/3359384060 shutdown_connections 2024-04-20T21:40:33.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.901+0000 7f01c6c00640 1 -- 172.21.15.22:0/3359384060 wait complete. 2024-04-20T21:40:33.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.902+0000 7f01c6c00640 1 Processor -- start 2024-04-20T21:40:33.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.902+0000 7f01c6c00640 1 -- start start 2024-04-20T21:40:33.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.903+0000 7f01c6c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c0102080 0x7f01c0074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.903+0000 7f01c6c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01c0107150 0x7f01c00747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.903+0000 7f01c6c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c0074ce0 0x7f01c01a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.903+0000 7f01c6c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0112230 con 0x7f01c0107150 2024-04-20T21:40:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.903+0000 7f01c6c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01120b0 con 0x7f01c0074ce0 2024-04-20T21:40:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.903+0000 7f01c6c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01123b0 con 0x7f01c0102080 2024-04-20T21:40:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.903+0000 7f01c4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01c0107150 0x7f01c00747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.903+0000 7f01c4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01c0107150 0x7f01c00747a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40710/0 (socket says 172.21.15.22:40710) 2024-04-20T21:40:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.903+0000 7f01c4e00640 1 -- 172.21.15.22:0/3648744383 learned_addr learned my addr 172.21.15.22:0/3648744383 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.903+0000 7f01c5800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c0102080 0x7f01c0074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.903+0000 7f01c6200640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c0074ce0 0x7f01c01a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.904+0000 7f01c5800640 1 -- 172.21.15.22:0/3648744383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c0074ce0 msgr2=0x7f01c01a6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:33.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.904+0000 7f01c5800640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c0074ce0 0x7f01c01a6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.904+0000 7f01c5800640 1 -- 172.21.15.22:0/3648744383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01c0107150 msgr2=0x7f01c00747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:33.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.904+0000 7f01c5800640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01c0107150 0x7f01c00747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.904+0000 7f01c5800640 1 -- 172.21.15.22:0/3648744383 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c01a7000 con 0x7f01c0102080 2024-04-20T21:40:33.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.904+0000 7f01c6200640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c0074ce0 0x7f01c01a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:33.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.904+0000 7f01c4e00640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01c0107150 0x7f01c00747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:33.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.904+0000 7f01c5800640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c0102080 0x7f01c0074260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009fd0 tx=0x7f01b400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:33.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.905+0000 7f01be000640 1 -- 172.21.15.22:0/3648744383 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4004b20 con 0x7f01c0102080 2024-04-20T21:40:33.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.905+0000 7f01c6c00640 1 -- 172.21.15.22:0/3648744383 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c01a72f0 con 0x7f01c0102080 2024-04-20T21:40:33.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.905+0000 7f01be000640 1 -- 172.21.15.22:0/3648744383 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b4004cc0 con 0x7f01c0102080 2024-04-20T21:40:33.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.905+0000 7f01be000640 1 -- 172.21.15.22:0/3648744383 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b40107b0 con 0x7f01c0102080 2024-04-20T21:40:33.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.905+0000 7f01c6c00640 1 -- 172.21.15.22:0/3648744383 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c01a77f0 con 0x7f01c0102080 2024-04-20T21:40:33.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.908+0000 7f01cae7b740 1 Processor -- start 2024-04-20T21:40:33.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.908+0000 7f01cae7b740 1 -- start start 2024-04-20T21:40:33.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.908+0000 7f01be000640 1 -- 172.21.15.22:0/3648744383 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f01b4005020 con 0x7f01c0102080 2024-04-20T21:40:33.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.908+0000 7f01cae7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.908+0000 7f01cae7b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.908+0000 7f01cae7b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263e839e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.908+0000 7f01cae7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e263ddcd20 con 0x55e263e780f0 2024-04-20T21:40:33.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.908+0000 7f01cae7b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e263d833e0 con 0x55e263e815f0 2024-04-20T21:40:33.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.908+0000 7f01cae7b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e263dd8ce0 con 0x55e263e78a10 2024-04-20T21:40:33.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.909+0000 7f01c6200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.909+0000 7f01c5800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.909+0000 7f01c6200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e784d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40722/0 (socket says 172.21.15.22:40722) 2024-04-20T21:40:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.909+0000 7f01c6200640 1 -- 172.21.15.22:0/1363963180 learned_addr learned my addr 172.21.15.22:0/1363963180 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.909+0000 7f01c4e00640 1 --2- 172.21.15.22:0/1363963180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263e839e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.909+0000 7f01c6200640 1 -- 172.21.15.22:0/1363963180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 msgr2=0x55e263e810b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.909+0000 7f01c6200640 1 --2- 172.21.15.22:0/1363963180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e810b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.909+0000 7f01c6200640 1 -- 172.21.15.22:0/1363963180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 msgr2=0x55e263e839e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.909+0000 7f01c6200640 1 --2- 172.21.15.22:0/1363963180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263e839e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.909+0000 7f01c6200640 1 -- 172.21.15.22:0/1363963180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e263da2980 con 0x55e263e780f0 2024-04-20T21:40:33.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.910+0000 7f01c6200640 1 --2- 172.21.15.22:0/1363963180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e784d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f01b8011440 tx=0x7f01b8011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4672bbff135f2392 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:33.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.910+0000 7f01b2a00640 1 -- 172.21.15.22:0/1363963180 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8004020 con 0x55e263e780f0 2024-04-20T21:40:33.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.910+0000 7f01b2a00640 1 -- 172.21.15.22:0/1363963180 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b80041c0 con 0x55e263e780f0 2024-04-20T21:40:33.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.910+0000 7f01b2a00640 1 -- 172.21.15.22:0/1363963180 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b800f940 con 0x55e263e780f0 2024-04-20T21:40:33.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.910+0000 7f01be000640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f01a80a3d20 0x7f01a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.910+0000 7f01be000640 1 -- 172.21.15.22:0/3648744383 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f01b40d0fe0 con 0x7f01c0102080 2024-04-20T21:40:33.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.910+0000 7f01c5800640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f01a80a3d20 0x7f01a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.911+0000 7f01cae7b740 1 -- 172.21.15.22:0/1363963180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 msgr2=0x55e263e784d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:33.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.911+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1363963180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e784d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f01b8011440 tx=0x7f01b8011900 comp rx=0 tx=0).stop 2024-04-20T21:40:33.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.911+0000 7f01cae7b740 1 -- 172.21.15.22:0/1363963180 shutdown_connections 2024-04-20T21:40:33.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.911+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1363963180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263e839e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.911+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1363963180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.911+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1363963180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e784d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.911+0000 7f01cae7b740 1 -- 172.21.15.22:0/1363963180 >> 172.21.15.22:0/1363963180 conn(0x55e263e72820 msgr2=0x55e263e72c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:33.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.911+0000 7f01cae7b740 1 -- 172.21.15.22:0/1363963180 shutdown_connections 2024-04-20T21:40:33.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.911+0000 7f01cae7b740 1 -- 172.21.15.22:0/1363963180 wait complete. 2024-04-20T21:40:33.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.911+0000 7f01c5800640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f01a80a3d20 0x7f01a80a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f01b40ce790 tx=0x7f01b40d4b30 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:33.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.912+0000 7f01cae7b740 1 -- start start 2024-04-20T21:40:33.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.912+0000 7f01cae7b740 1 Processor -- start 2024-04-20T21:40:33.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.912+0000 7f01cae7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e7e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.912+0000 7f01cae7b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e7ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.912+0000 7f01cae7b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263e7f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.912+0000 7f01cae7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e263e843e0 con 0x55e263e780f0 2024-04-20T21:40:33.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.912+0000 7f01cae7b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e263d967a0 con 0x55e263e815f0 2024-04-20T21:40:33.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.912+0000 7f01cae7b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e263e84560 con 0x55e263e78a10 2024-04-20T21:40:33.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.912+0000 7f01cae7b740 10 client.0 _ll_register_callbacks cb 0x55e263d98b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01c6200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e7e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01c4e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e7ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01c4e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e7ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:34156/0 (socket says 172.21.15.22:34156) 2024-04-20T21:40:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01c4e00640 1 -- 172.21.15.22:0/1234484108 learned_addr learned my addr 172.21.15.22:0/1234484108 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01c6200640 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263e7f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01c4e00640 1 -- 172.21.15.22:0/1234484108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 msgr2=0x55e263e7f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01c4e00640 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263e7f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01c4e00640 1 -- 172.21.15.22:0/1234484108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 msgr2=0x55e263e7e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:33.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01c4e00640 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e7e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01c4e00640 1 -- 172.21.15.22:0/1234484108 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e263bb77e0 con 0x55e263e78a10 2024-04-20T21:40:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01c6200640 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263e7f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01c4e00640 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e7ebc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f01ac02c180 tx=0x7f01ac005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4705fe3098cf9af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.913+0000 7f01a7400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:40:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.914+0000 7f01a7400640 1 -- 172.21.15.22:0/1234484108 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac02c890 con 0x55e263e78a10 2024-04-20T21:40:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.914+0000 7f01a7400640 1 -- 172.21.15.22:0/1234484108 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f01ac02ca30 con 0x55e263e78a10 2024-04-20T21:40:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.914+0000 7f01cae7b740 1 -- 172.21.15.22:0/1234484108 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b80041c0 con 0x55e263e78a10 2024-04-20T21:40:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.914+0000 7f01a7400640 1 -- 172.21.15.22:0/1234484108 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f01ac02cc30 con 0x55e263e78a10 2024-04-20T21:40:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.914+0000 7f01cae7b740 10 client.26392 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.914+0000 7f01cae7b740 1 -- 172.21.15.22:0/1234484108 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e263c17f70 con 0x55e263e78a10 2024-04-20T21:40:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.915+0000 7f01a7400640 1 -- 172.21.15.22:0/1234484108 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b80041c0 con 0x55e263e78a10 2024-04-20T21:40:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.915+0000 7f01a7400640 1 -- 172.21.15.22:0/1234484108 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f01ac046070 con 0x55e263e78a10 2024-04-20T21:40:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.915+0000 7f01cae7b740 10 client.26392 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.915+0000 7f01cae7b740 10 client.26392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.915+0000 7f01cae7b740 20 client.26392 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.915+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e263e91a20 0x55e263e93e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.915+0000 7f01cae7b740 4 client.26392 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.915+0000 7f01cae7b740 1 -- 172.21.15.22:0/1234484108 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e263dea740 con 0x55e263e91a20 2024-04-20T21:40:33.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.916+0000 7f01c6200640 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e263e91a20 0x55e263e93e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.916+0000 7f01c6200640 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e263e91a20 0x55e263e93e80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12e99abd01ed6c7 server_cookie=6c60e758ea672224 in_seq=0 out_seq=0 2024-04-20T21:40:33.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.916+0000 7f01a7400640 10 client.26392 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:33.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01a7400640 1 -- 172.21.15.22:0/1234484108 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e263dea740 con 0x55e263e91a20 2024-04-20T21:40:33.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01a7400640 10 client.26392 handle_command_reply: tid=0 2024-04-20T21:40:33.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01cae7b740 1 client.26392 shutdown 2024-04-20T21:40:33.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01cae7b740 1 -- 172.21.15.22:0/1234484108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 msgr2=0x55e263e7ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:33.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e7ebc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f01ac02c180 tx=0x7f01ac005de0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01cae7b740 1 -- 172.21.15.22:0/1234484108 shutdown_connections 2024-04-20T21:40:33.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e263e91a20 0x55e263e93e80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263e7f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01a7400640 0 client.26392 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:33.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e7ebc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1234484108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e7e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01cae7b740 1 -- 172.21.15.22:0/1234484108 >> 172.21.15.22:0/1234484108 conn(0x55e263e72820 msgr2=0x55e263e74570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:33.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01cae7b740 1 -- 172.21.15.22:0/1234484108 shutdown_connections 2024-04-20T21:40:33.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.917+0000 7f01cae7b740 1 -- 172.21.15.22:0/1234484108 wait complete. 2024-04-20T21:40:33.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.918+0000 7f01cae7b740 20 client.26392 trim_cache size 0 max 16384 2024-04-20T21:40:33.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.937+0000 7f01cae7b740 1 Processor -- start 2024-04-20T21:40:33.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.937+0000 7f01cae7b740 1 -- start start 2024-04-20T21:40:33.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.937+0000 7f01cae7b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e780f0 0x55e263e80db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.937+0000 7f01cae7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e78a10 0x55e263e8d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.937+0000 7f01cae7b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e815f0 0x55e263e71880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.937+0000 7f01cae7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01ade80 con 0x55e263e78a10 2024-04-20T21:40:33.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.937+0000 7f01cae7b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b06e0 con 0x55e263e780f0 2024-04-20T21:40:33.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.937+0000 7f01cae7b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01ae4d0 con 0x55e263e815f0 2024-04-20T21:40:33.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.938+0000 7f01c6200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e780f0 0x55e263e80db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.938+0000 7f01c6200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e780f0 0x55e263e80db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33028/0 (socket says 172.21.15.22:33028) 2024-04-20T21:40:33.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.938+0000 7f01c6200640 1 -- 172.21.15.22:0/1399692314 learned_addr learned my addr 172.21.15.22:0/1399692314 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:33.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.938+0000 7f01c4e00640 1 --2- 172.21.15.22:0/1399692314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e78a10 0x55e263e8d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.938+0000 7f01c6200640 1 --2- 172.21.15.22:0/1399692314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e815f0 0x55e263e71880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.938+0000 7f01c6200640 1 -- 172.21.15.22:0/1399692314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e815f0 msgr2=0x55e263e71880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:33.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.938+0000 7f01c6200640 1 --2- 172.21.15.22:0/1399692314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e815f0 0x55e263e71880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.938+0000 7f01c6200640 1 -- 172.21.15.22:0/1399692314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e78a10 msgr2=0x55e263e8d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:33.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.938+0000 7f01c6200640 1 --2- 172.21.15.22:0/1399692314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e78a10 0x55e263e8d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.938+0000 7f01c6200640 1 -- 172.21.15.22:0/1399692314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e263aaf170 con 0x55e263e780f0 2024-04-20T21:40:33.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.939+0000 7f01c6200640 1 --2- 172.21.15.22:0/1399692314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e780f0 0x55e263e80db0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009fb0 tx=0x7f01b8003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff3c319894cb792c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:33.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.939+0000 7f01b2000640 1 -- 172.21.15.22:0/1399692314 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b801d040 con 0x55e263e780f0 2024-04-20T21:40:33.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.939+0000 7f01b2000640 1 -- 172.21.15.22:0/1399692314 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b8004190 con 0x55e263e780f0 2024-04-20T21:40:33.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.939+0000 7f01b2000640 1 -- 172.21.15.22:0/1399692314 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b802d260 con 0x55e263e780f0 2024-04-20T21:40:33.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.940+0000 7f01cae7b740 1 -- 172.21.15.22:0/1399692314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e780f0 msgr2=0x55e263e80db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:33.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.940+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1399692314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e780f0 0x55e263e80db0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009fb0 tx=0x7f01b8003f10 comp rx=0 tx=0).stop 2024-04-20T21:40:33.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.940+0000 7f01cae7b740 1 -- 172.21.15.22:0/1399692314 shutdown_connections 2024-04-20T21:40:33.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.940+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1399692314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e815f0 0x55e263e71880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.940+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1399692314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e78a10 0x55e263e8d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.940+0000 7f01cae7b740 1 --2- 172.21.15.22:0/1399692314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e780f0 0x55e263e80db0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.940+0000 7f01cae7b740 1 -- 172.21.15.22:0/1399692314 >> 172.21.15.22:0/1399692314 conn(0x55e263e72820 msgr2=0x55e263e71170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:33.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.940+0000 7f01cae7b740 1 -- 172.21.15.22:0/1399692314 shutdown_connections 2024-04-20T21:40:33.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.940+0000 7f01cae7b740 1 -- 172.21.15.22:0/1399692314 wait complete. 2024-04-20T21:40:33.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.940+0000 7f01cae7b740 1 -- start start 2024-04-20T21:40:33.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.940+0000 7f01cae7b740 1 Processor -- start 2024-04-20T21:40:33.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.941+0000 7f01cae7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e85520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.941+0000 7f01cae7b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e85a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.941+0000 7f01cae7b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263ef7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.941+0000 7f01cae7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01aeba0 con 0x55e263e780f0 2024-04-20T21:40:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.941+0000 7f01cae7b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b0010 con 0x55e263e815f0 2024-04-20T21:40:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.941+0000 7f01cae7b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01af270 con 0x55e263e78a10 2024-04-20T21:40:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.941+0000 7f01cae7b740 10 client.0 _ll_register_callbacks cb 0x7f01ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.941+0000 7f01c6200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e85520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.941+0000 7f01c4e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e85a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.941+0000 7f01c4e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e85a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:34168/0 (socket says 172.21.15.22:34168) 2024-04-20T21:40:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.941+0000 7f01c4e00640 1 -- 172.21.15.22:0/821779667 learned_addr learned my addr 172.21.15.22:0/821779667 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.941+0000 7f01c6200640 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263ef7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.942+0000 7f01c4e00640 1 -- 172.21.15.22:0/821779667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 msgr2=0x55e263ef7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.942+0000 7f01c4e00640 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263ef7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.942+0000 7f01c4e00640 1 -- 172.21.15.22:0/821779667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 msgr2=0x55e263e85520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.942+0000 7f01c4e00640 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e85520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.942+0000 7f01c4e00640 1 -- 172.21.15.22:0/821779667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e263e86330 con 0x55e263e78a10 2024-04-20T21:40:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.942+0000 7f01c6200640 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263ef7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:40:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.942+0000 7f01c4e00640 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e85a60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55e263e7fc50 tx=0x7f01ac005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1890befdab938a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.942+0000 7f01a7400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:40:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.942+0000 7f01a7400640 1 -- 172.21.15.22:0/821779667 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac0037f0 con 0x55e263e78a10 2024-04-20T21:40:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.942+0000 7f01a7400640 1 -- 172.21.15.22:0/821779667 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f01ac004f60 con 0x55e263e78a10 2024-04-20T21:40:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.942+0000 7f01cae7b740 1 -- 172.21.15.22:0/821779667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b8004190 con 0x55e263e78a10 2024-04-20T21:40:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.943+0000 7f01a7400640 1 -- 172.21.15.22:0/821779667 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f01ac005850 con 0x55e263e78a10 2024-04-20T21:40:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.943+0000 7f01cae7b740 10 client.26398 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.943+0000 7f01cae7b740 1 -- 172.21.15.22:0/821779667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e263ef86b0 con 0x55e263e78a10 2024-04-20T21:40:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.943+0000 7f01a7400640 1 -- 172.21.15.22:0/821779667 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b8004190 con 0x55e263e78a10 2024-04-20T21:40:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.943+0000 7f01a7400640 1 -- 172.21.15.22:0/821779667 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f01ac057020 con 0x55e263e78a10 2024-04-20T21:40:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.943+0000 7f01cae7b740 10 client.26398 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.943+0000 7f01cae7b740 10 client.26398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.943+0000 7f01cae7b740 20 client.26398 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.943+0000 7f01cae7b740 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e263ef8eb0 0x55e263efb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.943+0000 7f01cae7b740 4 client.26398 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:40:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.943+0000 7f01cae7b740 1 -- 172.21.15.22:0/821779667 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e263de45b0 con 0x55e263ef8eb0 2024-04-20T21:40:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.943+0000 7f01c6200640 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e263ef8eb0 0x55e263efb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.944+0000 7f01c6200640 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e263ef8eb0 0x55e263efb2d0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e192c1ef4e1575 server_cookie=fde248f1aec0c283 in_seq=0 out_seq=0 2024-04-20T21:40:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.944+0000 7f01a7400640 10 client.26398 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.945+0000 7f01a7400640 1 -- 172.21.15.22:0/821779667 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e263de45b0 con 0x55e263ef8eb0 2024-04-20T21:40:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.945+0000 7f01a7400640 10 client.26398 handle_command_reply: tid=0 2024-04-20T21:40:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.945+0000 7f01cae7b740 1 client.26398 shutdown 2024-04-20T21:40:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.946+0000 7f01cae7b740 1 -- 172.21.15.22:0/821779667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 msgr2=0x55e263e85a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.946+0000 7f01cae7b740 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e85a60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55e263e7fc50 tx=0x7f01ac005c00 comp rx=0 tx=0).stop 2024-04-20T21:40:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.946+0000 7f01cae7b740 1 -- 172.21.15.22:0/821779667 shutdown_connections 2024-04-20T21:40:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.946+0000 7f01cae7b740 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e263ef8eb0 0x55e263efb2d0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.946+0000 7f01cae7b740 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e263e815f0 0x55e263ef7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.946+0000 7f01cae7b740 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e263e78a10 0x55e263e85a60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.946+0000 7f01cae7b740 1 --2- 172.21.15.22:0/821779667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e263e780f0 0x55e263e85520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.946+0000 7f01a7400640 0 client.26398 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.946+0000 7f01cae7b740 1 -- 172.21.15.22:0/821779667 >> 172.21.15.22:0/821779667 conn(0x55e263e72820 msgr2=0x55e263e71a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.946+0000 7f01cae7b740 1 -- 172.21.15.22:0/821779667 shutdown_connections 2024-04-20T21:40:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.946+0000 7f01cae7b740 1 -- 172.21.15.22:0/821779667 wait complete. 2024-04-20T21:40:33.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.946+0000 7f01cae7b740 20 client.26398 trim_cache size 0 max 16384 2024-04-20T21:40:33.952 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:40:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:40:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "230d5556-86e3-4f52-8c05-d5362c019fa1", 2024-04-20T21:40:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:40:33.952 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:40:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.947+0000 7f01c6c00640 1 -- 172.21.15.22:0/3648744383 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f01a80a3d20 msgr2=0x7f01a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.947+0000 7f01c6c00640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f01a80a3d20 0x7f01a80a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f01b40ce790 tx=0x7f01b40d4b30 comp rx=0 tx=0).stop 2024-04-20T21:40:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.947+0000 7f01c6c00640 1 -- 172.21.15.22:0/3648744383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c0102080 msgr2=0x7f01c0074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.947+0000 7f01c6c00640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c0102080 0x7f01c0074260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009fd0 tx=0x7f01b400ef90 comp rx=0 tx=0).stop 2024-04-20T21:40:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.947+0000 7f01c6c00640 1 -- 172.21.15.22:0/3648744383 shutdown_connections 2024-04-20T21:40:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.947+0000 7f01c6c00640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f01a80a3d20 0x7f01a80a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.947+0000 7f01c6c00640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c0074ce0 0x7f01c01a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.947+0000 7f01c6c00640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01c0107150 0x7f01c00747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.947+0000 7f01c6c00640 1 --2- 172.21.15.22:0/3648744383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c0102080 0x7f01c0074260 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.947+0000 7f01c6c00640 1 -- 172.21.15.22:0/3648744383 >> 172.21.15.22:0/3648744383 conn(0x7f01c00fd900 msgr2=0x7f01c00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.947+0000 7f01c6c00640 1 -- 172.21.15.22:0/3648744383 shutdown_connections 2024-04-20T21:40:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:33.947+0000 7f01c6c00640 1 -- 172.21.15.22:0/3648744383 wait complete. 2024-04-20T21:40:33.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:40:33.966 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:40:34.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.174+0000 7fca81400640 1 -- 172.21.15.22:0/2073963574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c107270 msgr2=0x7fca7c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:34.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.174+0000 7fca81400640 1 --2- 172.21.15.22:0/2073963574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c107270 0x7fca7c10ee40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca74009900 tx=0x7fca7402bc30 comp rx=0 tx=0).stop 2024-04-20T21:40:34.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.175+0000 7fca81400640 1 -- 172.21.15.22:0/2073963574 shutdown_connections 2024-04-20T21:40:34.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.175+0000 7fca81400640 1 --2- 172.21.15.22:0/2073963574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c107270 0x7fca7c10ee40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.175+0000 7fca81400640 1 --2- 172.21.15.22:0/2073963574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca7c102b50 0x7fca7c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.175+0000 7fca81400640 1 --2- 172.21.15.22:0/2073963574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c1021a0 0x7fca7c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.175+0000 7fca81400640 1 -- 172.21.15.22:0/2073963574 >> 172.21.15.22:0/2073963574 conn(0x7fca7c0793b0 msgr2=0x7fca7c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:34.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.175+0000 7fca81400640 1 -- 172.21.15.22:0/2073963574 shutdown_connections 2024-04-20T21:40:34.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.175+0000 7fca81400640 1 -- 172.21.15.22:0/2073963574 wait complete. 2024-04-20T21:40:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.176+0000 7fca81400640 1 Processor -- start 2024-04-20T21:40:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.176+0000 7fca81400640 1 -- start start 2024-04-20T21:40:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.176+0000 7fca81400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c1021a0 0x7fca7c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.176+0000 7fca81400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c107270 0x7fca7c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.176+0000 7fca81400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca7c19a810 0x7fca7c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.176+0000 7fca81400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c112360 con 0x7fca7c1021a0 2024-04-20T21:40:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.176+0000 7fca81400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1121e0 con 0x7fca7c19a810 2024-04-20T21:40:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.176+0000 7fca81400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1124e0 con 0x7fca7c107270 2024-04-20T21:40:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.177+0000 7fca7a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c1021a0 0x7fca7c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.177+0000 7fca7a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c1021a0 0x7fca7c1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40784/0 (socket says 172.21.15.22:40784) 2024-04-20T21:40:34.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.177+0000 7fca7a800640 1 -- 172.21.15.22:0/41662703 learned_addr learned my addr 172.21.15.22:0/41662703 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:34.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.177+0000 7fca7b200640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca7c19a810 0x7fca7c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.177+0000 7fca79e00640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c107270 0x7fca7c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.177+0000 7fca7a800640 1 -- 172.21.15.22:0/41662703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c107270 msgr2=0x7fca7c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:34.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.177+0000 7fca7a800640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c107270 0x7fca7c1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.177+0000 7fca7a800640 1 -- 172.21.15.22:0/41662703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca7c19a810 msgr2=0x7fca7c19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:34.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.177+0000 7fca7a800640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca7c19a810 0x7fca7c19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.177+0000 7fca7a800640 1 -- 172.21.15.22:0/41662703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c19b3a0 con 0x7fca7c1021a0 2024-04-20T21:40:34.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.177+0000 7fca7b200640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca7c19a810 0x7fca7c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:34.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.177+0000 7fca7a800640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c1021a0 0x7fca7c1a06a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009d50 tx=0x7fca6c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:34.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.178+0000 7fca73400640 1 -- 172.21.15.22:0/41662703 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c003b70 con 0x7fca7c1021a0 2024-04-20T21:40:34.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.178+0000 7fca81400640 1 -- 172.21.15.22:0/41662703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7c19b690 con 0x7fca7c1021a0 2024-04-20T21:40:34.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.178+0000 7fca73400640 1 -- 172.21.15.22:0/41662703 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca6c003d10 con 0x7fca7c1021a0 2024-04-20T21:40:34.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.178+0000 7fca73400640 1 -- 172.21.15.22:0/41662703 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c010a20 con 0x7fca7c1021a0 2024-04-20T21:40:34.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.178+0000 7fca81400640 1 -- 172.21.15.22:0/41662703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7c1a7870 con 0x7fca7c1021a0 2024-04-20T21:40:34.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.181+0000 7fca73400640 1 -- 172.21.15.22:0/41662703 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fca6c003ec0 con 0x7fca7c1021a0 2024-04-20T21:40:34.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.181+0000 7fca84166740 1 Processor -- start 2024-04-20T21:40:34.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.182+0000 7fca84166740 1 -- start start 2024-04-20T21:40:34.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.182+0000 7fca84166740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f91a0 0x55d0173f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.182+0000 7fca84166740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0173f9ac0 0x55d017402160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.182+0000 7fca84166740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0174026a0 0x55d017404a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.182+0000 7fca84166740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d01735f150 con 0x55d0173f91a0 2024-04-20T21:40:34.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.182+0000 7fca84166740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d017305370 con 0x55d0174026a0 2024-04-20T21:40:34.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.182+0000 7fca84166740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d01735b110 con 0x55d0173f9ac0 2024-04-20T21:40:34.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.182+0000 7fca7b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f91a0 0x55d0173f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.182+0000 7fca7b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f91a0 0x55d0173f9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40786/0 (socket says 172.21.15.22:40786) 2024-04-20T21:40:34.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.182+0000 7fca7b200640 1 -- 172.21.15.22:0/2420505869 learned_addr learned my addr 172.21.15.22:0/2420505869 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:34.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.182+0000 7fca7a800640 1 --2- 172.21.15.22:0/2420505869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0173f9ac0 0x55d017402160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.183+0000 7fca79e00640 1 --2- 172.21.15.22:0/2420505869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0174026a0 0x55d017404a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.183+0000 7fca7b200640 1 -- 172.21.15.22:0/2420505869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0173f9ac0 msgr2=0x55d017402160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.183+0000 7fca7b200640 1 --2- 172.21.15.22:0/2420505869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0173f9ac0 0x55d017402160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.183+0000 7fca7b200640 1 -- 172.21.15.22:0/2420505869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0174026a0 msgr2=0x55d017404a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.183+0000 7fca7b200640 1 --2- 172.21.15.22:0/2420505869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0174026a0 0x55d017404a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.183+0000 7fca7b200640 1 -- 172.21.15.22:0/2420505869 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d01713b7d0 con 0x55d0173f91a0 2024-04-20T21:40:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.183+0000 7fca73400640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fca600a3d20 0x7fca600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.183+0000 7fca7b200640 1 --2- 172.21.15.22:0/2420505869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f91a0 0x55d0173f9580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fca74009900 tx=0x7fca740095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f5c5d6e549ea15 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.183+0000 7fca73400640 1 -- 172.21.15.22:0/41662703 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fca6c0d0ed0 con 0x7fca7c1021a0 2024-04-20T21:40:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.184+0000 7fca7a800640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fca600a3d20 0x7fca600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.184+0000 7fca67400640 1 -- 172.21.15.22:0/2420505869 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74045070 con 0x55d0173f91a0 2024-04-20T21:40:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.184+0000 7fca67400640 1 -- 172.21.15.22:0/2420505869 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca7402c5d0 con 0x55d0173f91a0 2024-04-20T21:40:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.184+0000 7fca67400640 1 -- 172.21.15.22:0/2420505869 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74004f80 con 0x55d0173f91a0 2024-04-20T21:40:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.185+0000 7fca84166740 1 -- 172.21.15.22:0/2420505869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f91a0 msgr2=0x55d0173f9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.185+0000 7fca7a800640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fca600a3d20 0x7fca600a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0ce680 tx=0x7fca6c0d4a20 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.185+0000 7fca84166740 1 --2- 172.21.15.22:0/2420505869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f91a0 0x55d0173f9580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fca74009900 tx=0x7fca740095f0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.185+0000 7fca84166740 1 -- 172.21.15.22:0/2420505869 shutdown_connections 2024-04-20T21:40:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.185+0000 7fca84166740 1 --2- 172.21.15.22:0/2420505869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0174026a0 0x55d017404a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.185+0000 7fca84166740 1 --2- 172.21.15.22:0/2420505869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0173f9ac0 0x55d017402160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.185+0000 7fca84166740 1 --2- 172.21.15.22:0/2420505869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f91a0 0x55d0173f9580 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.185+0000 7fca84166740 1 -- 172.21.15.22:0/2420505869 >> 172.21.15.22:0/2420505869 conn(0x55d0173f38d0 msgr2=0x55d0173f3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.185+0000 7fca84166740 1 -- 172.21.15.22:0/2420505869 shutdown_connections 2024-04-20T21:40:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.185+0000 7fca84166740 1 -- 172.21.15.22:0/2420505869 wait complete. 2024-04-20T21:40:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.185+0000 7fca84166740 1 -- start start 2024-04-20T21:40:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.186+0000 7fca84166740 1 Processor -- start 2024-04-20T21:40:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.186+0000 7fca84166740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d0173ff740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.186+0000 7fca84166740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d0173ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.186+0000 7fca84166740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d0174001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.186+0000 7fca84166740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d017405430 con 0x55d0173f9ac0 2024-04-20T21:40:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.186+0000 7fca84166740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d017318740 con 0x55d0173f91a0 2024-04-20T21:40:34.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.186+0000 7fca84166740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d0174055b0 con 0x55d0174026a0 2024-04-20T21:40:34.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.186+0000 7fca84166740 10 client.0 _ll_register_callbacks cb 0x55d017313b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:34.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.187+0000 7fca79e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d0173ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.187+0000 7fca79e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d0173ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40792/0 (socket says 172.21.15.22:40792) 2024-04-20T21:40:34.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.187+0000 7fca79e00640 1 -- 172.21.15.22:0/3193484190 learned_addr learned my addr 172.21.15.22:0/3193484190 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:34.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.187+0000 7fca7b200640 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d0173ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.187+0000 7fca7b200640 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d0174001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.187+0000 7fca79e00640 1 -- 172.21.15.22:0/3193484190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 msgr2=0x55d0174001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.187+0000 7fca79e00640 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d0174001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.187+0000 7fca79e00640 1 -- 172.21.15.22:0/3193484190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 msgr2=0x55d0173ff740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.187+0000 7fca79e00640 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d0173ff740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.187+0000 7fca79e00640 1 -- 172.21.15.22:0/3193484190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0170713f0 con 0x55d0173f9ac0 2024-04-20T21:40:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.187+0000 7fca79e00640 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d0173ffc80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fca68012700 tx=0x7fca68012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb435df59abae6c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.187+0000 7fca5be00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:40:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.188+0000 7fca5be00640 1 -- 172.21.15.22:0/3193484190 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68004030 con 0x55d0173f9ac0 2024-04-20T21:40:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.188+0000 7fca5be00640 1 -- 172.21.15.22:0/3193484190 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fca680041d0 con 0x55d0173f9ac0 2024-04-20T21:40:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.188+0000 7fca84166740 1 -- 172.21.15.22:0/3193484190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca7402c5d0 con 0x55d0173f9ac0 2024-04-20T21:40:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.188+0000 7fca5be00640 1 -- 172.21.15.22:0/3193484190 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fca680043d0 con 0x55d0173f9ac0 2024-04-20T21:40:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.188+0000 7fca84166740 10 client.16236 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.188+0000 7fca84166740 1 -- 172.21.15.22:0/3193484190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d0170310f0 con 0x55d0173f9ac0 2024-04-20T21:40:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.188+0000 7fca5be00640 1 -- 172.21.15.22:0/3193484190 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca7402c5d0 con 0x55d0173f9ac0 2024-04-20T21:40:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.189+0000 7fca5be00640 1 -- 172.21.15.22:0/3193484190 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fca68003aa0 con 0x55d0173f9ac0 2024-04-20T21:40:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.189+0000 7fca84166740 10 client.16236 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.189+0000 7fca84166740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.189+0000 7fca84166740 20 client.16236 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:34.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.189+0000 7fca84166740 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d017412ad0 0x55d017414f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.189+0000 7fca84166740 4 client.16236 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:34.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.189+0000 7fca84166740 1 -- 172.21.15.22:0/3193484190 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d01736cb70 con 0x55d017412ad0 2024-04-20T21:40:34.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.189+0000 7fca7b200640 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d017412ad0 0x55d017414f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.190+0000 7fca7b200640 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d017412ad0 0x55d017414f10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e42ccff8e41ba0 server_cookie=b698aadbe792c015 in_seq=0 out_seq=0 2024-04-20T21:40:34.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.190+0000 7fca5be00640 10 client.16236 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:34.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca5be00640 1 -- 172.21.15.22:0/3193484190 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d01736cb70 con 0x55d017412ad0 2024-04-20T21:40:34.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca5be00640 10 client.16236 handle_command_reply: tid=0 2024-04-20T21:40:34.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca84166740 1 client.16236 shutdown 2024-04-20T21:40:34.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca84166740 1 -- 172.21.15.22:0/3193484190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 msgr2=0x55d0173ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca84166740 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d0173ffc80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fca68012700 tx=0x7fca68012bc0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca84166740 1 -- 172.21.15.22:0/3193484190 shutdown_connections 2024-04-20T21:40:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca84166740 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d0174001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca84166740 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d0173ffc80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca84166740 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d017412ad0 0x55d017414f10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca84166740 1 --2- 172.21.15.22:0/3193484190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d0173ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca5be00640 0 client.16236 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca84166740 1 -- 172.21.15.22:0/3193484190 >> 172.21.15.22:0/3193484190 conn(0x55d0173f38d0 msgr2=0x55d0173f5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:34.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca84166740 1 -- 172.21.15.22:0/3193484190 shutdown_connections 2024-04-20T21:40:34.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca84166740 1 -- 172.21.15.22:0/3193484190 wait complete. 2024-04-20T21:40:34.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.191+0000 7fca84166740 20 client.16236 trim_cache size 0 max 16384 2024-04-20T21:40:34.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.211+0000 7fca84166740 1 Processor -- start 2024-04-20T21:40:34.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.212+0000 7fca84166740 1 -- start start 2024-04-20T21:40:34.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.212+0000 7fca84166740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d017401e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.212+0000 7fca84166740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d0173fe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.212+0000 7fca84166740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d01740e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.212+0000 7fca84166740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1ae0d0 con 0x55d0173f9ac0 2024-04-20T21:40:34.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.212+0000 7fca84166740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1b0930 con 0x55d0173f91a0 2024-04-20T21:40:34.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.212+0000 7fca84166740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1ae720 con 0x55d0174026a0 2024-04-20T21:40:34.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.212+0000 7fca7b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d017401e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.212+0000 7fca79e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d0173fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.212+0000 7fca7b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d017401e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:33098/0 (socket says 172.21.15.22:33098) 2024-04-20T21:40:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.212+0000 7fca7b200640 1 -- 172.21.15.22:0/2928310895 learned_addr learned my addr 172.21.15.22:0/2928310895 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.212+0000 7fca7b200640 1 --2- 172.21.15.22:0/2928310895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d01740e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.213+0000 7fca7b200640 1 -- 172.21.15.22:0/2928310895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 msgr2=0x55d01740e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.213+0000 7fca7b200640 1 --2- 172.21.15.22:0/2928310895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d01740e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.213+0000 7fca7b200640 1 -- 172.21.15.22:0/2928310895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 msgr2=0x55d0173fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.213+0000 7fca7b200640 1 --2- 172.21.15.22:0/2928310895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d0173fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.213+0000 7fca7b200640 1 -- 172.21.15.22:0/2928310895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d017404a90 con 0x55d0173f91a0 2024-04-20T21:40:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.213+0000 7fca7b200640 1 --2- 172.21.15.22:0/2928310895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d017401e70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fca7402ca20 tx=0x7fca7404c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e091091c2199ae94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.213+0000 7fca66a00640 1 -- 172.21.15.22:0/2928310895 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74045040 con 0x55d0173f91a0 2024-04-20T21:40:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.214+0000 7fca66a00640 1 -- 172.21.15.22:0/2928310895 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca74004a70 con 0x55d0173f91a0 2024-04-20T21:40:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.214+0000 7fca66a00640 1 -- 172.21.15.22:0/2928310895 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74055770 con 0x55d0173f91a0 2024-04-20T21:40:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.214+0000 7fca84166740 1 -- 172.21.15.22:0/2928310895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 msgr2=0x55d017401e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.214+0000 7fca84166740 1 --2- 172.21.15.22:0/2928310895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d017401e70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fca7402ca20 tx=0x7fca7404c040 comp rx=0 tx=0).stop 2024-04-20T21:40:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.214+0000 7fca84166740 1 -- 172.21.15.22:0/2928310895 shutdown_connections 2024-04-20T21:40:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.214+0000 7fca84166740 1 --2- 172.21.15.22:0/2928310895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d01740e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.214+0000 7fca84166740 1 --2- 172.21.15.22:0/2928310895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d0173fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.214+0000 7fca84166740 1 --2- 172.21.15.22:0/2928310895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d017401e70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.214+0000 7fca84166740 1 -- 172.21.15.22:0/2928310895 >> 172.21.15.22:0/2928310895 conn(0x55d0173f38d0 msgr2=0x55d0173f28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.214+0000 7fca84166740 1 -- 172.21.15.22:0/2928310895 shutdown_connections 2024-04-20T21:40:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.214+0000 7fca84166740 1 -- 172.21.15.22:0/2928310895 wait complete. 2024-04-20T21:40:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.215+0000 7fca84166740 1 -- start start 2024-04-20T21:40:34.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.215+0000 7fca84166740 1 Processor -- start 2024-04-20T21:40:34.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.215+0000 7fca84166740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d017406240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.215+0000 7fca84166740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d017406780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.215+0000 7fca84166740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d017406cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.215+0000 7fca84166740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1aedf0 con 0x55d0173f9ac0 2024-04-20T21:40:34.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.215+0000 7fca84166740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1b0260 con 0x55d0173f91a0 2024-04-20T21:40:34.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca84166740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1af4c0 con 0x55d0174026a0 2024-04-20T21:40:34.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca84166740 10 client.0 _ll_register_callbacks cb 0x7fca68009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:34.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca79e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d017406780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca79e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d017406780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40812/0 (socket says 172.21.15.22:40812) 2024-04-20T21:40:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca79e00640 1 -- 172.21.15.22:0/79210264 learned_addr learned my addr 172.21.15.22:0/79210264 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:40:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca7b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d017406240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca7b200640 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d017406cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca79e00640 1 -- 172.21.15.22:0/79210264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 msgr2=0x55d017406cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca79e00640 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d017406cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca79e00640 1 -- 172.21.15.22:0/79210264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 msgr2=0x55d017406240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca79e00640 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d017406240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca79e00640 1 -- 172.21.15.22:0/79210264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d017478050 con 0x55d0173f9ac0 2024-04-20T21:40:34.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.216+0000 7fca7b200640 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d017406cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:34.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.217+0000 7fca79e00640 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d017406780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d0173fefa0 tx=0x7fca68009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4aac3fefbf954 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:34.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.217+0000 7fca5be00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:40:34.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.217+0000 7fca5be00640 1 -- 172.21.15.22:0/79210264 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68028040 con 0x55d0173f9ac0 2024-04-20T21:40:34.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.217+0000 7fca5be00640 1 -- 172.21.15.22:0/79210264 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fca6800fb30 con 0x55d0173f9ac0 2024-04-20T21:40:34.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.217+0000 7fca84166740 1 -- 172.21.15.22:0/79210264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca74004a70 con 0x55d0173f9ac0 2024-04-20T21:40:34.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.217+0000 7fca5be00640 1 -- 172.21.15.22:0/79210264 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fca68039050 con 0x55d0173f9ac0 2024-04-20T21:40:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.217+0000 7fca84166740 10 client.16242 fetch_fsmap learned FSMap version 20 2024-04-20T21:40:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.217+0000 7fca84166740 1 -- 172.21.15.22:0/79210264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d017479330 con 0x55d0173f9ac0 2024-04-20T21:40:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.218+0000 7fca5be00640 1 -- 172.21.15.22:0/79210264 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca74004a70 con 0x55d0173f9ac0 2024-04-20T21:40:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.218+0000 7fca5be00640 1 -- 172.21.15.22:0/79210264 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fca68034030 con 0x55d0173f9ac0 2024-04-20T21:40:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.218+0000 7fca84166740 10 client.16242 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:40:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.218+0000 7fca84166740 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.218+0000 7fca84166740 20 client.16242 populate_metadata read hostname 'smithi022' 2024-04-20T21:40:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.218+0000 7fca84166740 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d017479a90 0x55d01747beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:34.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.218+0000 7fca84166740 4 client.16242 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:40:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.218+0000 7fca84166740 1 -- 172.21.15.22:0/79210264 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d0173669e0 con 0x55d017479a90 2024-04-20T21:40:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.218+0000 7fca7b200640 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d017479a90 0x55d01747beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.219+0000 7fca7b200640 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d017479a90 0x55d01747beb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=450a69cbc92704bd server_cookie=ca4fdcd9591c8362 in_seq=0 out_seq=0 2024-04-20T21:40:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.219+0000 7fca5be00640 10 client.16242 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.219+0000 7fca5be00640 1 -- 172.21.15.22:0/79210264 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d0173669e0 con 0x55d017479a90 2024-04-20T21:40:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.219+0000 7fca5be00640 10 client.16242 handle_command_reply: tid=0 2024-04-20T21:40:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.219+0000 7fca84166740 1 client.16242 shutdown 2024-04-20T21:40:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.219+0000 7fca84166740 1 -- 172.21.15.22:0/79210264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 msgr2=0x55d017406780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.219+0000 7fca84166740 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d017406780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d0173fefa0 tx=0x7fca68009fa0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.220+0000 7fca84166740 1 -- 172.21.15.22:0/79210264 shutdown_connections 2024-04-20T21:40:34.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.220+0000 7fca84166740 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0174026a0 0x55d017406cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.220+0000 7fca84166740 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0173f9ac0 0x55d017406780 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.220+0000 7fca84166740 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d017479a90 0x55d01747beb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.220+0000 7fca84166740 1 --2- 172.21.15.22:0/79210264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0173f91a0 0x55d017406240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.220+0000 7fca5be00640 0 client.16242 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:40:34.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.220+0000 7fca84166740 1 -- 172.21.15.22:0/79210264 >> 172.21.15.22:0/79210264 conn(0x55d0173f38d0 msgr2=0x55d0173f2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:34.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.220+0000 7fca84166740 1 -- 172.21.15.22:0/79210264 shutdown_connections 2024-04-20T21:40:34.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.220+0000 7fca84166740 1 -- 172.21.15.22:0/79210264 wait complete. 2024-04-20T21:40:34.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.220+0000 7fca84166740 20 client.16242 trim_cache size 0 max 16384 2024-04-20T21:40:34.226 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:40:34.226 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:40:34.226 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:40:34.226 INFO:teuthology.orchestra.run.smithi022.stdout: "230d5556-86e3-4f52-8c05-d5362c019fa1": { 2024-04-20T21:40:34.227 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:40:34.227 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "230d5556-86e3-4f52-8c05-d5362c019fa1", 2024-04-20T21:40:34.227 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:40:34.227 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:40:34.227 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:40:34.227 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:40:34.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.220+0000 7fca81400640 1 -- 172.21.15.22:0/41662703 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fca600a3d20 msgr2=0x7fca600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:34.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.220+0000 7fca81400640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fca600a3d20 0x7fca600a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0ce680 tx=0x7fca6c0d4a20 comp rx=0 tx=0).stop 2024-04-20T21:40:34.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.221+0000 7fca81400640 1 -- 172.21.15.22:0/41662703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c1021a0 msgr2=0x7fca7c1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:34.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.221+0000 7fca81400640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c1021a0 0x7fca7c1a06a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009d50 tx=0x7fca6c00c6a0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.221+0000 7fca81400640 1 -- 172.21.15.22:0/41662703 shutdown_connections 2024-04-20T21:40:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.221+0000 7fca81400640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fca600a3d20 0x7fca600a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.221+0000 7fca81400640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca7c19a810 0x7fca7c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.221+0000 7fca81400640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c107270 0x7fca7c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.221+0000 7fca81400640 1 --2- 172.21.15.22:0/41662703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca7c1021a0 0x7fca7c1a06a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.221+0000 7fca81400640 1 -- 172.21.15.22:0/41662703 >> 172.21.15.22:0/41662703 conn(0x7fca7c0793b0 msgr2=0x7fca7c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.221+0000 7fca81400640 1 -- 172.21.15.22:0/41662703 shutdown_connections 2024-04-20T21:40:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:40:34.221+0000 7fca81400640 1 -- 172.21.15.22:0/41662703 wait complete. 2024-04-20T21:40:34.237 INFO:tasks.cephfs.filesystem:scrub status for tag:230d5556-86e3-4f52-8c05-d5362c019fa1 - {'path': '/', 'tag': '230d5556-86e3-4f52-8c05-d5362c019fa1', 'options': 'recursive,force'} 2024-04-20T21:40:34.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:34 smithi022 ceph-mon[16843]: pgmap v207: 97 pgs: 97 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.2 MiB/s wr, 41 op/s 2024-04-20T21:40:34.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:34 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:40:34.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:34 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:34.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:34 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:40:34.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:34 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:34.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:34 smithi122 ceph-mon[22147]: pgmap v207: 97 pgs: 97 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.2 MiB/s wr, 41 op/s 2024-04-20T21:40:34.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:34 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:40:34.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:34 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:34.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:34 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:40:34.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:34 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:34 smithi033 ceph-mon[21626]: pgmap v207: 97 pgs: 97 active+clean; 313 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.2 MiB/s wr, 41 op/s 2024-04-20T21:40:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:34 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:40:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:34 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:34 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:40:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:34 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:36 smithi033 ceph-mon[21626]: pgmap v208: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.4 MiB/s wr, 57 op/s 2024-04-20T21:40:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:36 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:40:36.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:36 smithi022 ceph-mon[16843]: pgmap v208: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.4 MiB/s wr, 57 op/s 2024-04-20T21:40:36.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:36 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:40:36.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:36 smithi122 ceph-mon[22147]: pgmap v208: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.4 MiB/s wr, 57 op/s 2024-04-20T21:40:36.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:36 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:40:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:38 smithi033 ceph-mon[21626]: pgmap v209: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 880 KiB/s wr, 38 op/s 2024-04-20T21:40:38.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:38 smithi022 ceph-mon[16843]: pgmap v209: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 880 KiB/s wr, 38 op/s 2024-04-20T21:40:38.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:38 smithi122 ceph-mon[22147]: pgmap v209: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 880 KiB/s wr, 38 op/s 2024-04-20T21:40:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:40 smithi033 ceph-mon[21626]: pgmap v210: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 880 KiB/s wr, 38 op/s 2024-04-20T21:40:40.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:40 smithi022 ceph-mon[16843]: pgmap v210: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 880 KiB/s wr, 38 op/s 2024-04-20T21:40:40.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:40 smithi122 ceph-mon[22147]: pgmap v210: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 880 KiB/s wr, 38 op/s 2024-04-20T21:40:42.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:42 smithi033 ceph-mon[21626]: pgmap v211: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 902 KiB/s wr, 42 op/s 2024-04-20T21:40:42.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:42 smithi022 ceph-mon[16843]: pgmap v211: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 902 KiB/s wr, 42 op/s 2024-04-20T21:40:42.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:42 smithi122 ceph-mon[22147]: pgmap v211: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 902 KiB/s wr, 42 op/s 2024-04-20T21:40:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:44 smithi033 ceph-mon[21626]: pgmap v212: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 191 KiB/s wr, 19 op/s 2024-04-20T21:40:44.577 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:44 smithi122 ceph-mon[22147]: pgmap v212: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 191 KiB/s wr, 19 op/s 2024-04-20T21:40:44.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:44 smithi022 ceph-mon[16843]: pgmap v212: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 191 KiB/s wr, 19 op/s 2024-04-20T21:40:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:46 smithi033 ceph-mon[21626]: pgmap v213: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 201 KiB/s wr, 21 op/s 2024-04-20T21:40:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:46 smithi022 ceph-mon[16843]: pgmap v213: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 201 KiB/s wr, 21 op/s 2024-04-20T21:40:46.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:46 smithi122 ceph-mon[22147]: pgmap v213: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 201 KiB/s wr, 21 op/s 2024-04-20T21:40:48.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:48 smithi022 ceph-mon[16843]: pgmap v214: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 5 op/s 2024-04-20T21:40:48.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:48 smithi122 ceph-mon[22147]: pgmap v214: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 5 op/s 2024-04-20T21:40:48.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:48 smithi033 ceph-mon[21626]: pgmap v214: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 5 op/s 2024-04-20T21:40:50.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:50 smithi022 ceph-mon[16843]: pgmap v215: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 5 op/s 2024-04-20T21:40:50.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:50 smithi122 ceph-mon[22147]: pgmap v215: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 5 op/s 2024-04-20T21:40:50.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:50 smithi033 ceph-mon[21626]: pgmap v215: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 5 op/s 2024-04-20T21:40:52.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:52 smithi022 ceph-mon[16843]: pgmap v216: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 6 op/s 2024-04-20T21:40:52.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:52 smithi122 ceph-mon[22147]: pgmap v216: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 6 op/s 2024-04-20T21:40:52.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:52 smithi033 ceph-mon[21626]: pgmap v216: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 6 op/s 2024-04-20T21:40:54.605 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:54 smithi033 ceph-mon[21626]: pgmap v217: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 3 op/s 2024-04-20T21:40:54.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:54 smithi022 ceph-mon[16843]: pgmap v217: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 3 op/s 2024-04-20T21:40:54.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:54 smithi122 ceph-mon[22147]: pgmap v217: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 3 op/s 2024-04-20T21:40:56.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:56 smithi022 ceph-mon[16843]: pgmap v218: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 4 op/s 2024-04-20T21:40:56.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:56 smithi122 ceph-mon[22147]: pgmap v218: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 4 op/s 2024-04-20T21:40:56.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:56 smithi033 ceph-mon[21626]: pgmap v218: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 4 op/s 2024-04-20T21:40:58.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:40:58 smithi022 ceph-mon[16843]: pgmap v219: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-04-20T21:40:58.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:40:58 smithi122 ceph-mon[22147]: pgmap v219: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-04-20T21:40:58.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:40:58 smithi033 ceph-mon[21626]: pgmap v219: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-04-20T21:41:00.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:00 smithi022 ceph-mon[16843]: pgmap v220: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-04-20T21:41:00.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:00 smithi122 ceph-mon[22147]: pgmap v220: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-04-20T21:41:00.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:00 smithi033 ceph-mon[21626]: pgmap v220: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-04-20T21:41:02.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:02 smithi022 ceph-mon[16843]: pgmap v221: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 3 op/s 2024-04-20T21:41:02.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:02 smithi122 ceph-mon[22147]: pgmap v221: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 3 op/s 2024-04-20T21:41:02.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:02 smithi033 ceph-mon[21626]: pgmap v221: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 3 op/s 2024-04-20T21:41:04.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:41:04.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:41:04.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.469+0000 7faa06e00640 1 -- 172.21.15.22:0/1899389336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa001094e0 msgr2=0x7faa001110b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:04.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.469+0000 7faa06e00640 1 --2- 172.21.15.22:0/1899389336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa001094e0 0x7faa001110b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009860 tx=0x7fa9f802bb70 comp rx=0 tx=0).stop 2024-04-20T21:41:04.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.469+0000 7faa06e00640 1 -- 172.21.15.22:0/1899389336 shutdown_connections 2024-04-20T21:41:04.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.469+0000 7faa06e00640 1 --2- 172.21.15.22:0/1899389336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa001094e0 0x7faa001110b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.469+0000 7faa06e00640 1 --2- 172.21.15.22:0/1899389336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa00104dc0 0x7faa00108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.469+0000 7faa06e00640 1 --2- 172.21.15.22:0/1899389336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa00104410 0x7faa001047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.469+0000 7faa06e00640 1 -- 172.21.15.22:0/1899389336 >> 172.21.15.22:0/1899389336 conn(0x7faa000ffcf0 msgr2=0x7faa00102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:04.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.469+0000 7faa06e00640 1 -- 172.21.15.22:0/1899389336 shutdown_connections 2024-04-20T21:41:04.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.469+0000 7faa06e00640 1 -- 172.21.15.22:0/1899389336 wait complete. 2024-04-20T21:41:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.470+0000 7faa06e00640 1 Processor -- start 2024-04-20T21:41:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.470+0000 7faa06e00640 1 -- start start 2024-04-20T21:41:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.471+0000 7faa06e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa00104410 0x7faa001a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.471+0000 7faa06e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa001094e0 0x7faa001a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.471+0000 7faa06e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0019a8e0 0x7faa0019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.471+0000 7faa06e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faa001145a0 con 0x7faa001094e0 2024-04-20T21:41:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.471+0000 7faa06e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa00114420 con 0x7faa00104410 2024-04-20T21:41:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.471+0000 7faa06e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa00114720 con 0x7faa0019a8e0 2024-04-20T21:41:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.471+0000 7fa9f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa001094e0 0x7faa001a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.471+0000 7faa04e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0019a8e0 0x7faa0019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.471+0000 7fa9ffe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa00104410 0x7faa001a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.471+0000 7fa9f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa001094e0 0x7faa001a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42960/0 (socket says 172.21.15.22:42960) 2024-04-20T21:41:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.471+0000 7faa04e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0019a8e0 0x7faa0019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:49082/0 (socket says 172.21.15.22:49082) 2024-04-20T21:41:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.471+0000 7fa9f7400640 1 -- 172.21.15.22:0/3631136258 learned_addr learned my addr 172.21.15.22:0/3631136258 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.472+0000 7faa04e00640 1 -- 172.21.15.22:0/3631136258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa00104410 msgr2=0x7faa001a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.472+0000 7faa04e00640 1 --2- 172.21.15.22:0/3631136258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa00104410 0x7faa001a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.472+0000 7faa04e00640 1 -- 172.21.15.22:0/3631136258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa001094e0 msgr2=0x7faa001a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.472+0000 7faa04e00640 1 --2- 172.21.15.22:0/3631136258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa001094e0 0x7faa001a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.472+0000 7faa04e00640 1 -- 172.21.15.22:0/3631136258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0019b400 con 0x7faa0019a8e0 2024-04-20T21:41:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.472+0000 7faa04e00640 1 --2- 172.21.15.22:0/3631136258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0019a8e0 0x7faa0019ad90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa9f802c0b0 tx=0x7fa9f8004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.472+0000 7fa9fd600640 1 -- 172.21.15.22:0/3631136258 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8041030 con 0x7faa0019a8e0 2024-04-20T21:41:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.472+0000 7fa9fd600640 1 -- 172.21.15.22:0/3631136258 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f802c510 con 0x7faa0019a8e0 2024-04-20T21:41:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.472+0000 7faa06e00640 1 -- 172.21.15.22:0/3631136258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0019b690 con 0x7faa0019a8e0 2024-04-20T21:41:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.472+0000 7fa9fd600640 1 -- 172.21.15.22:0/3631136258 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8039040 con 0x7faa0019a8e0 2024-04-20T21:41:04.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.473+0000 7faa06e00640 1 -- 172.21.15.22:0/3631136258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001a7880 con 0x7faa0019a8e0 2024-04-20T21:41:04.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.476+0000 7fa9fd600640 1 -- 172.21.15.22:0/3631136258 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa9f8044070 con 0x7faa0019a8e0 2024-04-20T21:41:04.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.476+0000 7faa099f9740 1 Processor -- start 2024-04-20T21:41:04.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.476+0000 7faa099f9740 1 -- start start 2024-04-20T21:41:04.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.476+0000 7faa099f9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 0x55b6b0f52580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.476+0000 7faa099f9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f5b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.476+0000 7faa099f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f5da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.476+0000 7faa099f9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b6b0eb8150 con 0x55b6b0f521a0 2024-04-20T21:41:04.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.476+0000 7faa099f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b6b0e5e370 con 0x55b6b0f5b6a0 2024-04-20T21:41:04.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.476+0000 7faa099f9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b6b0eb4110 con 0x55b6b0f52ac0 2024-04-20T21:41:04.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.477+0000 7fa9f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 0x55b6b0f52580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.477+0000 7fa9f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 0x55b6b0f52580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42966/0 (socket says 172.21.15.22:42966) 2024-04-20T21:41:04.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.477+0000 7fa9f7400640 1 -- 172.21.15.22:0/1526683926 learned_addr learned my addr 172.21.15.22:0/1526683926 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:04.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.477+0000 7fa9ffe00640 1 --2- 172.21.15.22:0/1526683926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f5b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.477+0000 7fa9f7400640 1 --2- 172.21.15.22:0/1526683926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f5da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.477+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1526683926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f5b6a0 msgr2=0x55b6b0f5da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:04.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.477+0000 7fa9ffe00640 1 --2- 172.21.15.22:0/1526683926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f5da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.477+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1526683926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 msgr2=0x55b6b0f52580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:04.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.477+0000 7fa9ffe00640 1 --2- 172.21.15.22:0/1526683926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 0x55b6b0f52580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.477+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1526683926 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b0c947d0 con 0x55b6b0f52ac0 2024-04-20T21:41:04.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.477+0000 7fa9f7400640 1 --2- 172.21.15.22:0/1526683926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 0x55b6b0f52580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:04.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.477+0000 7fa9ffe00640 1 --2- 172.21.15.22:0/1526683926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f5b160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0005fd0 tx=0x7fa9f00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fb8f7925228f0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:04.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.478+0000 7fa9f5600640 1 -- 172.21.15.22:0/1526683926 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0004d20 con 0x55b6b0f52ac0 2024-04-20T21:41:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.478+0000 7fa9f5600640 1 -- 172.21.15.22:0/1526683926 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f0004050 con 0x55b6b0f52ac0 2024-04-20T21:41:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.478+0000 7fa9f5600640 1 -- 172.21.15.22:0/1526683926 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0005240 con 0x55b6b0f52ac0 2024-04-20T21:41:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.478+0000 7fa9fd600640 1 --2- 172.21.15.22:0/3631136258 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa9e40a3d20 0x7fa9e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.478+0000 7faa099f9740 1 -- 172.21.15.22:0/1526683926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f52ac0 msgr2=0x55b6b0f5b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.478+0000 7faa099f9740 1 --2- 172.21.15.22:0/1526683926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f5b160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0005fd0 tx=0x7fa9f00118d0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.479+0000 7faa099f9740 1 -- 172.21.15.22:0/1526683926 shutdown_connections 2024-04-20T21:41:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.479+0000 7fa9fd600640 1 -- 172.21.15.22:0/3631136258 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fa9f80f1a60 con 0x7faa0019a8e0 2024-04-20T21:41:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.479+0000 7faa099f9740 1 --2- 172.21.15.22:0/1526683926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f5da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.479+0000 7faa099f9740 1 --2- 172.21.15.22:0/1526683926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f5b160 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.479+0000 7faa099f9740 1 --2- 172.21.15.22:0/1526683926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 0x55b6b0f52580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.479+0000 7faa099f9740 1 -- 172.21.15.22:0/1526683926 >> 172.21.15.22:0/1526683926 conn(0x55b6b0f4c8d0 msgr2=0x55b6b0f4ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.479+0000 7fa9f7400640 1 --2- 172.21.15.22:0/3631136258 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa9e40a3d20 0x7fa9e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.479+0000 7faa099f9740 1 -- 172.21.15.22:0/1526683926 shutdown_connections 2024-04-20T21:41:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.479+0000 7faa099f9740 1 -- 172.21.15.22:0/1526683926 wait complete. 2024-04-20T21:41:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.479+0000 7fa9f7400640 1 --2- 172.21.15.22:0/3631136258 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa9e40a3d20 0x7fa9e40a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55b6b0f4da60 tx=0x7fa9e8009610 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.479+0000 7faa099f9740 1 -- start start 2024-04-20T21:41:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.480+0000 7faa099f9740 1 Processor -- start 2024-04-20T21:41:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.480+0000 7faa099f9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 0x55b6b0f58690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.480+0000 7faa099f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f58bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.480+0000 7faa099f9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f59110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.480+0000 7faa099f9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b6b0f5e490 con 0x55b6b0f521a0 2024-04-20T21:41:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.480+0000 7faa099f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b6b0e71740 con 0x55b6b0f52ac0 2024-04-20T21:41:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.480+0000 7faa099f9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b6b0f5e610 con 0x55b6b0f5b6a0 2024-04-20T21:41:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.480+0000 7faa099f9740 10 client.0 _ll_register_callbacks cb 0x55b6b0e6cb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.480+0000 7fa9ffe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 0x55b6b0f58690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.480+0000 7fa9ffe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 0x55b6b0f58690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42978/0 (socket says 172.21.15.22:42978) 2024-04-20T21:41:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.480+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1038415123 learned_addr learned my addr 172.21.15.22:0/1038415123 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.480+0000 7fa9f7400640 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f58bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.481+0000 7faa04e00640 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f59110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.481+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1038415123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f5b6a0 msgr2=0x55b6b0f59110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.481+0000 7fa9ffe00640 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f59110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.481+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1038415123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f52ac0 msgr2=0x55b6b0f58bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.481+0000 7fa9ffe00640 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f58bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.481+0000 7fa9ffe00640 1 -- 172.21.15.22:0/1038415123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b0bca3f0 con 0x55b6b0f521a0 2024-04-20T21:41:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.481+0000 7fa9ffe00640 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 0x55b6b0f58690 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0005f40 tx=0x7fa9f0004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc6776fb67f521e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.481+0000 7fa9ee000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:41:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.481+0000 7fa9ee000640 1 -- 172.21.15.22:0/1038415123 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0028070 con 0x55b6b0f521a0 2024-04-20T21:41:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.482+0000 7fa9ee000640 1 -- 172.21.15.22:0/1038415123 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fa9f0005040 con 0x55b6b0f521a0 2024-04-20T21:41:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.482+0000 7faa099f9740 1 -- 172.21.15.22:0/1038415123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f0004050 con 0x55b6b0f521a0 2024-04-20T21:41:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.482+0000 7fa9ee000640 1 -- 172.21.15.22:0/1038415123 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa9f0004e10 con 0x55b6b0f521a0 2024-04-20T21:41:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.482+0000 7faa099f9740 10 client.16254 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:04.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.482+0000 7faa099f9740 1 -- 172.21.15.22:0/1038415123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b6b0b8a0f0 con 0x55b6b0f521a0 2024-04-20T21:41:04.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.482+0000 7fa9ee000640 1 -- 172.21.15.22:0/1038415123 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f0004050 con 0x55b6b0f521a0 2024-04-20T21:41:04.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.482+0000 7fa9ee000640 1 -- 172.21.15.22:0/1038415123 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fa9f0024730 con 0x55b6b0f521a0 2024-04-20T21:41:04.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.482+0000 7faa099f9740 10 client.16254 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:04.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.482+0000 7faa099f9740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:04.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.482+0000 7faa099f9740 20 client.16254 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:04.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.483+0000 7faa099f9740 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b6b0f6b9a0 0x55b6b0f6dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.483+0000 7faa099f9740 4 client.16254 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:04.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.483+0000 7faa099f9740 1 -- 172.21.15.22:0/1038415123 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6b0ec5b70 con 0x55b6b0f6b9a0 2024-04-20T21:41:04.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.483+0000 7fa9f7400640 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b6b0f6b9a0 0x55b6b0f6dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.484+0000 7fa9f7400640 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b6b0f6b9a0 0x55b6b0f6dd90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84503c274dfbab89 server_cookie=d2036b9bf5115701 in_seq=0 out_seq=0 2024-04-20T21:41:04.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.484+0000 7fa9ee000640 10 client.16254 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:04.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.484+0000 7fa9ee000640 1 -- 172.21.15.22:0/1038415123 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6b0ec5b70 con 0x55b6b0f6b9a0 2024-04-20T21:41:04.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.484+0000 7fa9ee000640 10 client.16254 handle_command_reply: tid=0 2024-04-20T21:41:04.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7faa099f9740 1 client.16254 shutdown 2024-04-20T21:41:04.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7faa099f9740 1 -- 172.21.15.22:0/1038415123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 msgr2=0x55b6b0f58690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:04.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7faa099f9740 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 0x55b6b0f58690 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0005f40 tx=0x7fa9f0004410 comp rx=0 tx=0).stop 2024-04-20T21:41:04.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7faa099f9740 1 -- 172.21.15.22:0/1038415123 shutdown_connections 2024-04-20T21:41:04.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7faa099f9740 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f59110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7faa099f9740 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b6b0f6b9a0 0x55b6b0f6dd90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7faa099f9740 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f58bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7faa099f9740 1 --2- 172.21.15.22:0/1038415123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f521a0 0x55b6b0f58690 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7fa9ee000640 0 client.16254 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:04.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7faa099f9740 1 -- 172.21.15.22:0/1038415123 >> 172.21.15.22:0/1038415123 conn(0x55b6b0f4c8d0 msgr2=0x55b6b0f4e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:04.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7faa099f9740 1 -- 172.21.15.22:0/1038415123 shutdown_connections 2024-04-20T21:41:04.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7faa099f9740 1 -- 172.21.15.22:0/1038415123 wait complete. 2024-04-20T21:41:04.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.485+0000 7faa099f9740 20 client.16254 trim_cache size 0 max 16384 2024-04-20T21:41:04.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.505+0000 7faa099f9740 1 Processor -- start 2024-04-20T21:41:04.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.505+0000 7faa099f9740 1 -- start start 2024-04-20T21:41:04.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.505+0000 7faa099f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f521a0 0x55b6b0f5adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.505+0000 7faa099f9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f67380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.505+0000 7faa099f9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.505+0000 7faa099f9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faa001adf00 con 0x55b6b0f5b6a0 2024-04-20T21:41:04.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.505+0000 7faa099f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b0760 con 0x55b6b0f521a0 2024-04-20T21:41:04.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.505+0000 7faa099f9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa001ae550 con 0x55b6b0f52ac0 2024-04-20T21:41:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.506+0000 7faa04e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.506+0000 7faa04e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42986/0 (socket says 172.21.15.22:42986) 2024-04-20T21:41:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.506+0000 7faa04e00640 1 -- 172.21.15.22:0/1736419734 learned_addr learned my addr 172.21.15.22:0/1736419734 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.506+0000 7fa9f7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f67380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.506+0000 7fa9ffe00640 1 --2- 172.21.15.22:0/1736419734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f521a0 0x55b6b0f5adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.506+0000 7fa9f7400640 1 -- 172.21.15.22:0/1736419734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f521a0 msgr2=0x55b6b0f5adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.506+0000 7fa9f7400640 1 --2- 172.21.15.22:0/1736419734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f521a0 0x55b6b0f5adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.506+0000 7fa9f7400640 1 -- 172.21.15.22:0/1736419734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f5b6a0 msgr2=0x55b6b0f539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.506+0000 7fa9f7400640 1 --2- 172.21.15.22:0/1736419734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.506+0000 7fa9f7400640 1 -- 172.21.15.22:0/1736419734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b0f4c700 con 0x55b6b0f52ac0 2024-04-20T21:41:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.506+0000 7faa04e00640 1 --2- 172.21.15.22:0/1736419734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.506+0000 7fa9f7400640 1 --2- 172.21.15.22:0/1736419734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f67380 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55b6b0f6e560 tx=0x7fa9e8013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9de2d542b6b55bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.507+0000 7fa9f4c00640 1 -- 172.21.15.22:0/1736419734 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e802d250 con 0x55b6b0f52ac0 2024-04-20T21:41:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.507+0000 7fa9f4c00640 1 -- 172.21.15.22:0/1736419734 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e8033070 con 0x55b6b0f52ac0 2024-04-20T21:41:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.507+0000 7fa9f4c00640 1 -- 172.21.15.22:0/1736419734 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800fc80 con 0x55b6b0f52ac0 2024-04-20T21:41:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.507+0000 7faa099f9740 1 -- 172.21.15.22:0/1736419734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f52ac0 msgr2=0x55b6b0f67380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.507+0000 7faa099f9740 1 --2- 172.21.15.22:0/1736419734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f67380 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55b6b0f6e560 tx=0x7fa9e8013040 comp rx=0 tx=0).stop 2024-04-20T21:41:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.507+0000 7faa099f9740 1 -- 172.21.15.22:0/1736419734 shutdown_connections 2024-04-20T21:41:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.507+0000 7faa099f9740 1 --2- 172.21.15.22:0/1736419734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.507+0000 7faa099f9740 1 --2- 172.21.15.22:0/1736419734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f67380 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.507+0000 7faa099f9740 1 --2- 172.21.15.22:0/1736419734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f521a0 0x55b6b0f5adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.507+0000 7faa099f9740 1 -- 172.21.15.22:0/1736419734 >> 172.21.15.22:0/1736419734 conn(0x55b6b0f4c8d0 msgr2=0x55b6b0f56e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.507+0000 7faa099f9740 1 -- 172.21.15.22:0/1736419734 shutdown_connections 2024-04-20T21:41:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.507+0000 7faa099f9740 1 -- 172.21.15.22:0/1736419734 wait complete. 2024-04-20T21:41:04.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.508+0000 7faa099f9740 1 -- start start 2024-04-20T21:41:04.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.508+0000 7faa099f9740 1 Processor -- start 2024-04-20T21:41:04.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.508+0000 7faa099f9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f521a0 0x55b6b0f63a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.508+0000 7faa099f9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f54d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.508+0000 7faa099f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f55250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.508+0000 7faa099f9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faa001aec20 con 0x55b6b0f52ac0 2024-04-20T21:41:04.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.508+0000 7faa099f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b0090 con 0x55b6b0f5b6a0 2024-04-20T21:41:04.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.508+0000 7faa099f9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa001af2f0 con 0x55b6b0f521a0 2024-04-20T21:41:04.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.508+0000 7faa099f9740 10 client.0 _ll_register_callbacks cb 0x7fa9e8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:04.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.509+0000 7fa9f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f54d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.509+0000 7fa9f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f54d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43000/0 (socket says 172.21.15.22:43000) 2024-04-20T21:41:04.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.509+0000 7fa9ffe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f521a0 0x55b6b0f63a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.509+0000 7fa9f7400640 1 -- 172.21.15.22:0/661570836 learned_addr learned my addr 172.21.15.22:0/661570836 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:04.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.509+0000 7faa04e00640 1 --2- 172.21.15.22:0/661570836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f55250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.509+0000 7fa9ffe00640 1 -- 172.21.15.22:0/661570836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f5b6a0 msgr2=0x55b6b0f55250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:04.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.509+0000 7fa9ffe00640 1 --2- 172.21.15.22:0/661570836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f55250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.509+0000 7fa9ffe00640 1 -- 172.21.15.22:0/661570836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f52ac0 msgr2=0x55b6b0f54d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:04.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.509+0000 7fa9ffe00640 1 --2- 172.21.15.22:0/661570836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f54d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.509+0000 7fa9ffe00640 1 -- 172.21.15.22:0/661570836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b0f5da90 con 0x55b6b0f521a0 2024-04-20T21:41:04.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.509+0000 7fa9ffe00640 1 --2- 172.21.15.22:0/661570836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f521a0 0x55b6b0f63a80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009730 tx=0x7fa9f0004c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=931aa3012720cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:04.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.509+0000 7fa9ee000640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:41:04.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.510+0000 7fa9ee000640 1 -- 172.21.15.22:0/661570836 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0028040 con 0x55b6b0f521a0 2024-04-20T21:41:04.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.510+0000 7faa099f9740 1 -- 172.21.15.22:0/661570836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e8033070 con 0x55b6b0f521a0 2024-04-20T21:41:04.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.510+0000 7fa9ee000640 1 -- 172.21.15.22:0/661570836 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fa9f0024440 con 0x55b6b0f521a0 2024-04-20T21:41:04.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.510+0000 7fa9ee000640 1 -- 172.21.15.22:0/661570836 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa9f0024640 con 0x55b6b0f521a0 2024-04-20T21:41:04.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.510+0000 7faa099f9740 10 client.26434 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:04.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.510+0000 7faa099f9740 1 -- 172.21.15.22:0/661570836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b6b0fd1a00 con 0x55b6b0f521a0 2024-04-20T21:41:04.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.510+0000 7fa9ee000640 1 -- 172.21.15.22:0/661570836 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e8033070 con 0x55b6b0f521a0 2024-04-20T21:41:04.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.510+0000 7fa9ee000640 1 -- 172.21.15.22:0/661570836 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fa9f000f480 con 0x55b6b0f521a0 2024-04-20T21:41:04.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.510+0000 7faa099f9740 10 client.26434 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:04.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.511+0000 7faa099f9740 10 client.26434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:04.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.511+0000 7faa099f9740 20 client.26434 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.511+0000 7faa099f9740 1 --2- 172.21.15.22:0/661570836 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b6b0fd20d0 0x55b6b0fd44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.511+0000 7faa099f9740 4 client.26434 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:41:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.511+0000 7faa099f9740 1 -- 172.21.15.22:0/661570836 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6b0ebf9e0 con 0x55b6b0fd20d0 2024-04-20T21:41:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.511+0000 7fa9f7400640 1 --2- 172.21.15.22:0/661570836 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b6b0fd20d0 0x55b6b0fd44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7fa9f7400640 1 --2- 172.21.15.22:0/661570836 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b6b0fd20d0 0x55b6b0fd44c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82ce81df7c293ad3 server_cookie=bffd54a9a1904944 in_seq=0 out_seq=0 2024-04-20T21:41:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7fa9ee000640 10 client.26434 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7fa9ee000640 1 -- 172.21.15.22:0/661570836 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b6b0ebf9e0 con 0x55b6b0fd20d0 2024-04-20T21:41:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7fa9ee000640 10 client.26434 handle_command_reply: tid=0 2024-04-20T21:41:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7faa099f9740 1 client.26434 shutdown 2024-04-20T21:41:04.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7faa099f9740 1 -- 172.21.15.22:0/661570836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f521a0 msgr2=0x55b6b0f63a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:04.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7faa099f9740 1 --2- 172.21.15.22:0/661570836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f521a0 0x55b6b0f63a80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009730 tx=0x7fa9f0004c10 comp rx=0 tx=0).stop 2024-04-20T21:41:04.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7faa099f9740 1 -- 172.21.15.22:0/661570836 shutdown_connections 2024-04-20T21:41:04.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7faa099f9740 1 --2- 172.21.15.22:0/661570836 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b6b0fd20d0 0x55b6b0fd44c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7faa099f9740 1 --2- 172.21.15.22:0/661570836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6b0f5b6a0 0x55b6b0f55250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7faa099f9740 1 --2- 172.21.15.22:0/661570836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6b0f52ac0 0x55b6b0f54d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7fa9ee000640 0 client.26434 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7faa099f9740 1 --2- 172.21.15.22:0/661570836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6b0f521a0 0x55b6b0f63a80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.512+0000 7faa099f9740 1 -- 172.21.15.22:0/661570836 >> 172.21.15.22:0/661570836 conn(0x55b6b0f4c8d0 msgr2=0x55b6b0f576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa099f9740 1 -- 172.21.15.22:0/661570836 shutdown_connections 2024-04-20T21:41:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa099f9740 1 -- 172.21.15.22:0/661570836 wait complete. 2024-04-20T21:41:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa099f9740 20 client.26434 trim_cache size 0 max 16384 2024-04-20T21:41:04.519 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:41:04.520 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:41:04.520 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:41:04.520 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:41:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa06e00640 1 -- 172.21.15.22:0/3631136258 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa9e40a3d20 msgr2=0x7fa9e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa06e00640 1 --2- 172.21.15.22:0/3631136258 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa9e40a3d20 0x7fa9e40a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55b6b0f4da60 tx=0x7fa9e8009610 comp rx=0 tx=0).stop 2024-04-20T21:41:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa06e00640 1 -- 172.21.15.22:0/3631136258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0019a8e0 msgr2=0x7faa0019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa06e00640 1 --2- 172.21.15.22:0/3631136258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0019a8e0 0x7faa0019ad90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa9f802c0b0 tx=0x7fa9f8004530 comp rx=0 tx=0).stop 2024-04-20T21:41:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa06e00640 1 -- 172.21.15.22:0/3631136258 shutdown_connections 2024-04-20T21:41:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa06e00640 1 --2- 172.21.15.22:0/3631136258 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa9e40a3d20 0x7fa9e40a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa06e00640 1 --2- 172.21.15.22:0/3631136258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0019a8e0 0x7faa0019ad90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa06e00640 1 --2- 172.21.15.22:0/3631136258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa001094e0 0x7faa001a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa06e00640 1 --2- 172.21.15.22:0/3631136258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa00104410 0x7faa001a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.513+0000 7faa06e00640 1 -- 172.21.15.22:0/3631136258 >> 172.21.15.22:0/3631136258 conn(0x7faa000ffcf0 msgr2=0x7faa00101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:04.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.514+0000 7faa06e00640 1 -- 172.21.15.22:0/3631136258 shutdown_connections 2024-04-20T21:41:04.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.514+0000 7faa06e00640 1 -- 172.21.15.22:0/3631136258 wait complete. 2024-04-20T21:41:04.531 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:41:04.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:04.606 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:04 smithi022 ceph-mon[16843]: pgmap v222: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.8 KiB/s wr, 1 op/s 2024-04-20T21:41:04.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.749+0000 7fb33dc00640 1 -- 172.21.15.22:0/3584738292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb338108400 msgr2=0x7fb338108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:04.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.749+0000 7fb33dc00640 1 --2- 172.21.15.22:0/3584738292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb338108400 0x7fb338108860 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb324009900 tx=0x7fb32402bcd0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.749+0000 7fb33dc00640 1 -- 172.21.15.22:0/3584738292 shutdown_connections 2024-04-20T21:41:04.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.749+0000 7fb33dc00640 1 --2- 172.21.15.22:0/3584738292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb338108da0 0x7fb338115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.749+0000 7fb33dc00640 1 --2- 172.21.15.22:0/3584738292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb338108400 0x7fb338108860 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.749+0000 7fb33dc00640 1 --2- 172.21.15.22:0/3584738292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb33810e470 0x7fb33810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.749+0000 7fb33dc00640 1 -- 172.21.15.22:0/3584738292 >> 172.21.15.22:0/3584738292 conn(0x7fb338077ae0 msgr2=0x7fb338079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:04.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.749+0000 7fb33dc00640 1 -- 172.21.15.22:0/3584738292 shutdown_connections 2024-04-20T21:41:04.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.749+0000 7fb33dc00640 1 -- 172.21.15.22:0/3584738292 wait complete. 2024-04-20T21:41:04.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.750+0000 7fb33dc00640 1 Processor -- start 2024-04-20T21:41:04.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.750+0000 7fb33dc00640 1 -- start start 2024-04-20T21:41:04.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.751+0000 7fb33dc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb338108400 0x7fb3381a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.751+0000 7fb33dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb338108da0 0x7fb3381a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.751+0000 7fb33dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb33810e470 0x7fb33819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.751+0000 7fb33dc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb338118780 con 0x7fb338108da0 2024-04-20T21:41:04.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.751+0000 7fb33dc00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb338118600 con 0x7fb338108400 2024-04-20T21:41:04.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.751+0000 7fb33dc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb338118900 con 0x7fb33810e470 2024-04-20T21:41:04.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.751+0000 7fb337400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb338108da0 0x7fb3381a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.751+0000 7fb337e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb338108400 0x7fb3381a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.751+0000 7fb337400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb338108da0 0x7fb3381a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43012/0 (socket says 172.21.15.22:43012) 2024-04-20T21:41:04.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.751+0000 7fb337e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb338108400 0x7fb3381a4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60912/0 (socket says 172.21.15.22:60912) 2024-04-20T21:41:04.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.751+0000 7fb337400640 1 -- 172.21.15.22:0/75975078 learned_addr learned my addr 172.21.15.22:0/75975078 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:04.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.751+0000 7fb33d200640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb33810e470 0x7fb33819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.752+0000 7fb337e00640 1 -- 172.21.15.22:0/75975078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb33810e470 msgr2=0x7fb33819ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:04.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.752+0000 7fb337e00640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb33810e470 0x7fb33819ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.752+0000 7fb337e00640 1 -- 172.21.15.22:0/75975078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb338108da0 msgr2=0x7fb3381a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:04.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.752+0000 7fb337e00640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb338108da0 0x7fb3381a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:04.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.752+0000 7fb337e00640 1 -- 172.21.15.22:0/75975078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33819f1c0 con 0x7fb338108400 2024-04-20T21:41:04.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.752+0000 7fb337e00640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb338108400 0x7fb3381a4840 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb32800e9f0 tx=0x7fb32800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:04.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.752+0000 7fb334c00640 1 -- 172.21.15.22:0/75975078 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32800ccc0 con 0x7fb338108400 2024-04-20T21:41:04.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.752+0000 7fb33dc00640 1 -- 172.21.15.22:0/75975078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33819f4b0 con 0x7fb338108400 2024-04-20T21:41:04.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.752+0000 7fb334c00640 1 -- 172.21.15.22:0/75975078 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb328010910 con 0x7fb338108400 2024-04-20T21:41:04.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.752+0000 7fb334c00640 1 -- 172.21.15.22:0/75975078 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328018960 con 0x7fb338108400 2024-04-20T21:41:04.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.752+0000 7fb33dc00640 1 -- 172.21.15.22:0/75975078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3381ab700 con 0x7fb338108400 2024-04-20T21:41:04.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.753+0000 7fb33dc00640 1 -- 172.21.15.22:0/75975078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3380690f0 con 0x7fb338108400 2024-04-20T21:41:04.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.756+0000 7fb334c00640 1 -- 172.21.15.22:0/75975078 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb328010ac0 con 0x7fb338108400 2024-04-20T21:41:04.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.757+0000 7fb334c00640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb31c0a3da0 0x7fb31c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:04.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.757+0000 7fb334c00640 1 -- 172.21.15.22:0/75975078 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fb3280d0c30 con 0x7fb338108400 2024-04-20T21:41:04.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.757+0000 7fb337400640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb31c0a3da0 0x7fb31c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:04.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.758+0000 7fb337400640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb31c0a3da0 0x7fb31c0a6260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb338107830 tx=0x7fb324005a20 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:04.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:04.759+0000 7fb334c00640 1 -- 172.21.15.22:0/75975078 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb32809cbd0 con 0x7fb338108400 2024-04-20T21:41:04.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:04 smithi122 ceph-mon[22147]: pgmap v222: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.8 KiB/s wr, 1 op/s 2024-04-20T21:41:04.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:04 smithi033 ceph-mon[21626]: pgmap v222: 97 pgs: 97 active+clean; 305 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.8 KiB/s wr, 1 op/s 2024-04-20T21:41:05.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.012+0000 7fb33dc00640 1 -- 172.21.15.22:0/75975078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb33819ff20 con 0x7fb338108400 2024-04-20T21:41:05.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.014+0000 7fb334c00640 1 -- 172.21.15.22:0/75975078 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7fb3280a0ed0 con 0x7fb338108400 2024-04-20T21:41:05.013 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:41:05.013 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:41:05.014 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:41:05.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.017+0000 7fb33dc00640 1 -- 172.21.15.22:0/75975078 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb31c0a3da0 msgr2=0x7fb31c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.017+0000 7fb33dc00640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb31c0a3da0 0x7fb31c0a6260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb338107830 tx=0x7fb324005a20 comp rx=0 tx=0).stop 2024-04-20T21:41:05.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.017+0000 7fb33dc00640 1 -- 172.21.15.22:0/75975078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb338108400 msgr2=0x7fb3381a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.017+0000 7fb33dc00640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb338108400 0x7fb3381a4840 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb32800e9f0 tx=0x7fb32800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.017+0000 7fb33dc00640 1 -- 172.21.15.22:0/75975078 shutdown_connections 2024-04-20T21:41:05.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.017+0000 7fb33dc00640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fb31c0a3da0 0x7fb31c0a6260 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.017+0000 7fb33dc00640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb33810e470 0x7fb33819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.017+0000 7fb33dc00640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb338108da0 0x7fb3381a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.017+0000 7fb33dc00640 1 --2- 172.21.15.22:0/75975078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb338108400 0x7fb3381a4840 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.017+0000 7fb33dc00640 1 -- 172.21.15.22:0/75975078 >> 172.21.15.22:0/75975078 conn(0x7fb338077ae0 msgr2=0x7fb3380791a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:05.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.017+0000 7fb33dc00640 1 -- 172.21.15.22:0/75975078 shutdown_connections 2024-04-20T21:41:05.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.017+0000 7fb33dc00640 1 -- 172.21.15.22:0/75975078 wait complete. 2024-04-20T21:41:05.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:41:05.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.253+0000 7f08c7200640 1 -- 172.21.15.22:0/3704790418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c8076db0 msgr2=0x7f08c8112760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.253+0000 7f08c7200640 1 --2- 172.21.15.22:0/3704790418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c8076db0 0x7f08c8112760 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009930 tx=0x7f08b002bc30 comp rx=0 tx=0).stop 2024-04-20T21:41:05.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.253+0000 7f08c7200640 1 -- 172.21.15.22:0/3704790418 shutdown_connections 2024-04-20T21:41:05.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.253+0000 7f08c7200640 1 --2- 172.21.15.22:0/3704790418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c8112ca0 0x7f08c8115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.253+0000 7f08c7200640 1 --2- 172.21.15.22:0/3704790418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c8076db0 0x7f08c8112760 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.253+0000 7f08c7200640 1 --2- 172.21.15.22:0/3704790418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8076490 0x7f08c8076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.253+0000 7f08c7200640 1 -- 172.21.15.22:0/3704790418 >> 172.21.15.22:0/3704790418 conn(0x7f08c80ffad0 msgr2=0x7f08c8101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:05.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.254+0000 7f08c7200640 1 -- 172.21.15.22:0/3704790418 shutdown_connections 2024-04-20T21:41:05.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.254+0000 7f08c7200640 1 -- 172.21.15.22:0/3704790418 wait complete. 2024-04-20T21:41:05.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.254+0000 7f08c7200640 1 Processor -- start 2024-04-20T21:41:05.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08c7200640 1 -- start start 2024-04-20T21:41:05.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08c7200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8076490 0x7f08c819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08c7200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c8076db0 0x7f08c819f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08c7200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c8112ca0 0x7f08c81a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08c7200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8118860 con 0x7f08c8076db0 2024-04-20T21:41:05.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08c7200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08c81186e0 con 0x7f08c8076490 2024-04-20T21:41:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08c7200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08c81189e0 con 0x7f08c8112ca0 2024-04-20T21:41:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08bd400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c8076db0 0x7f08c819f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08c6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c8112ca0 0x7f08c81a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08c6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c8112ca0 0x7f08c81a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:49156/0 (socket says 172.21.15.22:49156) 2024-04-20T21:41:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08bd400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c8076db0 0x7f08c819f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43026/0 (socket says 172.21.15.22:43026) 2024-04-20T21:41:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08c6800640 1 -- 172.21.15.22:0/2986501132 learned_addr learned my addr 172.21.15.22:0/2986501132 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.255+0000 7f08c5e00640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8076490 0x7f08c819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.256+0000 7f08c6800640 1 -- 172.21.15.22:0/2986501132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8076490 msgr2=0x7f08c819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.256+0000 7f08c6800640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8076490 0x7f08c819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.256+0000 7f08c6800640 1 -- 172.21.15.22:0/2986501132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c8076db0 msgr2=0x7f08c819f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.256+0000 7f08c6800640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c8076db0 0x7f08c819f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.256+0000 7f08c6800640 1 -- 172.21.15.22:0/2986501132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c81a3d30 con 0x7f08c8112ca0 2024-04-20T21:41:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.256+0000 7f08bd400640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c8076db0 0x7f08c819f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.256+0000 7f08c6800640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c8112ca0 0x7f08c81a3600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f08c000e970 tx=0x7f08c000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.256+0000 7f08bf400640 1 -- 172.21.15.22:0/2986501132 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c000ccc0 con 0x7f08c8112ca0 2024-04-20T21:41:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.256+0000 7f08c7200640 1 -- 172.21.15.22:0/2986501132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c81a4020 con 0x7f08c8112ca0 2024-04-20T21:41:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.256+0000 7f08bf400640 1 -- 172.21.15.22:0/2986501132 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c0005500 con 0x7f08c8112ca0 2024-04-20T21:41:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.256+0000 7f08bf400640 1 -- 172.21.15.22:0/2986501132 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0010690 con 0x7f08c8112ca0 2024-04-20T21:41:05.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.257+0000 7f08c7200640 1 -- 172.21.15.22:0/2986501132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c81ab8b0 con 0x7f08c8112ca0 2024-04-20T21:41:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.260+0000 7f08cfe5c740 1 Processor -- start 2024-04-20T21:41:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.260+0000 7f08bf400640 1 -- 172.21.15.22:0/2986501132 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f08c00046c0 con 0x7f08c8112ca0 2024-04-20T21:41:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.260+0000 7f08cfe5c740 1 -- start start 2024-04-20T21:41:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08cfe5c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87decb170 0x55e87decb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08cfe5c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87ded4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08cfe5c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87ded4670 0x55e87ded6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08cfe5c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e87de30050 con 0x55e87decb170 2024-04-20T21:41:05.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08cfe5c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e87ddd6490 con 0x55e87ded4670 2024-04-20T21:41:05.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08cfe5c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e87de2c010 con 0x55e87decba90 2024-04-20T21:41:05.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08bd400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87decb170 0x55e87decb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08bd400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87decb170 0x55e87decb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43036/0 (socket says 172.21.15.22:43036) 2024-04-20T21:41:05.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08bd400640 1 -- 172.21.15.22:0/4006010857 learned_addr learned my addr 172.21.15.22:0/4006010857 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:05.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08c5e00640 1 --2- 172.21.15.22:0/4006010857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87ded4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08bd400640 1 --2- 172.21.15.22:0/4006010857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87ded4670 0x55e87ded6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08c5e00640 1 -- 172.21.15.22:0/4006010857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87ded4670 msgr2=0x55e87ded6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.261+0000 7f08c5e00640 1 --2- 172.21.15.22:0/4006010857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87ded4670 0x55e87ded6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.262+0000 7f08c5e00640 1 -- 172.21.15.22:0/4006010857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87decb170 msgr2=0x55e87decb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:05.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.262+0000 7f08c5e00640 1 --2- 172.21.15.22:0/4006010857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87decb170 0x55e87decb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.262+0000 7f08c5e00640 1 -- 172.21.15.22:0/4006010857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e87ddde070 con 0x55e87decba90 2024-04-20T21:41:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.262+0000 7f08bd400640 1 --2- 172.21.15.22:0/4006010857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87decb170 0x55e87decb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:41:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.262+0000 7f08bf400640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f08ac0a3d20 0x7f08ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.262+0000 7f08c5e00640 1 --2- 172.21.15.22:0/4006010857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87ded4130 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009f80 tx=0x7f08b80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc6e4e22e9e2542f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.262+0000 7f08b7400640 1 -- 172.21.15.22:0/4006010857 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8004d20 con 0x55e87decba90 2024-04-20T21:41:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.262+0000 7f08bf400640 1 -- 172.21.15.22:0/2986501132 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f08c00d0c00 con 0x7f08c8112ca0 2024-04-20T21:41:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.262+0000 7f08b7400640 1 -- 172.21.15.22:0/4006010857 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b8004050 con 0x55e87decba90 2024-04-20T21:41:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.262+0000 7f08b7400640 1 -- 172.21.15.22:0/4006010857 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8005220 con 0x55e87decba90 2024-04-20T21:41:05.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.262+0000 7f08bd400640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f08ac0a3d20 0x7f08ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.263+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4006010857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 msgr2=0x55e87ded4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.263+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/4006010857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87ded4130 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009f80 tx=0x7f08b80118d0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.263+0000 7f08bd400640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f08ac0a3d20 0x7f08ac0a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08b0000c00 tx=0x7f08b0005b50 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:05.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.263+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4006010857 shutdown_connections 2024-04-20T21:41:05.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.263+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/4006010857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87ded4670 0x55e87ded6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.263+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/4006010857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87ded4130 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.263+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/4006010857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87decb170 0x55e87decb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.263+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4006010857 >> 172.21.15.22:0/4006010857 conn(0x55e87dec58a0 msgr2=0x55e87dec5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:05.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.263+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4006010857 shutdown_connections 2024-04-20T21:41:05.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.263+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4006010857 wait complete. 2024-04-20T21:41:05.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.264+0000 7f08cfe5c740 1 -- start start 2024-04-20T21:41:05.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.264+0000 7f08cfe5c740 1 Processor -- start 2024-04-20T21:41:05.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.264+0000 7f08cfe5c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decb170 0x55e87ded1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.264+0000 7f08cfe5c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87decba90 0x55e87ded1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.264+0000 7f08cfe5c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87ded4670 0x55e87ded2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.264+0000 7f08cfe5c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e87ded7460 con 0x55e87decba90 2024-04-20T21:41:05.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.264+0000 7f08cfe5c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e87dde9860 con 0x55e87ded4670 2024-04-20T21:41:05.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.264+0000 7f08cfe5c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e87ded75e0 con 0x55e87decb170 2024-04-20T21:41:05.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.264+0000 7f08cfe5c740 10 client.0 _ll_register_callbacks cb 0x55e87ddb7fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:05.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.265+0000 7f08c5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decb170 0x55e87ded1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.265+0000 7f08c5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decb170 0x55e87ded1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:49176/0 (socket says 172.21.15.22:49176) 2024-04-20T21:41:05.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.265+0000 7f08c5e00640 1 -- 172.21.15.22:0/2166011355 learned_addr learned my addr 172.21.15.22:0/2166011355 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:05.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.265+0000 7f08bd400640 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87decba90 0x55e87ded1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.265+0000 7f08c6800640 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87ded4670 0x55e87ded2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.265+0000 7f08c5e00640 1 -- 172.21.15.22:0/2166011355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87ded4670 msgr2=0x55e87ded2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:05.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.265+0000 7f08c5e00640 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87ded4670 0x55e87ded2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.265+0000 7f08c5e00640 1 -- 172.21.15.22:0/2166011355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87decba90 msgr2=0x55e87ded1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.265+0000 7f08c5e00640 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87decba90 0x55e87ded1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.265+0000 7f08c5e00640 1 -- 172.21.15.22:0/2166011355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e87dded950 con 0x55e87decb170 2024-04-20T21:41:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.265+0000 7f08c5e00640 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decb170 0x55e87ded1690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08b801b040 tx=0x7f08b8004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ef4325e685d4c38 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.265+0000 7f08abe00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:41:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.266+0000 7f08abe00640 1 -- 172.21.15.22:0/2166011355 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8028020 con 0x55e87decb170 2024-04-20T21:41:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.266+0000 7f08abe00640 1 -- 172.21.15.22:0/2166011355 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f08b8004410 con 0x55e87decb170 2024-04-20T21:41:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.266+0000 7f08cfe5c740 1 -- 172.21.15.22:0/2166011355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b8004050 con 0x55e87decb170 2024-04-20T21:41:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.266+0000 7f08abe00640 1 -- 172.21.15.22:0/2166011355 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f08b8033050 con 0x55e87decb170 2024-04-20T21:41:05.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.266+0000 7f08cfe5c740 10 client.26452 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:05.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.266+0000 7f08cfe5c740 1 -- 172.21.15.22:0/2166011355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e87ddbb820 con 0x55e87decb170 2024-04-20T21:41:05.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.266+0000 7f08abe00640 1 -- 172.21.15.22:0/2166011355 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b8004050 con 0x55e87decb170 2024-04-20T21:41:05.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.267+0000 7f08abe00640 1 -- 172.21.15.22:0/2166011355 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f08b800f4e0 con 0x55e87decb170 2024-04-20T21:41:05.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.267+0000 7f08cfe5c740 10 client.26452 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:05.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.267+0000 7f08cfe5c740 10 client.26452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:05.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.267+0000 7f08cfe5c740 20 client.26452 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:05.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.267+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e87dee4670 0x55e87dee6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.267+0000 7f08cfe5c740 4 client.26452 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:05.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.267+0000 7f08cfe5c740 1 -- 172.21.15.22:0/2166011355 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e87de3da70 con 0x55e87dee4670 2024-04-20T21:41:05.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.267+0000 7f08bd400640 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e87dee4670 0x55e87dee6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.268+0000 7f08bd400640 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e87dee4670 0x55e87dee6a60 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4764ec3948384e6 server_cookie=8c152e6dd4e41108 in_seq=0 out_seq=0 2024-04-20T21:41:05.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.268+0000 7f08abe00640 10 client.26452 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:05.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.268+0000 7f08abe00640 1 -- 172.21.15.22:0/2166011355 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e87de3da70 con 0x55e87dee4670 2024-04-20T21:41:05.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.268+0000 7f08abe00640 10 client.26452 handle_command_reply: tid=0 2024-04-20T21:41:05.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.268+0000 7f08cfe5c740 1 client.26452 shutdown 2024-04-20T21:41:05.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.269+0000 7f08cfe5c740 1 -- 172.21.15.22:0/2166011355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decb170 msgr2=0x55e87ded1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.269+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decb170 0x55e87ded1690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08b801b040 tx=0x7f08b8004cf0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.269+0000 7f08cfe5c740 1 -- 172.21.15.22:0/2166011355 shutdown_connections 2024-04-20T21:41:05.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.269+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e87dee4670 0x55e87dee6a60 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.269+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87ded4670 0x55e87ded2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.269+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87decba90 0x55e87ded1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.269+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/2166011355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decb170 0x55e87ded1690 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.269+0000 7f08abe00640 0 client.26452 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:05.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.269+0000 7f08cfe5c740 1 -- 172.21.15.22:0/2166011355 >> 172.21.15.22:0/2166011355 conn(0x55e87dec58a0 msgr2=0x55e87dec75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:05.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.269+0000 7f08cfe5c740 1 -- 172.21.15.22:0/2166011355 shutdown_connections 2024-04-20T21:41:05.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.269+0000 7f08cfe5c740 1 -- 172.21.15.22:0/2166011355 wait complete. 2024-04-20T21:41:05.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.269+0000 7f08cfe5c740 20 client.26452 trim_cache size 0 max 16384 2024-04-20T21:41:05.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.284+0000 7f08cfe5c740 1 Processor -- start 2024-04-20T21:41:05.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.284+0000 7f08cfe5c740 1 -- start start 2024-04-20T21:41:05.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08cfe5c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87decb170 0x55e87dec4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08cfe5c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87dee0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08cfe5c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87ded4670 0x55e87ded0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08cfe5c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08c81b2260 con 0x55e87ded4670 2024-04-20T21:41:05.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08cfe5c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08c81b4ac0 con 0x55e87decb170 2024-04-20T21:41:05.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08cfe5c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08c81b28b0 con 0x55e87decba90 2024-04-20T21:41:05.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08c5e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87decb170 0x55e87dec4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08c6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87ded4670 0x55e87ded0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08bd400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87dee0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08c5e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87decb170 0x55e87dec4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60952/0 (socket says 172.21.15.22:60952) 2024-04-20T21:41:05.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08c5e00640 1 -- 172.21.15.22:0/561920994 learned_addr learned my addr 172.21.15.22:0/561920994 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:05.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08bd400640 1 -- 172.21.15.22:0/561920994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87decb170 msgr2=0x55e87dec4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:05.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08bd400640 1 --2- 172.21.15.22:0/561920994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87decb170 0x55e87dec4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08bd400640 1 -- 172.21.15.22:0/561920994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87ded4670 msgr2=0x55e87ded0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:05.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08bd400640 1 --2- 172.21.15.22:0/561920994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87ded4670 0x55e87ded0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.285+0000 7f08bd400640 1 -- 172.21.15.22:0/561920994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e87decb8c0 con 0x55e87decba90 2024-04-20T21:41:05.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.286+0000 7f08bd400640 1 --2- 172.21.15.22:0/561920994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87dee0350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f08b0006010 tx=0x7f08b0043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a331ebd9b822764 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:05.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.286+0000 7f08b6a00640 1 -- 172.21.15.22:0/561920994 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b002ca00 con 0x55e87decba90 2024-04-20T21:41:05.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.286+0000 7f08b6a00640 1 -- 172.21.15.22:0/561920994 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b002cba0 con 0x55e87decba90 2024-04-20T21:41:05.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.286+0000 7f08b6a00640 1 -- 172.21.15.22:0/561920994 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0003a50 con 0x55e87decba90 2024-04-20T21:41:05.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.286+0000 7f08cfe5c740 1 -- 172.21.15.22:0/561920994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 msgr2=0x55e87dee0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.286+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/561920994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87dee0350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f08b0006010 tx=0x7f08b0043000 comp rx=0 tx=0).stop 2024-04-20T21:41:05.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.287+0000 7f08cfe5c740 1 -- 172.21.15.22:0/561920994 shutdown_connections 2024-04-20T21:41:05.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.287+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/561920994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87ded4670 0x55e87ded0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.287+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/561920994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87dee0350 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.287+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/561920994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87decb170 0x55e87dec4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.287+0000 7f08cfe5c740 1 -- 172.21.15.22:0/561920994 >> 172.21.15.22:0/561920994 conn(0x55e87dec58a0 msgr2=0x55e87ded1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:05.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.287+0000 7f08cfe5c740 1 -- 172.21.15.22:0/561920994 shutdown_connections 2024-04-20T21:41:05.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.287+0000 7f08cfe5c740 1 -- 172.21.15.22:0/561920994 wait complete. 2024-04-20T21:41:05.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.287+0000 7f08cfe5c740 1 -- start start 2024-04-20T21:41:05.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.287+0000 7f08cfe5c740 1 Processor -- start 2024-04-20T21:41:05.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08cfe5c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87decb170 0x55e87dedc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08cfe5c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87ded7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08cfe5c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87ded4670 0x55e87ded8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08cfe5c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08c81b2f80 con 0x55e87ded4670 2024-04-20T21:41:05.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08cfe5c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08c81b43f0 con 0x55e87decb170 2024-04-20T21:41:05.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08cfe5c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08c81b3650 con 0x55e87decba90 2024-04-20T21:41:05.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08cfe5c740 10 client.0 _ll_register_callbacks cb 0x7f08b8003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:05.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08c6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87ded4670 0x55e87ded8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08bd400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87ded7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08c6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87ded4670 0x55e87ded8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43054/0 (socket says 172.21.15.22:43054) 2024-04-20T21:41:05.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08c6800640 1 -- 172.21.15.22:0/4054254687 learned_addr learned my addr 172.21.15.22:0/4054254687 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:05.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08c5e00640 1 --2- 172.21.15.22:0/4054254687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87decb170 0x55e87dedc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08c6800640 1 -- 172.21.15.22:0/4054254687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 msgr2=0x55e87ded7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:05.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08c6800640 1 --2- 172.21.15.22:0/4054254687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87ded7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08c6800640 1 -- 172.21.15.22:0/4054254687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87decb170 msgr2=0x55e87dedc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:05.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.288+0000 7f08c6800640 1 --2- 172.21.15.22:0/4054254687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87decb170 0x55e87dedc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.289+0000 7f08c6800640 1 -- 172.21.15.22:0/4054254687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e87db423f0 con 0x55e87ded4670 2024-04-20T21:41:05.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.289+0000 7f08c6800640 1 --2- 172.21.15.22:0/4054254687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87ded4670 0x55e87ded8360 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f08c00ce300 tx=0x7f08c009c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd9c2b645991f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:05.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.289+0000 7f08abe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:41:05.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.289+0000 7f08abe00640 1 -- 172.21.15.22:0/4054254687 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c00a0ea0 con 0x55e87ded4670 2024-04-20T21:41:05.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.289+0000 7f08abe00640 1 -- 172.21.15.22:0/4054254687 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f08c00a52d0 con 0x55e87ded4670 2024-04-20T21:41:05.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.289+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4054254687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b002cba0 con 0x55e87ded4670 2024-04-20T21:41:05.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.289+0000 7f08abe00640 1 -- 172.21.15.22:0/4054254687 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f08c0082050 con 0x55e87ded4670 2024-04-20T21:41:05.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.289+0000 7f08cfe5c740 10 client.16284 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:05.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.289+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4054254687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e87dc0c7d0 con 0x55e87ded4670 2024-04-20T21:41:05.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.290+0000 7f08abe00640 1 -- 172.21.15.22:0/4054254687 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b002cba0 con 0x55e87ded4670 2024-04-20T21:41:05.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.290+0000 7f08abe00640 1 -- 172.21.15.22:0/4054254687 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f08c009b9b0 con 0x55e87ded4670 2024-04-20T21:41:05.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.290+0000 7f08cfe5c740 10 client.16284 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:05.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.290+0000 7f08cfe5c740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:05.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.290+0000 7f08cfe5c740 20 client.16284 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:05.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.290+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/4054254687 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e87df4ac40 0x55e87df4d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:05.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.290+0000 7f08cfe5c740 4 client.16284 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:41:05.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.290+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4054254687 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e87de378e0 con 0x55e87df4ac40 2024-04-20T21:41:05.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.290+0000 7f08c5e00640 1 --2- 172.21.15.22:0/4054254687 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e87df4ac40 0x55e87df4d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:05.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.291+0000 7f08c5e00640 1 --2- 172.21.15.22:0/4054254687 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e87df4ac40 0x55e87df4d060 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edef4288b638db27 server_cookie=e624067675bc3ff5 in_seq=0 out_seq=0 2024-04-20T21:41:05.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.291+0000 7f08abe00640 10 client.16284 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:05.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.291+0000 7f08abe00640 1 -- 172.21.15.22:0/4054254687 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e87de378e0 con 0x55e87df4ac40 2024-04-20T21:41:05.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.291+0000 7f08abe00640 10 client.16284 handle_command_reply: tid=0 2024-04-20T21:41:05.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.291+0000 7f08cfe5c740 1 client.16284 shutdown 2024-04-20T21:41:05.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.291+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4054254687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87ded4670 msgr2=0x55e87ded8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.291+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/4054254687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87ded4670 0x55e87ded8360 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f08c00ce300 tx=0x7f08c009c510 comp rx=0 tx=0).stop 2024-04-20T21:41:05.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.291+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4054254687 shutdown_connections 2024-04-20T21:41:05.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.291+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/4054254687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e87ded4670 0x55e87ded8360 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.291+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/4054254687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e87decba90 0x55e87ded7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.291+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/4054254687 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e87df4ac40 0x55e87df4d060 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.292+0000 7f08cfe5c740 1 --2- 172.21.15.22:0/4054254687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e87decb170 0x55e87dedc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.292+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4054254687 >> 172.21.15.22:0/4054254687 conn(0x55e87dec58a0 msgr2=0x55e87ded15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:05.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.292+0000 7f08abe00640 0 client.16284 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:05.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.292+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4054254687 shutdown_connections 2024-04-20T21:41:05.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.292+0000 7f08cfe5c740 1 -- 172.21.15.22:0/4054254687 wait complete. 2024-04-20T21:41:05.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.292+0000 7f08cfe5c740 20 client.16284 trim_cache size 0 max 16384 2024-04-20T21:41:05.298 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:41:05.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.292+0000 7f08c7200640 1 -- 172.21.15.22:0/2986501132 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f08ac0a3d20 msgr2=0x7f08ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.293+0000 7f08c7200640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f08ac0a3d20 0x7f08ac0a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08b0000c00 tx=0x7f08b0005b50 comp rx=0 tx=0).stop 2024-04-20T21:41:05.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.293+0000 7f08c7200640 1 -- 172.21.15.22:0/2986501132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c8112ca0 msgr2=0x7f08c81a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:05.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.293+0000 7f08c7200640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c8112ca0 0x7f08c81a3600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f08c000e970 tx=0x7f08c000ee30 comp rx=0 tx=0).stop 2024-04-20T21:41:05.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.293+0000 7f08c7200640 1 -- 172.21.15.22:0/2986501132 shutdown_connections 2024-04-20T21:41:05.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.293+0000 7f08c7200640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f08ac0a3d20 0x7f08ac0a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.293+0000 7f08c7200640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c8112ca0 0x7f08c81a3600 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.293+0000 7f08c7200640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c8076db0 0x7f08c819f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.293+0000 7f08c7200640 1 --2- 172.21.15.22:0/2986501132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8076490 0x7f08c819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:05.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.293+0000 7f08c7200640 1 -- 172.21.15.22:0/2986501132 >> 172.21.15.22:0/2986501132 conn(0x7f08c80ffad0 msgr2=0x7f08c8101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:05.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.293+0000 7f08c7200640 1 -- 172.21.15.22:0/2986501132 shutdown_connections 2024-04-20T21:41:05.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:05.293+0000 7f08c7200640 1 -- 172.21.15.22:0/2986501132 wait complete. 2024-04-20T21:41:05.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:05 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/75975078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:41:05.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:05 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/75975078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:41:05.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:05 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/75975078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:41:06.310 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:41:06.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:41:06.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:41:06.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.533+0000 7f7359400640 1 -- 172.21.15.22:0/363575745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7354102fc0 msgr2=0x7f73541033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.533+0000 7f7359400640 1 --2- 172.21.15.22:0/363575745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7354102fc0 0x7f73541033a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7344009be0 tx=0x7f734402bf80 comp rx=0 tx=0).stop 2024-04-20T21:41:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.533+0000 7f7359400640 1 -- 172.21.15.22:0/363575745 shutdown_connections 2024-04-20T21:41:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.533+0000 7f7359400640 1 --2- 172.21.15.22:0/363575745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f735410c3e0 0x7f735410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.533+0000 7f7359400640 1 --2- 172.21.15.22:0/363575745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73541038e0 0x7f735410bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.533+0000 7f7359400640 1 --2- 172.21.15.22:0/363575745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7354102fc0 0x7f73541033a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.533+0000 7f7359400640 1 -- 172.21.15.22:0/363575745 >> 172.21.15.22:0/363575745 conn(0x7f73540fc7e0 msgr2=0x7f73540fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.533+0000 7f7359400640 1 -- 172.21.15.22:0/363575745 shutdown_connections 2024-04-20T21:41:06.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.534+0000 7f7359400640 1 -- 172.21.15.22:0/363575745 wait complete. 2024-04-20T21:41:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.534+0000 7f7359400640 1 Processor -- start 2024-04-20T21:41:06.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.535+0000 7f7359400640 1 -- start start 2024-04-20T21:41:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.535+0000 7f7359400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7354102fc0 0x7f7354100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.535+0000 7f7359400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73541038e0 0x7f7354101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.535+0000 7f7359400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f735410c3e0 0x7f7354101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.535+0000 7f7359400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f73541121f0 con 0x7f735410c3e0 2024-04-20T21:41:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.535+0000 7f7359400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7354112070 con 0x7f73541038e0 2024-04-20T21:41:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.535+0000 7f7359400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7354112370 con 0x7f7354102fc0 2024-04-20T21:41:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.535+0000 7f7358a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f735410c3e0 0x7f7354101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.535+0000 7f7358a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f735410c3e0 0x7f7354101690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43066/0 (socket says 172.21.15.22:43066) 2024-04-20T21:41:06.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.535+0000 7f7358a00640 1 -- 172.21.15.22:0/402479077 learned_addr learned my addr 172.21.15.22:0/402479077 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.535+0000 7f7353e00640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7354102fc0 0x7f7354100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.535+0000 7f7353400640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73541038e0 0x7f7354101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.536+0000 7f7353e00640 1 -- 172.21.15.22:0/402479077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73541038e0 msgr2=0x7f7354101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.536+0000 7f7353e00640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73541038e0 0x7f7354101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.536+0000 7f7353e00640 1 -- 172.21.15.22:0/402479077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f735410c3e0 msgr2=0x7f7354101690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.536+0000 7f7353e00640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f735410c3e0 0x7f7354101690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.536+0000 7f7353e00640 1 -- 172.21.15.22:0/402479077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73541a6bf0 con 0x7f7354102fc0 2024-04-20T21:41:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.536+0000 7f7353e00640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7354102fc0 0x7f7354100c10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f734402c590 tx=0x7f7344004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.536+0000 7f7350c00640 1 -- 172.21.15.22:0/402479077 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344043020 con 0x7f7354102fc0 2024-04-20T21:41:06.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.536+0000 7f7350c00640 1 -- 172.21.15.22:0/402479077 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73440045f0 con 0x7f7354102fc0 2024-04-20T21:41:06.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.536+0000 7f7350c00640 1 -- 172.21.15.22:0/402479077 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734403e6b0 con 0x7f7354102fc0 2024-04-20T21:41:06.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.537+0000 7f7359400640 1 -- 172.21.15.22:0/402479077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73541a6e80 con 0x7f7354102fc0 2024-04-20T21:41:06.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.537+0000 7f7359400640 1 -- 172.21.15.22:0/402479077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73541a7390 con 0x7f7354102fc0 2024-04-20T21:41:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.540+0000 7f735d660740 1 Processor -- start 2024-04-20T21:41:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.540+0000 7f735d660740 1 -- start start 2024-04-20T21:41:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.540+0000 7f7350c00640 1 -- 172.21.15.22:0/402479077 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7344046020 con 0x7f7354102fc0 2024-04-20T21:41:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.540+0000 7f735d660740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.540+0000 7f735d660740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 0x5612d54cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.540+0000 7f735d660740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54cd5f0 0x5612d54cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.540+0000 7f735d660740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5612d5428d20 con 0x5612d54c40f0 2024-04-20T21:41:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.540+0000 7f735d660740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5612d53cf3e0 con 0x5612d54c4a10 2024-04-20T21:41:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.540+0000 7f735d660740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5612d5424ce0 con 0x5612d54cd5f0 2024-04-20T21:41:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.541+0000 7f7353e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 0x5612d54cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.541+0000 7f7358a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.541+0000 7f7353e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 0x5612d54cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:60992/0 (socket says 172.21.15.22:60992) 2024-04-20T21:41:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.541+0000 7f7353e00640 1 -- 172.21.15.22:0/3747953087 learned_addr learned my addr 172.21.15.22:0/3747953087 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.541+0000 7f7353400640 1 --2- 172.21.15.22:0/3747953087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54cd5f0 0x5612d54cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.541+0000 7f7358a00640 1 -- 172.21.15.22:0/3747953087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54cd5f0 msgr2=0x5612d54cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.541+0000 7f7358a00640 1 --2- 172.21.15.22:0/3747953087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54cd5f0 0x5612d54cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.541+0000 7f7358a00640 1 -- 172.21.15.22:0/3747953087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 msgr2=0x5612d54cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.541+0000 7f7358a00640 1 --2- 172.21.15.22:0/3747953087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 0x5612d54cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.541+0000 7f7358a00640 1 -- 172.21.15.22:0/3747953087 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612d53ee980 con 0x5612d54c40f0 2024-04-20T21:41:06.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.541+0000 7f7353e00640 1 --2- 172.21.15.22:0/3747953087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 0x5612d54cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:06.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.542+0000 7f7358a00640 1 --2- 172.21.15.22:0/3747953087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54c44d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f734c009bf0 tx=0x7f734c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7883c301c2e22871 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.542+0000 7f7350c00640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f73380a3dd0 0x7f73380a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.542+0000 7f7349200640 1 -- 172.21.15.22:0/3747953087 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c009e60 con 0x5612d54c40f0 2024-04-20T21:41:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.542+0000 7f7349200640 1 -- 172.21.15.22:0/3747953087 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f734c0047a0 con 0x5612d54c40f0 2024-04-20T21:41:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.542+0000 7f7349200640 1 -- 172.21.15.22:0/3747953087 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c019650 con 0x5612d54c40f0 2024-04-20T21:41:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.542+0000 7f7350c00640 1 -- 172.21.15.22:0/402479077 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8644+0+0 (secure 0 0 0) 0x7f73440f2e20 con 0x7f7354102fc0 2024-04-20T21:41:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.542+0000 7f7353e00640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f73380a3dd0 0x7f73380a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.543+0000 7f735d660740 1 -- 172.21.15.22:0/3747953087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 msgr2=0x5612d54c44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.543+0000 7f735d660740 1 --2- 172.21.15.22:0/3747953087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54c44d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f734c009bf0 tx=0x7f734c011810 comp rx=0 tx=0).stop 2024-04-20T21:41:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.543+0000 7f735d660740 1 -- 172.21.15.22:0/3747953087 shutdown_connections 2024-04-20T21:41:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.543+0000 7f735d660740 1 --2- 172.21.15.22:0/3747953087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54cd5f0 0x5612d54cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.543+0000 7f735d660740 1 --2- 172.21.15.22:0/3747953087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 0x5612d54cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.543+0000 7f735d660740 1 --2- 172.21.15.22:0/3747953087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54c44d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.543+0000 7f735d660740 1 -- 172.21.15.22:0/3747953087 >> 172.21.15.22:0/3747953087 conn(0x5612d54be820 msgr2=0x5612d54bec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.543+0000 7f735d660740 1 -- 172.21.15.22:0/3747953087 shutdown_connections 2024-04-20T21:41:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.543+0000 7f7353e00640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f73380a3dd0 0x7f73380a6290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f734404e000 tx=0x7f73440f68d0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.543+0000 7f735d660740 1 -- 172.21.15.22:0/3747953087 wait complete. 2024-04-20T21:41:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.543+0000 7f735d660740 1 -- start start 2024-04-20T21:41:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.544+0000 7f735d660740 1 Processor -- start 2024-04-20T21:41:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.544+0000 7f735d660740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c40f0 0x5612d54ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.544+0000 7f735d660740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 0x5612d54cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.544+0000 7f735d660740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54cd5f0 0x5612d54cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.544+0000 7f735d660740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5612d54d03e0 con 0x5612d54cd5f0 2024-04-20T21:41:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.544+0000 7f735d660740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5612d53e27a0 con 0x5612d54c4a10 2024-04-20T21:41:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.544+0000 7f735d660740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5612d54d0560 con 0x5612d54c40f0 2024-04-20T21:41:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.544+0000 7f735d660740 10 client.0 _ll_register_callbacks cb 0x5612d53e4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.544+0000 7f7358a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54cd5f0 0x5612d54cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.544+0000 7f7358a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c40f0 0x5612d54ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.544+0000 7f7358a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c40f0 0x5612d54ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:49226/0 (socket says 172.21.15.22:49226) 2024-04-20T21:41:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.544+0000 7f7358a00640 1 -- 172.21.15.22:0/3977318206 learned_addr learned my addr 172.21.15.22:0/3977318206 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.545+0000 7f7353400640 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 0x5612d54cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.545+0000 7f7353400640 1 -- 172.21.15.22:0/3977318206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c40f0 msgr2=0x5612d54ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.545+0000 7f7353400640 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c40f0 0x5612d54ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.545+0000 7f7353400640 1 -- 172.21.15.22:0/3977318206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54cd5f0 msgr2=0x5612d54cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.545+0000 7f7353400640 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54cd5f0 0x5612d54cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.545+0000 7f7353400640 1 -- 172.21.15.22:0/3977318206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612d52037e0 con 0x5612d54c4a10 2024-04-20T21:41:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.545+0000 7f7358a00640 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54cd5f0 0x5612d54cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.545+0000 7f7353400640 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 0x5612d54cab90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f733c005fd0 tx=0x7f733c011b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef93df00ebdfc12c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.545+0000 7f7341600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:41:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.546+0000 7f7341600640 1 -- 172.21.15.22:0/3977318206 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c01e020 con 0x5612d54c4a10 2024-04-20T21:41:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.546+0000 7f7341600640 1 -- 172.21.15.22:0/3977318206 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8644+0+0 (secure 0 0 0) 0x7f733c00fb90 con 0x5612d54c4a10 2024-04-20T21:41:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.546+0000 7f735d660740 1 -- 172.21.15.22:0/3977318206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f734c0047a0 con 0x5612d54c4a10 2024-04-20T21:41:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.546+0000 7f7341600640 1 -- 172.21.15.22:0/3977318206 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f733c026050 con 0x5612d54c4a10 2024-04-20T21:41:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.546+0000 7f735d660740 10 client.26447 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.546+0000 7f735d660740 1 -- 172.21.15.22:0/3977318206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5612d5263f70 con 0x5612d54c4a10 2024-04-20T21:41:06.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.547+0000 7f7341600640 1 -- 172.21.15.22:0/3977318206 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f734c0047a0 con 0x5612d54c4a10 2024-04-20T21:41:06.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.547+0000 7f7341600640 1 -- 172.21.15.22:0/3977318206 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f733c019070 con 0x5612d54c4a10 2024-04-20T21:41:06.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.547+0000 7f735d660740 10 client.26447 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:06.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.547+0000 7f735d660740 10 client.26447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:06.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.547+0000 7f735d660740 20 client.26447 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:06.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.547+0000 7f735d660740 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5612d54ddaa0 0x5612d54dfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.547+0000 7f735d660740 4 client.26447 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:06.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.547+0000 7f735d660740 1 -- 172.21.15.22:0/3977318206 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612d5436740 con 0x5612d54ddaa0 2024-04-20T21:41:06.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.547+0000 7f7358a00640 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5612d54ddaa0 0x5612d54dfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.548+0000 7f7358a00640 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5612d54ddaa0 0x5612d54dfec0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17212da429f2b86e server_cookie=996e123d2d292734 in_seq=0 out_seq=0 2024-04-20T21:41:06.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.548+0000 7f7341600640 10 client.26447 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:06.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f7341600640 1 -- 172.21.15.22:0/3977318206 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612d5436740 con 0x5612d54ddaa0 2024-04-20T21:41:06.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f7341600640 10 client.26447 handle_command_reply: tid=0 2024-04-20T21:41:06.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f735d660740 1 client.26447 shutdown 2024-04-20T21:41:06.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f735d660740 1 -- 172.21.15.22:0/3977318206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 msgr2=0x5612d54cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f735d660740 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 0x5612d54cab90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f733c005fd0 tx=0x7f733c011b50 comp rx=0 tx=0).stop 2024-04-20T21:41:06.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f735d660740 1 -- 172.21.15.22:0/3977318206 shutdown_connections 2024-04-20T21:41:06.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f735d660740 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54cd5f0 0x5612d54cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f735d660740 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5612d54ddaa0 0x5612d54dfec0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f735d660740 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54c4a10 0x5612d54cab90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f735d660740 1 --2- 172.21.15.22:0/3977318206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c40f0 0x5612d54ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f7341600640 0 client.26447 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:06.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f735d660740 1 -- 172.21.15.22:0/3977318206 >> 172.21.15.22:0/3977318206 conn(0x5612d54be820 msgr2=0x5612d54c0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f735d660740 1 -- 172.21.15.22:0/3977318206 shutdown_connections 2024-04-20T21:41:06.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f735d660740 1 -- 172.21.15.22:0/3977318206 wait complete. 2024-04-20T21:41:06.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.549+0000 7f735d660740 20 client.26447 trim_cache size 0 max 16384 2024-04-20T21:41:06.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.569+0000 7f735d660740 1 Processor -- start 2024-04-20T21:41:06.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.569+0000 7f735d660740 1 -- start start 2024-04-20T21:41:06.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.569+0000 7f735d660740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.569+0000 7f735d660740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c4a10 0x5612d54d92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.569+0000 7f735d660740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54cd5f0 0x5612d54e5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.569+0000 7f735d660740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f73541adf50 con 0x5612d54c40f0 2024-04-20T21:41:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.569+0000 7f735d660740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73541b07b0 con 0x5612d54cd5f0 2024-04-20T21:41:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.569+0000 7f735d660740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73541ae5a0 con 0x5612d54c4a10 2024-04-20T21:41:06.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.570+0000 7f7353400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c4a10 0x5612d54d92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.570+0000 7f7358a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.570+0000 7f7353400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c4a10 0x5612d54d92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:49232/0 (socket says 172.21.15.22:49232) 2024-04-20T21:41:06.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.570+0000 7f7353400640 1 -- 172.21.15.22:0/965374803 learned_addr learned my addr 172.21.15.22:0/965374803 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.570+0000 7f7353400640 1 -- 172.21.15.22:0/965374803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54cd5f0 msgr2=0x5612d54e5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.570+0000 7f7358a00640 1 --2- 172.21.15.22:0/965374803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54cd5f0 0x5612d54e5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.570+0000 7f7353400640 1 --2- 172.21.15.22:0/965374803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54cd5f0 0x5612d54e5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.570+0000 7f7353400640 1 -- 172.21.15.22:0/965374803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 msgr2=0x5612d54ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.570+0000 7f7353400640 1 --2- 172.21.15.22:0/965374803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.570+0000 7f7353400640 1 -- 172.21.15.22:0/965374803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612d50fb170 con 0x5612d54c4a10 2024-04-20T21:41:06.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.570+0000 7f7358a00640 1 --2- 172.21.15.22:0/965374803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54cd5f0 0x5612d54e5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:41:06.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.571+0000 7f7353400640 1 --2- 172.21.15.22:0/965374803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c4a10 0x5612d54d92a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f733c009da0 tx=0x7f733c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c0df348ff876f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.571+0000 7f7343e00640 1 -- 172.21.15.22:0/965374803 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c000e60 con 0x5612d54c4a10 2024-04-20T21:41:06.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.571+0000 7f7343e00640 1 -- 172.21.15.22:0/965374803 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f733c0196e0 con 0x5612d54c4a10 2024-04-20T21:41:06.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.571+0000 7f7343e00640 1 -- 172.21.15.22:0/965374803 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c01e040 con 0x5612d54c4a10 2024-04-20T21:41:06.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.572+0000 7f735d660740 1 -- 172.21.15.22:0/965374803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c4a10 msgr2=0x5612d54d92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.572+0000 7f735d660740 1 --2- 172.21.15.22:0/965374803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c4a10 0x5612d54d92a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f733c009da0 tx=0x7f733c005880 comp rx=0 tx=0).stop 2024-04-20T21:41:06.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.572+0000 7f735d660740 1 -- 172.21.15.22:0/965374803 shutdown_connections 2024-04-20T21:41:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.572+0000 7f735d660740 1 --2- 172.21.15.22:0/965374803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54cd5f0 0x5612d54e5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.572+0000 7f735d660740 1 --2- 172.21.15.22:0/965374803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c4a10 0x5612d54d92a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.572+0000 7f735d660740 1 --2- 172.21.15.22:0/965374803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.572+0000 7f735d660740 1 -- 172.21.15.22:0/965374803 >> 172.21.15.22:0/965374803 conn(0x5612d54be820 msgr2=0x5612d54bda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.572+0000 7f735d660740 1 -- 172.21.15.22:0/965374803 shutdown_connections 2024-04-20T21:41:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.572+0000 7f735d660740 1 -- 172.21.15.22:0/965374803 wait complete. 2024-04-20T21:41:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.572+0000 7f735d660740 1 -- start start 2024-04-20T21:41:06.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.572+0000 7f735d660740 1 Processor -- start 2024-04-20T21:41:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.573+0000 7f735d660740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.573+0000 7f735d660740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c4a10 0x5612d54d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.573+0000 7f735d660740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54cd5f0 0x5612d54d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.573+0000 7f735d660740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f73541aec70 con 0x5612d54c40f0 2024-04-20T21:41:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.573+0000 7f735d660740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73541b00e0 con 0x5612d54cd5f0 2024-04-20T21:41:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.573+0000 7f735d660740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73541af340 con 0x5612d54c4a10 2024-04-20T21:41:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.573+0000 7f735d660740 10 client.0 _ll_register_callbacks cb 0x7f733c009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:06.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.573+0000 7f7358a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.573+0000 7f7358a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43098/0 (socket says 172.21.15.22:43098) 2024-04-20T21:41:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.573+0000 7f7358a00640 1 -- 172.21.15.22:0/3277381734 learned_addr learned my addr 172.21.15.22:0/3277381734 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.573+0000 7f7353400640 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c4a10 0x5612d54d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.574+0000 7f7358a00640 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54cd5f0 0x5612d54d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.574+0000 7f7358a00640 1 -- 172.21.15.22:0/3277381734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c4a10 msgr2=0x5612d54d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.574+0000 7f7358a00640 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c4a10 0x5612d54d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.574+0000 7f7358a00640 1 -- 172.21.15.22:0/3277381734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54cd5f0 msgr2=0x5612d54d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.574+0000 7f7358a00640 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54cd5f0 0x5612d54d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.574+0000 7f7358a00640 1 -- 172.21.15.22:0/3277381734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612d5543000 con 0x5612d54c40f0 2024-04-20T21:41:06.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.574+0000 7f7358a00640 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54d11b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f734c009610 tx=0x7f734c031040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66579592671af40 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.574+0000 7f7341600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:41:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.574+0000 7f7341600640 1 -- 172.21.15.22:0/3277381734 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c02fb50 con 0x5612d54c40f0 2024-04-20T21:41:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.574+0000 7f735d660740 1 -- 172.21.15.22:0/3277381734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f733c0196e0 con 0x5612d54c40f0 2024-04-20T21:41:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.574+0000 7f7341600640 1 -- 172.21.15.22:0/3277381734 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8644+0+0 (secure 0 0 0) 0x7f734c02fcf0 con 0x5612d54c40f0 2024-04-20T21:41:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.575+0000 7f7341600640 1 -- 172.21.15.22:0/3277381734 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f734c019850 con 0x5612d54c40f0 2024-04-20T21:41:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.575+0000 7f735d660740 10 client.16302 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:06.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.575+0000 7f735d660740 1 -- 172.21.15.22:0/3277381734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5612d5544230 con 0x5612d54c40f0 2024-04-20T21:41:06.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.575+0000 7f7341600640 1 -- 172.21.15.22:0/3277381734 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f733c0196e0 con 0x5612d54c40f0 2024-04-20T21:41:06.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.575+0000 7f7341600640 1 -- 172.21.15.22:0/3277381734 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f734c02f790 con 0x5612d54c40f0 2024-04-20T21:41:06.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.575+0000 7f735d660740 10 client.16302 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:06.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.575+0000 7f735d660740 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:06.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.575+0000 7f735d660740 20 client.16302 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:06.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.575+0000 7f735d660740 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5612d55447e0 0x5612d5546c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.575+0000 7f735d660740 4 client.16302 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:41:06.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.575+0000 7f735d660740 1 -- 172.21.15.22:0/3277381734 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5612d54305b0 con 0x5612d55447e0 2024-04-20T21:41:06.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.576+0000 7f7353400640 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5612d55447e0 0x5612d5546c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.576+0000 7f7353400640 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5612d55447e0 0x5612d5546c00 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f95bd28011462aea server_cookie=22f6dd57eea068f4 in_seq=0 out_seq=0 2024-04-20T21:41:06.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.576+0000 7f7341600640 10 client.16302 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:06.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.577+0000 7f7341600640 1 -- 172.21.15.22:0/3277381734 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5612d54305b0 con 0x5612d55447e0 2024-04-20T21:41:06.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.577+0000 7f7341600640 10 client.16302 handle_command_reply: tid=0 2024-04-20T21:41:06.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.577+0000 7f735d660740 1 client.16302 shutdown 2024-04-20T21:41:06.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.578+0000 7f735d660740 1 -- 172.21.15.22:0/3277381734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 msgr2=0x5612d54d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.578+0000 7f735d660740 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54d11b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f734c009610 tx=0x7f734c031040 comp rx=0 tx=0).stop 2024-04-20T21:41:06.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.578+0000 7f735d660740 1 -- 172.21.15.22:0/3277381734 shutdown_connections 2024-04-20T21:41:06.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.578+0000 7f735d660740 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5612d55447e0 0x5612d5546c00 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.578+0000 7f735d660740 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612d54cd5f0 0x5612d54d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.578+0000 7f735d660740 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612d54c4a10 0x5612d54d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.578+0000 7f735d660740 1 --2- 172.21.15.22:0/3277381734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612d54c40f0 0x5612d54d11b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.578+0000 7f7341600640 0 client.16302 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:06.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.578+0000 7f735d660740 1 -- 172.21.15.22:0/3277381734 >> 172.21.15.22:0/3277381734 conn(0x5612d54be820 msgr2=0x5612d54ca610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.578+0000 7f735d660740 1 -- 172.21.15.22:0/3277381734 shutdown_connections 2024-04-20T21:41:06.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.578+0000 7f735d660740 1 -- 172.21.15.22:0/3277381734 wait complete. 2024-04-20T21:41:06.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.578+0000 7f735d660740 20 client.16302 trim_cache size 0 max 16384 2024-04-20T21:41:06.584 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:41:06.585 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:41:06.585 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "cbb1d2e8-e79b-4833-912f-97640fed471f", 2024-04-20T21:41:06.585 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:41:06.585 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:41:06.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.579+0000 7f7359400640 1 -- 172.21.15.22:0/402479077 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f73380a3dd0 msgr2=0x7f73380a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.579+0000 7f7359400640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f73380a3dd0 0x7f73380a6290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f734404e000 tx=0x7f73440f68d0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.579+0000 7f7359400640 1 -- 172.21.15.22:0/402479077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7354102fc0 msgr2=0x7f7354100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.579+0000 7f7359400640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7354102fc0 0x7f7354100c10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f734402c590 tx=0x7f7344004270 comp rx=0 tx=0).stop 2024-04-20T21:41:06.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.579+0000 7f7359400640 1 -- 172.21.15.22:0/402479077 shutdown_connections 2024-04-20T21:41:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.579+0000 7f7359400640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f73380a3dd0 0x7f73380a6290 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.579+0000 7f7359400640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f735410c3e0 0x7f7354101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.579+0000 7f7359400640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73541038e0 0x7f7354101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.579+0000 7f7359400640 1 --2- 172.21.15.22:0/402479077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7354102fc0 0x7f7354100c10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.579+0000 7f7359400640 1 -- 172.21.15.22:0/402479077 >> 172.21.15.22:0/402479077 conn(0x7f73540fc7e0 msgr2=0x7f73540fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.580+0000 7f7359400640 1 -- 172.21.15.22:0/402479077 shutdown_connections 2024-04-20T21:41:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.580+0000 7f7359400640 1 -- 172.21.15.22:0/402479077 wait complete. 2024-04-20T21:41:06.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:41:06.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:41:06.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:06 smithi022 ceph-mon[16843]: pgmap v223: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 59 KiB/s wr, 20 op/s 2024-04-20T21:41:06.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:06 smithi122 ceph-mon[22147]: pgmap v223: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 59 KiB/s wr, 20 op/s 2024-04-20T21:41:06.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.797+0000 7f82cbe00640 1 -- 172.21.15.22:0/1571381959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82cc10e6c0 msgr2=0x7f82cc110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.797+0000 7f82cbe00640 1 --2- 172.21.15.22:0/1571381959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82cc10e6c0 0x7f82cc110ab0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009900 tx=0x7f82c402bcc0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.797+0000 7f82cbe00640 1 -- 172.21.15.22:0/1571381959 shutdown_connections 2024-04-20T21:41:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.797+0000 7f82cbe00640 1 --2- 172.21.15.22:0/1571381959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82cc10e6c0 0x7f82cc110ab0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.797+0000 7f82cbe00640 1 --2- 172.21.15.22:0/1571381959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82cc074c30 0x7f82cc10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.797+0000 7f82cbe00640 1 --2- 172.21.15.22:0/1571381959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82cc074310 0x7f82cc0746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.797+0000 7f82cbe00640 1 -- 172.21.15.22:0/1571381959 >> 172.21.15.22:0/1571381959 conn(0x7f82cc0fd930 msgr2=0x7f82cc0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.797+0000 7f82cbe00640 1 -- 172.21.15.22:0/1571381959 shutdown_connections 2024-04-20T21:41:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.797+0000 7f82cbe00640 1 -- 172.21.15.22:0/1571381959 wait complete. 2024-04-20T21:41:06.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.798+0000 7f82cbe00640 1 Processor -- start 2024-04-20T21:41:06.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.799+0000 7f82cbe00640 1 -- start start 2024-04-20T21:41:06.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.799+0000 7f82cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82cc074310 0x7f82cc1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.799+0000 7f82cbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82cc074c30 0x7f82cc1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.799+0000 7f82cbe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82cc10e6c0 0x7f82cc19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.799+0000 7f82cbe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1143a0 con 0x7f82cc074310 2024-04-20T21:41:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.799+0000 7f82cbe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc114220 con 0x7f82cc10e6c0 2024-04-20T21:41:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.799+0000 7f82cbe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc114520 con 0x7f82cc074c30 2024-04-20T21:41:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.799+0000 7f82caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82cc074310 0x7f82cc1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.799+0000 7f82caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82cc074310 0x7f82cc1a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43118/0 (socket says 172.21.15.22:43118) 2024-04-20T21:41:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.799+0000 7f82caa00640 1 -- 172.21.15.22:0/2480003303 learned_addr learned my addr 172.21.15.22:0/2480003303 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.799+0000 7f82cb400640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82cc10e6c0 0x7f82cc19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.799+0000 7f82ca000640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82cc074c30 0x7f82cc1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.800+0000 7f82caa00640 1 -- 172.21.15.22:0/2480003303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82cc074c30 msgr2=0x7f82cc1a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.800+0000 7f82caa00640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82cc074c30 0x7f82cc1a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.800+0000 7f82caa00640 1 -- 172.21.15.22:0/2480003303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82cc10e6c0 msgr2=0x7f82cc19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.800+0000 7f82caa00640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82cc10e6c0 0x7f82cc19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.800+0000 7f82caa00640 1 -- 172.21.15.22:0/2480003303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82cc19af60 con 0x7f82cc074310 2024-04-20T21:41:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.800+0000 7f82caa00640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82cc074310 0x7f82cc1a05e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00e9f0 tx=0x7f82bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.800+0000 7f82c3400640 1 -- 172.21.15.22:0/2480003303 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00cd80 con 0x7f82cc074310 2024-04-20T21:41:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.801+0000 7f82c3400640 1 -- 172.21.15.22:0/2480003303 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc010910 con 0x7f82cc074310 2024-04-20T21:41:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.801+0000 7f82c3400640 1 -- 172.21.15.22:0/2480003303 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc018990 con 0x7f82cc074310 2024-04-20T21:41:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.801+0000 7f82cbe00640 1 -- 172.21.15.22:0/2480003303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82cc19b250 con 0x7f82cc074310 2024-04-20T21:41:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.801+0000 7f82cbe00640 1 -- 172.21.15.22:0/2480003303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82cc1a7560 con 0x7f82cc074310 2024-04-20T21:41:06.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:06 smithi033 ceph-mon[21626]: pgmap v223: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 59 KiB/s wr, 20 op/s 2024-04-20T21:41:06.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.804+0000 7f82d4768740 1 Processor -- start 2024-04-20T21:41:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.804+0000 7f82d4768740 1 -- start start 2024-04-20T21:41:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.804+0000 7f82c3400640 1 -- 172.21.15.22:0/2480003303 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f82bc010ac0 con 0x7f82cc074310 2024-04-20T21:41:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.805+0000 7f82d4768740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd97b380 0x555afd97b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.805+0000 7f82d4768740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97bca0 0x555afd984340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.805+0000 7f82d4768740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd984880 0x555afd986c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.805+0000 7f82d4768740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555afd8e1150 con 0x555afd97bca0 2024-04-20T21:41:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.805+0000 7f82d4768740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555afd887370 con 0x555afd97b380 2024-04-20T21:41:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.805+0000 7f82d4768740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555afd8dd110 con 0x555afd984880 2024-04-20T21:41:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.805+0000 7f82caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97bca0 0x555afd984340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.805+0000 7f82cb400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd97b380 0x555afd97b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.805+0000 7f82caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97bca0 0x555afd984340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43132/0 (socket says 172.21.15.22:43132) 2024-04-20T21:41:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.805+0000 7f82caa00640 1 -- 172.21.15.22:0/2778465130 learned_addr learned my addr 172.21.15.22:0/2778465130 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.805+0000 7f82ca000640 1 --2- 172.21.15.22:0/2778465130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd984880 0x555afd986c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.806+0000 7f82cb400640 1 -- 172.21.15.22:0/2778465130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd984880 msgr2=0x555afd986c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.806+0000 7f82cb400640 1 --2- 172.21.15.22:0/2778465130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd984880 0x555afd986c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.806+0000 7f82cb400640 1 -- 172.21.15.22:0/2778465130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97bca0 msgr2=0x555afd984340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.806+0000 7f82cb400640 1 --2- 172.21.15.22:0/2778465130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97bca0 0x555afd984340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.806+0000 7f82cb400640 1 -- 172.21.15.22:0/2778465130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555afd6bd7d0 con 0x555afd97b380 2024-04-20T21:41:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.806+0000 7f82caa00640 1 --2- 172.21.15.22:0/2778465130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97bca0 0x555afd984340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:06.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.806+0000 7f82cb400640 1 --2- 172.21.15.22:0/2778465130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd97b380 0x555afd97b760 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009230 tx=0x7f82c4005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6439249c36421824 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.806+0000 7f82bbe00640 1 -- 172.21.15.22:0/2778465130 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4045070 con 0x555afd97b380 2024-04-20T21:41:06.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.806+0000 7f82c3400640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f82b00a3da0 0x7f82b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.806+0000 7f82bbe00640 1 -- 172.21.15.22:0/2778465130 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c4004040 con 0x555afd97b380 2024-04-20T21:41:06.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.806+0000 7f82bbe00640 1 -- 172.21.15.22:0/2778465130 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4004bc0 con 0x555afd97b380 2024-04-20T21:41:06.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.807+0000 7f82c3400640 1 -- 172.21.15.22:0/2480003303 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8644+0+0 (secure 0 0 0) 0x7f82bc0d0b60 con 0x7f82cc074310 2024-04-20T21:41:06.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.807+0000 7f82caa00640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f82b00a3da0 0x7f82b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.807+0000 7f82d4768740 1 -- 172.21.15.22:0/2778465130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd97b380 msgr2=0x555afd97b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.807+0000 7f82d4768740 1 --2- 172.21.15.22:0/2778465130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd97b380 0x555afd97b760 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009230 tx=0x7f82c4005d50 comp rx=0 tx=0).stop 2024-04-20T21:41:06.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.807+0000 7f82d4768740 1 -- 172.21.15.22:0/2778465130 shutdown_connections 2024-04-20T21:41:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.807+0000 7f82d4768740 1 --2- 172.21.15.22:0/2778465130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd984880 0x555afd986c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.807+0000 7f82d4768740 1 --2- 172.21.15.22:0/2778465130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97bca0 0x555afd984340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.807+0000 7f82d4768740 1 --2- 172.21.15.22:0/2778465130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd97b380 0x555afd97b760 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.807+0000 7f82d4768740 1 -- 172.21.15.22:0/2778465130 >> 172.21.15.22:0/2778465130 conn(0x555afd975ab0 msgr2=0x555afd975e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.807+0000 7f82caa00640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f82b00a3da0 0x7f82b00a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82bc098f30 tx=0x7f82bc0d4610 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.807+0000 7f82d4768740 1 -- 172.21.15.22:0/2778465130 shutdown_connections 2024-04-20T21:41:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.807+0000 7f82d4768740 1 -- 172.21.15.22:0/2778465130 wait complete. 2024-04-20T21:41:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.808+0000 7f82d4768740 1 -- start start 2024-04-20T21:41:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.808+0000 7f82d4768740 1 Processor -- start 2024-04-20T21:41:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82d4768740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97b380 0x555afd9818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82d4768740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 0x555afd981df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82d4768740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd984880 0x555afd982330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82d4768740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555afd987610 con 0x555afd97b380 2024-04-20T21:41:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82d4768740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555afd89a740 con 0x555afd984880 2024-04-20T21:41:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82d4768740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555afd987790 con 0x555afd97bca0 2024-04-20T21:41:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82d4768740 10 client.0 _ll_register_callbacks cb 0x555afd895b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82cb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97b380 0x555afd9818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82ca000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 0x555afd981df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82cb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97b380 0x555afd9818b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43140/0 (socket says 172.21.15.22:43140) 2024-04-20T21:41:06.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82cb400640 1 -- 172.21.15.22:0/490638271 learned_addr learned my addr 172.21.15.22:0/490638271 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:06.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82cb400640 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd984880 0x555afd982330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.809+0000 7f82ca000640 1 -- 172.21.15.22:0/490638271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd984880 msgr2=0x555afd982330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.810+0000 7f82ca000640 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd984880 0x555afd982330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.810+0000 7f82ca000640 1 -- 172.21.15.22:0/490638271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97b380 msgr2=0x555afd9818b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.810+0000 7f82ca000640 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97b380 0x555afd9818b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.810+0000 7f82ca000640 1 -- 172.21.15.22:0/490638271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555afd5f33f0 con 0x555afd97bca0 2024-04-20T21:41:06.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.810+0000 7f82cb400640 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97b380 0x555afd9818b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.810+0000 7f82ca000640 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 0x555afd981df0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f82b4005fd0 tx=0x7f82b4011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91da8aedfe4653f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.810+0000 7f82b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:41:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.810+0000 7f82b8c00640 1 -- 172.21.15.22:0/490638271 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4010b70 con 0x555afd97bca0 2024-04-20T21:41:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.810+0000 7f82b8c00640 1 -- 172.21.15.22:0/490638271 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8644+0+0 (secure 0 0 0) 0x7f82b4010d10 con 0x555afd97bca0 2024-04-20T21:41:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.810+0000 7f82d4768740 1 -- 172.21.15.22:0/490638271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c4004040 con 0x555afd97bca0 2024-04-20T21:41:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.811+0000 7f82b8c00640 1 -- 172.21.15.22:0/490638271 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f82b4022430 con 0x555afd97bca0 2024-04-20T21:41:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.811+0000 7f82d4768740 10 client.26488 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.811+0000 7f82d4768740 1 -- 172.21.15.22:0/490638271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555afd5b30f0 con 0x555afd97bca0 2024-04-20T21:41:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.811+0000 7f82b8c00640 1 -- 172.21.15.22:0/490638271 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c4004040 con 0x555afd97bca0 2024-04-20T21:41:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.811+0000 7f82b8c00640 1 -- 172.21.15.22:0/490638271 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f82b401a5f0 con 0x555afd97bca0 2024-04-20T21:41:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.811+0000 7f82d4768740 10 client.26488 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.811+0000 7f82d4768740 10 client.26488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.811+0000 7f82d4768740 20 client.26488 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.811+0000 7f82d4768740 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555afd994c00 0x555afd997060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.811+0000 7f82d4768740 4 client.26488 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.811+0000 7f82d4768740 1 -- 172.21.15.22:0/490638271 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555afd8eeb70 con 0x555afd994c00 2024-04-20T21:41:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.812+0000 7f82cb400640 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555afd994c00 0x555afd997060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.812+0000 7f82cb400640 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555afd994c00 0x555afd997060 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7c72cef4bae5cc server_cookie=c4a7e8284458edc5 in_seq=0 out_seq=0 2024-04-20T21:41:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.812+0000 7f82b8c00640 10 client.26488 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.813+0000 7f82b8c00640 1 -- 172.21.15.22:0/490638271 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555afd8eeb70 con 0x555afd994c00 2024-04-20T21:41:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.813+0000 7f82b8c00640 10 client.26488 handle_command_reply: tid=0 2024-04-20T21:41:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.813+0000 7f82d4768740 1 client.26488 shutdown 2024-04-20T21:41:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.814+0000 7f82d4768740 1 -- 172.21.15.22:0/490638271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 msgr2=0x555afd981df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.814+0000 7f82d4768740 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 0x555afd981df0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f82b4005fd0 tx=0x7f82b4011b50 comp rx=0 tx=0).stop 2024-04-20T21:41:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.814+0000 7f82d4768740 1 -- 172.21.15.22:0/490638271 shutdown_connections 2024-04-20T21:41:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.814+0000 7f82d4768740 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555afd994c00 0x555afd997060 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.814+0000 7f82d4768740 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd984880 0x555afd982330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.814+0000 7f82b8c00640 0 client.26488 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.814+0000 7f82d4768740 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 0x555afd981df0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.814+0000 7f82d4768740 1 --2- 172.21.15.22:0/490638271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97b380 0x555afd9818b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.814+0000 7f82d4768740 1 -- 172.21.15.22:0/490638271 >> 172.21.15.22:0/490638271 conn(0x555afd975ab0 msgr2=0x555afd977800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.814+0000 7f82d4768740 1 -- 172.21.15.22:0/490638271 shutdown_connections 2024-04-20T21:41:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.814+0000 7f82d4768740 1 -- 172.21.15.22:0/490638271 wait complete. 2024-04-20T21:41:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.814+0000 7f82d4768740 20 client.26488 trim_cache size 0 max 16384 2024-04-20T21:41:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.833+0000 7f82d4768740 1 Processor -- start 2024-04-20T21:41:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.833+0000 7f82d4768740 1 -- start start 2024-04-20T21:41:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.834+0000 7f82d4768740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97b380 0x555afd983fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.834+0000 7f82d4768740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97bca0 0x555afd9807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.834+0000 7f82d4768740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd984880 0x555afd97cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.834+0000 7f82d4768740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1adea0 con 0x555afd97bca0 2024-04-20T21:41:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.834+0000 7f82d4768740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1b0700 con 0x555afd984880 2024-04-20T21:41:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.834+0000 7f82d4768740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1ae4f0 con 0x555afd97b380 2024-04-20T21:41:06.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.834+0000 7f82cb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97b380 0x555afd983fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.834+0000 7f82cb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97b380 0x555afd983fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:49274/0 (socket says 172.21.15.22:49274) 2024-04-20T21:41:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.834+0000 7f82cb400640 1 -- 172.21.15.22:0/37151122 learned_addr learned my addr 172.21.15.22:0/37151122 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.834+0000 7f82ca000640 1 --2- 172.21.15.22:0/37151122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97bca0 0x555afd9807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.834+0000 7f82cb400640 1 --2- 172.21.15.22:0/37151122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd984880 0x555afd97cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.835+0000 7f82cb400640 1 -- 172.21.15.22:0/37151122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd984880 msgr2=0x555afd97cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.835+0000 7f82cb400640 1 --2- 172.21.15.22:0/37151122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd984880 0x555afd97cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.835+0000 7f82cb400640 1 -- 172.21.15.22:0/37151122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97bca0 msgr2=0x555afd9807f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.835+0000 7f82cb400640 1 --2- 172.21.15.22:0/37151122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97bca0 0x555afd9807f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.835+0000 7f82cb400640 1 -- 172.21.15.22:0/37151122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555afd986c70 con 0x555afd97b380 2024-04-20T21:41:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.835+0000 7f82cb400640 1 --2- 172.21.15.22:0/37151122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97b380 0x555afd983fe0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f82c402c1e0 tx=0x7f82c4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4db15b52b700f3a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.835+0000 7f82bb400640 1 -- 172.21.15.22:0/37151122 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4045040 con 0x555afd97b380 2024-04-20T21:41:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.835+0000 7f82bb400640 1 -- 172.21.15.22:0/37151122 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c402cdf0 con 0x555afd97b380 2024-04-20T21:41:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.835+0000 7f82bb400640 1 -- 172.21.15.22:0/37151122 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c40568d0 con 0x555afd97b380 2024-04-20T21:41:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.836+0000 7f82d4768740 1 -- 172.21.15.22:0/37151122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97b380 msgr2=0x555afd983fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.836+0000 7f82d4768740 1 --2- 172.21.15.22:0/37151122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97b380 0x555afd983fe0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f82c402c1e0 tx=0x7f82c4005fd0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.836+0000 7f82d4768740 1 -- 172.21.15.22:0/37151122 shutdown_connections 2024-04-20T21:41:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.836+0000 7f82d4768740 1 --2- 172.21.15.22:0/37151122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd984880 0x555afd97cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.836+0000 7f82d4768740 1 --2- 172.21.15.22:0/37151122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd97bca0 0x555afd9807f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.836+0000 7f82d4768740 1 --2- 172.21.15.22:0/37151122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97b380 0x555afd983fe0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.836+0000 7f82d4768740 1 -- 172.21.15.22:0/37151122 >> 172.21.15.22:0/37151122 conn(0x555afd975ab0 msgr2=0x555afd982100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.836+0000 7f82d4768740 1 -- 172.21.15.22:0/37151122 shutdown_connections 2024-04-20T21:41:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.836+0000 7f82d4768740 1 -- 172.21.15.22:0/37151122 wait complete. 2024-04-20T21:41:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.836+0000 7f82d4768740 1 -- start start 2024-04-20T21:41:06.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.837+0000 7f82d4768740 1 Processor -- start 2024-04-20T21:41:06.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.837+0000 7f82d4768740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd97b380 0x555afd98cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.837+0000 7f82d4768740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 0x555afd988260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.837+0000 7f82d4768740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd984880 0x555afd988870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.837+0000 7f82d4768740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1aebc0 con 0x555afd984880 2024-04-20T21:41:06.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.837+0000 7f82d4768740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1b0030 con 0x555afd97b380 2024-04-20T21:41:06.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.837+0000 7f82d4768740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1af290 con 0x555afd97bca0 2024-04-20T21:41:06.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.837+0000 7f82d4768740 10 client.0 _ll_register_callbacks cb 0x7f82b4009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:06.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82ca000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 0x555afd988260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82cb400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd97b380 0x555afd98cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82ca000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 0x555afd988260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:49288/0 (socket says 172.21.15.22:49288) 2024-04-20T21:41:06.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82ca000640 1 -- 172.21.15.22:0/3764778638 learned_addr learned my addr 172.21.15.22:0/3764778638 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:06.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82cb400640 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd984880 0x555afd988870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82ca000640 1 -- 172.21.15.22:0/3764778638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd97b380 msgr2=0x555afd98cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82ca000640 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd97b380 0x555afd98cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82ca000640 1 -- 172.21.15.22:0/3764778638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd984880 msgr2=0x555afd988870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:06.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82ca000640 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd984880 0x555afd988870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82ca000640 1 -- 172.21.15.22:0/3764778638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555afd989070 con 0x555afd97bca0 2024-04-20T21:41:06.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82cb400640 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd984880 0x555afd988870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:41:06.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82ca000640 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 0x555afd988260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f82b40108d0 tx=0x7f82b4010da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e724c0189f0c887 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:06.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.838+0000 7f82b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:41:06.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.839+0000 7f82b8c00640 1 -- 172.21.15.22:0/3764778638 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b401a5d0 con 0x555afd97bca0 2024-04-20T21:41:06.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.839+0000 7f82d4768740 1 -- 172.21.15.22:0/3764778638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c402cdf0 con 0x555afd97bca0 2024-04-20T21:41:06.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.839+0000 7f82b8c00640 1 -- 172.21.15.22:0/3764778638 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8644+0+0 (secure 0 0 0) 0x7f82b4017040 con 0x555afd97bca0 2024-04-20T21:41:06.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.839+0000 7f82b8c00640 1 -- 172.21.15.22:0/3764778638 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f82b401a770 con 0x555afd97bca0 2024-04-20T21:41:06.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.839+0000 7f82d4768740 10 client.26494 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.839+0000 7f82d4768740 1 -- 172.21.15.22:0/3764778638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555afd9fb2e0 con 0x555afd97bca0 2024-04-20T21:41:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.839+0000 7f82b8c00640 1 -- 172.21.15.22:0/3764778638 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c402cdf0 con 0x555afd97bca0 2024-04-20T21:41:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.840+0000 7f82b8c00640 1 -- 172.21.15.22:0/3764778638 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f82b4005250 con 0x555afd97bca0 2024-04-20T21:41:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.840+0000 7f82d4768740 10 client.26494 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.840+0000 7f82d4768740 10 client.26494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.840+0000 7f82d4768740 20 client.26494 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.840+0000 7f82d4768740 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555afd9fba90 0x555afd9fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.840+0000 7f82d4768740 4 client.26494 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:41:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.840+0000 7f82d4768740 1 -- 172.21.15.22:0/3764778638 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555afd8e89e0 con 0x555afd9fba90 2024-04-20T21:41:06.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.840+0000 7f82cb400640 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555afd9fba90 0x555afd9fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:06.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82cb400640 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555afd9fba90 0x555afd9fdeb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6063b254e1517d6 server_cookie=ef1529dd708e39e7 in_seq=0 out_seq=0 2024-04-20T21:41:06.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82b8c00640 10 client.26494 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:06.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82b8c00640 1 -- 172.21.15.22:0/3764778638 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555afd8e89e0 con 0x555afd9fba90 2024-04-20T21:41:06.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82b8c00640 10 client.26494 handle_command_reply: tid=0 2024-04-20T21:41:06.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82d4768740 1 client.26494 shutdown 2024-04-20T21:41:06.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82d4768740 1 -- 172.21.15.22:0/3764778638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 msgr2=0x555afd988260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82d4768740 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 0x555afd988260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f82b40108d0 tx=0x7f82b4010da0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82d4768740 1 -- 172.21.15.22:0/3764778638 shutdown_connections 2024-04-20T21:41:06.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82d4768740 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555afd984880 0x555afd988870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82d4768740 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555afd97bca0 0x555afd988260 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82d4768740 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x555afd9fba90 0x555afd9fdeb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82d4768740 1 --2- 172.21.15.22:0/3764778638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555afd97b380 0x555afd98cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82d4768740 1 -- 172.21.15.22:0/3764778638 >> 172.21.15.22:0/3764778638 conn(0x555afd975ab0 msgr2=0x555afd974280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82b8c00640 0 client.26494 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:06.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82d4768740 1 -- 172.21.15.22:0/3764778638 shutdown_connections 2024-04-20T21:41:06.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.841+0000 7f82d4768740 1 -- 172.21.15.22:0/3764778638 wait complete. 2024-04-20T21:41:06.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.842+0000 7f82d4768740 20 client.26494 trim_cache size 0 max 16384 2024-04-20T21:41:06.848 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:41:06.848 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:41:06.848 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:41:06.848 INFO:teuthology.orchestra.run.smithi022.stdout: "cbb1d2e8-e79b-4833-912f-97640fed471f": { 2024-04-20T21:41:06.848 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:41:06.849 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "cbb1d2e8-e79b-4833-912f-97640fed471f", 2024-04-20T21:41:06.849 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:41:06.849 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:41:06.849 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:41:06.849 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:41:06.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.842+0000 7f82cbe00640 1 -- 172.21.15.22:0/2480003303 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f82b00a3da0 msgr2=0x7f82b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.842+0000 7f82cbe00640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f82b00a3da0 0x7f82b00a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82bc098f30 tx=0x7f82bc0d4610 comp rx=0 tx=0).stop 2024-04-20T21:41:06.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.842+0000 7f82cbe00640 1 -- 172.21.15.22:0/2480003303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82cc074310 msgr2=0x7f82cc1a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:06.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.842+0000 7f82cbe00640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82cc074310 0x7f82cc1a05e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00e9f0 tx=0x7f82bc00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.842+0000 7f82cbe00640 1 -- 172.21.15.22:0/2480003303 shutdown_connections 2024-04-20T21:41:06.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.842+0000 7f82cbe00640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f82b00a3da0 0x7f82b00a6260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.842+0000 7f82cbe00640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82cc10e6c0 0x7f82cc19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.842+0000 7f82cbe00640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82cc074c30 0x7f82cc1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.842+0000 7f82cbe00640 1 --2- 172.21.15.22:0/2480003303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82cc074310 0x7f82cc1a05e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:06.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.842+0000 7f82cbe00640 1 -- 172.21.15.22:0/2480003303 >> 172.21.15.22:0/2480003303 conn(0x7f82cc0fd930 msgr2=0x7f82cc0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:06.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.843+0000 7f82cbe00640 1 -- 172.21.15.22:0/2480003303 shutdown_connections 2024-04-20T21:41:06.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:06.843+0000 7f82cbe00640 1 -- 172.21.15.22:0/2480003303 wait complete. 2024-04-20T21:41:06.859 INFO:tasks.cephfs.filesystem:scrub status for tag:cbb1d2e8-e79b-4833-912f-97640fed471f - {'path': '/', 'tag': 'cbb1d2e8-e79b-4833-912f-97640fed471f', 'options': 'recursive,force'} 2024-04-20T21:41:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:07 smithi022 ceph-mon[16843]: osdmap e96: 12 total, 12 up, 12 in 2024-04-20T21:41:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:07 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:41:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:07 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:07 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:41:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:07 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:07.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:07 smithi122 ceph-mon[22147]: osdmap e96: 12 total, 12 up, 12 in 2024-04-20T21:41:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:07 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:41:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:07 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:07 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:41:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:07 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:07.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:07 smithi033 ceph-mon[21626]: osdmap e96: 12 total, 12 up, 12 in 2024-04-20T21:41:07.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:07 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:41:07.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:07 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:07.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:07 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:41:07.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:07 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:08.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:08 smithi022 ceph-mon[16843]: pgmap v225: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 23 op/s 2024-04-20T21:41:08.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:08 smithi122 ceph-mon[22147]: pgmap v225: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 23 op/s 2024-04-20T21:41:08.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:08 smithi033 ceph-mon[21626]: pgmap v225: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 23 op/s 2024-04-20T21:41:09.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:09 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:41:09.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:09 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:41:09.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:09 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:41:10.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:10 smithi022 ceph-mon[16843]: pgmap v226: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 23 op/s 2024-04-20T21:41:10.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:10 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:41:10.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:10 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:41:10.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:10 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:41:10.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:10 smithi122 ceph-mon[22147]: pgmap v226: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 23 op/s 2024-04-20T21:41:10.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:10 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:41:10.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:10 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:41:10.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:10 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:41:10.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:10 smithi033 ceph-mon[21626]: pgmap v226: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 23 op/s 2024-04-20T21:41:10.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:10 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:41:10.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:10 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:41:10.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:10 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:41:11.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:11 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:41:11.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:11 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:41:11.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:11 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:41:12.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:12 smithi022 ceph-mon[16843]: pgmap v227: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 KiB/s wr, 23 op/s 2024-04-20T21:41:12.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:12 smithi122 ceph-mon[22147]: pgmap v227: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 KiB/s wr, 23 op/s 2024-04-20T21:41:12.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:12 smithi033 ceph-mon[21626]: pgmap v227: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 KiB/s wr, 23 op/s 2024-04-20T21:41:14.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:14 smithi122 ceph-mon[22147]: pgmap v228: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 KiB/s wr, 23 op/s 2024-04-20T21:41:14.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:14 smithi122 ceph-mon[22147]: osdmap e97: 12 total, 12 up, 12 in 2024-04-20T21:41:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:14 smithi033 ceph-mon[21626]: pgmap v228: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 KiB/s wr, 23 op/s 2024-04-20T21:41:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:14 smithi033 ceph-mon[21626]: osdmap e97: 12 total, 12 up, 12 in 2024-04-20T21:41:14.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:14 smithi022 ceph-mon[16843]: pgmap v228: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 KiB/s wr, 23 op/s 2024-04-20T21:41:14.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:14 smithi022 ceph-mon[16843]: osdmap e97: 12 total, 12 up, 12 in 2024-04-20T21:41:16.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:16 smithi122 ceph-mon[22147]: pgmap v230: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 3 op/s 2024-04-20T21:41:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:16 smithi033 ceph-mon[21626]: pgmap v230: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 3 op/s 2024-04-20T21:41:16.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:16 smithi022 ceph-mon[16843]: pgmap v230: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 3 op/s 2024-04-20T21:41:18.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:18 smithi122 ceph-mon[22147]: pgmap v231: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.6 KiB/s wr, 2 op/s 2024-04-20T21:41:18.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:18 smithi033 ceph-mon[21626]: pgmap v231: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.6 KiB/s wr, 2 op/s 2024-04-20T21:41:18.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:18 smithi022 ceph-mon[16843]: pgmap v231: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.6 KiB/s wr, 2 op/s 2024-04-20T21:41:20.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:20 smithi122 ceph-mon[22147]: pgmap v232: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.6 KiB/s wr, 2 op/s 2024-04-20T21:41:20.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:20 smithi033 ceph-mon[21626]: pgmap v232: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.6 KiB/s wr, 2 op/s 2024-04-20T21:41:20.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:20 smithi022 ceph-mon[16843]: pgmap v232: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.6 KiB/s wr, 2 op/s 2024-04-20T21:41:22.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:22 smithi122 ceph-mon[22147]: pgmap v233: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:41:22.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:22 smithi033 ceph-mon[21626]: pgmap v233: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:41:22.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:22 smithi022 ceph-mon[16843]: pgmap v233: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:41:23.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:23 smithi122 ceph-mon[22147]: pgmap v234: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:41:23.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:23 smithi033 ceph-mon[21626]: pgmap v234: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:41:23.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:23 smithi022 ceph-mon[16843]: pgmap v234: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:41:26.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:26 smithi022 ceph-mon[16843]: pgmap v235: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 94 B/s wr, 2 op/s 2024-04-20T21:41:26.506 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:26 smithi033 ceph-mon[21626]: pgmap v235: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 94 B/s wr, 2 op/s 2024-04-20T21:41:26.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:26 smithi122 ceph-mon[22147]: pgmap v235: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 94 B/s wr, 2 op/s 2024-04-20T21:41:28.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:28 smithi022 ceph-mon[16843]: pgmap v236: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:28.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:28 smithi122 ceph-mon[22147]: pgmap v236: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:28.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:28 smithi033 ceph-mon[21626]: pgmap v236: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:30 smithi022 ceph-mon[16843]: pgmap v237: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:30.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:30 smithi122 ceph-mon[22147]: pgmap v237: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:30.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:30 smithi033 ceph-mon[21626]: pgmap v237: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:32.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:32 smithi022 ceph-mon[16843]: pgmap v238: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:41:32.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:32 smithi122 ceph-mon[22147]: pgmap v238: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:41:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:32 smithi033 ceph-mon[21626]: pgmap v238: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:41:34.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:34 smithi022 ceph-mon[16843]: pgmap v239: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:34.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:34 smithi122 ceph-mon[22147]: pgmap v239: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:34 smithi033 ceph-mon[21626]: pgmap v239: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:36.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:36 smithi022 ceph-mon[16843]: pgmap v240: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:41:36.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:36 smithi122 ceph-mon[22147]: pgmap v240: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:41:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:36 smithi033 ceph-mon[21626]: pgmap v240: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:41:36.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:41:36.861 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:41:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.091+0000 7fd3b5200640 1 -- 172.21.15.22:0/3660281134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b0108ea0 msgr2=0x7fd3b01152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.091+0000 7fd3b5200640 1 --2- 172.21.15.22:0/3660281134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b0108ea0 0x7fd3b01152c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009ca0 tx=0x7fd3a802c190 comp rx=0 tx=0).stop 2024-04-20T21:41:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.092+0000 7fd3b5200640 1 -- 172.21.15.22:0/3660281134 shutdown_connections 2024-04-20T21:41:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.092+0000 7fd3b5200640 1 --2- 172.21.15.22:0/3660281134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b0108ea0 0x7fd3b01152c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.092+0000 7fd3b5200640 1 --2- 172.21.15.22:0/3660281134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0108500 0x7fd3b0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.092+0000 7fd3b5200640 1 --2- 172.21.15.22:0/3660281134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3b010d3d0 0x7fd3b010d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.092+0000 7fd3b5200640 1 -- 172.21.15.22:0/3660281134 >> 172.21.15.22:0/3660281134 conn(0x7fd3b0077d80 msgr2=0x7fd3b0106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.092+0000 7fd3b5200640 1 -- 172.21.15.22:0/3660281134 shutdown_connections 2024-04-20T21:41:37.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.092+0000 7fd3b5200640 1 -- 172.21.15.22:0/3660281134 wait complete. 2024-04-20T21:41:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.093+0000 7fd3b5200640 1 Processor -- start 2024-04-20T21:41:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.093+0000 7fd3b5200640 1 -- start start 2024-04-20T21:41:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.093+0000 7fd3b5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0108500 0x7fd3b01a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3b5200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3b0108ea0 0x7fd3b01a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3b5200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b010d3d0 0x7fd3b019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3b5200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01187f0 con 0x7fd3b0108ea0 2024-04-20T21:41:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3b5200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b0118670 con 0x7fd3b010d3d0 2024-04-20T21:41:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3b5200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b0118970 con 0x7fd3b0108500 2024-04-20T21:41:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3a6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3b0108ea0 0x7fd3b01a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3a6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3b0108ea0 0x7fd3b01a4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38294/0 (socket says 172.21.15.22:38294) 2024-04-20T21:41:37.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3a6a00640 1 -- 172.21.15.22:0/1053955845 learned_addr learned my addr 172.21.15.22:0/1053955845 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:37.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3af400640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0108500 0x7fd3b01a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3afe00640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b010d3d0 0x7fd3b019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3a6a00640 1 -- 172.21.15.22:0/1053955845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0108500 msgr2=0x7fd3b01a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0108500 0x7fd3b01a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3a6a00640 1 -- 172.21.15.22:0/1053955845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b010d3d0 msgr2=0x7fd3b019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b010d3d0 0x7fd3b019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.094+0000 7fd3a6a00640 1 -- 172.21.15.22:0/1053955845 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b019f2d0 con 0x7fd3b0108ea0 2024-04-20T21:41:37.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.095+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3b0108ea0 0x7fd3b01a4e90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd39800e9f0 tx=0x7fd39800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.095+0000 7fd3acc00640 1 -- 172.21.15.22:0/1053955845 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39800cd80 con 0x7fd3b0108ea0 2024-04-20T21:41:37.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.095+0000 7fd3acc00640 1 -- 172.21.15.22:0/1053955845 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd398010910 con 0x7fd3b0108ea0 2024-04-20T21:41:37.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.095+0000 7fd3b5200640 1 -- 172.21.15.22:0/1053955845 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b019f5c0 con 0x7fd3b0108ea0 2024-04-20T21:41:37.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.095+0000 7fd3acc00640 1 -- 172.21.15.22:0/1053955845 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398018990 con 0x7fd3b0108ea0 2024-04-20T21:41:37.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.095+0000 7fd3b5200640 1 -- 172.21.15.22:0/1053955845 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b01ab810 con 0x7fd3b0108ea0 2024-04-20T21:41:37.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.099+0000 7fd3acc00640 1 -- 172.21.15.22:0/1053955845 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd398010ac0 con 0x7fd3b0108ea0 2024-04-20T21:41:37.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.099+0000 7fd3b9418740 1 Processor -- start 2024-04-20T21:41:37.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.099+0000 7fd3b9418740 1 -- start start 2024-04-20T21:41:37.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.099+0000 7fd3b9418740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137380 0x55cfda137760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.099+0000 7fd3b9418740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137ca0 0x55cfda140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.099+0000 7fd3b9418740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda140880 0x55cfda142c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.099+0000 7fd3b9418740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cfda09d150 con 0x55cfda137ca0 2024-04-20T21:41:37.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.099+0000 7fd3b9418740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cfda043370 con 0x55cfda137380 2024-04-20T21:41:37.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.099+0000 7fd3b9418740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cfda099110 con 0x55cfda140880 2024-04-20T21:41:37.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.099+0000 7fd3af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137ca0 0x55cfda140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.100+0000 7fd3af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137ca0 0x55cfda140340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38300/0 (socket says 172.21.15.22:38300) 2024-04-20T21:41:37.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.100+0000 7fd3af400640 1 -- 172.21.15.22:0/2885370760 learned_addr learned my addr 172.21.15.22:0/2885370760 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:37.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.100+0000 7fd3afe00640 1 --2- 172.21.15.22:0/2885370760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137380 0x55cfda137760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.100+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/2885370760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda140880 0x55cfda142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.100+0000 7fd3af400640 1 -- 172.21.15.22:0/2885370760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda140880 msgr2=0x55cfda142c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.100+0000 7fd3af400640 1 --2- 172.21.15.22:0/2885370760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda140880 0x55cfda142c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.100+0000 7fd3af400640 1 -- 172.21.15.22:0/2885370760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137380 msgr2=0x55cfda137760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.100+0000 7fd3af400640 1 --2- 172.21.15.22:0/2885370760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137380 0x55cfda137760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.100+0000 7fd3af400640 1 -- 172.21.15.22:0/2885370760 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfd9e797d0 con 0x55cfda137ca0 2024-04-20T21:41:37.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.100+0000 7fd3acc00640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd3940a3da0 0x7fd3940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.100+0000 7fd3af400640 1 --2- 172.21.15.22:0/2885370760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137ca0 0x55cfda140340 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009c70 tx=0x7fd3a0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fbd320c5d74acd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.100+0000 7fd3acc00640 1 -- 172.21.15.22:0/1053955845 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7fd3980d0b30 con 0x7fd3b0108ea0 2024-04-20T21:41:37.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.101+0000 7fd3a4c00640 1 -- 172.21.15.22:0/2885370760 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0004230 con 0x55cfda137ca0 2024-04-20T21:41:37.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.101+0000 7fd3afe00640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd3940a3da0 0x7fd3940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.101+0000 7fd3a4c00640 1 -- 172.21.15.22:0/2885370760 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a00043d0 con 0x55cfda137ca0 2024-04-20T21:41:37.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.101+0000 7fd3a4c00640 1 -- 172.21.15.22:0/2885370760 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0019740 con 0x55cfda137ca0 2024-04-20T21:41:37.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.101+0000 7fd3b9418740 1 -- 172.21.15.22:0/2885370760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137ca0 msgr2=0x55cfda140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.101+0000 7fd3b9418740 1 --2- 172.21.15.22:0/2885370760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137ca0 0x55cfda140340 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009c70 tx=0x7fd3a0011810 comp rx=0 tx=0).stop 2024-04-20T21:41:37.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.102+0000 7fd3afe00640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd3940a3da0 0x7fd3940a6260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8005e30 tx=0x7fd3a8005d80 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.102+0000 7fd3b9418740 1 -- 172.21.15.22:0/2885370760 shutdown_connections 2024-04-20T21:41:37.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.102+0000 7fd3b9418740 1 --2- 172.21.15.22:0/2885370760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda140880 0x55cfda142c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.102+0000 7fd3b9418740 1 --2- 172.21.15.22:0/2885370760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137ca0 0x55cfda140340 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.102+0000 7fd3b9418740 1 --2- 172.21.15.22:0/2885370760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137380 0x55cfda137760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.102+0000 7fd3b9418740 1 -- 172.21.15.22:0/2885370760 >> 172.21.15.22:0/2885370760 conn(0x55cfda131ab0 msgr2=0x55cfda131e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.102+0000 7fd3b9418740 1 -- 172.21.15.22:0/2885370760 shutdown_connections 2024-04-20T21:41:37.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.102+0000 7fd3b9418740 1 -- 172.21.15.22:0/2885370760 wait complete. 2024-04-20T21:41:37.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.102+0000 7fd3b9418740 1 -- start start 2024-04-20T21:41:37.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.102+0000 7fd3b9418740 1 Processor -- start 2024-04-20T21:41:37.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.103+0000 7fd3b9418740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 0x55cfda13d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.103+0000 7fd3b9418740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137ca0 0x55cfda13dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.103+0000 7fd3b9418740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda140880 0x55cfda13e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.103+0000 7fd3b9418740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cfda143670 con 0x55cfda137380 2024-04-20T21:41:37.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.103+0000 7fd3b9418740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cfda056740 con 0x55cfda137ca0 2024-04-20T21:41:37.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.103+0000 7fd3b9418740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cfda1437f0 con 0x55cfda140880 2024-04-20T21:41:37.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.103+0000 7fd3b9418740 10 client.0 _ll_register_callbacks cb 0x55cfda051b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:37.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.103+0000 7fd3af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 0x55cfda13d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.103+0000 7fd3af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 0x55cfda13d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38308/0 (socket says 172.21.15.22:38308) 2024-04-20T21:41:37.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.103+0000 7fd3af400640 1 -- 172.21.15.22:0/966946066 learned_addr learned my addr 172.21.15.22:0/966946066 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:37.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.103+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137ca0 0x55cfda13dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.103+0000 7fd3afe00640 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda140880 0x55cfda13e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.104+0000 7fd3af400640 1 -- 172.21.15.22:0/966946066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda140880 msgr2=0x55cfda13e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.104+0000 7fd3af400640 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda140880 0x55cfda13e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.104+0000 7fd3af400640 1 -- 172.21.15.22:0/966946066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137ca0 msgr2=0x55cfda13dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.104+0000 7fd3af400640 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137ca0 0x55cfda13dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.104+0000 7fd3af400640 1 -- 172.21.15.22:0/966946066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfd9daf3f0 con 0x55cfda137380 2024-04-20T21:41:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.104+0000 7fd3af400640 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 0x55cfda13d850 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0005db0 tx=0x7fd3a00056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4723e34eedb27168 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.104+0000 7fd39d600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:41:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.104+0000 7fd39d600640 1 -- 172.21.15.22:0/966946066 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0027070 con 0x55cfda137380 2024-04-20T21:41:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.104+0000 7fd39d600640 1 -- 172.21.15.22:0/966946066 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7fd3a0003800 con 0x55cfda137380 2024-04-20T21:41:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.104+0000 7fd3b9418740 1 -- 172.21.15.22:0/966946066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a00043d0 con 0x55cfda137380 2024-04-20T21:41:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.105+0000 7fd39d600640 1 -- 172.21.15.22:0/966946066 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd3a0033050 con 0x55cfda137380 2024-04-20T21:41:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.105+0000 7fd3b9418740 10 client.16332 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:37.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.105+0000 7fd3b9418740 1 -- 172.21.15.22:0/966946066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55cfd9d6f0f0 con 0x55cfda137380 2024-04-20T21:41:37.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.105+0000 7fd39d600640 1 -- 172.21.15.22:0/966946066 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a00043d0 con 0x55cfda137380 2024-04-20T21:41:37.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.105+0000 7fd39d600640 1 -- 172.21.15.22:0/966946066 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fd3a0018070 con 0x55cfda137380 2024-04-20T21:41:37.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.105+0000 7fd3b9418740 10 client.16332 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:37.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.105+0000 7fd3b9418740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:37.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.105+0000 7fd3b9418740 20 client.16332 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:37.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.105+0000 7fd3b9418740 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cfda150b40 0x55cfda152f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.105+0000 7fd3b9418740 4 client.16332 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:37.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.105+0000 7fd3b9418740 1 -- 172.21.15.22:0/966946066 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfda0aab70 con 0x55cfda150b40 2024-04-20T21:41:37.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.106+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cfda150b40 0x55cfda152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.106+0000 7fd39d600640 10 client.16332 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:37.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.106+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cfda150b40 0x55cfda152f30 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84d293a4269a4a6 server_cookie=e700f7eb4528aa04 in_seq=0 out_seq=0 2024-04-20T21:41:37.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.107+0000 7fd39d600640 1 -- 172.21.15.22:0/966946066 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfda0aab70 con 0x55cfda150b40 2024-04-20T21:41:37.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.107+0000 7fd39d600640 10 client.16332 handle_command_reply: tid=0 2024-04-20T21:41:37.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.107+0000 7fd3b9418740 1 client.16332 shutdown 2024-04-20T21:41:37.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.107+0000 7fd3b9418740 1 -- 172.21.15.22:0/966946066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 msgr2=0x55cfda13d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.107+0000 7fd3b9418740 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 0x55cfda13d850 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0005db0 tx=0x7fd3a00056d0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.107+0000 7fd3b9418740 1 -- 172.21.15.22:0/966946066 shutdown_connections 2024-04-20T21:41:37.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.107+0000 7fd3b9418740 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda140880 0x55cfda13e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.107+0000 7fd3b9418740 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cfda150b40 0x55cfda152f30 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.107+0000 7fd3b9418740 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137ca0 0x55cfda13dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.107+0000 7fd3b9418740 1 --2- 172.21.15.22:0/966946066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 0x55cfda13d850 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.107+0000 7fd39d600640 0 client.16332 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:37.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.107+0000 7fd3b9418740 1 -- 172.21.15.22:0/966946066 >> 172.21.15.22:0/966946066 conn(0x55cfda131ab0 msgr2=0x55cfda133800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.108+0000 7fd3b9418740 1 -- 172.21.15.22:0/966946066 shutdown_connections 2024-04-20T21:41:37.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.108+0000 7fd3b9418740 1 -- 172.21.15.22:0/966946066 wait complete. 2024-04-20T21:41:37.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.108+0000 7fd3b9418740 20 client.16332 trim_cache size 0 max 16384 2024-04-20T21:41:37.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.127+0000 7fd3b9418740 1 Processor -- start 2024-04-20T21:41:37.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.127+0000 7fd3b9418740 1 -- start start 2024-04-20T21:41:37.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.127+0000 7fd3b9418740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137380 0x55cfda13ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.127+0000 7fd3b9418740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 0x55cfda14c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.127+0000 7fd3b9418740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda140880 0x55cfda131e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.127+0000 7fd3b9418740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01b2210 con 0x55cfda140880 2024-04-20T21:41:37.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.127+0000 7fd3b9418740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01b4a70 con 0x55cfda137380 2024-04-20T21:41:37.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.127+0000 7fd3b9418740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01b2860 con 0x55cfda137ca0 2024-04-20T21:41:37.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.128+0000 7fd3af400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137380 0x55cfda13ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.128+0000 7fd3a6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 0x55cfda14c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.128+0000 7fd3a6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 0x55cfda14c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42496/0 (socket says 172.21.15.22:42496) 2024-04-20T21:41:37.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.128+0000 7fd3a6a00640 1 -- 172.21.15.22:0/979064872 learned_addr learned my addr 172.21.15.22:0/979064872 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:37.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.128+0000 7fd3afe00640 1 --2- 172.21.15.22:0/979064872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda140880 0x55cfda131e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.128+0000 7fd3a6a00640 1 -- 172.21.15.22:0/979064872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137380 msgr2=0x55cfda13ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.128+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/979064872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137380 0x55cfda13ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.128+0000 7fd3a6a00640 1 -- 172.21.15.22:0/979064872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda140880 msgr2=0x55cfda131e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.128+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/979064872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda140880 0x55cfda131e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.128+0000 7fd3a6a00640 1 -- 172.21.15.22:0/979064872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfda1532a0 con 0x55cfda137ca0 2024-04-20T21:41:37.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.128+0000 7fd3afe00640 1 --2- 172.21.15.22:0/979064872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda140880 0x55cfda131e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:41:37.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.128+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/979064872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 0x55cfda14c590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd3980ce230 tx=0x7fd3980d45e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4083ea342728b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.129+0000 7fd39fe00640 1 -- 172.21.15.22:0/979064872 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39809c430 con 0x55cfda137ca0 2024-04-20T21:41:37.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.129+0000 7fd39fe00640 1 -- 172.21.15.22:0/979064872 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3980d6070 con 0x55cfda137ca0 2024-04-20T21:41:37.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.129+0000 7fd39fe00640 1 -- 172.21.15.22:0/979064872 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3980ce450 con 0x55cfda137ca0 2024-04-20T21:41:37.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.129+0000 7fd3b9418740 1 -- 172.21.15.22:0/979064872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 msgr2=0x55cfda14c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.129+0000 7fd3b9418740 1 --2- 172.21.15.22:0/979064872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 0x55cfda14c590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd3980ce230 tx=0x7fd3980d45e0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.129+0000 7fd3b9418740 1 -- 172.21.15.22:0/979064872 shutdown_connections 2024-04-20T21:41:37.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.129+0000 7fd3b9418740 1 --2- 172.21.15.22:0/979064872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda140880 0x55cfda131e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.129+0000 7fd3b9418740 1 --2- 172.21.15.22:0/979064872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 0x55cfda14c590 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.130+0000 7fd3b9418740 1 --2- 172.21.15.22:0/979064872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda137380 0x55cfda13ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.130+0000 7fd3b9418740 1 -- 172.21.15.22:0/979064872 >> 172.21.15.22:0/979064872 conn(0x55cfda131ab0 msgr2=0x55cfda130280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.130+0000 7fd3b9418740 1 -- 172.21.15.22:0/979064872 shutdown_connections 2024-04-20T21:41:37.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.130+0000 7fd3b9418740 1 -- 172.21.15.22:0/979064872 wait complete. 2024-04-20T21:41:37.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.130+0000 7fd3b9418740 1 -- start start 2024-04-20T21:41:37.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.130+0000 7fd3b9418740 1 Processor -- start 2024-04-20T21:41:37.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3b9418740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 0x55cfda144700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3b9418740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 0x55cfda144c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3b9418740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda140880 0x55cfda145180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3b9418740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01b2f30 con 0x55cfda137380 2024-04-20T21:41:37.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3b9418740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01b43a0 con 0x55cfda140880 2024-04-20T21:41:37.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3b9418740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01b3600 con 0x55cfda137ca0 2024-04-20T21:41:37.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3b9418740 10 client.0 _ll_register_callbacks cb 0x7fd3a8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:37.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 0x55cfda144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3a6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 0x55cfda144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 0x55cfda144700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38326/0 (socket says 172.21.15.22:38326) 2024-04-20T21:41:37.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3af400640 1 -- 172.21.15.22:0/3371488843 learned_addr learned my addr 172.21.15.22:0/3371488843 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3afe00640 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda140880 0x55cfda145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 0x55cfda144c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42498/0 (socket says 172.21.15.22:42498) 2024-04-20T21:41:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3a6a00640 1 -- 172.21.15.22:0/3371488843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda140880 msgr2=0x55cfda145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda140880 0x55cfda145180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3a6a00640 1 -- 172.21.15.22:0/3371488843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 msgr2=0x55cfda144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 0x55cfda144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3a6a00640 1 -- 172.21.15.22:0/3371488843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfda137ad0 con 0x55cfda137ca0 2024-04-20T21:41:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.131+0000 7fd3af400640 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 0x55cfda144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:37.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.132+0000 7fd3a6a00640 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 0x55cfda144c40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55cfda1394d0 tx=0x7fd39809c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd2407a45c56d660 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.132+0000 7fd39d600640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:41:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd39d600640 1 -- 172.21.15.22:0/3371488843 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3980ce650 con 0x55cfda137ca0 2024-04-20T21:41:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd39d600640 1 -- 172.21.15.22:0/3371488843 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7fd3980a5200 con 0x55cfda137ca0 2024-04-20T21:41:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd3b9418740 1 -- 172.21.15.22:0/3371488843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3980d6070 con 0x55cfda137ca0 2024-04-20T21:41:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd39d600640 1 -- 172.21.15.22:0/3371488843 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd3980a5690 con 0x55cfda137ca0 2024-04-20T21:41:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd3b9418740 10 client.26512 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd3b9418740 1 -- 172.21.15.22:0/3371488843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55cfda1b7370 con 0x55cfda137ca0 2024-04-20T21:41:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd39d600640 1 -- 172.21.15.22:0/3371488843 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3980d6070 con 0x55cfda137ca0 2024-04-20T21:41:37.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd39d600640 1 -- 172.21.15.22:0/3371488843 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fd3980ba050 con 0x55cfda137ca0 2024-04-20T21:41:37.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd3b9418740 10 client.26512 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:37.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd3b9418740 10 client.26512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:37.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd3b9418740 20 client.26512 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:37.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd3b9418740 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cfda1b77d0 0x55cfda1b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd3b9418740 4 client.26512 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:41:37.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.133+0000 7fd3b9418740 1 -- 172.21.15.22:0/3371488843 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cfda0a49e0 con 0x55cfda1b77d0 2024-04-20T21:41:37.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.134+0000 7fd3af400640 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cfda1b77d0 0x55cfda1b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.134+0000 7fd3af400640 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cfda1b77d0 0x55cfda1b9bf0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c72674e73ef35c8a server_cookie=51df7623040d69b2 in_seq=0 out_seq=0 2024-04-20T21:41:37.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.134+0000 7fd39d600640 10 client.26512 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:37.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd39d600640 1 -- 172.21.15.22:0/3371488843 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cfda0a49e0 con 0x55cfda1b77d0 2024-04-20T21:41:37.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd39d600640 10 client.26512 handle_command_reply: tid=0 2024-04-20T21:41:37.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd3b9418740 1 client.26512 shutdown 2024-04-20T21:41:37.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd3b9418740 1 -- 172.21.15.22:0/3371488843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 msgr2=0x55cfda144c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd3b9418740 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 0x55cfda144c40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55cfda1394d0 tx=0x7fd39809c430 comp rx=0 tx=0).stop 2024-04-20T21:41:37.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd3b9418740 1 -- 172.21.15.22:0/3371488843 shutdown_connections 2024-04-20T21:41:37.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd3b9418740 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cfda1b77d0 0x55cfda1b9bf0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd3b9418740 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cfda140880 0x55cfda145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd39d600640 0 client.26512 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:37.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd3b9418740 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfda137ca0 0x55cfda144c40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd3b9418740 1 --2- 172.21.15.22:0/3371488843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfda137380 0x55cfda144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd3b9418740 1 -- 172.21.15.22:0/3371488843 >> 172.21.15.22:0/3371488843 conn(0x55cfda131ab0 msgr2=0x55cfda130b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd3b9418740 1 -- 172.21.15.22:0/3371488843 shutdown_connections 2024-04-20T21:41:37.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd3b9418740 1 -- 172.21.15.22:0/3371488843 wait complete. 2024-04-20T21:41:37.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.135+0000 7fd3b9418740 20 client.26512 trim_cache size 0 max 16384 2024-04-20T21:41:37.142 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:41:37.142 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:41:37.142 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:41:37.142 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:41:37.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.136+0000 7fd3b5200640 1 -- 172.21.15.22:0/1053955845 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd3940a3da0 msgr2=0x7fd3940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.136+0000 7fd3b5200640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd3940a3da0 0x7fd3940a6260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8005e30 tx=0x7fd3a8005d80 comp rx=0 tx=0).stop 2024-04-20T21:41:37.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.136+0000 7fd3b5200640 1 -- 172.21.15.22:0/1053955845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3b0108ea0 msgr2=0x7fd3b01a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.136+0000 7fd3b5200640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3b0108ea0 0x7fd3b01a4e90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd39800e9f0 tx=0x7fd39800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.136+0000 7fd3b5200640 1 -- 172.21.15.22:0/1053955845 shutdown_connections 2024-04-20T21:41:37.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.136+0000 7fd3b5200640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd3940a3da0 0x7fd3940a6260 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.136+0000 7fd3b5200640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b010d3d0 0x7fd3b019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.136+0000 7fd3b5200640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd3b0108ea0 0x7fd3b01a4e90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.136+0000 7fd3b5200640 1 --2- 172.21.15.22:0/1053955845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0108500 0x7fd3b01a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.136+0000 7fd3b5200640 1 -- 172.21.15.22:0/1053955845 >> 172.21.15.22:0/1053955845 conn(0x7fd3b0077d80 msgr2=0x7fd3b00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.136+0000 7fd3b5200640 1 -- 172.21.15.22:0/1053955845 shutdown_connections 2024-04-20T21:41:37.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.136+0000 7fd3b5200640 1 -- 172.21.15.22:0/1053955845 wait complete. 2024-04-20T21:41:37.155 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:41:37.155 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:37.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.370+0000 7f231f200640 1 -- 172.21.15.22:0/1432537403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320108da0 msgr2=0x7f2320115210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.370+0000 7f231f200640 1 --2- 172.21.15.22:0/1432537403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320108da0 0x7f2320115210 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2318009ca0 tx=0x7f231802c180 comp rx=0 tx=0).stop 2024-04-20T21:41:37.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.370+0000 7f231f200640 1 -- 172.21.15.22:0/1432537403 shutdown_connections 2024-04-20T21:41:37.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.370+0000 7f231f200640 1 --2- 172.21.15.22:0/1432537403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320108da0 0x7f2320115210 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.370+0000 7f231f200640 1 --2- 172.21.15.22:0/1432537403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2320108400 0x7f2320108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.370+0000 7f231f200640 1 --2- 172.21.15.22:0/1432537403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232010e470 0x7f232010e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.370+0000 7f231f200640 1 -- 172.21.15.22:0/1432537403 >> 172.21.15.22:0/1432537403 conn(0x7f2320077ae0 msgr2=0x7f2320079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.370+0000 7f231f200640 1 -- 172.21.15.22:0/1432537403 shutdown_connections 2024-04-20T21:41:37.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.370+0000 7f231f200640 1 -- 172.21.15.22:0/1432537403 wait complete. 2024-04-20T21:41:37.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.371+0000 7f231f200640 1 Processor -- start 2024-04-20T21:41:37.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.371+0000 7f231f200640 1 -- start start 2024-04-20T21:41:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231f200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320108400 0x7f23201a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2320108da0 0x7f23201a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f232010e470 0x7f232019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2320118750 con 0x7f232010e470 2024-04-20T21:41:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231f200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23201185d0 con 0x7f2320108400 2024-04-20T21:41:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f23201188d0 con 0x7f2320108da0 2024-04-20T21:41:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320108400 0x7f23201a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f232010e470 0x7f232019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231de00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320108400 0x7f23201a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34056/0 (socket says 172.21.15.22:34056) 2024-04-20T21:41:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231de00640 1 -- 172.21.15.22:0/894682984 learned_addr learned my addr 172.21.15.22:0/894682984 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231d400640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2320108da0 0x7f23201a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231e800640 1 -- 172.21.15.22:0/894682984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2320108da0 msgr2=0x7f23201a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231e800640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2320108da0 0x7f23201a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231e800640 1 -- 172.21.15.22:0/894682984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320108400 msgr2=0x7f23201a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231e800640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320108400 0x7f23201a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.372+0000 7f231e800640 1 -- 172.21.15.22:0/894682984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232019f190 con 0x7f232010e470 2024-04-20T21:41:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.373+0000 7f231e800640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f232010e470 0x7f232019ea10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2318009bf0 tx=0x7f2318035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.373+0000 7f2316a00640 1 -- 172.21.15.22:0/894682984 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318039040 con 0x7f232010e470 2024-04-20T21:41:37.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.373+0000 7f231f200640 1 -- 172.21.15.22:0/894682984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232019f420 con 0x7f232010e470 2024-04-20T21:41:37.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.373+0000 7f2316a00640 1 -- 172.21.15.22:0/894682984 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2318044070 con 0x7f232010e470 2024-04-20T21:41:37.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.373+0000 7f2316a00640 1 -- 172.21.15.22:0/894682984 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23180328d0 con 0x7f232010e470 2024-04-20T21:41:37.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.373+0000 7f231f200640 1 -- 172.21.15.22:0/894682984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23201ab6d0 con 0x7f232010e470 2024-04-20T21:41:37.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.375+0000 7f231f200640 1 -- 172.21.15.22:0/894682984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23200690f0 con 0x7f232010e470 2024-04-20T21:41:37.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.378+0000 7f2316a00640 1 -- 172.21.15.22:0/894682984 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2318032a70 con 0x7f232010e470 2024-04-20T21:41:37.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.379+0000 7f2316a00640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f23040a3d20 0x7f23040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.380+0000 7f2316a00640 1 -- 172.21.15.22:0/894682984 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f23180f3190 con 0x7f232010e470 2024-04-20T21:41:37.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.380+0000 7f231de00640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f23040a3d20 0x7f23040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.381+0000 7f231de00640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f23040a3d20 0x7f23040a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f232010dd60 tx=0x7f2310005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.383+0000 7f2316a00640 1 -- 172.21.15.22:0/894682984 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f231804e020 con 0x7f232010e470 2024-04-20T21:41:37.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.629+0000 7f231f200640 1 -- 172.21.15.22:0/894682984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f232019fef0 con 0x7f232010e470 2024-04-20T21:41:37.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.631+0000 7f2316a00640 1 -- 172.21.15.22:0/894682984 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f23180bf910 con 0x7f232010e470 2024-04-20T21:41:37.630 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:41:37.630 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:41:37.631 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:41:37.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.634+0000 7f231f200640 1 -- 172.21.15.22:0/894682984 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f23040a3d20 msgr2=0x7f23040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.634+0000 7f231f200640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f23040a3d20 0x7f23040a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f232010dd60 tx=0x7f2310005f90 comp rx=0 tx=0).stop 2024-04-20T21:41:37.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.634+0000 7f231f200640 1 -- 172.21.15.22:0/894682984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f232010e470 msgr2=0x7f232019ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.634+0000 7f231f200640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f232010e470 0x7f232019ea10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2318009bf0 tx=0x7f2318035d80 comp rx=0 tx=0).stop 2024-04-20T21:41:37.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.634+0000 7f231f200640 1 -- 172.21.15.22:0/894682984 shutdown_connections 2024-04-20T21:41:37.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.634+0000 7f231f200640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f23040a3d20 0x7f23040a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.634+0000 7f231f200640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f232010e470 0x7f232019ea10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.634+0000 7f231f200640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2320108da0 0x7f23201a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.634+0000 7f231f200640 1 --2- 172.21.15.22:0/894682984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320108400 0x7f23201a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.634+0000 7f231f200640 1 -- 172.21.15.22:0/894682984 >> 172.21.15.22:0/894682984 conn(0x7f2320077ae0 msgr2=0x7f2320079150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.634+0000 7f231f200640 1 -- 172.21.15.22:0/894682984 shutdown_connections 2024-04-20T21:41:37.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.634+0000 7f231f200640 1 -- 172.21.15.22:0/894682984 wait complete. 2024-04-20T21:41:37.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:41:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.877+0000 7fcafe400640 1 -- 172.21.15.22:0/2879118120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf8102fc0 msgr2=0x7fcaf81033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.877+0000 7fcafe400640 1 --2- 172.21.15.22:0/2879118120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf8102fc0 0x7fcaf81033a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009900 tx=0x7fcae002bcd0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.877+0000 7fcafe400640 1 -- 172.21.15.22:0/2879118120 shutdown_connections 2024-04-20T21:41:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.877+0000 7fcafe400640 1 --2- 172.21.15.22:0/2879118120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf810c3e0 0x7fcaf810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.877+0000 7fcafe400640 1 --2- 172.21.15.22:0/2879118120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf81038e0 0x7fcaf810bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.877+0000 7fcafe400640 1 --2- 172.21.15.22:0/2879118120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf8102fc0 0x7fcaf81033a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.877+0000 7fcafe400640 1 -- 172.21.15.22:0/2879118120 >> 172.21.15.22:0/2879118120 conn(0x7fcaf80fc7e0 msgr2=0x7fcaf80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.877+0000 7fcafe400640 1 -- 172.21.15.22:0/2879118120 shutdown_connections 2024-04-20T21:41:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.877+0000 7fcafe400640 1 -- 172.21.15.22:0/2879118120 wait complete. 2024-04-20T21:41:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.878+0000 7fcafe400640 1 Processor -- start 2024-04-20T21:41:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.878+0000 7fcafe400640 1 -- start start 2024-04-20T21:41:37.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.879+0000 7fcafe400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf8102fc0 0x7fcaf81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.879+0000 7fcafe400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf81038e0 0x7fcaf81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.879+0000 7fcafe400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf810c3e0 0x7fcaf819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.879+0000 7fcafe400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81121c0 con 0x7fcaf810c3e0 2024-04-20T21:41:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.879+0000 7fcafe400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8112040 con 0x7fcaf81038e0 2024-04-20T21:41:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.879+0000 7fcafe400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8112340 con 0x7fcaf8102fc0 2024-04-20T21:41:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.879+0000 7fcaf7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf81038e0 0x7fcaf81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.879+0000 7fcafda00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf810c3e0 0x7fcaf819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.879+0000 7fcaf7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf81038e0 0x7fcaf81a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34062/0 (socket says 172.21.15.22:34062) 2024-04-20T21:41:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.879+0000 7fcafd000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf8102fc0 0x7fcaf81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.879+0000 7fcaf7e00640 1 -- 172.21.15.22:0/882446316 learned_addr learned my addr 172.21.15.22:0/882446316 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.880+0000 7fcaf7e00640 1 -- 172.21.15.22:0/882446316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf8102fc0 msgr2=0x7fcaf81a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.880+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf8102fc0 0x7fcaf81a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.880+0000 7fcaf7e00640 1 -- 172.21.15.22:0/882446316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf810c3e0 msgr2=0x7fcaf819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.880+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf810c3e0 0x7fcaf819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.880+0000 7fcaf7e00640 1 -- 172.21.15.22:0/882446316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf819afc0 con 0x7fcaf81038e0 2024-04-20T21:41:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.880+0000 7fcafd000640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf8102fc0 0x7fcaf81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.880+0000 7fcafda00640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf810c3e0 0x7fcaf819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.880+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf81038e0 0x7fcaf81a0b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcae800e9f0 tx=0x7fcae800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.880+0000 7fcaf5600640 1 -- 172.21.15.22:0/882446316 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae800cd80 con 0x7fcaf81038e0 2024-04-20T21:41:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.880+0000 7fcafe400640 1 -- 172.21.15.22:0/882446316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf819b2b0 con 0x7fcaf81038e0 2024-04-20T21:41:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.880+0000 7fcaf5600640 1 -- 172.21.15.22:0/882446316 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae80035f0 con 0x7fcaf81038e0 2024-04-20T21:41:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.880+0000 7fcaf5600640 1 -- 172.21.15.22:0/882446316 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8010670 con 0x7fcaf81038e0 2024-04-20T21:41:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.881+0000 7fcafe400640 1 -- 172.21.15.22:0/882446316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf81a75c0 con 0x7fcaf81038e0 2024-04-20T21:41:37.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.884+0000 7fcb02764740 1 Processor -- start 2024-04-20T21:41:37.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.884+0000 7fcb02764740 1 -- start start 2024-04-20T21:41:37.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.884+0000 7fcaf5600640 1 -- 172.21.15.22:0/882446316 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fcae80037a0 con 0x7fcaf81038e0 2024-04-20T21:41:37.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.884+0000 7fcb02764740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2d8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.884+0000 7fcb02764740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2e1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.884+0000 7fcb02764740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 0x55bf1c2e3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcb02764740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bf1c23d050 con 0x55bf1c2d8a90 2024-04-20T21:41:37.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcb02764740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bf1c1e3490 con 0x55bf1c2e1670 2024-04-20T21:41:37.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcb02764740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bf1c239010 con 0x55bf1c2d8170 2024-04-20T21:41:37.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcafd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcafda00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2d8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcafd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2e1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38376/0 (socket says 172.21.15.22:38376) 2024-04-20T21:41:37.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcafd000640 1 -- 172.21.15.22:0/3017782786 learned_addr learned my addr 172.21.15.22:0/3017782786 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:37.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/3017782786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 0x55bf1c2e3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcafda00640 1 -- 172.21.15.22:0/3017782786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 msgr2=0x55bf1c2e3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcafda00640 1 --2- 172.21.15.22:0/3017782786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 0x55bf1c2e3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcafda00640 1 -- 172.21.15.22:0/3017782786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 msgr2=0x55bf1c2e1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcafda00640 1 --2- 172.21.15.22:0/3017782786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2e1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.885+0000 7fcafda00640 1 -- 172.21.15.22:0/3017782786 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf1c1eb070 con 0x55bf1c2d8170 2024-04-20T21:41:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.886+0000 7fcafda00640 1 --2- 172.21.15.22:0/3017782786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2d8550 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0005fd0 tx=0x7fcaf0011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c839f0da8702cc88 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.886+0000 7fcaee000640 1 -- 172.21.15.22:0/3017782786 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0004d20 con 0x55bf1c2d8170 2024-04-20T21:41:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.886+0000 7fcaf5600640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fcadc0a3da0 0x7fcadc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.886+0000 7fcaee000640 1 -- 172.21.15.22:0/3017782786 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaf000f9a0 con 0x55bf1c2d8170 2024-04-20T21:41:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.886+0000 7fcaee000640 1 -- 172.21.15.22:0/3017782786 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0004510 con 0x55bf1c2d8170 2024-04-20T21:41:37.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.886+0000 7fcaf5600640 1 -- 172.21.15.22:0/882446316 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7fcae80d0ac0 con 0x7fcaf81038e0 2024-04-20T21:41:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.886+0000 7fcafd000640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fcadc0a3da0 0x7fcadc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.887+0000 7fcb02764740 1 -- 172.21.15.22:0/3017782786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 msgr2=0x55bf1c2d8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.887+0000 7fcb02764740 1 --2- 172.21.15.22:0/3017782786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2d8550 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0005fd0 tx=0x7fcaf0011bf0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.887+0000 7fcb02764740 1 -- 172.21.15.22:0/3017782786 shutdown_connections 2024-04-20T21:41:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.887+0000 7fcb02764740 1 --2- 172.21.15.22:0/3017782786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 0x55bf1c2e3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.887+0000 7fcb02764740 1 --2- 172.21.15.22:0/3017782786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2e1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.887+0000 7fcb02764740 1 --2- 172.21.15.22:0/3017782786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2d8550 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.887+0000 7fcb02764740 1 -- 172.21.15.22:0/3017782786 >> 172.21.15.22:0/3017782786 conn(0x55bf1c2d28a0 msgr2=0x55bf1c2d2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.887+0000 7fcb02764740 1 -- 172.21.15.22:0/3017782786 shutdown_connections 2024-04-20T21:41:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.887+0000 7fcb02764740 1 -- 172.21.15.22:0/3017782786 wait complete. 2024-04-20T21:41:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.887+0000 7fcafd000640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fcadc0a3da0 0x7fcadc0a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcae0006010 tx=0x7fcae002cf90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.887+0000 7fcb02764740 1 -- start start 2024-04-20T21:41:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.888+0000 7fcb02764740 1 Processor -- start 2024-04-20T21:41:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.888+0000 7fcb02764740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2de680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.888+0000 7fcb02764740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2debc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.888+0000 7fcb02764740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2e1670 0x55bf1c2df100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.888+0000 7fcb02764740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bf1c2e4460 con 0x55bf1c2e1670 2024-04-20T21:41:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.888+0000 7fcb02764740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bf1c1f6860 con 0x55bf1c2d8a90 2024-04-20T21:41:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.888+0000 7fcb02764740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bf1c2e45e0 con 0x55bf1c2d8170 2024-04-20T21:41:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.888+0000 7fcb02764740 10 client.0 _ll_register_callbacks cb 0x55bf1c1c4fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.888+0000 7fcafd000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.889+0000 7fcafda00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2e1670 0x55bf1c2df100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.889+0000 7fcafd000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2de680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42524/0 (socket says 172.21.15.22:42524) 2024-04-20T21:41:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.889+0000 7fcafd000640 1 -- 172.21.15.22:0/3906153282 learned_addr learned my addr 172.21.15.22:0/3906153282 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.889+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/3906153282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2debc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.889+0000 7fcafd000640 1 -- 172.21.15.22:0/3906153282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2d8a90 msgr2=0x55bf1c2debc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.889+0000 7fcafd000640 1 --2- 172.21.15.22:0/3906153282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2debc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.889+0000 7fcafd000640 1 -- 172.21.15.22:0/3906153282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2e1670 msgr2=0x55bf1c2df100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.889+0000 7fcafd000640 1 --2- 172.21.15.22:0/3906153282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2e1670 0x55bf1c2df100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.889+0000 7fcafd000640 1 -- 172.21.15.22:0/3906153282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf1c1fa950 con 0x55bf1c2d8170 2024-04-20T21:41:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.889+0000 7fcafd000640 1 --2- 172.21.15.22:0/3906153282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2de680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcae0035000 tx=0x7fcae0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeb38440dc9544c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.889+0000 7fcae6a00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:41:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.890+0000 7fcae6a00640 1 -- 172.21.15.22:0/3906153282 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0005340 con 0x55bf1c2d8170 2024-04-20T21:41:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.890+0000 7fcb02764740 1 -- 172.21.15.22:0/3906153282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaf000f9a0 con 0x55bf1c2d8170 2024-04-20T21:41:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.890+0000 7fcae6a00640 1 -- 172.21.15.22:0/3906153282 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7fcae00054e0 con 0x55bf1c2d8170 2024-04-20T21:41:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.890+0000 7fcae6a00640 1 -- 172.21.15.22:0/3906153282 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcae00035f0 con 0x55bf1c2d8170 2024-04-20T21:41:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.890+0000 7fcb02764740 10 client.26530 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.890+0000 7fcb02764740 1 -- 172.21.15.22:0/3906153282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bf1c1c8820 con 0x55bf1c2d8170 2024-04-20T21:41:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.890+0000 7fcae6a00640 1 -- 172.21.15.22:0/3906153282 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaf000f9a0 con 0x55bf1c2d8170 2024-04-20T21:41:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.890+0000 7fcae6a00640 1 -- 172.21.15.22:0/3906153282 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fcae0051020 con 0x55bf1c2d8170 2024-04-20T21:41:37.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.890+0000 7fcb02764740 10 client.26530 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.890+0000 7fcb02764740 10 client.26530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.891+0000 7fcb02764740 20 client.26530 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.891+0000 7fcb02764740 1 --2- 172.21.15.22:0/3906153282 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55bf1c2f1640 0x55bf1c2f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.891+0000 7fcb02764740 4 client.26530 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.891+0000 7fcb02764740 1 -- 172.21.15.22:0/3906153282 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf1c24aa70 con 0x55bf1c2f1640 2024-04-20T21:41:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.891+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/3906153282 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55bf1c2f1640 0x55bf1c2f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.891+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/3906153282 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55bf1c2f1640 0x55bf1c2f3ac0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb1da3dcc16e4202 server_cookie=5c1c16007f6202b9 in_seq=0 out_seq=0 2024-04-20T21:41:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.891+0000 7fcae6a00640 10 client.26530 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.892+0000 7fcae6a00640 1 -- 172.21.15.22:0/3906153282 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf1c24aa70 con 0x55bf1c2f1640 2024-04-20T21:41:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.892+0000 7fcae6a00640 10 client.26530 handle_command_reply: tid=0 2024-04-20T21:41:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.892+0000 7fcb02764740 1 client.26530 shutdown 2024-04-20T21:41:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.893+0000 7fcb02764740 1 -- 172.21.15.22:0/3906153282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 msgr2=0x55bf1c2de680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.893+0000 7fcb02764740 1 --2- 172.21.15.22:0/3906153282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2de680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcae0035000 tx=0x7fcae0009260 comp rx=0 tx=0).stop 2024-04-20T21:41:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.893+0000 7fcb02764740 1 -- 172.21.15.22:0/3906153282 shutdown_connections 2024-04-20T21:41:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.893+0000 7fcb02764740 1 --2- 172.21.15.22:0/3906153282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2e1670 0x55bf1c2df100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.893+0000 7fcb02764740 1 --2- 172.21.15.22:0/3906153282 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55bf1c2f1640 0x55bf1c2f3ac0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.893+0000 7fcb02764740 1 --2- 172.21.15.22:0/3906153282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2debc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.893+0000 7fcb02764740 1 --2- 172.21.15.22:0/3906153282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2de680 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.893+0000 7fcb02764740 1 -- 172.21.15.22:0/3906153282 >> 172.21.15.22:0/3906153282 conn(0x55bf1c2d28a0 msgr2=0x55bf1c2d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.893+0000 7fcae6a00640 0 client.26530 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.893+0000 7fcb02764740 1 -- 172.21.15.22:0/3906153282 shutdown_connections 2024-04-20T21:41:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.893+0000 7fcb02764740 1 -- 172.21.15.22:0/3906153282 wait complete. 2024-04-20T21:41:37.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.893+0000 7fcb02764740 20 client.26530 trim_cache size 0 max 16384 2024-04-20T21:41:37.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.908+0000 7fcb02764740 1 Processor -- start 2024-04-20T21:41:37.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.908+0000 7fcb02764740 1 -- start start 2024-04-20T21:41:37.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcb02764740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8170 0x55bf1c2e0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcb02764740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2ed380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcb02764740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 0x55bf1c2f9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcb02764740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81add10 con 0x55bf1c2d8170 2024-04-20T21:41:37.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcb02764740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81b0570 con 0x55bf1c2e1670 2024-04-20T21:41:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcb02764740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81ae360 con 0x55bf1c2d8a90 2024-04-20T21:41:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcafd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8170 0x55bf1c2e0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcafd000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8170 0x55bf1c2e0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38390/0 (socket says 172.21.15.22:38390) 2024-04-20T21:41:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcafd000640 1 -- 172.21.15.22:0/4079272384 learned_addr learned my addr 172.21.15.22:0/4079272384 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/4079272384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2ed380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcafda00640 1 --2- 172.21.15.22:0/4079272384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 0x55bf1c2f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcaf7e00640 1 -- 172.21.15.22:0/4079272384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 msgr2=0x55bf1c2f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/4079272384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 0x55bf1c2f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcaf7e00640 1 -- 172.21.15.22:0/4079272384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8170 msgr2=0x55bf1c2e0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/4079272384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8170 0x55bf1c2e0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcaf7e00640 1 -- 172.21.15.22:0/4079272384 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf1c2d88c0 con 0x55bf1c2d8a90 2024-04-20T21:41:37.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.909+0000 7fcafd000640 1 --2- 172.21.15.22:0/4079272384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8170 0x55bf1c2e0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:37.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.910+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/4079272384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2ed380 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcae80ce160 tx=0x7fcae80bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f773df99d3040ca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.910+0000 7fcaed600640 1 -- 172.21.15.22:0/4079272384 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae809d0b0 con 0x55bf1c2d8a90 2024-04-20T21:41:37.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.910+0000 7fcaed600640 1 -- 172.21.15.22:0/4079272384 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae809c9b0 con 0x55bf1c2d8a90 2024-04-20T21:41:37.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.910+0000 7fcaed600640 1 -- 172.21.15.22:0/4079272384 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8099020 con 0x55bf1c2d8a90 2024-04-20T21:41:37.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.911+0000 7fcb02764740 1 -- 172.21.15.22:0/4079272384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8a90 msgr2=0x55bf1c2ed380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.911+0000 7fcb02764740 1 --2- 172.21.15.22:0/4079272384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2ed380 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcae80ce160 tx=0x7fcae80bd040 comp rx=0 tx=0).stop 2024-04-20T21:41:37.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.911+0000 7fcb02764740 1 -- 172.21.15.22:0/4079272384 shutdown_connections 2024-04-20T21:41:37.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.911+0000 7fcb02764740 1 --2- 172.21.15.22:0/4079272384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 0x55bf1c2f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.911+0000 7fcb02764740 1 --2- 172.21.15.22:0/4079272384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2ed380 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.911+0000 7fcb02764740 1 --2- 172.21.15.22:0/4079272384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8170 0x55bf1c2e0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.911+0000 7fcb02764740 1 -- 172.21.15.22:0/4079272384 >> 172.21.15.22:0/4079272384 conn(0x55bf1c2d28a0 msgr2=0x55bf1c2d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.911+0000 7fcb02764740 1 -- 172.21.15.22:0/4079272384 shutdown_connections 2024-04-20T21:41:37.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.911+0000 7fcb02764740 1 -- 172.21.15.22:0/4079272384 wait complete. 2024-04-20T21:41:37.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.911+0000 7fcb02764740 1 -- start start 2024-04-20T21:41:37.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.911+0000 7fcb02764740 1 Processor -- start 2024-04-20T21:41:37.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.912+0000 7fcb02764740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2e5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.912+0000 7fcb02764740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2e5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.912+0000 7fcb02764740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 0x55bf1c2e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.912+0000 7fcb02764740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81aea30 con 0x55bf1c2d8a90 2024-04-20T21:41:37.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.912+0000 7fcb02764740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81afea0 con 0x55bf1c2e1670 2024-04-20T21:41:37.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.912+0000 7fcb02764740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81af100 con 0x55bf1c2d8170 2024-04-20T21:41:37.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.912+0000 7fcb02764740 10 client.0 _ll_register_callbacks cb 0x7fcae002c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:37.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.912+0000 7fcaf7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.912+0000 7fcaf7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2e5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38406/0 (socket says 172.21.15.22:38406) 2024-04-20T21:41:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.912+0000 7fcaf7e00640 1 -- 172.21.15.22:0/3794185576 learned_addr learned my addr 172.21.15.22:0/3794185576 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.912+0000 7fcafd000640 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.912+0000 7fcafda00640 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 0x55bf1c2e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcaf7e00640 1 -- 172.21.15.22:0/3794185576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 msgr2=0x55bf1c2e5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2e5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcaf7e00640 1 -- 172.21.15.22:0/3794185576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 msgr2=0x55bf1c2e5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:37.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 0x55bf1c2e5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcaf7e00640 1 -- 172.21.15.22:0/3794185576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf1bf4f3f0 con 0x55bf1c2d8a90 2024-04-20T21:41:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcaf7e00640 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2e5940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcae809c6b0 tx=0x7fcae80bd120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b84c742c66c29e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcae6a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:41:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcae6a00640 1 -- 172.21.15.22:0/3794185576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80a0d60 con 0x55bf1c2d8a90 2024-04-20T21:41:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcae6a00640 1 -- 172.21.15.22:0/3794185576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7fcae8099020 con 0x55bf1c2d8a90 2024-04-20T21:41:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcb02764740 1 -- 172.21.15.22:0/3794185576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae809c9b0 con 0x55bf1c2d8a90 2024-04-20T21:41:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcae6a00640 1 -- 172.21.15.22:0/3794185576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcae80ce3e0 con 0x55bf1c2d8a90 2024-04-20T21:41:37.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcb02764740 10 client.16362 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.913+0000 7fcb02764740 1 -- 172.21.15.22:0/3794185576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bf1c0197d0 con 0x55bf1c2d8a90 2024-04-20T21:41:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.914+0000 7fcae6a00640 1 -- 172.21.15.22:0/3794185576 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae80b8080 con 0x55bf1c2d8a90 2024-04-20T21:41:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.914+0000 7fcae6a00640 1 -- 172.21.15.22:0/3794185576 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fcae809dce0 con 0x55bf1c2d8a90 2024-04-20T21:41:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.914+0000 7fcb02764740 10 client.16362 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.914+0000 7fcb02764740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.914+0000 7fcb02764740 20 client.16362 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.914+0000 7fcb02764740 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55bf1c3609f0 0x55bf1c362de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.914+0000 7fcb02764740 4 client.16362 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:41:37.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.914+0000 7fcb02764740 1 -- 172.21.15.22:0/3794185576 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf1c2448e0 con 0x55bf1c3609f0 2024-04-20T21:41:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.914+0000 7fcafd000640 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55bf1c3609f0 0x55bf1c362de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcafd000640 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55bf1c3609f0 0x55bf1c362de0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e70061b3f834e54 server_cookie=e3fb132361622d1a in_seq=0 out_seq=0 2024-04-20T21:41:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcae6a00640 10 client.16362 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcae6a00640 1 -- 172.21.15.22:0/3794185576 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf1c2448e0 con 0x55bf1c3609f0 2024-04-20T21:41:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcae6a00640 10 client.16362 handle_command_reply: tid=0 2024-04-20T21:41:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcb02764740 1 client.16362 shutdown 2024-04-20T21:41:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcb02764740 1 -- 172.21.15.22:0/3794185576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 msgr2=0x55bf1c2e5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcb02764740 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2e5940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcae809c6b0 tx=0x7fcae80bd120 comp rx=0 tx=0).stop 2024-04-20T21:41:37.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcb02764740 1 -- 172.21.15.22:0/3794185576 shutdown_connections 2024-04-20T21:41:37.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcb02764740 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55bf1c3609f0 0x55bf1c362de0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcb02764740 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1c2e1670 0x55bf1c2e5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcb02764740 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bf1c2d8a90 0x55bf1c2e5940 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcae6a00640 0 client.16362 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:37.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.915+0000 7fcb02764740 1 --2- 172.21.15.22:0/3794185576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf1c2d8170 0x55bf1c2e5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.916+0000 7fcb02764740 1 -- 172.21.15.22:0/3794185576 >> 172.21.15.22:0/3794185576 conn(0x55bf1c2d28a0 msgr2=0x55bf1c2d10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.916+0000 7fcb02764740 1 -- 172.21.15.22:0/3794185576 shutdown_connections 2024-04-20T21:41:37.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.916+0000 7fcb02764740 1 -- 172.21.15.22:0/3794185576 wait complete. 2024-04-20T21:41:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.916+0000 7fcb02764740 20 client.16362 trim_cache size 0 max 16384 2024-04-20T21:41:37.923 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:41:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.916+0000 7fcafe400640 1 -- 172.21.15.22:0/882446316 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fcadc0a3da0 msgr2=0x7fcadc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.916+0000 7fcafe400640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fcadc0a3da0 0x7fcadc0a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcae0006010 tx=0x7fcae002cf90 comp rx=0 tx=0).stop 2024-04-20T21:41:37.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.916+0000 7fcafe400640 1 -- 172.21.15.22:0/882446316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf81038e0 msgr2=0x7fcaf81a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:37.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.916+0000 7fcafe400640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf81038e0 0x7fcaf81a0b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcae800e9f0 tx=0x7fcae800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.916+0000 7fcafe400640 1 -- 172.21.15.22:0/882446316 shutdown_connections 2024-04-20T21:41:37.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.917+0000 7fcafe400640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fcadc0a3da0 0x7fcadc0a6260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.917+0000 7fcafe400640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaf810c3e0 0x7fcaf819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.917+0000 7fcafe400640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf81038e0 0x7fcaf81a0b80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.917+0000 7fcafe400640 1 --2- 172.21.15.22:0/882446316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf8102fc0 0x7fcaf81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.917+0000 7fcafe400640 1 -- 172.21.15.22:0/882446316 >> 172.21.15.22:0/882446316 conn(0x7fcaf80fc7e0 msgr2=0x7fcaf8101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.917+0000 7fcafe400640 1 -- 172.21.15.22:0/882446316 shutdown_connections 2024-04-20T21:41:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:37.917+0000 7fcafe400640 1 -- 172.21.15.22:0/882446316 wait complete. 2024-04-20T21:41:38.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:38 smithi022 ceph-mon[16843]: pgmap v241: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:38.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:38 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/894682984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:41:38.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:38 smithi122 ceph-mon[22147]: pgmap v241: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:38.639 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:38 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/894682984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:41:38.640 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:38 smithi033 ceph-mon[21626]: pgmap v241: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:38.640 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:38 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/894682984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:41:38.935 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:41:38.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:41:38.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:41:39.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.164+0000 7f4831200640 1 -- 172.21.15.22:0/3532736211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f482c1016d0 msgr2=0x7f482c105b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.164+0000 7f4831200640 1 --2- 172.21.15.22:0/3532736211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f482c1016d0 0x7f482c105b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4818009ca0 tx=0x7f481802c1d0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.164+0000 7f4831200640 1 -- 172.21.15.22:0/3532736211 shutdown_connections 2024-04-20T21:41:39.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.164+0000 7f4831200640 1 --2- 172.21.15.22:0/3532736211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f482c1061b0 0x7f482c106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.164+0000 7f4831200640 1 --2- 172.21.15.22:0/3532736211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f482c1016d0 0x7f482c105b90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.164+0000 7f4831200640 1 --2- 172.21.15.22:0/3532736211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c100db0 0x7f482c101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.164+0000 7f4831200640 1 -- 172.21.15.22:0/3532736211 >> 172.21.15.22:0/3532736211 conn(0x7f482c0fc7e0 msgr2=0x7f482c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.165+0000 7f4831200640 1 -- 172.21.15.22:0/3532736211 shutdown_connections 2024-04-20T21:41:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.165+0000 7f4831200640 1 -- 172.21.15.22:0/3532736211 wait complete. 2024-04-20T21:41:39.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.165+0000 7f4831200640 1 Processor -- start 2024-04-20T21:41:39.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.166+0000 7f4831200640 1 -- start start 2024-04-20T21:41:39.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.166+0000 7f4831200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f482c100db0 0x7f482c1a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.166+0000 7f4831200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c1016d0 0x7f482c1a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.166+0000 7f4831200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f482c1061b0 0x7f482c19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.166+0000 7f4831200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1187d0 con 0x7f482c100db0 2024-04-20T21:41:39.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.166+0000 7f4831200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f482c118650 con 0x7f482c1016d0 2024-04-20T21:41:39.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.166+0000 7f4831200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f482c118950 con 0x7f482c1061b0 2024-04-20T21:41:39.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.166+0000 7f482b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f482c100db0 0x7f482c1a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.166+0000 7f482b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f482c100db0 0x7f482c1a4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38416/0 (socket says 172.21.15.22:38416) 2024-04-20T21:41:39.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.166+0000 7f482b400640 1 -- 172.21.15.22:0/3502668984 learned_addr learned my addr 172.21.15.22:0/3502668984 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:39.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.166+0000 7f482aa00640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c1016d0 0x7f482c1a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.167+0000 7f482be00640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f482c1061b0 0x7f482c19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.167+0000 7f482b400640 1 -- 172.21.15.22:0/3502668984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f482c1061b0 msgr2=0x7f482c19eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.167+0000 7f482b400640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f482c1061b0 0x7f482c19eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.167+0000 7f482b400640 1 -- 172.21.15.22:0/3502668984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c1016d0 msgr2=0x7f482c1a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.167+0000 7f482b400640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c1016d0 0x7f482c1a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.167+0000 7f482b400640 1 -- 172.21.15.22:0/3502668984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482c19f3f0 con 0x7f482c100db0 2024-04-20T21:41:39.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.167+0000 7f482b400640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f482c100db0 0x7f482c1a4a70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f481c00e9f0 tx=0x7f481c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.167+0000 7f4823e00640 1 -- 172.21.15.22:0/3502668984 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c00cd80 con 0x7f482c100db0 2024-04-20T21:41:39.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.167+0000 7f4823e00640 1 -- 172.21.15.22:0/3502668984 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f481c010910 con 0x7f482c100db0 2024-04-20T21:41:39.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.167+0000 7f4823e00640 1 -- 172.21.15.22:0/3502668984 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c018990 con 0x7f482c100db0 2024-04-20T21:41:39.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.168+0000 7f4831200640 1 -- 172.21.15.22:0/3502668984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482c19f6e0 con 0x7f482c100db0 2024-04-20T21:41:39.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.168+0000 7f4831200640 1 -- 172.21.15.22:0/3502668984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f482c1ab9f0 con 0x7f482c100db0 2024-04-20T21:41:39.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.171+0000 7f48355c9740 1 Processor -- start 2024-04-20T21:41:39.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.171+0000 7f4823e00640 1 -- 172.21.15.22:0/3502668984 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f481c010ac0 con 0x7f482c100db0 2024-04-20T21:41:39.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.171+0000 7f48355c9740 1 -- start start 2024-04-20T21:41:39.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.171+0000 7f48355c9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.171+0000 7f48355c9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 0x55d6b28aa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.171+0000 7f48355c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28aa5f0 0x55d6b28ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.171+0000 7f48355c9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d6b2805d20 con 0x55d6b28a10f0 2024-04-20T21:41:39.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.171+0000 7f48355c9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d6b27ac3e0 con 0x55d6b28a1a10 2024-04-20T21:41:39.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.171+0000 7f48355c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d6b2801ce0 con 0x55d6b28aa5f0 2024-04-20T21:41:39.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.172+0000 7f482be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.172+0000 7f482be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38420/0 (socket says 172.21.15.22:38420) 2024-04-20T21:41:39.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.172+0000 7f482be00640 1 -- 172.21.15.22:0/169497647 learned_addr learned my addr 172.21.15.22:0/169497647 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:39.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.172+0000 7f482b400640 1 --2- 172.21.15.22:0/169497647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 0x55d6b28aa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.172+0000 7f482aa00640 1 --2- 172.21.15.22:0/169497647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28aa5f0 0x55d6b28ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.172+0000 7f482be00640 1 -- 172.21.15.22:0/169497647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28aa5f0 msgr2=0x55d6b28ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.172+0000 7f482be00640 1 --2- 172.21.15.22:0/169497647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28aa5f0 0x55d6b28ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.172+0000 7f482be00640 1 -- 172.21.15.22:0/169497647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 msgr2=0x55d6b28aa0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.172+0000 7f482be00640 1 --2- 172.21.15.22:0/169497647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 0x55d6b28aa0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.172+0000 7f482be00640 1 -- 172.21.15.22:0/169497647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6b27cb980 con 0x55d6b28a10f0 2024-04-20T21:41:39.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.172+0000 7f482b400640 1 --2- 172.21.15.22:0/169497647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 0x55d6b28aa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:39.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.173+0000 7f4823e00640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f48100a3da0 0x7f48100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.173+0000 7f482be00640 1 --2- 172.21.15.22:0/169497647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a14d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4824009c20 tx=0x7f4824011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c80accc95b7542 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.173+0000 7f4823e00640 1 -- 172.21.15.22:0/3502668984 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f481c0d0ad0 con 0x7f482c100db0 2024-04-20T21:41:39.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.173+0000 7f482b400640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f48100a3da0 0x7f48100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.173+0000 7f4820c00640 1 -- 172.21.15.22:0/169497647 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482401e070 con 0x55d6b28a10f0 2024-04-20T21:41:39.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.173+0000 7f4820c00640 1 -- 172.21.15.22:0/169497647 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4824019b10 con 0x55d6b28a10f0 2024-04-20T21:41:39.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.173+0000 7f4820c00640 1 -- 172.21.15.22:0/169497647 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482400eae0 con 0x55d6b28a10f0 2024-04-20T21:41:39.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.174+0000 7f48355c9740 1 -- 172.21.15.22:0/169497647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 msgr2=0x55d6b28a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.174+0000 7f482b400640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f48100a3da0 0x7f48100a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f481c0ce280 tx=0x7f481c0d4610 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.174+0000 7f48355c9740 1 --2- 172.21.15.22:0/169497647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a14d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4824009c20 tx=0x7f4824011810 comp rx=0 tx=0).stop 2024-04-20T21:41:39.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.174+0000 7f48355c9740 1 -- 172.21.15.22:0/169497647 shutdown_connections 2024-04-20T21:41:39.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.174+0000 7f48355c9740 1 --2- 172.21.15.22:0/169497647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28aa5f0 0x55d6b28ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.174+0000 7f48355c9740 1 --2- 172.21.15.22:0/169497647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 0x55d6b28aa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.174+0000 7f48355c9740 1 --2- 172.21.15.22:0/169497647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a14d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.174+0000 7f48355c9740 1 -- 172.21.15.22:0/169497647 >> 172.21.15.22:0/169497647 conn(0x55d6b289b820 msgr2=0x55d6b289bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.174+0000 7f48355c9740 1 -- 172.21.15.22:0/169497647 shutdown_connections 2024-04-20T21:41:39.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.174+0000 7f48355c9740 1 -- 172.21.15.22:0/169497647 wait complete. 2024-04-20T21:41:39.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.175+0000 7f48355c9740 1 -- start start 2024-04-20T21:41:39.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.175+0000 7f48355c9740 1 Processor -- start 2024-04-20T21:41:39.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.175+0000 7f48355c9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.175+0000 7f48355c9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 0x55d6b28a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.175+0000 7f48355c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28aa5f0 0x55d6b28a80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.175+0000 7f48355c9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d6b28ad3e0 con 0x55d6b28a10f0 2024-04-20T21:41:39.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.175+0000 7f48355c9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d6b27bf7a0 con 0x55d6b28a1a10 2024-04-20T21:41:39.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.175+0000 7f48355c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d6b28ad560 con 0x55d6b28aa5f0 2024-04-20T21:41:39.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.175+0000 7f48355c9740 10 client.0 _ll_register_callbacks cb 0x55d6b27c1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:39.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482aa00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 0x55d6b28a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482aa00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 0x55d6b28a7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34116/0 (socket says 172.21.15.22:34116) 2024-04-20T21:41:39.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38426/0 (socket says 172.21.15.22:38426) 2024-04-20T21:41:39.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482aa00640 1 -- 172.21.15.22:0/1110571961 learned_addr learned my addr 172.21.15.22:0/1110571961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:39.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482be00640 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28aa5f0 0x55d6b28a80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482aa00640 1 -- 172.21.15.22:0/1110571961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28aa5f0 msgr2=0x55d6b28a80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482aa00640 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28aa5f0 0x55d6b28a80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482aa00640 1 -- 172.21.15.22:0/1110571961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 msgr2=0x55d6b28a7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482aa00640 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482aa00640 1 -- 172.21.15.22:0/1110571961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6b25e07e0 con 0x55d6b28a1a10 2024-04-20T21:41:39.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482be00640 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28aa5f0 0x55d6b28a80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:41:39.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f482aa00640 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 0x55d6b28a7b80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f481802c710 tx=0x7f4818003e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b016dc6a9968f819 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.176+0000 7f4815600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:41:39.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.177+0000 7f4815600640 1 -- 172.21.15.22:0/1110571961 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818044070 con 0x55d6b28a1a10 2024-04-20T21:41:39.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.177+0000 7f4815600640 1 -- 172.21.15.22:0/1110571961 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f4818004240 con 0x55d6b28a1a10 2024-04-20T21:41:39.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.177+0000 7f48355c9740 1 -- 172.21.15.22:0/1110571961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4824019b10 con 0x55d6b28a1a10 2024-04-20T21:41:39.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.177+0000 7f4815600640 1 -- 172.21.15.22:0/1110571961 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4818039040 con 0x55d6b28a1a10 2024-04-20T21:41:39.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.177+0000 7f48355c9740 10 client.26522 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:39.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.177+0000 7f48355c9740 1 -- 172.21.15.22:0/1110571961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d6b2640f70 con 0x55d6b28a1a10 2024-04-20T21:41:39.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.177+0000 7f4815600640 1 -- 172.21.15.22:0/1110571961 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4824019b10 con 0x55d6b28a1a10 2024-04-20T21:41:39.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.178+0000 7f4815600640 1 -- 172.21.15.22:0/1110571961 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f4818034440 con 0x55d6b28a1a10 2024-04-20T21:41:39.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.178+0000 7f48355c9740 10 client.26522 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:39.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.178+0000 7f48355c9740 10 client.26522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:39.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.178+0000 7f48355c9740 20 client.26522 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:39.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.178+0000 7f48355c9740 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d6b28ba990 0x55d6b28bcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.178+0000 7f48355c9740 4 client.26522 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:39.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.178+0000 7f48355c9740 1 -- 172.21.15.22:0/1110571961 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6b2813740 con 0x55d6b28ba990 2024-04-20T21:41:39.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.178+0000 7f482be00640 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d6b28ba990 0x55d6b28bcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.179+0000 7f482be00640 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d6b28ba990 0x55d6b28bcd80 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d89fed66f4b9c377 server_cookie=fee9667d9eedf209 in_seq=0 out_seq=0 2024-04-20T21:41:39.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.179+0000 7f4815600640 10 client.26522 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:39.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f4815600640 1 -- 172.21.15.22:0/1110571961 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6b2813740 con 0x55d6b28ba990 2024-04-20T21:41:39.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f4815600640 10 client.26522 handle_command_reply: tid=0 2024-04-20T21:41:39.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f48355c9740 1 client.26522 shutdown 2024-04-20T21:41:39.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f48355c9740 1 -- 172.21.15.22:0/1110571961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 msgr2=0x55d6b28a7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f48355c9740 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 0x55d6b28a7b80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f481802c710 tx=0x7f4818003e50 comp rx=0 tx=0).stop 2024-04-20T21:41:39.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f48355c9740 1 -- 172.21.15.22:0/1110571961 shutdown_connections 2024-04-20T21:41:39.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f48355c9740 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28aa5f0 0x55d6b28a80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f48355c9740 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d6b28ba990 0x55d6b28bcd80 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f48355c9740 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a1a10 0x55d6b28a7b80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f48355c9740 1 --2- 172.21.15.22:0/1110571961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f4815600640 0 client.26522 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:39.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f48355c9740 1 -- 172.21.15.22:0/1110571961 >> 172.21.15.22:0/1110571961 conn(0x55d6b289b820 msgr2=0x55d6b289d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f48355c9740 1 -- 172.21.15.22:0/1110571961 shutdown_connections 2024-04-20T21:41:39.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.180+0000 7f48355c9740 1 -- 172.21.15.22:0/1110571961 wait complete. 2024-04-20T21:41:39.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.181+0000 7f48355c9740 20 client.26522 trim_cache size 0 max 16384 2024-04-20T21:41:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.200+0000 7f48355c9740 1 Processor -- start 2024-04-20T21:41:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.200+0000 7f48355c9740 1 -- start start 2024-04-20T21:41:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.200+0000 7f48355c9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.200+0000 7f48355c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 0x55d6b28b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.200+0000 7f48355c9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28aa5f0 0x55d6b289a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.200+0000 7f48355c9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1b22d0 con 0x55d6b28a10f0 2024-04-20T21:41:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.200+0000 7f48355c9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1b4b30 con 0x55d6b28aa5f0 2024-04-20T21:41:39.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.200+0000 7f48355c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1b2920 con 0x55d6b28a1a10 2024-04-20T21:41:39.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.201+0000 7f482be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.201+0000 7f482aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 0x55d6b28b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.201+0000 7f482be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38436/0 (socket says 172.21.15.22:38436) 2024-04-20T21:41:39.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.201+0000 7f482be00640 1 -- 172.21.15.22:0/3985990901 learned_addr learned my addr 172.21.15.22:0/3985990901 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:39.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.201+0000 7f482aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 0x55d6b28b62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42578/0 (socket says 172.21.15.22:42578) 2024-04-20T21:41:39.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.201+0000 7f482be00640 1 --2- 172.21.15.22:0/3985990901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28aa5f0 0x55d6b289a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.201+0000 7f482be00640 1 -- 172.21.15.22:0/3985990901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 msgr2=0x55d6b28b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.201+0000 7f482be00640 1 --2- 172.21.15.22:0/3985990901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 0x55d6b28b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.201+0000 7f482be00640 1 -- 172.21.15.22:0/3985990901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28aa5f0 msgr2=0x55d6b289a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.201+0000 7f482be00640 1 --2- 172.21.15.22:0/3985990901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28aa5f0 0x55d6b289a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.201+0000 7f482be00640 1 -- 172.21.15.22:0/3985990901 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6b24d8170 con 0x55d6b28a10f0 2024-04-20T21:41:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.201+0000 7f482aa00640 1 --2- 172.21.15.22:0/3985990901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 0x55d6b28b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.202+0000 7f482be00640 1 --2- 172.21.15.22:0/3985990901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a9d70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f48240098a0 tx=0x7f48240099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e265633dd31ae2a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.202+0000 7f4817e00640 1 -- 172.21.15.22:0/3985990901 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482401e040 con 0x55d6b28a10f0 2024-04-20T21:41:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.202+0000 7f4817e00640 1 -- 172.21.15.22:0/3985990901 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4824009dd0 con 0x55d6b28a10f0 2024-04-20T21:41:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.202+0000 7f4817e00640 1 -- 172.21.15.22:0/3985990901 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482400e900 con 0x55d6b28a10f0 2024-04-20T21:41:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.203+0000 7f48355c9740 1 -- 172.21.15.22:0/3985990901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 msgr2=0x55d6b28a9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.203+0000 7f48355c9740 1 --2- 172.21.15.22:0/3985990901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a9d70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f48240098a0 tx=0x7f48240099f0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.203+0000 7f48355c9740 1 -- 172.21.15.22:0/3985990901 shutdown_connections 2024-04-20T21:41:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.203+0000 7f48355c9740 1 --2- 172.21.15.22:0/3985990901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28aa5f0 0x55d6b289a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.203+0000 7f48355c9740 1 --2- 172.21.15.22:0/3985990901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 0x55d6b28b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.203+0000 7f48355c9740 1 --2- 172.21.15.22:0/3985990901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28a10f0 0x55d6b28a9d70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.203+0000 7f48355c9740 1 -- 172.21.15.22:0/3985990901 >> 172.21.15.22:0/3985990901 conn(0x55d6b289b820 msgr2=0x55d6b289a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.203+0000 7f48355c9740 1 -- 172.21.15.22:0/3985990901 shutdown_connections 2024-04-20T21:41:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.203+0000 7f48355c9740 1 -- 172.21.15.22:0/3985990901 wait complete. 2024-04-20T21:41:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.203+0000 7f48355c9740 1 -- start start 2024-04-20T21:41:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.204+0000 7f48355c9740 1 Processor -- start 2024-04-20T21:41:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.204+0000 7f48355c9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a10f0 0x55d6b28a4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.204+0000 7f48355c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 0x55d6b28a4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.204+0000 7f48355c9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28aa5f0 0x55d6b28a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.204+0000 7f48355c9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1b2ff0 con 0x55d6b28aa5f0 2024-04-20T21:41:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.204+0000 7f48355c9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1b4460 con 0x55d6b28a10f0 2024-04-20T21:41:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.204+0000 7f48355c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1b36c0 con 0x55d6b28a1a10 2024-04-20T21:41:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.204+0000 7f48355c9740 10 client.0 _ll_register_callbacks cb 0x7f481c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f482be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28aa5f0 0x55d6b28a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f482be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a10f0 0x55d6b28a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f482aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 0x55d6b28a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f482aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 0x55d6b28a4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42582/0 (socket says 172.21.15.22:42582) 2024-04-20T21:41:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f482be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a10f0 0x55d6b28a4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34130/0 (socket says 172.21.15.22:34130) 2024-04-20T21:41:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f482aa00640 1 -- 172.21.15.22:0/2313938521 learned_addr learned my addr 172.21.15.22:0/2313938521 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f482aa00640 1 -- 172.21.15.22:0/2313938521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a10f0 msgr2=0x55d6b28a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f482aa00640 1 --2- 172.21.15.22:0/2313938521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a10f0 0x55d6b28a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f482aa00640 1 -- 172.21.15.22:0/2313938521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28aa5f0 msgr2=0x55d6b28a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f482aa00640 1 --2- 172.21.15.22:0/2313938521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28aa5f0 0x55d6b28a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f482aa00640 1 -- 172.21.15.22:0/2313938521 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6b291ffc0 con 0x55d6b28a1a10 2024-04-20T21:41:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f482aa00640 1 --2- 172.21.15.22:0/2313938521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 0x55d6b28a4990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f481802c210 tx=0x7f4818034c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bfb53d7728e439e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.205+0000 7f4815600640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:41:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.206+0000 7f4815600640 1 -- 172.21.15.22:0/2313938521 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818044040 con 0x55d6b28a1a10 2024-04-20T21:41:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.206+0000 7f48355c9740 1 -- 172.21.15.22:0/2313938521 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4824009dd0 con 0x55d6b28a1a10 2024-04-20T21:41:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.206+0000 7f4815600640 1 -- 172.21.15.22:0/2313938521 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f4818032a10 con 0x55d6b28a1a10 2024-04-20T21:41:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.206+0000 7f4815600640 1 -- 172.21.15.22:0/2313938521 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4818009540 con 0x55d6b28a1a10 2024-04-20T21:41:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.206+0000 7f48355c9740 10 client.26551 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:39.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.206+0000 7f48355c9740 1 -- 172.21.15.22:0/2313938521 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d6b29212e0 con 0x55d6b28a1a10 2024-04-20T21:41:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.206+0000 7f4815600640 1 -- 172.21.15.22:0/2313938521 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4824009dd0 con 0x55d6b28a1a10 2024-04-20T21:41:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.207+0000 7f4815600640 1 -- 172.21.15.22:0/2313938521 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f4818057020 con 0x55d6b28a1a10 2024-04-20T21:41:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.207+0000 7f48355c9740 10 client.26551 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.207+0000 7f48355c9740 10 client.26551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.207+0000 7f48355c9740 20 client.26551 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.207+0000 7f48355c9740 1 --2- 172.21.15.22:0/2313938521 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d6b2921a90 0x55d6b2923eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.207+0000 7f48355c9740 4 client.26551 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:41:39.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.207+0000 7f48355c9740 1 -- 172.21.15.22:0/2313938521 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d6b280d5b0 con 0x55d6b2921a90 2024-04-20T21:41:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.207+0000 7f482be00640 1 --2- 172.21.15.22:0/2313938521 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d6b2921a90 0x55d6b2923eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.207+0000 7f482be00640 1 --2- 172.21.15.22:0/2313938521 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d6b2921a90 0x55d6b2923eb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7762fda62c6164 server_cookie=fd55963e86a33f8f in_seq=0 out_seq=0 2024-04-20T21:41:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.207+0000 7f4815600640 10 client.26551 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f4815600640 1 -- 172.21.15.22:0/2313938521 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d6b280d5b0 con 0x55d6b2921a90 2024-04-20T21:41:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f4815600640 10 client.26551 handle_command_reply: tid=0 2024-04-20T21:41:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f48355c9740 1 client.26551 shutdown 2024-04-20T21:41:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f48355c9740 1 -- 172.21.15.22:0/2313938521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 msgr2=0x55d6b28a4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f48355c9740 1 --2- 172.21.15.22:0/2313938521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 0x55d6b28a4990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f481802c210 tx=0x7f4818034c80 comp rx=0 tx=0).stop 2024-04-20T21:41:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f48355c9740 1 -- 172.21.15.22:0/2313938521 shutdown_connections 2024-04-20T21:41:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f48355c9740 1 --2- 172.21.15.22:0/2313938521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6b28aa5f0 0x55d6b28a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f48355c9740 1 --2- 172.21.15.22:0/2313938521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6b28a1a10 0x55d6b28a4990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f48355c9740 1 --2- 172.21.15.22:0/2313938521 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55d6b2921a90 0x55d6b2923eb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f48355c9740 1 --2- 172.21.15.22:0/2313938521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b28a10f0 0x55d6b28a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f48355c9740 1 -- 172.21.15.22:0/2313938521 >> 172.21.15.22:0/2313938521 conn(0x55d6b289b820 msgr2=0x55d6b289aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f4815600640 0 client.26551 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f48355c9740 1 -- 172.21.15.22:0/2313938521 shutdown_connections 2024-04-20T21:41:39.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f48355c9740 1 -- 172.21.15.22:0/2313938521 wait complete. 2024-04-20T21:41:39.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.209+0000 7f48355c9740 20 client.26551 trim_cache size 0 max 16384 2024-04-20T21:41:39.215 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:41:39.215 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:41:39.215 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "61c65d19-2eb0-4599-adb9-e3190aadc92a", 2024-04-20T21:41:39.215 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:41:39.215 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:41:39.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.210+0000 7f4831200640 1 -- 172.21.15.22:0/3502668984 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f48100a3da0 msgr2=0x7f48100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.210+0000 7f4831200640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f48100a3da0 0x7f48100a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f481c0ce280 tx=0x7f481c0d4610 comp rx=0 tx=0).stop 2024-04-20T21:41:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.210+0000 7f4831200640 1 -- 172.21.15.22:0/3502668984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f482c100db0 msgr2=0x7f482c1a4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.210+0000 7f4831200640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f482c100db0 0x7f482c1a4a70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f481c00e9f0 tx=0x7f481c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.210+0000 7f4831200640 1 -- 172.21.15.22:0/3502668984 shutdown_connections 2024-04-20T21:41:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.210+0000 7f4831200640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f48100a3da0 0x7f48100a6260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.210+0000 7f4831200640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f482c1061b0 0x7f482c19eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.210+0000 7f4831200640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c1016d0 0x7f482c1a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.210+0000 7f4831200640 1 --2- 172.21.15.22:0/3502668984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f482c100db0 0x7f482c1a4a70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.210+0000 7f4831200640 1 -- 172.21.15.22:0/3502668984 >> 172.21.15.22:0/3502668984 conn(0x7f482c0fc7e0 msgr2=0x7f482c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.210+0000 7f4831200640 1 -- 172.21.15.22:0/3502668984 shutdown_connections 2024-04-20T21:41:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.210+0000 7f4831200640 1 -- 172.21.15.22:0/3502668984 wait complete. 2024-04-20T21:41:39.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:41:39.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:41:39.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.447+0000 7f001f000640 1 -- 172.21.15.22:0/1742529488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00181038e0 msgr2=0x7f001810bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.447+0000 7f001f000640 1 --2- 172.21.15.22:0/1742529488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00181038e0 0x7f001810bdc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0004009900 tx=0x7f000402bcc0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.447+0000 7f001f000640 1 -- 172.21.15.22:0/1742529488 shutdown_connections 2024-04-20T21:41:39.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.447+0000 7f001f000640 1 --2- 172.21.15.22:0/1742529488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f001810c3e0 0x7f001810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.448+0000 7f001f000640 1 --2- 172.21.15.22:0/1742529488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00181038e0 0x7f001810bdc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.448+0000 7f001f000640 1 --2- 172.21.15.22:0/1742529488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0018102fc0 0x7f00181033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.448+0000 7f001f000640 1 -- 172.21.15.22:0/1742529488 >> 172.21.15.22:0/1742529488 conn(0x7f00180fc7e0 msgr2=0x7f00180fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.448+0000 7f001f000640 1 -- 172.21.15.22:0/1742529488 shutdown_connections 2024-04-20T21:41:39.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.448+0000 7f001f000640 1 -- 172.21.15.22:0/1742529488 wait complete. 2024-04-20T21:41:39.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.449+0000 7f001f000640 1 Processor -- start 2024-04-20T21:41:39.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.449+0000 7f001f000640 1 -- start start 2024-04-20T21:41:39.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.449+0000 7f001f000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102fc0 0x7f0018198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.449+0000 7f001f000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00181038e0 0x7f0018198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.449+0000 7f001f000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f001810c3e0 0x7f001819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.449+0000 7f001f000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00181121c0 con 0x7f001810c3e0 2024-04-20T21:41:39.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.449+0000 7f001f000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0018112040 con 0x7f00181038e0 2024-04-20T21:41:39.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.449+0000 7f001f000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0018112340 con 0x7f0018102fc0 2024-04-20T21:41:39.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.450+0000 7f001dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102fc0 0x7f0018198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.450+0000 7f001dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102fc0 0x7f0018198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42602/0 (socket says 172.21.15.22:42602) 2024-04-20T21:41:39.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.450+0000 7f001dc00640 1 -- 172.21.15.22:0/4182804911 learned_addr learned my addr 172.21.15.22:0/4182804911 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:39.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.450+0000 7f001e600640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f001810c3e0 0x7f001819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.450+0000 7f001d200640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00181038e0 0x7f0018198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.450+0000 7f001e600640 1 -- 172.21.15.22:0/4182804911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102fc0 msgr2=0x7f0018198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.450+0000 7f001e600640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102fc0 0x7f0018198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.450+0000 7f001e600640 1 -- 172.21.15.22:0/4182804911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00181038e0 msgr2=0x7f0018198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.450+0000 7f001e600640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00181038e0 0x7f0018198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.450+0000 7f001e600640 1 -- 172.21.15.22:0/4182804911 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001819d6c0 con 0x7f001810c3e0 2024-04-20T21:41:39.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.450+0000 7f001e600640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f001810c3e0 0x7f001819cf90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f001000e970 tx=0x7f001000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.451+0000 7f0016000640 1 -- 172.21.15.22:0/4182804911 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001000ccc0 con 0x7f001810c3e0 2024-04-20T21:41:39.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.451+0000 7f001f000640 1 -- 172.21.15.22:0/4182804911 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001819d9b0 con 0x7f001810c3e0 2024-04-20T21:41:39.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.451+0000 7f0016000640 1 -- 172.21.15.22:0/4182804911 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0010005020 con 0x7f001810c3e0 2024-04-20T21:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.451+0000 7f0016000640 1 -- 172.21.15.22:0/4182804911 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010004a00 con 0x7f001810c3e0 2024-04-20T21:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.451+0000 7f001f000640 1 -- 172.21.15.22:0/4182804911 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00181a5240 con 0x7f001810c3e0 2024-04-20T21:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.455+0000 7f00233a0740 1 Processor -- start 2024-04-20T21:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.455+0000 7f0016000640 1 -- 172.21.15.22:0/4182804911 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f00100051d0 con 0x7f001810c3e0 2024-04-20T21:41:39.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.456+0000 7f00233a0740 1 -- start start 2024-04-20T21:41:39.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.456+0000 7f00233a0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3467760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.456+0000 7f00233a0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3467ca0 0x5601b3470340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.456+0000 7f00233a0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3470880 0x5601b3472c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.456+0000 7f00233a0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5601b33cd150 con 0x5601b3467380 2024-04-20T21:41:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.456+0000 7f00233a0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5601b3373370 con 0x5601b3470880 2024-04-20T21:41:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.456+0000 7f00233a0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5601b33c9110 con 0x5601b3467ca0 2024-04-20T21:41:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.456+0000 7f001d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3467760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.456+0000 7f001dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3467ca0 0x5601b3470340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.457+0000 7f001dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3467ca0 0x5601b3470340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:42610/0 (socket says 172.21.15.22:42610) 2024-04-20T21:41:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.457+0000 7f001dc00640 1 -- 172.21.15.22:0/1744186176 learned_addr learned my addr 172.21.15.22:0/1744186176 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.457+0000 7f001d200640 1 --2- 172.21.15.22:0/1744186176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3470880 0x5601b3472c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.457+0000 7f001dc00640 1 -- 172.21.15.22:0/1744186176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3470880 msgr2=0x5601b3472c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.457+0000 7f001dc00640 1 --2- 172.21.15.22:0/1744186176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3470880 0x5601b3472c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.457+0000 7f001dc00640 1 -- 172.21.15.22:0/1744186176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 msgr2=0x5601b3467760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.457+0000 7f001dc00640 1 --2- 172.21.15.22:0/1744186176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3467760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.457+0000 7f001dc00640 1 -- 172.21.15.22:0/1744186176 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b31a97d0 con 0x5601b3467ca0 2024-04-20T21:41:39.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.457+0000 7f001d200640 1 --2- 172.21.15.22:0/1744186176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3470880 0x5601b3472c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.457+0000 7f001dc00640 1 --2- 172.21.15.22:0/1744186176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3467ca0 0x5601b3470340 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f000c0115f0 tx=0x7f000c011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bef27f16c92452b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.458+0000 7f0016000640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f00000a3d20 0x7f00000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.458+0000 7f000aa00640 1 -- 172.21.15.22:0/1744186176 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c00f8f0 con 0x5601b3467ca0 2024-04-20T21:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.458+0000 7f000aa00640 1 -- 172.21.15.22:0/1744186176 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f000c019420 con 0x5601b3467ca0 2024-04-20T21:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.458+0000 7f000aa00640 1 -- 172.21.15.22:0/1744186176 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c018630 con 0x5601b3467ca0 2024-04-20T21:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.458+0000 7f0016000640 1 -- 172.21.15.22:0/4182804911 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f00100d0720 con 0x7f001810c3e0 2024-04-20T21:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.458+0000 7f001d200640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f00000a3d20 0x7f00000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.458+0000 7f00233a0740 1 -- 172.21.15.22:0/1744186176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3467ca0 msgr2=0x5601b3470340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.458+0000 7f00233a0740 1 --2- 172.21.15.22:0/1744186176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3467ca0 0x5601b3470340 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f000c0115f0 tx=0x7f000c011ab0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.459+0000 7f00233a0740 1 -- 172.21.15.22:0/1744186176 shutdown_connections 2024-04-20T21:41:39.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.459+0000 7f00233a0740 1 --2- 172.21.15.22:0/1744186176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3470880 0x5601b3472c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.459+0000 7f00233a0740 1 --2- 172.21.15.22:0/1744186176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3467ca0 0x5601b3470340 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.459+0000 7f00233a0740 1 --2- 172.21.15.22:0/1744186176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3467760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.459+0000 7f00233a0740 1 -- 172.21.15.22:0/1744186176 >> 172.21.15.22:0/1744186176 conn(0x5601b3461ab0 msgr2=0x5601b3461e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.459+0000 7f001d200640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f00000a3d20 0x7f00000a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0004009230 tx=0x7f0004005df0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.459+0000 7f00233a0740 1 -- 172.21.15.22:0/1744186176 shutdown_connections 2024-04-20T21:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.459+0000 7f00233a0740 1 -- 172.21.15.22:0/1744186176 wait complete. 2024-04-20T21:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.459+0000 7f00233a0740 1 -- start start 2024-04-20T21:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.459+0000 7f00233a0740 1 Processor -- start 2024-04-20T21:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.460+0000 7f00233a0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b346d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.460+0000 7f00233a0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b346de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.460+0000 7f00233a0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 0x5601b346e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.460+0000 7f00233a0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5601b3473670 con 0x5601b3467380 2024-04-20T21:41:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.460+0000 7f00233a0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5601b3386740 con 0x5601b3467ca0 2024-04-20T21:41:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.460+0000 7f00233a0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5601b34737f0 con 0x5601b3470880 2024-04-20T21:41:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.460+0000 7f00233a0740 10 client.0 _ll_register_callbacks cb 0x5601b3381b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.460+0000 7f001dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b346d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.460+0000 7f001dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b346d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38490/0 (socket says 172.21.15.22:38490) 2024-04-20T21:41:39.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.460+0000 7f001dc00640 1 -- 172.21.15.22:0/4093379626 learned_addr learned my addr 172.21.15.22:0/4093379626 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:39.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.460+0000 7f001d200640 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b346de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.460+0000 7f001e600640 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 0x5601b346e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.461+0000 7f001dc00640 1 -- 172.21.15.22:0/4093379626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 msgr2=0x5601b346e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.461+0000 7f001dc00640 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 0x5601b346e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.461+0000 7f001dc00640 1 -- 172.21.15.22:0/4093379626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 msgr2=0x5601b346de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.461+0000 7f001dc00640 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b346de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.461+0000 7f001dc00640 1 -- 172.21.15.22:0/4093379626 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b30df3f0 con 0x5601b3467380 2024-04-20T21:41:39.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.461+0000 7f001dc00640 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b346d8f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f000c019da0 tx=0x7f000c019800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9a9f06ae132ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.461+0000 7effff400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:41:39.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.461+0000 7effff400640 1 -- 172.21.15.22:0/4093379626 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c00fad0 con 0x5601b3467380 2024-04-20T21:41:39.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.461+0000 7effff400640 1 -- 172.21.15.22:0/4093379626 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f000c00fc70 con 0x5601b3467380 2024-04-20T21:41:39.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.461+0000 7f00233a0740 1 -- 172.21.15.22:0/4093379626 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f000c019420 con 0x5601b3467380 2024-04-20T21:41:39.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.462+0000 7effff400640 1 -- 172.21.15.22:0/4093379626 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f000c0199f0 con 0x5601b3467380 2024-04-20T21:41:39.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.462+0000 7f00233a0740 10 client.16392 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:39.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.462+0000 7f00233a0740 1 -- 172.21.15.22:0/4093379626 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5601b309f0f0 con 0x5601b3467380 2024-04-20T21:41:39.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.462+0000 7effff400640 1 -- 172.21.15.22:0/4093379626 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f000c019420 con 0x5601b3467380 2024-04-20T21:41:39.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.462+0000 7effff400640 1 -- 172.21.15.22:0/4093379626 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f000c005820 con 0x5601b3467380 2024-04-20T21:41:39.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.462+0000 7f00233a0740 10 client.16392 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:39.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.462+0000 7f00233a0740 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.462+0000 7f00233a0740 20 client.16392 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.462+0000 7f00233a0740 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5601b3485060 0x5601b3487450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.462+0000 7f00233a0740 4 client.16392 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.462+0000 7f00233a0740 1 -- 172.21.15.22:0/4093379626 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601b33dab70 con 0x5601b3485060 2024-04-20T21:41:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.463+0000 7f001d200640 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5601b3485060 0x5601b3487450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.463+0000 7f001d200640 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5601b3485060 0x5601b3487450 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220234e675b5a2aa server_cookie=a889c6c8b88ec986 in_seq=0 out_seq=0 2024-04-20T21:41:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.463+0000 7effff400640 10 client.16392 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.464+0000 7effff400640 1 -- 172.21.15.22:0/4093379626 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601b33dab70 con 0x5601b3485060 2024-04-20T21:41:39.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.464+0000 7effff400640 10 client.16392 handle_command_reply: tid=0 2024-04-20T21:41:39.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.464+0000 7f00233a0740 1 client.16392 shutdown 2024-04-20T21:41:39.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.464+0000 7f00233a0740 1 -- 172.21.15.22:0/4093379626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 msgr2=0x5601b346d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.464+0000 7f00233a0740 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b346d8f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f000c019da0 tx=0x7f000c019800 comp rx=0 tx=0).stop 2024-04-20T21:41:39.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.465+0000 7f00233a0740 1 -- 172.21.15.22:0/4093379626 shutdown_connections 2024-04-20T21:41:39.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.465+0000 7f00233a0740 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 0x5601b346e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.465+0000 7f00233a0740 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5601b3485060 0x5601b3487450 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.465+0000 7f00233a0740 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b346de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.465+0000 7f00233a0740 1 --2- 172.21.15.22:0/4093379626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b346d8f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.465+0000 7effff400640 0 client.16392 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:39.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.465+0000 7f00233a0740 1 -- 172.21.15.22:0/4093379626 >> 172.21.15.22:0/4093379626 conn(0x5601b3461ab0 msgr2=0x5601b3463800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.465+0000 7f00233a0740 1 -- 172.21.15.22:0/4093379626 shutdown_connections 2024-04-20T21:41:39.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.465+0000 7f00233a0740 1 -- 172.21.15.22:0/4093379626 wait complete. 2024-04-20T21:41:39.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.465+0000 7f00233a0740 20 client.16392 trim_cache size 0 max 16384 2024-04-20T21:41:39.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.484+0000 7f00233a0740 1 Processor -- start 2024-04-20T21:41:39.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.484+0000 7f00233a0740 1 -- start start 2024-04-20T21:41:39.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.485+0000 7f00233a0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3470020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.485+0000 7f00233a0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b346e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.485+0000 7f00233a0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 0x5601b346d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.485+0000 7f00233a0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00181a9ac0 con 0x5601b3467380 2024-04-20T21:41:39.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.485+0000 7f00233a0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00181ac320 con 0x5601b3467ca0 2024-04-20T21:41:39.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.485+0000 7f00233a0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00181aa110 con 0x5601b3470880 2024-04-20T21:41:39.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.485+0000 7f001d200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b346e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.485+0000 7f001d200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b346e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34164/0 (socket says 172.21.15.22:34164) 2024-04-20T21:41:39.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.485+0000 7f001d200640 1 -- 172.21.15.22:0/3943284160 learned_addr learned my addr 172.21.15.22:0/3943284160 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:39.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.485+0000 7f001dc00640 1 --2- 172.21.15.22:0/3943284160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3470020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.485+0000 7f001e600640 1 --2- 172.21.15.22:0/3943284160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 0x5601b346d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.486+0000 7f001d200640 1 -- 172.21.15.22:0/3943284160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 msgr2=0x5601b346d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.486+0000 7f001d200640 1 --2- 172.21.15.22:0/3943284160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 0x5601b346d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.486+0000 7f001d200640 1 -- 172.21.15.22:0/3943284160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 msgr2=0x5601b3470020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.486+0000 7f001d200640 1 --2- 172.21.15.22:0/3943284160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3470020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.486+0000 7f001d200640 1 -- 172.21.15.22:0/3943284160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b3467ad0 con 0x5601b3467ca0 2024-04-20T21:41:39.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.486+0000 7f001d200640 1 --2- 172.21.15.22:0/3943284160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b346e140 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0004009900 tx=0x7f0004009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fba2583e501951d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.486+0000 7f000a000640 1 -- 172.21.15.22:0/3943284160 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004004840 con 0x5601b3467ca0 2024-04-20T21:41:39.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.486+0000 7f000a000640 1 -- 172.21.15.22:0/3943284160 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f000405b070 con 0x5601b3467ca0 2024-04-20T21:41:39.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.486+0000 7f000a000640 1 -- 172.21.15.22:0/3943284160 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004055360 con 0x5601b3467ca0 2024-04-20T21:41:39.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.487+0000 7f00233a0740 1 -- 172.21.15.22:0/3943284160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 msgr2=0x5601b346e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.487+0000 7f00233a0740 1 --2- 172.21.15.22:0/3943284160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b346e140 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0004009900 tx=0x7f0004009260 comp rx=0 tx=0).stop 2024-04-20T21:41:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.487+0000 7f00233a0740 1 -- 172.21.15.22:0/3943284160 shutdown_connections 2024-04-20T21:41:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.487+0000 7f00233a0740 1 --2- 172.21.15.22:0/3943284160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 0x5601b346d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.487+0000 7f00233a0740 1 --2- 172.21.15.22:0/3943284160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b346e140 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.487+0000 7f00233a0740 1 --2- 172.21.15.22:0/3943284160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3470020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.487+0000 7f00233a0740 1 -- 172.21.15.22:0/3943284160 >> 172.21.15.22:0/3943284160 conn(0x5601b3461ab0 msgr2=0x5601b3462ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.487+0000 7f00233a0740 1 -- 172.21.15.22:0/3943284160 shutdown_connections 2024-04-20T21:41:39.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.487+0000 7f00233a0740 1 -- 172.21.15.22:0/3943284160 wait complete. 2024-04-20T21:41:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.487+0000 7f00233a0740 1 -- start start 2024-04-20T21:41:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.488+0000 7f00233a0740 1 Processor -- start 2024-04-20T21:41:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.488+0000 7f00233a0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3478af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.488+0000 7f00233a0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b3479030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.488+0000 7f00233a0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 0x5601b3479570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.488+0000 7f00233a0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00181aa7e0 con 0x5601b3467380 2024-04-20T21:41:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.488+0000 7f00233a0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00181abc50 con 0x5601b3467ca0 2024-04-20T21:41:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.488+0000 7f00233a0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00181aaeb0 con 0x5601b3470880 2024-04-20T21:41:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.488+0000 7f00233a0740 10 client.0 _ll_register_callbacks cb 0x7f000c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.488+0000 7f001dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3478af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.488+0000 7f001dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3478af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38500/0 (socket says 172.21.15.22:38500) 2024-04-20T21:41:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.488+0000 7f001dc00640 1 -- 172.21.15.22:0/1408887939 learned_addr learned my addr 172.21.15.22:0/1408887939 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:41:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.488+0000 7f001d200640 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b3479030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.489+0000 7f001e600640 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 0x5601b3479570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.489+0000 7f001dc00640 1 -- 172.21.15.22:0/1408887939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 msgr2=0x5601b3479570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.489+0000 7f001dc00640 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 0x5601b3479570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.489+0000 7f001dc00640 1 -- 172.21.15.22:0/1408887939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 msgr2=0x5601b3479030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.489+0000 7f001dc00640 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b3479030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.489+0000 7f001dc00640 1 -- 172.21.15.22:0/1408887939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b34ea650 con 0x5601b3467380 2024-04-20T21:41:39.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.489+0000 7f001d200640 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b3479030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:39.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.489+0000 7f001dc00640 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3478af0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f000c0225c0 tx=0x7f000c019870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db5e3840a01a677 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:39.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.489+0000 7effff400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:41:39.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.489+0000 7effff400640 1 -- 172.21.15.22:0/1408887939 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c011230 con 0x5601b3467380 2024-04-20T21:41:39.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.489+0000 7effff400640 1 -- 172.21.15.22:0/1408887939 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f000c022ae0 con 0x5601b3467380 2024-04-20T21:41:39.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.490+0000 7f00233a0740 1 -- 172.21.15.22:0/1408887939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f000405b070 con 0x5601b3467380 2024-04-20T21:41:39.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.490+0000 7effff400640 1 -- 172.21.15.22:0/1408887939 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f000c019420 con 0x5601b3467380 2024-04-20T21:41:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.490+0000 7f00233a0740 10 client.16398 fetch_fsmap learned FSMap version 20 2024-04-20T21:41:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.490+0000 7f00233a0740 1 -- 172.21.15.22:0/1408887939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5601b34eb900 con 0x5601b3467380 2024-04-20T21:41:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.490+0000 7effff400640 1 -- 172.21.15.22:0/1408887939 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f000405b070 con 0x5601b3467380 2024-04-20T21:41:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.490+0000 7effff400640 1 -- 172.21.15.22:0/1408887939 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f000c039030 con 0x5601b3467380 2024-04-20T21:41:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.490+0000 7f00233a0740 10 client.16398 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:41:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.490+0000 7f00233a0740 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.490+0000 7f00233a0740 20 client.16398 populate_metadata read hostname 'smithi022' 2024-04-20T21:41:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.490+0000 7f00233a0740 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5601b34ec050 0x5601b34ee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.490+0000 7f00233a0740 4 client.16398 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:41:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.490+0000 7f00233a0740 1 -- 172.21.15.22:0/1408887939 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5601b33d49e0 con 0x5601b34ec050 2024-04-20T21:41:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.491+0000 7f001d200640 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5601b34ec050 0x5601b34ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.491+0000 7f001d200640 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5601b34ec050 0x5601b34ee470 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eafda22bb760cc8 server_cookie=f9e396fc3210acf3 in_seq=0 out_seq=0 2024-04-20T21:41:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.491+0000 7effff400640 10 client.16398 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.491+0000 7effff400640 1 -- 172.21.15.22:0/1408887939 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5601b33d49e0 con 0x5601b34ec050 2024-04-20T21:41:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.491+0000 7effff400640 10 client.16398 handle_command_reply: tid=0 2024-04-20T21:41:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.491+0000 7f00233a0740 1 client.16398 shutdown 2024-04-20T21:41:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.492+0000 7f00233a0740 1 -- 172.21.15.22:0/1408887939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 msgr2=0x5601b3478af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.492+0000 7f00233a0740 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3478af0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f000c0225c0 tx=0x7f000c019870 comp rx=0 tx=0).stop 2024-04-20T21:41:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.492+0000 7f00233a0740 1 -- 172.21.15.22:0/1408887939 shutdown_connections 2024-04-20T21:41:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.492+0000 7f00233a0740 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5601b3470880 0x5601b3479570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.492+0000 7f00233a0740 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5601b34ec050 0x5601b34ee470 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.492+0000 7f00233a0740 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601b3467ca0 0x5601b3479030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.492+0000 7f00233a0740 1 --2- 172.21.15.22:0/1408887939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5601b3467380 0x5601b3478af0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.492+0000 7f00233a0740 1 -- 172.21.15.22:0/1408887939 >> 172.21.15.22:0/1408887939 conn(0x5601b3461ab0 msgr2=0x5601b3463340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.492+0000 7effff400640 0 client.16398 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:41:39.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.492+0000 7f00233a0740 1 -- 172.21.15.22:0/1408887939 shutdown_connections 2024-04-20T21:41:39.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.492+0000 7f00233a0740 1 -- 172.21.15.22:0/1408887939 wait complete. 2024-04-20T21:41:39.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.492+0000 7f00233a0740 20 client.16398 trim_cache size 0 max 16384 2024-04-20T21:41:39.499 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:41:39.499 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:41:39.499 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:41:39.499 INFO:teuthology.orchestra.run.smithi022.stdout: "61c65d19-2eb0-4599-adb9-e3190aadc92a": { 2024-04-20T21:41:39.500 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:41:39.500 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "61c65d19-2eb0-4599-adb9-e3190aadc92a", 2024-04-20T21:41:39.500 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:41:39.500 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:41:39.500 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:41:39.500 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:41:39.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.493+0000 7f001f000640 1 -- 172.21.15.22:0/4182804911 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f00000a3d20 msgr2=0x7f00000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.493+0000 7f001f000640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f00000a3d20 0x7f00000a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0004009230 tx=0x7f0004005df0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.493+0000 7f001f000640 1 -- 172.21.15.22:0/4182804911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f001810c3e0 msgr2=0x7f001819cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:39.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.493+0000 7f001f000640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f001810c3e0 0x7f001819cf90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f001000e970 tx=0x7f001000ee30 comp rx=0 tx=0).stop 2024-04-20T21:41:39.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.493+0000 7f001f000640 1 -- 172.21.15.22:0/4182804911 shutdown_connections 2024-04-20T21:41:39.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.493+0000 7f001f000640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f00000a3d20 0x7f00000a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.493+0000 7f001f000640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f001810c3e0 0x7f001819cf90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.493+0000 7f001f000640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00181038e0 0x7f0018198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.493+0000 7f001f000640 1 --2- 172.21.15.22:0/4182804911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102fc0 0x7f0018198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:39.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.493+0000 7f001f000640 1 -- 172.21.15.22:0/4182804911 >> 172.21.15.22:0/4182804911 conn(0x7f00180fc7e0 msgr2=0x7f00180fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:39.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.493+0000 7f001f000640 1 -- 172.21.15.22:0/4182804911 shutdown_connections 2024-04-20T21:41:39.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:41:39.493+0000 7f001f000640 1 -- 172.21.15.22:0/4182804911 wait complete. 2024-04-20T21:41:39.510 INFO:tasks.cephfs.filesystem:scrub status for tag:61c65d19-2eb0-4599-adb9-e3190aadc92a - {'path': '/', 'tag': '61c65d19-2eb0-4599-adb9-e3190aadc92a', 'options': 'recursive,force'} 2024-04-20T21:41:40.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:40 smithi022 ceph-mon[16843]: pgmap v242: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:40.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:40 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:41:40.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:40 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:40.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:40 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:41:40.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:40 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:40.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:40 smithi122 ceph-mon[22147]: pgmap v242: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:40.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:40 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:41:40.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:40 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:40.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:40 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:41:40.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:40 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:40 smithi033 ceph-mon[21626]: pgmap v242: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:40 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:41:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:40 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:40 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:41:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:40 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:42.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:42 smithi022 ceph-mon[16843]: pgmap v243: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:41:42.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:42 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:41:42.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:42 smithi122 ceph-mon[22147]: pgmap v243: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:41:42.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:42 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:41:42.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:42 smithi033 ceph-mon[21626]: pgmap v243: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:41:42.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:42 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:41:44.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:44 smithi022 ceph-mon[16843]: pgmap v244: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:41:44.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:44 smithi122 ceph-mon[22147]: pgmap v244: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:41:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:44 smithi033 ceph-mon[21626]: pgmap v244: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:41:46.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:46 smithi022 ceph-mon[16843]: pgmap v245: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:41:46.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:46 smithi122 ceph-mon[22147]: pgmap v245: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:41:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:46 smithi033 ceph-mon[21626]: pgmap v245: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:41:48.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:48 smithi022 ceph-mon[16843]: pgmap v246: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:41:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:48 smithi122 ceph-mon[22147]: pgmap v246: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:41:48.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:48 smithi033 ceph-mon[21626]: pgmap v246: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:41:50.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:50 smithi122 ceph-mon[22147]: pgmap v247: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:41:50.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:50 smithi033 ceph-mon[21626]: pgmap v247: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:41:50.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:50 smithi022 ceph-mon[16843]: pgmap v247: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:41:52.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:52 smithi122 ceph-mon[22147]: pgmap v248: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:41:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:52 smithi033 ceph-mon[21626]: pgmap v248: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:41:52.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:52 smithi022 ceph-mon[16843]: pgmap v248: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:41:54.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:54 smithi122 ceph-mon[22147]: pgmap v249: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:54.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:54 smithi033 ceph-mon[21626]: pgmap v249: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:54.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:54 smithi022 ceph-mon[16843]: pgmap v249: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:56.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:56 smithi122 ceph-mon[22147]: pgmap v250: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:41:56.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:56 smithi033 ceph-mon[21626]: pgmap v250: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:41:56.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:56 smithi022 ceph-mon[16843]: pgmap v250: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:41:58.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:41:58 smithi122 ceph-mon[22147]: pgmap v251: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:58.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:41:58 smithi033 ceph-mon[21626]: pgmap v251: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:41:58.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:41:58 smithi022 ceph-mon[16843]: pgmap v251: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:00.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:00 smithi122 ceph-mon[22147]: pgmap v252: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:00.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:00 smithi033 ceph-mon[21626]: pgmap v252: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:00.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:00 smithi022 ceph-mon[16843]: pgmap v252: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:02.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:02 smithi122 ceph-mon[22147]: pgmap v253: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:42:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:02 smithi033 ceph-mon[21626]: pgmap v253: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:42:02.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:02 smithi022 ceph-mon[16843]: pgmap v253: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:42:04.460 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:04 smithi022 ceph-mon[16843]: pgmap v254: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:04.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:04 smithi122 ceph-mon[22147]: pgmap v254: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:04.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:04 smithi033 ceph-mon[21626]: pgmap v254: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:06.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:06 smithi122 ceph-mon[22147]: pgmap v255: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-20T21:42:06.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:06 smithi033 ceph-mon[21626]: pgmap v255: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-20T21:42:06.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:06 smithi022 ceph-mon[16843]: pgmap v255: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-20T21:42:08.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:08 smithi122 ceph-mon[22147]: pgmap v256: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-20T21:42:08.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:08 smithi122 ceph-mon[22147]: osdmap e98: 12 total, 12 up, 12 in 2024-04-20T21:42:08.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:08 smithi033 ceph-mon[21626]: pgmap v256: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-20T21:42:08.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:08 smithi033 ceph-mon[21626]: osdmap e98: 12 total, 12 up, 12 in 2024-04-20T21:42:08.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:08 smithi022 ceph-mon[16843]: pgmap v256: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-20T21:42:08.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:08 smithi022 ceph-mon[16843]: osdmap e98: 12 total, 12 up, 12 in 2024-04-20T21:42:09.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:42:09.511 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:42:09.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.738+0000 7ff5b4200640 1 -- 172.21.15.22:0/819165275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5ac108f50 msgr2=0x7ff5ac1153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:09.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.738+0000 7ff5b4200640 1 --2- 172.21.15.22:0/819165275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5ac108f50 0x7ff5ac1153c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009890 tx=0x7ff5a402bb70 comp rx=0 tx=0).stop 2024-04-20T21:42:09.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.738+0000 7ff5b4200640 1 -- 172.21.15.22:0/819165275 shutdown_connections 2024-04-20T21:42:09.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.738+0000 7ff5b4200640 1 --2- 172.21.15.22:0/819165275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5ac108f50 0x7ff5ac1153c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.738+0000 7ff5b4200640 1 --2- 172.21.15.22:0/819165275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ac108590 0x7ff5ac108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.738+0000 7ff5b4200640 1 --2- 172.21.15.22:0/819165275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5ac10e600 0x7ff5ac10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.738+0000 7ff5b4200640 1 -- 172.21.15.22:0/819165275 >> 172.21.15.22:0/819165275 conn(0x7ff5ac0764a0 msgr2=0x7ff5ac0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:09.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.738+0000 7ff5b4200640 1 -- 172.21.15.22:0/819165275 shutdown_connections 2024-04-20T21:42:09.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.738+0000 7ff5b4200640 1 -- 172.21.15.22:0/819165275 wait complete. 2024-04-20T21:42:09.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.739+0000 7ff5b4200640 1 Processor -- start 2024-04-20T21:42:09.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.739+0000 7ff5b4200640 1 -- start start 2024-04-20T21:42:09.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.740+0000 7ff5b4200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5ac108590 0x7ff5ac19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.740+0000 7ff5b4200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ac108f50 0x7ff5ac19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.740+0000 7ff5b4200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5ac10e600 0x7ff5ac1a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.740+0000 7ff5b4200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1188d0 con 0x7ff5ac10e600 2024-04-20T21:42:09.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.740+0000 7ff5b4200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac118750 con 0x7ff5ac108590 2024-04-20T21:42:09.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.740+0000 7ff5b4200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac118a50 con 0x7ff5ac108f50 2024-04-20T21:42:09.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.740+0000 7ff5b1800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5ac108590 0x7ff5ac19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.740+0000 7ff5b1800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5ac108590 0x7ff5ac19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:37916/0 (socket says 172.21.15.22:37916) 2024-04-20T21:42:09.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.740+0000 7ff5b1800640 1 -- 172.21.15.22:0/3703311520 learned_addr learned my addr 172.21.15.22:0/3703311520 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:09.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.740+0000 7ff5b2200640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5ac10e600 0x7ff5ac1a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.740+0000 7ff5b0e00640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ac108f50 0x7ff5ac19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.741+0000 7ff5b1800640 1 -- 172.21.15.22:0/3703311520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ac108f50 msgr2=0x7ff5ac19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:09.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.741+0000 7ff5b1800640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ac108f50 0x7ff5ac19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.741+0000 7ff5b1800640 1 -- 172.21.15.22:0/3703311520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5ac10e600 msgr2=0x7ff5ac1a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:09.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.741+0000 7ff5b1800640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5ac10e600 0x7ff5ac1a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.741+0000 7ff5b1800640 1 -- 172.21.15.22:0/3703311520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac1a3dc0 con 0x7ff5ac108590 2024-04-20T21:42:09.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.741+0000 7ff5b1800640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5ac108590 0x7ff5ac19edd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000c420 tx=0x7ff5a0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:09.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.741+0000 7ff5aa000640 1 -- 172.21.15.22:0/3703311520 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0003970 con 0x7ff5ac108590 2024-04-20T21:42:09.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.742+0000 7ff5aa000640 1 -- 172.21.15.22:0/3703311520 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a0004020 con 0x7ff5ac108590 2024-04-20T21:42:09.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.742+0000 7ff5b4200640 1 -- 172.21.15.22:0/3703311520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ac1a40b0 con 0x7ff5ac108590 2024-04-20T21:42:09.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.742+0000 7ff5aa000640 1 -- 172.21.15.22:0/3703311520 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a000bad0 con 0x7ff5ac108590 2024-04-20T21:42:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.742+0000 7ff5b4200640 1 -- 172.21.15.22:0/3703311520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ac1ab940 con 0x7ff5ac108590 2024-04-20T21:42:09.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.746+0000 7ff5b6f8e740 1 Processor -- start 2024-04-20T21:42:09.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.746+0000 7ff5b6f8e740 1 -- start start 2024-04-20T21:42:09.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b6f8e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x563705792580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b6f8e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x56370579b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b6f8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x56370579da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b6f8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5637056f8150 con 0x56370579b6a0 2024-04-20T21:42:09.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b6f8e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56370569e370 con 0x5637057921a0 2024-04-20T21:42:09.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b6f8e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5637056f4110 con 0x563705792ac0 2024-04-20T21:42:09.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5aa000640 1 -- 172.21.15.22:0/3703311520 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff5a001d020 con 0x7ff5ac108590 2024-04-20T21:42:09.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b2200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x563705792580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b2200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x563705792580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:37924/0 (socket says 172.21.15.22:37924) 2024-04-20T21:42:09.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b2200640 1 -- 172.21.15.22:0/4155709968 learned_addr learned my addr 172.21.15.22:0/4155709968 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:09.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b0e00640 1 --2- 172.21.15.22:0/4155709968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x56370579da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b1800640 1 --2- 172.21.15.22:0/4155709968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x56370579b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b2200640 1 -- 172.21.15.22:0/4155709968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 msgr2=0x56370579b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b2200640 1 --2- 172.21.15.22:0/4155709968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x56370579b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b2200640 1 -- 172.21.15.22:0/4155709968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 msgr2=0x56370579da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b2200640 1 --2- 172.21.15.22:0/4155709968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x56370579da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b2200640 1 -- 172.21.15.22:0/4155709968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637054d47d0 con 0x5637057921a0 2024-04-20T21:42:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.747+0000 7ff5b0e00640 1 --2- 172.21.15.22:0/4155709968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x56370579da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.748+0000 7ff5b2200640 1 --2- 172.21.15.22:0/4155709968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x563705792580 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009890 tx=0x7ff5a4009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b173e4499976784 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.748+0000 7ff59ea00640 1 -- 172.21.15.22:0/4155709968 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4043020 con 0x5637057921a0 2024-04-20T21:42:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.748+0000 7ff59ea00640 1 -- 172.21.15.22:0/4155709968 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a4034cb0 con 0x5637057921a0 2024-04-20T21:42:09.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.748+0000 7ff59ea00640 1 -- 172.21.15.22:0/4155709968 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4039040 con 0x5637057921a0 2024-04-20T21:42:09.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.749+0000 7ff5aa000640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff5940a3da0 0x7ff5940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.749+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/4155709968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 msgr2=0x563705792580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:09.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.749+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/4155709968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x563705792580 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009890 tx=0x7ff5a4009490 comp rx=0 tx=0).stop 2024-04-20T21:42:09.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.749+0000 7ff5aa000640 1 -- 172.21.15.22:0/3703311520 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7ff5a00d06b0 con 0x7ff5ac108590 2024-04-20T21:42:09.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.749+0000 7ff5b1800640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff5940a3da0 0x7ff5940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.749+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/4155709968 shutdown_connections 2024-04-20T21:42:09.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.749+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/4155709968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x56370579da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.749+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/4155709968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x56370579b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.749+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/4155709968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x563705792580 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.749+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/4155709968 >> 172.21.15.22:0/4155709968 conn(0x56370578c8d0 msgr2=0x56370578ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:09.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.749+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/4155709968 shutdown_connections 2024-04-20T21:42:09.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.749+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/4155709968 wait complete. 2024-04-20T21:42:09.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.750+0000 7ff5b1800640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff5940a3da0 0x7ff5940a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x563705791c70 tx=0x7ff5a0004730 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:09.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.750+0000 7ff5b6f8e740 1 -- start start 2024-04-20T21:42:09.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.750+0000 7ff5b6f8e740 1 Processor -- start 2024-04-20T21:42:09.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.750+0000 7ff5b6f8e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x5637057986f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.750+0000 7ff5b6f8e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x563705798c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.750+0000 7ff5b6f8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x563705799170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.750+0000 7ff5b6f8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56370579e430 con 0x56370579b6a0 2024-04-20T21:42:09.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.750+0000 7ff5b6f8e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5637056b1740 con 0x5637057921a0 2024-04-20T21:42:09.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.750+0000 7ff5b6f8e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56370579e5b0 con 0x563705792ac0 2024-04-20T21:42:09.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.751+0000 7ff5b6f8e740 10 client.0 _ll_register_callbacks cb 0x5637056acb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:09.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.751+0000 7ff5b2200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x5637057986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.751+0000 7ff5b0e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x563705798c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.751+0000 7ff5b2200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x563705799170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.751+0000 7ff5b0e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x563705798c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50738/0 (socket says 172.21.15.22:50738) 2024-04-20T21:42:09.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.751+0000 7ff5b0e00640 1 -- 172.21.15.22:0/2164862552 learned_addr learned my addr 172.21.15.22:0/2164862552 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:09.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.751+0000 7ff5b0e00640 1 -- 172.21.15.22:0/2164862552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 msgr2=0x5637057986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:09.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.751+0000 7ff5b0e00640 1 --2- 172.21.15.22:0/2164862552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x5637057986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.751+0000 7ff5b0e00640 1 -- 172.21.15.22:0/2164862552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 msgr2=0x563705799170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:09.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.751+0000 7ff5b0e00640 1 --2- 172.21.15.22:0/2164862552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x563705799170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.751+0000 7ff5b0e00640 1 -- 172.21.15.22:0/2164862552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56370540a3f0 con 0x563705792ac0 2024-04-20T21:42:09.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.751+0000 7ff5b2200640 1 --2- 172.21.15.22:0/2164862552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x563705799170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:09.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.752+0000 7ff5b0e00640 1 --2- 172.21.15.22:0/2164862552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x563705798c30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff598012730 tx=0x7ff598012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25dc95275216bc68 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:09.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.752+0000 7ff593400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:42:09.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.752+0000 7ff593400640 1 -- 172.21.15.22:0/2164862552 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598004030 con 0x563705792ac0 2024-04-20T21:42:09.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.752+0000 7ff593400640 1 -- 172.21.15.22:0/2164862552 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7ff5980041d0 con 0x563705792ac0 2024-04-20T21:42:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.752+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/2164862552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a4034cb0 con 0x563705792ac0 2024-04-20T21:42:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.752+0000 7ff593400640 1 -- 172.21.15.22:0/2164862552 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff5980043d0 con 0x563705792ac0 2024-04-20T21:42:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.752+0000 7ff5b6f8e740 10 client.26581 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.752+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/2164862552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5637053ca0f0 con 0x563705792ac0 2024-04-20T21:42:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.753+0000 7ff593400640 1 -- 172.21.15.22:0/2164862552 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a4034cb0 con 0x563705792ac0 2024-04-20T21:42:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.753+0000 7ff593400640 1 -- 172.21.15.22:0/2164862552 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7ff598003bf0 con 0x563705792ac0 2024-04-20T21:42:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.753+0000 7ff5b6f8e740 10 client.26581 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.753+0000 7ff5b6f8e740 10 client.26581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.753+0000 7ff5b6f8e740 20 client.26581 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:09.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.753+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/2164862552 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5637057abb40 0x5637057adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.753+0000 7ff5b6f8e740 4 client.26581 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:09.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.753+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/2164862552 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563705705b70 con 0x5637057abb40 2024-04-20T21:42:09.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.753+0000 7ff5b2200640 1 --2- 172.21.15.22:0/2164862552 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5637057abb40 0x5637057adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.754+0000 7ff5b2200640 1 --2- 172.21.15.22:0/2164862552 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5637057abb40 0x5637057adf60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58586429c4cfd35c server_cookie=2a9f9ec70064698b in_seq=0 out_seq=0 2024-04-20T21:42:09.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.754+0000 7ff593400640 10 client.26581 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:09.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff593400640 1 -- 172.21.15.22:0/2164862552 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563705705b70 con 0x5637057abb40 2024-04-20T21:42:09.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff593400640 10 client.26581 handle_command_reply: tid=0 2024-04-20T21:42:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff5b6f8e740 1 client.26581 shutdown 2024-04-20T21:42:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/2164862552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 msgr2=0x563705798c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/2164862552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x563705798c30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff598012730 tx=0x7ff598012bf0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/2164862552 shutdown_connections 2024-04-20T21:42:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/2164862552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x563705799170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/2164862552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x563705798c30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/2164862552 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5637057abb40 0x5637057adf60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/2164862552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x5637057986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/2164862552 >> 172.21.15.22:0/2164862552 conn(0x56370578c8d0 msgr2=0x56370578e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:09.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff593400640 0 client.26581 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:09.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/2164862552 shutdown_connections 2024-04-20T21:42:09.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/2164862552 wait complete. 2024-04-20T21:42:09.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.755+0000 7ff5b6f8e740 20 client.26581 trim_cache size 0 max 16384 2024-04-20T21:42:09.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.774+0000 7ff5b6f8e740 1 Processor -- start 2024-04-20T21:42:09.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.775+0000 7ff5b6f8e740 1 -- start start 2024-04-20T21:42:09.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.775+0000 7ff5b6f8e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x56370579ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.775+0000 7ff5b6f8e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x5637057a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.775+0000 7ff5b6f8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x5637057b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.775+0000 7ff5b6f8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1b2420 con 0x56370579b6a0 2024-04-20T21:42:09.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.775+0000 7ff5b6f8e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1b4c80 con 0x5637057921a0 2024-04-20T21:42:09.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.775+0000 7ff5b6f8e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1b2a70 con 0x563705792ac0 2024-04-20T21:42:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.775+0000 7ff5b2200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x56370579ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.775+0000 7ff5b2200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x56370579ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:37944/0 (socket says 172.21.15.22:37944) 2024-04-20T21:42:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.775+0000 7ff5b2200640 1 -- 172.21.15.22:0/1718552424 learned_addr learned my addr 172.21.15.22:0/1718552424 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.776+0000 7ff5b0e00640 1 --2- 172.21.15.22:0/1718552424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x5637057a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.776+0000 7ff5b2200640 1 --2- 172.21.15.22:0/1718552424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x5637057b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.776+0000 7ff5b2200640 1 -- 172.21.15.22:0/1718552424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 msgr2=0x5637057a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.776+0000 7ff5b2200640 1 --2- 172.21.15.22:0/1718552424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x5637057a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.776+0000 7ff5b2200640 1 -- 172.21.15.22:0/1718552424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 msgr2=0x5637057b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:09.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.776+0000 7ff5b2200640 1 --2- 172.21.15.22:0/1718552424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x5637057b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.776+0000 7ff5b2200640 1 -- 172.21.15.22:0/1718552424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637057ae5c0 con 0x5637057921a0 2024-04-20T21:42:09.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.776+0000 7ff5b0e00640 1 --2- 172.21.15.22:0/1718552424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x5637057a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:09.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.776+0000 7ff5b2200640 1 --2- 172.21.15.22:0/1718552424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x56370579ae20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4005b60 tx=0x7ff5a40056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d99f08f54991a72e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:09.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.776+0000 7ff59e000640 1 -- 172.21.15.22:0/1718552424 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4039040 con 0x5637057921a0 2024-04-20T21:42:09.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.777+0000 7ff59e000640 1 -- 172.21.15.22:0/1718552424 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a4034b90 con 0x5637057921a0 2024-04-20T21:42:09.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.777+0000 7ff59e000640 1 -- 172.21.15.22:0/1718552424 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4043040 con 0x5637057921a0 2024-04-20T21:42:09.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.777+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1718552424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 msgr2=0x56370579ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:09.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.777+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/1718552424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x56370579ae20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4005b60 tx=0x7ff5a40056d0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.777+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1718552424 shutdown_connections 2024-04-20T21:42:09.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.777+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/1718552424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56370579b6a0 0x5637057b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.777+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/1718552424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563705792ac0 0x5637057a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.777+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/1718552424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x56370579ae20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.777+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1718552424 >> 172.21.15.22:0/1718552424 conn(0x56370578c8d0 msgr2=0x56370578b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:09.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.777+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1718552424 shutdown_connections 2024-04-20T21:42:09.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.778+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1718552424 wait complete. 2024-04-20T21:42:09.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.778+0000 7ff5b6f8e740 1 -- start start 2024-04-20T21:42:09.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.778+0000 7ff5b6f8e740 1 Processor -- start 2024-04-20T21:42:09.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b6f8e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x5637057a3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b6f8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563705792ac0 0x5637057a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b6f8e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56370579b6a0 0x5637057a0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b6f8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1b3140 con 0x563705792ac0 2024-04-20T21:42:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b6f8e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1b45b0 con 0x5637057921a0 2024-04-20T21:42:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b6f8e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1b3810 con 0x56370579b6a0 2024-04-20T21:42:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b6f8e740 10 client.0 _ll_register_callbacks cb 0x7ff598009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563705792ac0 0x5637057a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563705792ac0 0x5637057a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47130/0 (socket says 172.21.15.22:47130) 2024-04-20T21:42:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b0e00640 1 -- 172.21.15.22:0/1634468432 learned_addr learned my addr 172.21.15.22:0/1634468432 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b2200640 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x5637057a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b2200640 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56370579b6a0 0x5637057a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b0e00640 1 -- 172.21.15.22:0/1634468432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56370579b6a0 msgr2=0x5637057a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:09.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b0e00640 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56370579b6a0 0x5637057a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b0e00640 1 -- 172.21.15.22:0/1634468432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 msgr2=0x5637057a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:09.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b0e00640 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x5637057a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b0e00640 1 -- 172.21.15.22:0/1634468432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637057aa690 con 0x563705792ac0 2024-04-20T21:42:09.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.779+0000 7ff5b2200640 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x5637057a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:42:09.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.780+0000 7ff5b0e00640 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563705792ac0 0x5637057a3da0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x563705799cc0 tx=0x7ff5980057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd846ca22c157e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:09.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.780+0000 7ff593400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:42:09.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.780+0000 7ff593400640 1 -- 172.21.15.22:0/1634468432 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598003fe0 con 0x563705792ac0 2024-04-20T21:42:09.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.780+0000 7ff593400640 1 -- 172.21.15.22:0/1634468432 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7ff598004180 con 0x563705792ac0 2024-04-20T21:42:09.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.780+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1634468432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a4034b90 con 0x563705792ac0 2024-04-20T21:42:09.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.780+0000 7ff593400640 1 -- 172.21.15.22:0/1634468432 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff598024930 con 0x563705792ac0 2024-04-20T21:42:09.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.780+0000 7ff5b6f8e740 10 client.16416 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:09.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.780+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1634468432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5637058128d0 con 0x563705792ac0 2024-04-20T21:42:09.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.781+0000 7ff593400640 1 -- 172.21.15.22:0/1634468432 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a4034b90 con 0x563705792ac0 2024-04-20T21:42:09.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.781+0000 7ff593400640 1 -- 172.21.15.22:0/1634468432 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7ff598003660 con 0x563705792ac0 2024-04-20T21:42:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.781+0000 7ff5b6f8e740 10 client.16416 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.781+0000 7ff5b6f8e740 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.781+0000 7ff5b6f8e740 20 client.16416 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.781+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563705813080 0x5637058154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.781+0000 7ff5b6f8e740 4 client.16416 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:42:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.781+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1634468432 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5637056ff9e0 con 0x563705813080 2024-04-20T21:42:09.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.781+0000 7ff5b2200640 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563705813080 0x5637058154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:09.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff5b2200640 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563705813080 0x5637058154a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d157f84a877f4c42 server_cookie=128d861bc88b386 in_seq=0 out_seq=0 2024-04-20T21:42:09.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff593400640 10 client.16416 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:09.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff593400640 1 -- 172.21.15.22:0/1634468432 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5637056ff9e0 con 0x563705813080 2024-04-20T21:42:09.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff593400640 10 client.16416 handle_command_reply: tid=0 2024-04-20T21:42:09.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff5b6f8e740 1 client.16416 shutdown 2024-04-20T21:42:09.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1634468432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563705792ac0 msgr2=0x5637057a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:09.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563705792ac0 0x5637057a3da0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x563705799cc0 tx=0x7ff5980057b0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1634468432 shutdown_connections 2024-04-20T21:42:09.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56370579b6a0 0x5637057a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563705792ac0 0x5637057a3da0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x563705813080 0x5637058154a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff5b6f8e740 1 --2- 172.21.15.22:0/1634468432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637057921a0 0x5637057a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff593400640 0 client.16416 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:09.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1634468432 >> 172.21.15.22:0/1634468432 conn(0x56370578c8d0 msgr2=0x563705798690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:09.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.782+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1634468432 shutdown_connections 2024-04-20T21:42:09.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.783+0000 7ff5b6f8e740 1 -- 172.21.15.22:0/1634468432 wait complete. 2024-04-20T21:42:09.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.783+0000 7ff5b6f8e740 20 client.16416 trim_cache size 0 max 16384 2024-04-20T21:42:09.791 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:42:09.791 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:42:09.791 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:42:09.791 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:42:09.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.783+0000 7ff5b4200640 1 -- 172.21.15.22:0/3703311520 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff5940a3da0 msgr2=0x7ff5940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:09.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.783+0000 7ff5b4200640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff5940a3da0 0x7ff5940a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x563705791c70 tx=0x7ff5a0004730 comp rx=0 tx=0).stop 2024-04-20T21:42:09.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.783+0000 7ff5b4200640 1 -- 172.21.15.22:0/3703311520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5ac108590 msgr2=0x7ff5ac19edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:09.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.783+0000 7ff5b4200640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5ac108590 0x7ff5ac19edd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000c420 tx=0x7ff5a0009f90 comp rx=0 tx=0).stop 2024-04-20T21:42:09.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.783+0000 7ff5b4200640 1 -- 172.21.15.22:0/3703311520 shutdown_connections 2024-04-20T21:42:09.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.783+0000 7ff5b4200640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7ff5940a3da0 0x7ff5940a6260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.783+0000 7ff5b4200640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5ac10e600 0x7ff5ac1a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.783+0000 7ff5b4200640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5ac108f50 0x7ff5ac19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.783+0000 7ff5b4200640 1 --2- 172.21.15.22:0/3703311520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5ac108590 0x7ff5ac19edd0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:09.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.783+0000 7ff5b4200640 1 -- 172.21.15.22:0/3703311520 >> 172.21.15.22:0/3703311520 conn(0x7ff5ac0764a0 msgr2=0x7ff5ac077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:09.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.784+0000 7ff5b4200640 1 -- 172.21.15.22:0/3703311520 shutdown_connections 2024-04-20T21:42:09.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:09.784+0000 7ff5b4200640 1 -- 172.21.15.22:0/3703311520 wait complete. 2024-04-20T21:42:09.800 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:42:09.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.013+0000 7f2ca5e00640 1 -- 172.21.15.22:0/566929376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca0104b80 msgr2=0x7f2ca0108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.013+0000 7f2ca5e00640 1 --2- 172.21.15.22:0/566929376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca0104b80 0x7f2ca0108bc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009900 tx=0x7f2c8c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:42:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.013+0000 7f2ca5e00640 1 -- 172.21.15.22:0/566929376 shutdown_connections 2024-04-20T21:42:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.013+0000 7f2ca5e00640 1 --2- 172.21.15.22:0/566929376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca01092a0 0x7f2ca0110e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.013+0000 7f2ca5e00640 1 --2- 172.21.15.22:0/566929376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca0104b80 0x7f2ca0108bc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.013+0000 7f2ca5e00640 1 --2- 172.21.15.22:0/566929376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ca01041d0 0x7f2ca01045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.013+0000 7f2ca5e00640 1 -- 172.21.15.22:0/566929376 >> 172.21.15.22:0/566929376 conn(0x7f2ca00ffa30 msgr2=0x7f2ca0101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.013+0000 7f2ca5e00640 1 -- 172.21.15.22:0/566929376 shutdown_connections 2024-04-20T21:42:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.014+0000 7f2ca5e00640 1 -- 172.21.15.22:0/566929376 wait complete. 2024-04-20T21:42:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.014+0000 7f2ca5e00640 1 Processor -- start 2024-04-20T21:42:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.015+0000 7f2ca5e00640 1 -- start start 2024-04-20T21:42:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.015+0000 7f2ca5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca01041d0 0x7f2ca01a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.015+0000 7f2ca5e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca01092a0 0x7f2ca01a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.015+0000 7f2ca5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ca019a7b0 0x7f2ca019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.015+0000 7f2ca5e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0114330 con 0x7f2ca019a7b0 2024-04-20T21:42:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.015+0000 7f2ca5e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca01141b0 con 0x7f2ca01092a0 2024-04-20T21:42:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.015+0000 7f2ca5e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca01144b0 con 0x7f2ca01041d0 2024-04-20T21:42:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.015+0000 7f2ca5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ca019a7b0 0x7f2ca019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.015+0000 7f2ca5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ca019a7b0 0x7f2ca019ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47144/0 (socket says 172.21.15.22:47144) 2024-04-20T21:42:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.015+0000 7f2ca5400640 1 -- 172.21.15.22:0/1127583283 learned_addr learned my addr 172.21.15.22:0/1127583283 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.015+0000 7f2ca4a00640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca01041d0 0x7f2ca01a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.015+0000 7f2c9fe00640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca01092a0 0x7f2ca01a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.016+0000 7f2ca5400640 1 -- 172.21.15.22:0/1127583283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca01041d0 msgr2=0x7f2ca01a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.016+0000 7f2ca5400640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca01041d0 0x7f2ca01a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.016+0000 7f2ca5400640 1 -- 172.21.15.22:0/1127583283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca01092a0 msgr2=0x7f2ca01a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.016+0000 7f2ca5400640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca01092a0 0x7f2ca01a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.016+0000 7f2ca5400640 1 -- 172.21.15.22:0/1127583283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca019b390 con 0x7f2ca019a7b0 2024-04-20T21:42:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.016+0000 7f2ca4a00640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca01041d0 0x7f2ca01a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.016+0000 7f2ca5400640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ca019a7b0 0x7f2ca019ac60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800eb10 tx=0x7f2c9800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.017+0000 7f2c9d600640 1 -- 172.21.15.22:0/1127583283 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98004b20 con 0x7f2ca019a7b0 2024-04-20T21:42:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.017+0000 7f2c9d600640 1 -- 172.21.15.22:0/1127583283 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c98004cc0 con 0x7f2ca019a7b0 2024-04-20T21:42:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.017+0000 7f2ca5e00640 1 -- 172.21.15.22:0/1127583283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca01a7470 con 0x7f2ca019a7b0 2024-04-20T21:42:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.017+0000 7f2c9d600640 1 -- 172.21.15.22:0/1127583283 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98010760 con 0x7f2ca019a7b0 2024-04-20T21:42:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.017+0000 7f2ca5e00640 1 -- 172.21.15.22:0/1127583283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca01a7820 con 0x7f2ca019a7b0 2024-04-20T21:42:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.018+0000 7f2ca5e00640 1 -- 172.21.15.22:0/1127583283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca01056a0 con 0x7f2ca019a7b0 2024-04-20T21:42:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.020+0000 7f2c9d600640 1 -- 172.21.15.22:0/1127583283 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2c98005020 con 0x7f2ca019a7b0 2024-04-20T21:42:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.021+0000 7f2c9d600640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2c880a3d20 0x7f2c880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.022+0000 7f2c9d600640 1 -- 172.21.15.22:0/1127583283 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7f2c980d0ea0 con 0x7f2ca019a7b0 2024-04-20T21:42:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.022+0000 7f2ca4a00640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2c880a3d20 0x7f2c880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.023+0000 7f2ca4a00640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2c880a3d20 0x7f2c880a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009fc0 tx=0x7f2c94005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:10.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.026+0000 7f2c9d600640 1 -- 172.21.15.22:0/1127583283 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2c9809d460 con 0x7f2ca019a7b0 2024-04-20T21:42:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.277+0000 7f2ca5e00640 1 -- 172.21.15.22:0/1127583283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ca010c4d0 con 0x7f2ca019a7b0 2024-04-20T21:42:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.278+0000 7f2c9d600640 1 -- 172.21.15.22:0/1127583283 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f2c9809cd60 con 0x7f2ca019a7b0 2024-04-20T21:42:10.277 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:42:10.278 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:42:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:42:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.281+0000 7f2ca5e00640 1 -- 172.21.15.22:0/1127583283 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2c880a3d20 msgr2=0x7f2c880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.281+0000 7f2ca5e00640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2c880a3d20 0x7f2c880a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009fc0 tx=0x7f2c94005f90 comp rx=0 tx=0).stop 2024-04-20T21:42:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.281+0000 7f2ca5e00640 1 -- 172.21.15.22:0/1127583283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ca019a7b0 msgr2=0x7f2ca019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.281+0000 7f2ca5e00640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ca019a7b0 0x7f2ca019ac60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800eb10 tx=0x7f2c9800c2d0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.281+0000 7f2ca5e00640 1 -- 172.21.15.22:0/1127583283 shutdown_connections 2024-04-20T21:42:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.281+0000 7f2ca5e00640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2c880a3d20 0x7f2c880a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.281+0000 7f2ca5e00640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ca019a7b0 0x7f2ca019ac60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.281+0000 7f2ca5e00640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca01092a0 0x7f2ca01a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.281+0000 7f2ca5e00640 1 --2- 172.21.15.22:0/1127583283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca01041d0 0x7f2ca01a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.281+0000 7f2ca5e00640 1 -- 172.21.15.22:0/1127583283 >> 172.21.15.22:0/1127583283 conn(0x7f2ca00ffa30 msgr2=0x7f2ca0101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.281+0000 7f2ca5e00640 1 -- 172.21.15.22:0/1127583283 shutdown_connections 2024-04-20T21:42:10.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.281+0000 7f2ca5e00640 1 -- 172.21.15.22:0/1127583283 wait complete. 2024-04-20T21:42:10.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:42:10.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.519+0000 7f0b3f200640 1 -- 172.21.15.22:0/904311123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b40102090 msgr2=0x7f0b40102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.519+0000 7f0b3f200640 1 --2- 172.21.15.22:0/904311123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b40102090 0x7f0b40102470 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009900 tx=0x7f0b3002bc30 comp rx=0 tx=0).stop 2024-04-20T21:42:10.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.520+0000 7f0b3f200640 1 -- 172.21.15.22:0/904311123 shutdown_connections 2024-04-20T21:42:10.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.520+0000 7f0b3f200640 1 --2- 172.21.15.22:0/904311123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b40107160 0x7f0b4010ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.520+0000 7f0b3f200640 1 --2- 172.21.15.22:0/904311123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b40102a40 0x7f0b40106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.520+0000 7f0b3f200640 1 --2- 172.21.15.22:0/904311123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b40102090 0x7f0b40102470 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.520+0000 7f0b3f200640 1 -- 172.21.15.22:0/904311123 >> 172.21.15.22:0/904311123 conn(0x7f0b400fd930 msgr2=0x7f0b400ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:10.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.520+0000 7f0b3f200640 1 -- 172.21.15.22:0/904311123 shutdown_connections 2024-04-20T21:42:10.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.520+0000 7f0b3f200640 1 -- 172.21.15.22:0/904311123 wait complete. 2024-04-20T21:42:10.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.521+0000 7f0b3f200640 1 Processor -- start 2024-04-20T21:42:10.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.521+0000 7f0b3f200640 1 -- start start 2024-04-20T21:42:10.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.522+0000 7f0b3f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b40102090 0x7f0b401a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.522+0000 7f0b3f200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b40107160 0x7f0b401a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.522+0000 7f0b3f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b4019a8c0 0x7f0b4019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.522+0000 7f0b3f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0b40112260 con 0x7f0b40102090 2024-04-20T21:42:10.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.522+0000 7f0b3f200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401120e0 con 0x7f0b40107160 2024-04-20T21:42:10.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.522+0000 7f0b3f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401123e0 con 0x7f0b4019a8c0 2024-04-20T21:42:10.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.522+0000 7f0b3de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b40102090 0x7f0b401a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.522+0000 7f0b3de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b40102090 0x7f0b401a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47154/0 (socket says 172.21.15.22:47154) 2024-04-20T21:42:10.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.522+0000 7f0b3de00640 1 -- 172.21.15.22:0/364492553 learned_addr learned my addr 172.21.15.22:0/364492553 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:10.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.522+0000 7f0b3d400640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b40107160 0x7f0b401a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.522+0000 7f0b3e800640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b4019a8c0 0x7f0b4019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.523+0000 7f0b3de00640 1 -- 172.21.15.22:0/364492553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b4019a8c0 msgr2=0x7f0b4019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.523+0000 7f0b3de00640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b4019a8c0 0x7f0b4019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.523+0000 7f0b3de00640 1 -- 172.21.15.22:0/364492553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b40107160 msgr2=0x7f0b401a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.523+0000 7f0b3de00640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b40107160 0x7f0b401a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.523+0000 7f0b3de00640 1 -- 172.21.15.22:0/364492553 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4019b3e0 con 0x7f0b40102090 2024-04-20T21:42:10.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.523+0000 7f0b3d400640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b40107160 0x7f0b401a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:10.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.523+0000 7f0b3de00640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b40102090 0x7f0b401a0620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0b3002c240 tx=0x7f0b300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:10.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.523+0000 7f0b36a00640 1 -- 172.21.15.22:0/364492553 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30043020 con 0x7f0b40102090 2024-04-20T21:42:10.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.523+0000 7f0b3f200640 1 -- 172.21.15.22:0/364492553 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4019b670 con 0x7f0b40102090 2024-04-20T21:42:10.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.523+0000 7f0b36a00640 1 -- 172.21.15.22:0/364492553 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b300054e0 con 0x7f0b40102090 2024-04-20T21:42:10.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.523+0000 7f0b36a00640 1 -- 172.21.15.22:0/364492553 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30003b50 con 0x7f0b40102090 2024-04-20T21:42:10.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.524+0000 7f0b3f200640 1 -- 172.21.15.22:0/364492553 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b401a77d0 con 0x7f0b40102090 2024-04-20T21:42:10.526 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:10 smithi122 ceph-mon[22147]: pgmap v258: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:42:10.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.527+0000 7f0b47ddb740 1 Processor -- start 2024-04-20T21:42:10.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.527+0000 7f0b47ddb740 1 -- start start 2024-04-20T21:42:10.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.527+0000 7f0b36a00640 1 -- 172.21.15.22:0/364492553 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0b30046020 con 0x7f0b40102090 2024-04-20T21:42:10.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.527+0000 7f0b47ddb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407b3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.527+0000 7f0b47ddb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3a90 0x5639407bc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.527+0000 7f0b47ddb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407bc670 0x5639407bea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.527+0000 7f0b47ddb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563940718050 con 0x5639407bc670 2024-04-20T21:42:10.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.527+0000 7f0b47ddb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5639406be490 con 0x5639407b3170 2024-04-20T21:42:10.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.527+0000 7f0b47ddb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563940714010 con 0x5639407b3a90 2024-04-20T21:42:10.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.528+0000 7f0b3e800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.528+0000 7f0b3d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407bc670 0x5639407bea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.528+0000 7f0b3de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3a90 0x5639407bc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.528+0000 7f0b3e800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38006/0 (socket says 172.21.15.22:38006) 2024-04-20T21:42:10.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.528+0000 7f0b3e800640 1 -- 172.21.15.22:0/6737213 learned_addr learned my addr 172.21.15.22:0/6737213 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:10.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.528+0000 7f0b3e800640 1 -- 172.21.15.22:0/6737213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3a90 msgr2=0x5639407bc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:10.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.528+0000 7f0b3e800640 1 --2- 172.21.15.22:0/6737213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3a90 0x5639407bc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.528+0000 7f0b3e800640 1 -- 172.21.15.22:0/6737213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407bc670 msgr2=0x5639407bea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:10.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.528+0000 7f0b3e800640 1 --2- 172.21.15.22:0/6737213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407bc670 0x5639407bea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.528+0000 7f0b3e800640 1 -- 172.21.15.22:0/6737213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639406c6070 con 0x5639407b3170 2024-04-20T21:42:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.528+0000 7f0b3d400640 1 --2- 172.21.15.22:0/6737213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407bc670 0x5639407bea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.529+0000 7f0b3e800640 1 --2- 172.21.15.22:0/6737213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407b3550 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009c20 tx=0x7f0b38011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ca4d76e81f54e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.529+0000 7f0b36a00640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0b200a3d20 0x7f0b200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.529+0000 7f0b2f400640 1 -- 172.21.15.22:0/6737213 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3801e070 con 0x5639407b3170 2024-04-20T21:42:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.529+0000 7f0b2f400640 1 -- 172.21.15.22:0/6737213 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b38019b10 con 0x5639407b3170 2024-04-20T21:42:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.529+0000 7f0b2f400640 1 -- 172.21.15.22:0/6737213 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3800ea50 con 0x5639407b3170 2024-04-20T21:42:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.529+0000 7f0b36a00640 1 -- 172.21.15.22:0/364492553 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7f0b300f27a0 con 0x7f0b40102090 2024-04-20T21:42:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.529+0000 7f0b3de00640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0b200a3d20 0x7f0b200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.530+0000 7f0b47ddb740 1 -- 172.21.15.22:0/6737213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 msgr2=0x5639407b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.530+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/6737213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407b3550 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009c20 tx=0x7f0b38011810 comp rx=0 tx=0).stop 2024-04-20T21:42:10.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.530+0000 7f0b3de00640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0b200a3d20 0x7f0b200a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0b300f0030 tx=0x7f0b300f8550 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:10.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.530+0000 7f0b47ddb740 1 -- 172.21.15.22:0/6737213 shutdown_connections 2024-04-20T21:42:10.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.530+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/6737213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407bc670 0x5639407bea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.530+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/6737213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3a90 0x5639407bc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.530+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/6737213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407b3550 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.530+0000 7f0b47ddb740 1 -- 172.21.15.22:0/6737213 >> 172.21.15.22:0/6737213 conn(0x5639407ad8a0 msgr2=0x5639407adc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.530+0000 7f0b47ddb740 1 -- 172.21.15.22:0/6737213 shutdown_connections 2024-04-20T21:42:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.530+0000 7f0b47ddb740 1 -- 172.21.15.22:0/6737213 wait complete. 2024-04-20T21:42:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.531+0000 7f0b47ddb740 1 -- start start 2024-04-20T21:42:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.531+0000 7f0b47ddb740 1 Processor -- start 2024-04-20T21:42:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.531+0000 7f0b47ddb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.531+0000 7f0b47ddb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3a90 0x5639407b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.531+0000 7f0b47ddb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407bc670 0x5639407ba190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.531+0000 7f0b47ddb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5639407bf460 con 0x5639407bc670 2024-04-20T21:42:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.531+0000 7f0b47ddb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5639406d1860 con 0x5639407b3170 2024-04-20T21:42:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.531+0000 7f0b47ddb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5639407bf5e0 con 0x5639407b3a90 2024-04-20T21:42:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.531+0000 7f0b47ddb740 10 client.0 _ll_register_callbacks cb 0x56394069ffe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.532+0000 7f0b3d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3a90 0x5639407b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.532+0000 7f0b3e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407bc670 0x5639407ba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.532+0000 7f0b3d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3a90 0x5639407b9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50812/0 (socket says 172.21.15.22:50812) 2024-04-20T21:42:10.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.532+0000 7f0b3d400640 1 -- 172.21.15.22:0/523892051 learned_addr learned my addr 172.21.15.22:0/523892051 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:10.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.532+0000 7f0b3e800640 1 --2- 172.21.15.22:0/523892051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.532+0000 7f0b3d400640 1 -- 172.21.15.22:0/523892051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 msgr2=0x5639407b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:10.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.532+0000 7f0b3d400640 1 --2- 172.21.15.22:0/523892051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407b9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.532+0000 7f0b3d400640 1 -- 172.21.15.22:0/523892051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407bc670 msgr2=0x5639407ba190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:10.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.532+0000 7f0b3d400640 1 --2- 172.21.15.22:0/523892051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407bc670 0x5639407ba190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.532+0000 7f0b3d400640 1 -- 172.21.15.22:0/523892051 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639406d5950 con 0x5639407b3a90 2024-04-20T21:42:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.532+0000 7f0b3d400640 1 --2- 172.21.15.22:0/523892051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3a90 0x5639407b9c50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0b28011d30 tx=0x7f0b28013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b015e0701852d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.532+0000 7f0b27e00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:42:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.533+0000 7f0b27e00640 1 -- 172.21.15.22:0/523892051 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b280107e0 con 0x5639407b3a90 2024-04-20T21:42:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.533+0000 7f0b47ddb740 1 -- 172.21.15.22:0/523892051 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b38019b10 con 0x5639407b3a90 2024-04-20T21:42:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.533+0000 7f0b27e00640 1 -- 172.21.15.22:0/523892051 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7f0b28010980 con 0x5639407b3a90 2024-04-20T21:42:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.533+0000 7f0b27e00640 1 -- 172.21.15.22:0/523892051 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0b28010b80 con 0x5639407b3a90 2024-04-20T21:42:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.533+0000 7f0b47ddb740 10 client.26605 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.533+0000 7f0b47ddb740 1 -- 172.21.15.22:0/523892051 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5639406a3820 con 0x5639407b3a90 2024-04-20T21:42:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.533+0000 7f0b27e00640 1 -- 172.21.15.22:0/523892051 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b38019b10 con 0x5639407b3a90 2024-04-20T21:42:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.534+0000 7f0b27e00640 1 -- 172.21.15.22:0/523892051 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f0b2801a070 con 0x5639407b3a90 2024-04-20T21:42:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.534+0000 7f0b47ddb740 10 client.26605 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.534+0000 7f0b47ddb740 10 client.26605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.534+0000 7f0b47ddb740 20 client.26605 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.534+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/523892051 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5639407cc760 0x5639407ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.534+0000 7f0b47ddb740 4 client.26605 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.534+0000 7f0b47ddb740 1 -- 172.21.15.22:0/523892051 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563940725a70 con 0x5639407cc760 2024-04-20T21:42:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.534+0000 7f0b3e800640 1 --2- 172.21.15.22:0/523892051 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5639407cc760 0x5639407ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.535+0000 7f0b3e800640 1 --2- 172.21.15.22:0/523892051 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5639407cc760 0x5639407ceb80 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e813256ee20d5c server_cookie=9bef8f4ca9963ec1 in_seq=0 out_seq=0 2024-04-20T21:42:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.535+0000 7f0b27e00640 10 client.26605 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b27e00640 1 -- 172.21.15.22:0/523892051 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563940725a70 con 0x5639407cc760 2024-04-20T21:42:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b27e00640 10 client.26605 handle_command_reply: tid=0 2024-04-20T21:42:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b47ddb740 1 client.26605 shutdown 2024-04-20T21:42:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b47ddb740 1 -- 172.21.15.22:0/523892051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3a90 msgr2=0x5639407b9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/523892051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3a90 0x5639407b9c50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0b28011d30 tx=0x7f0b28013830 comp rx=0 tx=0).stop 2024-04-20T21:42:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b47ddb740 1 -- 172.21.15.22:0/523892051 shutdown_connections 2024-04-20T21:42:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/523892051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407bc670 0x5639407ba190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/523892051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3a90 0x5639407b9c50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/523892051 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5639407cc760 0x5639407ceb80 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/523892051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b47ddb740 1 -- 172.21.15.22:0/523892051 >> 172.21.15.22:0/523892051 conn(0x5639407ad8a0 msgr2=0x5639407af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b27e00640 0 client.26605 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b47ddb740 1 -- 172.21.15.22:0/523892051 shutdown_connections 2024-04-20T21:42:10.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b47ddb740 1 -- 172.21.15.22:0/523892051 wait complete. 2024-04-20T21:42:10.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.536+0000 7f0b47ddb740 20 client.26605 trim_cache size 0 max 16384 2024-04-20T21:42:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.551+0000 7f0b47ddb740 1 Processor -- start 2024-04-20T21:42:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.551+0000 7f0b47ddb740 1 -- start start 2024-04-20T21:42:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.551+0000 7f0b47ddb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407bbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.551+0000 7f0b47ddb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 0x5639407c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.551+0000 7f0b47ddb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407bc670 0x5639407b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.551+0000 7f0b47ddb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401adf60 con 0x5639407b3a90 2024-04-20T21:42:10.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.551+0000 7f0b47ddb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401b07c0 con 0x5639407b3170 2024-04-20T21:42:10.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.551+0000 7f0b47ddb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401ae5b0 con 0x5639407bc670 2024-04-20T21:42:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.552+0000 7f0b3d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 0x5639407c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.552+0000 7f0b3d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 0x5639407c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47166/0 (socket says 172.21.15.22:47166) 2024-04-20T21:42:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.552+0000 7f0b3d400640 1 -- 172.21.15.22:0/982774099 learned_addr learned my addr 172.21.15.22:0/982774099 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.552+0000 7f0b3e800640 1 --2- 172.21.15.22:0/982774099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407bbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.552+0000 7f0b3e800640 1 --2- 172.21.15.22:0/982774099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407bc670 0x5639407b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.552+0000 7f0b3d400640 1 -- 172.21.15.22:0/982774099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407bc670 msgr2=0x5639407b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.552+0000 7f0b3d400640 1 --2- 172.21.15.22:0/982774099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407bc670 0x5639407b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.552+0000 7f0b3d400640 1 -- 172.21.15.22:0/982774099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 msgr2=0x5639407bbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.552+0000 7f0b3d400640 1 --2- 172.21.15.22:0/982774099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407bbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.552+0000 7f0b3d400640 1 -- 172.21.15.22:0/982774099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56394042a3f0 con 0x5639407b3a90 2024-04-20T21:42:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.552+0000 7f0b3e800640 1 --2- 172.21.15.22:0/982774099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407bbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:42:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.553+0000 7f0b3d400640 1 --2- 172.21.15.22:0/982774099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 0x5639407c8320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0b28002230 tx=0x7f0b28009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5602c3d248e384 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.553+0000 7f0b2ea00640 1 -- 172.21.15.22:0/982774099 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28031070 con 0x5639407b3a90 2024-04-20T21:42:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.553+0000 7f0b2ea00640 1 -- 172.21.15.22:0/982774099 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b280049a0 con 0x5639407b3a90 2024-04-20T21:42:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.553+0000 7f0b2ea00640 1 -- 172.21.15.22:0/982774099 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2801c9d0 con 0x5639407b3a90 2024-04-20T21:42:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.554+0000 7f0b47ddb740 1 -- 172.21.15.22:0/982774099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 msgr2=0x5639407c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.554+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/982774099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 0x5639407c8320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0b28002230 tx=0x7f0b28009ee0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.554+0000 7f0b47ddb740 1 -- 172.21.15.22:0/982774099 shutdown_connections 2024-04-20T21:42:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.554+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/982774099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407bc670 0x5639407b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.554+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/982774099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 0x5639407c8320 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.554+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/982774099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407b3170 0x5639407bbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.554+0000 7f0b47ddb740 1 -- 172.21.15.22:0/982774099 >> 172.21.15.22:0/982774099 conn(0x5639407ad8a0 msgr2=0x5639407b96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.554+0000 7f0b47ddb740 1 -- 172.21.15.22:0/982774099 shutdown_connections 2024-04-20T21:42:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.554+0000 7f0b47ddb740 1 -- 172.21.15.22:0/982774099 wait complete. 2024-04-20T21:42:10.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.554+0000 7f0b47ddb740 1 -- start start 2024-04-20T21:42:10.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.554+0000 7f0b47ddb740 1 Processor -- start 2024-04-20T21:42:10.557 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:10 smithi033 ceph-mon[21626]: pgmap v258: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:42:10.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b47ddb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3170 0x5639407c0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b47ddb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 0x5639407c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b47ddb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407bc670 0x563940831ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b47ddb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401aec80 con 0x5639407b3a90 2024-04-20T21:42:10.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b47ddb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401b00f0 con 0x5639407bc670 2024-04-20T21:42:10.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b47ddb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401af350 con 0x5639407b3170 2024-04-20T21:42:10.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b47ddb740 10 client.0 _ll_register_callbacks cb 0x7f0b28011970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:10.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b3d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 0x5639407c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b3d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 0x5639407c0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47182/0 (socket says 172.21.15.22:47182) 2024-04-20T21:42:10.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b3d400640 1 -- 172.21.15.22:0/1926205421 learned_addr learned my addr 172.21.15.22:0/1926205421 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:10.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b3e800640 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3170 0x5639407c0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b3d400640 1 -- 172.21.15.22:0/1926205421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3170 msgr2=0x5639407c0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:10.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b3d400640 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3170 0x5639407c0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.555+0000 7f0b3d400640 1 -- 172.21.15.22:0/1926205421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407bc670 msgr2=0x563940831ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:10.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.556+0000 7f0b3e800640 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407bc670 0x563940831ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.556+0000 7f0b3d400640 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407bc670 0x563940831ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.556+0000 7f0b3d400640 1 -- 172.21.15.22:0/1926205421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639404f47d0 con 0x5639407b3a90 2024-04-20T21:42:10.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.556+0000 7f0b3e800640 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407bc670 0x563940831ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:42:10.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.556+0000 7f0b3d400640 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 0x5639407c0e50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0b2802f040 tx=0x7f0b2801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc17d6565195aac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:10.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.556+0000 7f0b27e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:42:10.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.556+0000 7f0b27e00640 1 -- 172.21.15.22:0/1926205421 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28031040 con 0x5639407b3a90 2024-04-20T21:42:10.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.556+0000 7f0b27e00640 1 -- 172.21.15.22:0/1926205421 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7f0b280105e0 con 0x5639407b3a90 2024-04-20T21:42:10.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.556+0000 7f0b47ddb740 1 -- 172.21.15.22:0/1926205421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b280049a0 con 0x5639407b3a90 2024-04-20T21:42:10.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.557+0000 7f0b27e00640 1 -- 172.21.15.22:0/1926205421 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0b28045050 con 0x5639407b3a90 2024-04-20T21:42:10.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.557+0000 7f0b47ddb740 10 client.16440 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:10.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.557+0000 7f0b47ddb740 1 -- 172.21.15.22:0/1926205421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5639408332b0 con 0x5639407b3a90 2024-04-20T21:42:10.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.557+0000 7f0b27e00640 1 -- 172.21.15.22:0/1926205421 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b280049a0 con 0x5639407b3a90 2024-04-20T21:42:10.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.557+0000 7f0b27e00640 1 -- 172.21.15.22:0/1926205421 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f0b2801a050 con 0x5639407b3a90 2024-04-20T21:42:10.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.557+0000 7f0b47ddb740 10 client.16440 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:10.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.557+0000 7f0b47ddb740 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:10.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.557+0000 7f0b47ddb740 20 client.16440 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:10.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.557+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5639408339b0 0x563940835dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:10.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.557+0000 7f0b47ddb740 4 client.16440 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:42:10.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.557+0000 7f0b47ddb740 1 -- 172.21.15.22:0/1926205421 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56394071f8e0 con 0x5639408339b0 2024-04-20T21:42:10.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.558+0000 7f0b3e800640 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5639408339b0 0x563940835dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:10.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.558+0000 7f0b3e800640 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5639408339b0 0x563940835dd0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=584783899cfcc365 server_cookie=d295a7c1741deaee in_seq=0 out_seq=0 2024-04-20T21:42:10.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.558+0000 7f0b27e00640 10 client.16440 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:10.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.558+0000 7f0b27e00640 1 -- 172.21.15.22:0/1926205421 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56394071f8e0 con 0x5639408339b0 2024-04-20T21:42:10.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.558+0000 7f0b27e00640 10 client.16440 handle_command_reply: tid=0 2024-04-20T21:42:10.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.558+0000 7f0b47ddb740 1 client.16440 shutdown 2024-04-20T21:42:10.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.559+0000 7f0b47ddb740 1 -- 172.21.15.22:0/1926205421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 msgr2=0x5639407c0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.559+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 0x5639407c0e50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0b2802f040 tx=0x7f0b2801c4c0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.559+0000 7f0b47ddb740 1 -- 172.21.15.22:0/1926205421 shutdown_connections 2024-04-20T21:42:10.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.559+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5639408339b0 0x563940835dd0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.559+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639407bc670 0x563940831ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.559+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639407b3a90 0x5639407c0e50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.559+0000 7f0b27e00640 0 client.16440 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:10.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.559+0000 7f0b47ddb740 1 --2- 172.21.15.22:0/1926205421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639407b3170 0x5639407c0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.559+0000 7f0b47ddb740 1 -- 172.21.15.22:0/1926205421 >> 172.21.15.22:0/1926205421 conn(0x5639407ad8a0 msgr2=0x5639407b9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:10.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.559+0000 7f0b47ddb740 1 -- 172.21.15.22:0/1926205421 shutdown_connections 2024-04-20T21:42:10.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.559+0000 7f0b47ddb740 1 -- 172.21.15.22:0/1926205421 wait complete. 2024-04-20T21:42:10.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.559+0000 7f0b47ddb740 20 client.16440 trim_cache size 0 max 16384 2024-04-20T21:42:10.567 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:42:10.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.560+0000 7f0b3f200640 1 -- 172.21.15.22:0/364492553 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0b200a3d20 msgr2=0x7f0b200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.560+0000 7f0b3f200640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0b200a3d20 0x7f0b200a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0b300f0030 tx=0x7f0b300f8550 comp rx=0 tx=0).stop 2024-04-20T21:42:10.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.560+0000 7f0b3f200640 1 -- 172.21.15.22:0/364492553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b40102090 msgr2=0x7f0b401a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:10.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.560+0000 7f0b3f200640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b40102090 0x7f0b401a0620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0b3002c240 tx=0x7f0b300042f0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.560+0000 7f0b3f200640 1 -- 172.21.15.22:0/364492553 shutdown_connections 2024-04-20T21:42:10.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.560+0000 7f0b3f200640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0b200a3d20 0x7f0b200a61e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.560+0000 7f0b3f200640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b4019a8c0 0x7f0b4019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.560+0000 7f0b3f200640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b40107160 0x7f0b401a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.560+0000 7f0b3f200640 1 --2- 172.21.15.22:0/364492553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b40102090 0x7f0b401a0620 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:10.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.560+0000 7f0b3f200640 1 -- 172.21.15.22:0/364492553 >> 172.21.15.22:0/364492553 conn(0x7f0b400fd930 msgr2=0x7f0b400ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:10.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.560+0000 7f0b3f200640 1 -- 172.21.15.22:0/364492553 shutdown_connections 2024-04-20T21:42:10.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:10.560+0000 7f0b3f200640 1 -- 172.21.15.22:0/364492553 wait complete. 2024-04-20T21:42:10.574 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:10 smithi022 ceph-mon[16843]: pgmap v258: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:42:11.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:11 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1127583283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:42:11.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:11 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:42:11.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:11 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1127583283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:42:11.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:11 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:42:11.579 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:42:11.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:42:11.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:42:11.646 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:11 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1127583283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:42:11.646 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:11 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:42:11.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.788+0000 7fe34dc00640 1 -- 172.21.15.22:0/633871451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe348102070 msgr2=0x7fe348102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:11.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.788+0000 7fe34dc00640 1 --2- 172.21.15.22:0/633871451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe348102070 0x7fe348102450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe338009900 tx=0x7fe33802bc50 comp rx=0 tx=0).stop 2024-04-20T21:42:11.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.788+0000 7fe34dc00640 1 -- 172.21.15.22:0/633871451 shutdown_connections 2024-04-20T21:42:11.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.788+0000 7fe34dc00640 1 --2- 172.21.15.22:0/633871451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe348107140 0x7fe34810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.788+0000 7fe34dc00640 1 --2- 172.21.15.22:0/633871451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe348102a20 0x7fe348106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.788+0000 7fe34dc00640 1 --2- 172.21.15.22:0/633871451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe348102070 0x7fe348102450 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.788+0000 7fe34dc00640 1 -- 172.21.15.22:0/633871451 >> 172.21.15.22:0/633871451 conn(0x7fe3480fd930 msgr2=0x7fe3480ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:11.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.788+0000 7fe34dc00640 1 -- 172.21.15.22:0/633871451 shutdown_connections 2024-04-20T21:42:11.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.788+0000 7fe34dc00640 1 -- 172.21.15.22:0/633871451 wait complete. 2024-04-20T21:42:11.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.789+0000 7fe34dc00640 1 Processor -- start 2024-04-20T21:42:11.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.790+0000 7fe34dc00640 1 -- start start 2024-04-20T21:42:11.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.790+0000 7fe34dc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe348102070 0x7fe3481a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.790+0000 7fe34dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe348107140 0x7fe3481a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.790+0000 7fe34dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe34819a800 0x7fe34819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.790+0000 7fe34dc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe348112200 con 0x7fe34819a800 2024-04-20T21:42:11.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.790+0000 7fe34dc00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe348112080 con 0x7fe348102070 2024-04-20T21:42:11.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.790+0000 7fe34dc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe348112380 con 0x7fe348107140 2024-04-20T21:42:11.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.790+0000 7fe34d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe34819a800 0x7fe34819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.790+0000 7fe347e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe348102070 0x7fe3481a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.790+0000 7fe347400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe348107140 0x7fe3481a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.790+0000 7fe34d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe34819a800 0x7fe34819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47196/0 (socket says 172.21.15.22:47196) 2024-04-20T21:42:11.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.790+0000 7fe34d200640 1 -- 172.21.15.22:0/970399707 learned_addr learned my addr 172.21.15.22:0/970399707 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:11.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.791+0000 7fe347e00640 1 -- 172.21.15.22:0/970399707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe348107140 msgr2=0x7fe3481a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:11.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.791+0000 7fe347e00640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe348107140 0x7fe3481a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.791+0000 7fe347e00640 1 -- 172.21.15.22:0/970399707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe34819a800 msgr2=0x7fe34819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:11.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.791+0000 7fe347e00640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe34819a800 0x7fe34819acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.791+0000 7fe347e00640 1 -- 172.21.15.22:0/970399707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34819b390 con 0x7fe348102070 2024-04-20T21:42:11.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.791+0000 7fe347400640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe348107140 0x7fe3481a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:11.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.791+0000 7fe347e00640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe348102070 0x7fe3481a0690 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe338036040 tx=0x7fe33802cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:11.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.792+0000 7fe344c00640 1 -- 172.21.15.22:0/970399707 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338042020 con 0x7fe348102070 2024-04-20T21:42:11.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.792+0000 7fe344c00640 1 -- 172.21.15.22:0/970399707 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe338004450 con 0x7fe348102070 2024-04-20T21:42:11.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.792+0000 7fe34dc00640 1 -- 172.21.15.22:0/970399707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34819b620 con 0x7fe348102070 2024-04-20T21:42:11.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.792+0000 7fe344c00640 1 -- 172.21.15.22:0/970399707 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338003d90 con 0x7fe348102070 2024-04-20T21:42:11.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.792+0000 7fe34dc00640 1 -- 172.21.15.22:0/970399707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3481a7850 con 0x7fe348102070 2024-04-20T21:42:11.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.796+0000 7fe351e36740 1 Processor -- start 2024-04-20T21:42:11.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.797+0000 7fe351e36740 1 -- start start 2024-04-20T21:42:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.797+0000 7fe344c00640 1 -- 172.21.15.22:0/970399707 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe338045020 con 0x7fe348102070 2024-04-20T21:42:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.797+0000 7fe351e36740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97ae0f0 0x55cdc97ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.797+0000 7fe351e36740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.797+0000 7fe351e36740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.797+0000 7fe351e36740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cdc9712d20 con 0x55cdc97ae0f0 2024-04-20T21:42:11.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.797+0000 7fe351e36740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cdc96b93e0 con 0x55cdc97b75f0 2024-04-20T21:42:11.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.797+0000 7fe351e36740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cdc970ece0 con 0x55cdc97aea10 2024-04-20T21:42:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.797+0000 7fe34d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97ae0f0 0x55cdc97ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.798+0000 7fe347400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.798+0000 7fe347e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.798+0000 7fe347400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97b99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38062/0 (socket says 172.21.15.22:38062) 2024-04-20T21:42:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.798+0000 7fe347400640 1 -- 172.21.15.22:0/1122274334 learned_addr learned my addr 172.21.15.22:0/1122274334 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.798+0000 7fe347e00640 1 -- 172.21.15.22:0/1122274334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 msgr2=0x55cdc97b99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.798+0000 7fe347e00640 1 --2- 172.21.15.22:0/1122274334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97b99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.798+0000 7fe347e00640 1 -- 172.21.15.22:0/1122274334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97ae0f0 msgr2=0x55cdc97ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.798+0000 7fe347e00640 1 --2- 172.21.15.22:0/1122274334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97ae0f0 0x55cdc97ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.798+0000 7fe347e00640 1 -- 172.21.15.22:0/1122274334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc96d8980 con 0x55cdc97aea10 2024-04-20T21:42:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.799+0000 7fe347e00640 1 --2- 172.21.15.22:0/1122274334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97b70b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe3380efd30 tx=0x7fe3380f85a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee0bc7a4d697bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.799+0000 7fe33d600640 1 -- 172.21.15.22:0/1122274334 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3380f2c00 con 0x55cdc97aea10 2024-04-20T21:42:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.799+0000 7fe344c00640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fe32c0a3da0 0x7fe32c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.799+0000 7fe33d600640 1 -- 172.21.15.22:0/1122274334 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3380ff030 con 0x55cdc97aea10 2024-04-20T21:42:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.799+0000 7fe33d600640 1 -- 172.21.15.22:0/1122274334 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338104840 con 0x55cdc97aea10 2024-04-20T21:42:11.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.799+0000 7fe344c00640 1 -- 172.21.15.22:0/970399707 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7fe3380f2550 con 0x7fe348102070 2024-04-20T21:42:11.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.800+0000 7fe34d200640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fe32c0a3da0 0x7fe32c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.800+0000 7fe351e36740 1 -- 172.21.15.22:0/1122274334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 msgr2=0x55cdc97b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:11.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.800+0000 7fe351e36740 1 --2- 172.21.15.22:0/1122274334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97b70b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe3380efd30 tx=0x7fe3380f85a0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.800+0000 7fe351e36740 1 -- 172.21.15.22:0/1122274334 shutdown_connections 2024-04-20T21:42:11.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.800+0000 7fe351e36740 1 --2- 172.21.15.22:0/1122274334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97b99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.800+0000 7fe351e36740 1 --2- 172.21.15.22:0/1122274334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97b70b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.800+0000 7fe351e36740 1 --2- 172.21.15.22:0/1122274334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97ae0f0 0x55cdc97ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.800+0000 7fe351e36740 1 -- 172.21.15.22:0/1122274334 >> 172.21.15.22:0/1122274334 conn(0x55cdc97a8820 msgr2=0x55cdc97a8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:11.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.800+0000 7fe351e36740 1 -- 172.21.15.22:0/1122274334 shutdown_connections 2024-04-20T21:42:11.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.800+0000 7fe34d200640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fe32c0a3da0 0x7fe32c0a6260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe3400049d0 tx=0x7fe340009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:11.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.801+0000 7fe351e36740 1 -- 172.21.15.22:0/1122274334 wait complete. 2024-04-20T21:42:11.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.801+0000 7fe351e36740 1 -- start start 2024-04-20T21:42:11.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.801+0000 7fe351e36740 1 Processor -- start 2024-04-20T21:42:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.802+0000 7fe351e36740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97ae0f0 0x55cdc97b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.802+0000 7fe351e36740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.802+0000 7fe351e36740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97b5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.802+0000 7fe351e36740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cdc97ba1a0 con 0x55cdc97ae0f0 2024-04-20T21:42:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.802+0000 7fe351e36740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cdc96cc7a0 con 0x55cdc97b75f0 2024-04-20T21:42:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.802+0000 7fe351e36740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cdc97ba320 con 0x55cdc97aea10 2024-04-20T21:42:11.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.802+0000 7fe351e36740 10 client.0 _ll_register_callbacks cb 0x55cdc96ceb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:11.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.802+0000 7fe347e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97ae0f0 0x55cdc97b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.802+0000 7fe347e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97ae0f0 0x55cdc97b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47216/0 (socket says 172.21.15.22:47216) 2024-04-20T21:42:11.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.802+0000 7fe347e00640 1 -- 172.21.15.22:0/3755641723 learned_addr learned my addr 172.21.15.22:0/3755641723 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:11.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.802+0000 7fe347400640 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.803+0000 7fe34d200640 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.803+0000 7fe347e00640 1 -- 172.21.15.22:0/3755641723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 msgr2=0x55cdc97b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:11.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.803+0000 7fe347e00640 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.803+0000 7fe347e00640 1 -- 172.21.15.22:0/3755641723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 msgr2=0x55cdc97b5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:11.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.803+0000 7fe347e00640 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97b5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.803+0000 7fe347e00640 1 -- 172.21.15.22:0/3755641723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc94ed7e0 con 0x55cdc97ae0f0 2024-04-20T21:42:11.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.803+0000 7fe347400640 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:11.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.803+0000 7fe347e00640 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97ae0f0 0x55cdc97b4690 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe3380be280 tx=0x7fe338000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbbf2e659a08c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:11.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.803+0000 7fe332000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:42:11.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.804+0000 7fe332000640 1 -- 172.21.15.22:0/3755641723 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3380ba820 con 0x55cdc97ae0f0 2024-04-20T21:42:11.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.804+0000 7fe332000640 1 -- 172.21.15.22:0/3755641723 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7fe3380efea0 con 0x55cdc97ae0f0 2024-04-20T21:42:11.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.804+0000 7fe351e36740 1 -- 172.21.15.22:0/3755641723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3380ff030 con 0x55cdc97ae0f0 2024-04-20T21:42:11.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.804+0000 7fe332000640 1 -- 172.21.15.22:0/3755641723 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe3380bdee0 con 0x55cdc97ae0f0 2024-04-20T21:42:11.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.804+0000 7fe351e36740 10 client.16452 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:11.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.804+0000 7fe351e36740 1 -- 172.21.15.22:0/3755641723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55cdc954df70 con 0x55cdc97ae0f0 2024-04-20T21:42:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.804+0000 7fe332000640 1 -- 172.21.15.22:0/3755641723 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3380ff030 con 0x55cdc97ae0f0 2024-04-20T21:42:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.804+0000 7fe332000640 1 -- 172.21.15.22:0/3755641723 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fe3380babf0 con 0x55cdc97ae0f0 2024-04-20T21:42:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.805+0000 7fe351e36740 10 client.16452 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.805+0000 7fe351e36740 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.805+0000 7fe351e36740 20 client.16452 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.805+0000 7fe351e36740 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cdc97c7590 0x55cdc97c9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.805+0000 7fe351e36740 4 client.16452 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.805+0000 7fe351e36740 1 -- 172.21.15.22:0/3755641723 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdc9720740 con 0x55cdc97c7590 2024-04-20T21:42:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.805+0000 7fe347400640 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cdc97c7590 0x55cdc97c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.806+0000 7fe347400640 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cdc97c7590 0x55cdc97c9980 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3127163535d035 server_cookie=cb988165e44ffad0 in_seq=0 out_seq=0 2024-04-20T21:42:11.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.806+0000 7fe332000640 10 client.16452 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:11.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe332000640 1 -- 172.21.15.22:0/3755641723 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cdc9720740 con 0x55cdc97c7590 2024-04-20T21:42:11.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe332000640 10 client.16452 handle_command_reply: tid=0 2024-04-20T21:42:11.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe351e36740 1 client.16452 shutdown 2024-04-20T21:42:11.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe351e36740 1 -- 172.21.15.22:0/3755641723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97ae0f0 msgr2=0x55cdc97b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:11.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe351e36740 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97ae0f0 0x55cdc97b4690 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe3380be280 tx=0x7fe338000bc0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe351e36740 1 -- 172.21.15.22:0/3755641723 shutdown_connections 2024-04-20T21:42:11.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe351e36740 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cdc97c7590 0x55cdc97c9980 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe351e36740 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97b5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe351e36740 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe351e36740 1 --2- 172.21.15.22:0/3755641723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97ae0f0 0x55cdc97b4690 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe351e36740 1 -- 172.21.15.22:0/3755641723 >> 172.21.15.22:0/3755641723 conn(0x55cdc97a8820 msgr2=0x55cdc97aa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:11.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe332000640 0 client.16452 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:11.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe351e36740 1 -- 172.21.15.22:0/3755641723 shutdown_connections 2024-04-20T21:42:11.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.807+0000 7fe351e36740 1 -- 172.21.15.22:0/3755641723 wait complete. 2024-04-20T21:42:11.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.808+0000 7fe351e36740 20 client.16452 trim_cache size 0 max 16384 2024-04-20T21:42:11.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.827+0000 7fe351e36740 1 Processor -- start 2024-04-20T21:42:11.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.828+0000 7fe351e36740 1 -- start start 2024-04-20T21:42:11.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.828+0000 7fe351e36740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97ae0f0 0x55cdc97b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.828+0000 7fe351e36740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.828+0000 7fe351e36740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97b75f0 0x55cdc97b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.828+0000 7fe351e36740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481adf00 con 0x55cdc97b75f0 2024-04-20T21:42:11.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.828+0000 7fe351e36740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481b0760 con 0x55cdc97ae0f0 2024-04-20T21:42:11.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.828+0000 7fe351e36740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481ae550 con 0x55cdc97aea10 2024-04-20T21:42:11.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.828+0000 7fe34d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97b75f0 0x55cdc97b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.828+0000 7fe347e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97ae0f0 0x55cdc97b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.828+0000 7fe347e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97ae0f0 0x55cdc97b6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:38090/0 (socket says 172.21.15.22:38090) 2024-04-20T21:42:11.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.828+0000 7fe347400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.828+0000 7fe347e00640 1 -- 172.21.15.22:0/4025001897 learned_addr learned my addr 172.21.15.22:0/4025001897 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:11.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.829+0000 7fe347e00640 1 -- 172.21.15.22:0/4025001897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 msgr2=0x55cdc97c2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:11.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.829+0000 7fe347e00640 1 --2- 172.21.15.22:0/4025001897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97c2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.829+0000 7fe347e00640 1 -- 172.21.15.22:0/4025001897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97b75f0 msgr2=0x55cdc97b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:11.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.829+0000 7fe347e00640 1 --2- 172.21.15.22:0/4025001897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97b75f0 0x55cdc97b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.829+0000 7fe347e00640 1 -- 172.21.15.22:0/4025001897 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc93e5170 con 0x55cdc97ae0f0 2024-04-20T21:42:11.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.829+0000 7fe347400640 1 --2- 172.21.15.22:0/4025001897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:11.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.829+0000 7fe347e00640 1 --2- 172.21.15.22:0/4025001897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97ae0f0 0x55cdc97b6dc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe3380efde0 tx=0x7fe338004ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0333019606c59ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:11.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.830+0000 7fe33cc00640 1 -- 172.21.15.22:0/4025001897 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3380be0c0 con 0x55cdc97ae0f0 2024-04-20T21:42:11.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.830+0000 7fe33cc00640 1 -- 172.21.15.22:0/4025001897 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3380ff030 con 0x55cdc97ae0f0 2024-04-20T21:42:11.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.830+0000 7fe33cc00640 1 -- 172.21.15.22:0/4025001897 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3380be350 con 0x55cdc97ae0f0 2024-04-20T21:42:11.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.830+0000 7fe351e36740 1 -- 172.21.15.22:0/4025001897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97ae0f0 msgr2=0x55cdc97b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:11.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.830+0000 7fe351e36740 1 --2- 172.21.15.22:0/4025001897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97ae0f0 0x55cdc97b6dc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe3380efde0 tx=0x7fe338004ee0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.831+0000 7fe351e36740 1 -- 172.21.15.22:0/4025001897 shutdown_connections 2024-04-20T21:42:11.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.831+0000 7fe351e36740 1 --2- 172.21.15.22:0/4025001897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97b75f0 0x55cdc97b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.831+0000 7fe351e36740 1 --2- 172.21.15.22:0/4025001897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97aea10 0x55cdc97c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.831+0000 7fe351e36740 1 --2- 172.21.15.22:0/4025001897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97ae0f0 0x55cdc97b6dc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.831+0000 7fe351e36740 1 -- 172.21.15.22:0/4025001897 >> 172.21.15.22:0/4025001897 conn(0x55cdc97a8820 msgr2=0x55cdc97a7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:11.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.831+0000 7fe351e36740 1 -- 172.21.15.22:0/4025001897 shutdown_connections 2024-04-20T21:42:11.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.831+0000 7fe351e36740 1 -- 172.21.15.22:0/4025001897 wait complete. 2024-04-20T21:42:11.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.831+0000 7fe351e36740 1 -- start start 2024-04-20T21:42:11.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.831+0000 7fe351e36740 1 Processor -- start 2024-04-20T21:42:11.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.832+0000 7fe351e36740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97ae0f0 0x55cdc97bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.832+0000 7fe351e36740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97aea10 0x55cdc97bac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.832+0000 7fe351e36740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97bb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.832+0000 7fe351e36740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481aec20 con 0x55cdc97aea10 2024-04-20T21:42:11.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.832+0000 7fe351e36740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481b0090 con 0x55cdc97b75f0 2024-04-20T21:42:11.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.832+0000 7fe351e36740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481af2f0 con 0x55cdc97ae0f0 2024-04-20T21:42:11.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.832+0000 7fe351e36740 10 client.0 _ll_register_callbacks cb 0x7fe334009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:11.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.832+0000 7fe347400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97aea10 0x55cdc97bac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.832+0000 7fe347400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97aea10 0x55cdc97bac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47236/0 (socket says 172.21.15.22:47236) 2024-04-20T21:42:11.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.832+0000 7fe347400640 1 -- 172.21.15.22:0/4094491979 learned_addr learned my addr 172.21.15.22:0/4094491979 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:11.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.832+0000 7fe347e00640 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97ae0f0 0x55cdc97bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.832+0000 7fe34d200640 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.833+0000 7fe347400640 1 -- 172.21.15.22:0/4094491979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97ae0f0 msgr2=0x55cdc97bf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:11.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.833+0000 7fe347400640 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97ae0f0 0x55cdc97bf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.833+0000 7fe347400640 1 -- 172.21.15.22:0/4094491979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 msgr2=0x55cdc97bb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:11.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.833+0000 7fe347400640 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97bb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.833+0000 7fe347400640 1 -- 172.21.15.22:0/4094491979 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc97bb980 con 0x55cdc97aea10 2024-04-20T21:42:11.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.833+0000 7fe347400640 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97aea10 0x55cdc97bac40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe334005e00 tx=0x7fe3340207d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc87400134dd1a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:11.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.833+0000 7fe332000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:42:11.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.833+0000 7fe332000640 1 -- 172.21.15.22:0/4094491979 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3340220b0 con 0x55cdc97aea10 2024-04-20T21:42:11.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.833+0000 7fe332000640 1 -- 172.21.15.22:0/4094491979 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7fe334022250 con 0x55cdc97aea10 2024-04-20T21:42:11.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.834+0000 7fe351e36740 1 -- 172.21.15.22:0/4094491979 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3380ff030 con 0x55cdc97aea10 2024-04-20T21:42:11.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.834+0000 7fe332000640 1 -- 172.21.15.22:0/4094491979 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe334022450 con 0x55cdc97aea10 2024-04-20T21:42:11.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.834+0000 7fe351e36740 10 client.16458 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:11.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.834+0000 7fe351e36740 1 -- 172.21.15.22:0/4094491979 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55cdc982db00 con 0x55cdc97aea10 2024-04-20T21:42:11.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.834+0000 7fe332000640 1 -- 172.21.15.22:0/4094491979 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3380ff030 con 0x55cdc97aea10 2024-04-20T21:42:11.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.834+0000 7fe332000640 1 -- 172.21.15.22:0/4094491979 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fe334031070 con 0x55cdc97aea10 2024-04-20T21:42:11.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.834+0000 7fe351e36740 10 client.16458 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:11.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.834+0000 7fe351e36740 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:11.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.834+0000 7fe351e36740 20 client.16458 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:11.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.835+0000 7fe351e36740 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cdc982e2b0 0x55cdc98306d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:11.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.835+0000 7fe351e36740 4 client.16458 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:42:11.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.835+0000 7fe351e36740 1 -- 172.21.15.22:0/4094491979 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cdc971a5b0 con 0x55cdc982e2b0 2024-04-20T21:42:11.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.835+0000 7fe347e00640 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cdc982e2b0 0x55cdc98306d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:11.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.835+0000 7fe347e00640 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cdc982e2b0 0x55cdc98306d0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6344e99a7577443b server_cookie=c88d1fb530082786 in_seq=0 out_seq=0 2024-04-20T21:42:11.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.835+0000 7fe332000640 10 client.16458 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:11.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.836+0000 7fe332000640 1 -- 172.21.15.22:0/4094491979 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cdc971a5b0 con 0x55cdc982e2b0 2024-04-20T21:42:11.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.836+0000 7fe332000640 10 client.16458 handle_command_reply: tid=0 2024-04-20T21:42:11.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.836+0000 7fe351e36740 1 client.16458 shutdown 2024-04-20T21:42:11.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.837+0000 7fe351e36740 1 -- 172.21.15.22:0/4094491979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97aea10 msgr2=0x55cdc97bac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:11.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.837+0000 7fe351e36740 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97aea10 0x55cdc97bac40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe334005e00 tx=0x7fe3340207d0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.837+0000 7fe351e36740 1 -- 172.21.15.22:0/4094491979 shutdown_connections 2024-04-20T21:42:11.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.837+0000 7fe351e36740 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55cdc982e2b0 0x55cdc98306d0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.837+0000 7fe351e36740 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc97b75f0 0x55cdc97bb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.837+0000 7fe351e36740 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc97aea10 0x55cdc97bac40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.837+0000 7fe351e36740 1 --2- 172.21.15.22:0/4094491979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc97ae0f0 0x55cdc97bf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.837+0000 7fe332000640 0 client.16458 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:11.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.837+0000 7fe351e36740 1 -- 172.21.15.22:0/4094491979 >> 172.21.15.22:0/4094491979 conn(0x55cdc97a8820 msgr2=0x55cdc97a7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:11.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.837+0000 7fe351e36740 1 -- 172.21.15.22:0/4094491979 shutdown_connections 2024-04-20T21:42:11.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.837+0000 7fe351e36740 1 -- 172.21.15.22:0/4094491979 wait complete. 2024-04-20T21:42:11.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.837+0000 7fe351e36740 20 client.16458 trim_cache size 0 max 16384 2024-04-20T21:42:11.842 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:42:11.843 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:42:11.843 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "8806d031-aa85-48e6-97a9-bea3f737cfbb", 2024-04-20T21:42:11.843 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:42:11.843 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:42:11.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.838+0000 7fe34dc00640 1 -- 172.21.15.22:0/970399707 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fe32c0a3da0 msgr2=0x7fe32c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:11.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.838+0000 7fe34dc00640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fe32c0a3da0 0x7fe32c0a6260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe3400049d0 tx=0x7fe340009500 comp rx=0 tx=0).stop 2024-04-20T21:42:11.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.838+0000 7fe34dc00640 1 -- 172.21.15.22:0/970399707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe348102070 msgr2=0x7fe3481a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:11.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.838+0000 7fe34dc00640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe348102070 0x7fe3481a0690 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe338036040 tx=0x7fe33802cb40 comp rx=0 tx=0).stop 2024-04-20T21:42:11.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.838+0000 7fe34dc00640 1 -- 172.21.15.22:0/970399707 shutdown_connections 2024-04-20T21:42:11.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.838+0000 7fe34dc00640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fe32c0a3da0 0x7fe32c0a6260 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.838+0000 7fe34dc00640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe34819a800 0x7fe34819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.838+0000 7fe34dc00640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe348107140 0x7fe3481a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.838+0000 7fe34dc00640 1 --2- 172.21.15.22:0/970399707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe348102070 0x7fe3481a0690 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:11.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.838+0000 7fe34dc00640 1 -- 172.21.15.22:0/970399707 >> 172.21.15.22:0/970399707 conn(0x7fe3480fd930 msgr2=0x7fe3480ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:11.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.839+0000 7fe34dc00640 1 -- 172.21.15.22:0/970399707 shutdown_connections 2024-04-20T21:42:11.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:11.839+0000 7fe34dc00640 1 -- 172.21.15.22:0/970399707 wait complete. 2024-04-20T21:42:11.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:42:11.857 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:42:12.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.054+0000 7f94af000640 1 -- 172.21.15.22:0/465829659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a8104280 msgr2=0x7f94a8104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:12.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.054+0000 7f94af000640 1 --2- 172.21.15.22:0/465829659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a8104280 0x7f94a8104660 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f949c009ca0 tx=0x7f949c02c0f0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.054+0000 7f94af000640 1 -- 172.21.15.22:0/465829659 shutdown_connections 2024-04-20T21:42:12.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.054+0000 7f94af000640 1 --2- 172.21.15.22:0/465829659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94a8109350 0x7f94a8110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.054+0000 7f94af000640 1 --2- 172.21.15.22:0/465829659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94a8104c30 0x7f94a8108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.054+0000 7f94af000640 1 --2- 172.21.15.22:0/465829659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a8104280 0x7f94a8104660 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.054+0000 7f94af000640 1 -- 172.21.15.22:0/465829659 >> 172.21.15.22:0/465829659 conn(0x7f94a80ffaa0 msgr2=0x7f94a8101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:12.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.054+0000 7f94af000640 1 -- 172.21.15.22:0/465829659 shutdown_connections 2024-04-20T21:42:12.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.054+0000 7f94af000640 1 -- 172.21.15.22:0/465829659 wait complete. 2024-04-20T21:42:12.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.055+0000 7f94af000640 1 Processor -- start 2024-04-20T21:42:12.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.056+0000 7f94af000640 1 -- start start 2024-04-20T21:42:12.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.056+0000 7f94af000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94a8104280 0x7f94a81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.056+0000 7f94af000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94a8109350 0x7f94a81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.056+0000 7f94af000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a819a760 0x7f94a819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.056+0000 7f94af000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81143e0 con 0x7f94a8109350 2024-04-20T21:42:12.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.056+0000 7f94af000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8114260 con 0x7f94a819a760 2024-04-20T21:42:12.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.056+0000 7f94af000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8114560 con 0x7f94a8104280 2024-04-20T21:42:12.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.056+0000 7f94adc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94a8104280 0x7f94a81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.056+0000 7f94ad200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94a8109350 0x7f94a81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.056+0000 7f94adc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94a8104280 0x7f94a81a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50902/0 (socket says 172.21.15.22:50902) 2024-04-20T21:42:12.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.056+0000 7f94adc00640 1 -- 172.21.15.22:0/3814466885 learned_addr learned my addr 172.21.15.22:0/3814466885 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:12.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.056+0000 7f94ae600640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a819a760 0x7f94a819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.057+0000 7f94adc00640 1 -- 172.21.15.22:0/3814466885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a819a760 msgr2=0x7f94a819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:12.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.057+0000 7f94adc00640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a819a760 0x7f94a819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.057+0000 7f94adc00640 1 -- 172.21.15.22:0/3814466885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94a8109350 msgr2=0x7f94a81a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:12.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.057+0000 7f94adc00640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94a8109350 0x7f94a81a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.057+0000 7f94adc00640 1 -- 172.21.15.22:0/3814466885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a819b360 con 0x7f94a8104280 2024-04-20T21:42:12.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.057+0000 7f94adc00640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94a8104280 0x7f94a81a06a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f949c005e30 tx=0x7f949c032c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:12.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.058+0000 7f94a6000640 1 -- 172.21.15.22:0/3814466885 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c041020 con 0x7f94a8104280 2024-04-20T21:42:12.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.058+0000 7f94a6000640 1 -- 172.21.15.22:0/3814466885 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f949c034cd0 con 0x7f94a8104280 2024-04-20T21:42:12.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.058+0000 7f94a6000640 1 -- 172.21.15.22:0/3814466885 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c039040 con 0x7f94a8104280 2024-04-20T21:42:12.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.058+0000 7f94af000640 1 -- 172.21.15.22:0/3814466885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a819b5f0 con 0x7f94a8104280 2024-04-20T21:42:12.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.058+0000 7f94af000640 1 -- 172.21.15.22:0/3814466885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81a76e0 con 0x7f94a8104280 2024-04-20T21:42:12.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.061+0000 7f94b3216740 1 Processor -- start 2024-04-20T21:42:12.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.061+0000 7f94a6000640 1 -- 172.21.15.22:0/3814466885 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f949c044020 con 0x7f94a8104280 2024-04-20T21:42:12.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.062+0000 7f94b3216740 1 -- start start 2024-04-20T21:42:12.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.062+0000 7f94b3216740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 0x5582de910760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.062+0000 7f94b3216740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de910ca0 0x5582de919340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.062+0000 7f94b3216740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de919880 0x5582de91bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.062+0000 7f94b3216740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5582de876150 con 0x5582de910ca0 2024-04-20T21:42:12.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.062+0000 7f94b3216740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5582de81c370 con 0x5582de919880 2024-04-20T21:42:12.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.062+0000 7f94b3216740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5582de872110 con 0x5582de910380 2024-04-20T21:42:12.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.062+0000 7f94ae600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 0x5582de910760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.062+0000 7f94ae600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 0x5582de910760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50916/0 (socket says 172.21.15.22:50916) 2024-04-20T21:42:12.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.062+0000 7f94ae600640 1 -- 172.21.15.22:0/1569055646 learned_addr learned my addr 172.21.15.22:0/1569055646 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.062+0000 7f94adc00640 1 --2- 172.21.15.22:0/1569055646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de910ca0 0x5582de919340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.062+0000 7f94ad200640 1 --2- 172.21.15.22:0/1569055646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de919880 0x5582de91bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.063+0000 7f94ae600640 1 -- 172.21.15.22:0/1569055646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de919880 msgr2=0x5582de91bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.063+0000 7f94ae600640 1 --2- 172.21.15.22:0/1569055646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de919880 0x5582de91bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.063+0000 7f94ae600640 1 -- 172.21.15.22:0/1569055646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de910ca0 msgr2=0x5582de919340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.063+0000 7f94ae600640 1 --2- 172.21.15.22:0/1569055646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de910ca0 0x5582de919340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.063+0000 7f94ae600640 1 -- 172.21.15.22:0/1569055646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582de6527d0 con 0x5582de910380 2024-04-20T21:42:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.063+0000 7f94ad200640 1 --2- 172.21.15.22:0/1569055646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de919880 0x5582de91bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:42:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.063+0000 7f94ae600640 1 --2- 172.21.15.22:0/1569055646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 0x5582de910760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009c20 tx=0x7f94a0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd032d46c31a4130 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.063+0000 7f949aa00640 1 -- 172.21.15.22:0/1569055646 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a001e070 con 0x5582de910380 2024-04-20T21:42:12.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.063+0000 7f949aa00640 1 -- 172.21.15.22:0/1569055646 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a0019b10 con 0x5582de910380 2024-04-20T21:42:12.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.063+0000 7f949aa00640 1 -- 172.21.15.22:0/1569055646 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a000e9d0 con 0x5582de910380 2024-04-20T21:42:12.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.063+0000 7f94a6000640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f94900a3da0 0x7f94900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.064+0000 7f94a6000640 1 -- 172.21.15.22:0/3814466885 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7f949c0f23d0 con 0x7f94a8104280 2024-04-20T21:42:12.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.064+0000 7f94adc00640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f94900a3da0 0x7f94900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.064+0000 7f94b3216740 1 -- 172.21.15.22:0/1569055646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 msgr2=0x5582de910760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:12.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.064+0000 7f94b3216740 1 --2- 172.21.15.22:0/1569055646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 0x5582de910760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009c20 tx=0x7f94a0011810 comp rx=0 tx=0).stop 2024-04-20T21:42:12.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.064+0000 7f94b3216740 1 -- 172.21.15.22:0/1569055646 shutdown_connections 2024-04-20T21:42:12.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.064+0000 7f94b3216740 1 --2- 172.21.15.22:0/1569055646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de919880 0x5582de91bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.064+0000 7f94b3216740 1 --2- 172.21.15.22:0/1569055646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de910ca0 0x5582de919340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.064+0000 7f94b3216740 1 --2- 172.21.15.22:0/1569055646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 0x5582de910760 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.064+0000 7f94b3216740 1 -- 172.21.15.22:0/1569055646 >> 172.21.15.22:0/1569055646 conn(0x5582de90aab0 msgr2=0x5582de90ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:12.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.064+0000 7f94b3216740 1 -- 172.21.15.22:0/1569055646 shutdown_connections 2024-04-20T21:42:12.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.064+0000 7f94adc00640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f94900a3da0 0x7f94900a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x5582de90fe50 tx=0x7f949c032b00 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:12.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.065+0000 7f94b3216740 1 -- 172.21.15.22:0/1569055646 wait complete. 2024-04-20T21:42:12.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.065+0000 7f94b3216740 1 -- start start 2024-04-20T21:42:12.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.065+0000 7f94b3216740 1 Processor -- start 2024-04-20T21:42:12.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94b3216740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910380 0x5582de916980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94b3216740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de910ca0 0x5582de916ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94b3216740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de919880 0x5582de917400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94b3216740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5582de91c670 con 0x5582de910ca0 2024-04-20T21:42:12.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94b3216740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5582de82f740 con 0x5582de910380 2024-04-20T21:42:12.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94b3216740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5582de91c7f0 con 0x5582de919880 2024-04-20T21:42:12.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94b3216740 10 client.0 _ll_register_callbacks cb 0x5582de82ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:12.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94ad200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de910ca0 0x5582de916ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94ad200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de910ca0 0x5582de916ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47272/0 (socket says 172.21.15.22:47272) 2024-04-20T21:42:12.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94ad200640 1 -- 172.21.15.22:0/573560323 learned_addr learned my addr 172.21.15.22:0/573560323 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:12.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94ae600640 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910380 0x5582de916980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94ae600640 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de919880 0x5582de917400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94ad200640 1 -- 172.21.15.22:0/573560323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de919880 msgr2=0x5582de917400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:12.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94ad200640 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de919880 0x5582de917400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94ad200640 1 -- 172.21.15.22:0/573560323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910380 msgr2=0x5582de916980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:12.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.066+0000 7f94ad200640 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910380 0x5582de916980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.067+0000 7f94ad200640 1 -- 172.21.15.22:0/573560323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582de5883f0 con 0x5582de910ca0 2024-04-20T21:42:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.067+0000 7f94ae600640 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de919880 0x5582de917400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:42:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.067+0000 7f94ad200640 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de910ca0 0x5582de916ec0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9494012700 tx=0x7f9494012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f163ff79f2a71102 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.067+0000 7f948f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:42:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.067+0000 7f948f400640 1 -- 172.21.15.22:0/573560323 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494004050 con 0x5582de910ca0 2024-04-20T21:42:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.067+0000 7f948f400640 1 -- 172.21.15.22:0/573560323 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7f94940041f0 con 0x5582de910ca0 2024-04-20T21:42:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.067+0000 7f94b3216740 1 -- 172.21.15.22:0/573560323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a0019b10 con 0x5582de910ca0 2024-04-20T21:42:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.067+0000 7f948f400640 1 -- 172.21.15.22:0/573560323 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f94940043f0 con 0x5582de910ca0 2024-04-20T21:42:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.067+0000 7f94b3216740 10 client.16470 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:12.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.067+0000 7f94b3216740 1 -- 172.21.15.22:0/573560323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5582de5480f0 con 0x5582de910ca0 2024-04-20T21:42:12.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.068+0000 7f948f400640 1 -- 172.21.15.22:0/573560323 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a0019b10 con 0x5582de910ca0 2024-04-20T21:42:12.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.068+0000 7f948f400640 1 -- 172.21.15.22:0/573560323 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f9494003bf0 con 0x5582de910ca0 2024-04-20T21:42:12.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.068+0000 7f94b3216740 10 client.16470 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:12.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.068+0000 7f94b3216740 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.068+0000 7f94b3216740 20 client.16470 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.068+0000 7f94b3216740 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5582de929d40 0x5582de92c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.068+0000 7f94b3216740 4 client.16470 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:12.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.068+0000 7f94b3216740 1 -- 172.21.15.22:0/573560323 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582de883b70 con 0x5582de929d40 2024-04-20T21:42:12.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.069+0000 7f94ae600640 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5582de929d40 0x5582de92c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.069+0000 7f94ae600640 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5582de929d40 0x5582de92c130 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43fc747dd69772d server_cookie=16707d0b0c811ed9 in_seq=0 out_seq=0 2024-04-20T21:42:12.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.069+0000 7f948f400640 10 client.16470 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:12.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.070+0000 7f948f400640 1 -- 172.21.15.22:0/573560323 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582de883b70 con 0x5582de929d40 2024-04-20T21:42:12.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.070+0000 7f948f400640 10 client.16470 handle_command_reply: tid=0 2024-04-20T21:42:12.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.070+0000 7f94b3216740 1 client.16470 shutdown 2024-04-20T21:42:12.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.070+0000 7f94b3216740 1 -- 172.21.15.22:0/573560323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de910ca0 msgr2=0x5582de916ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:12.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.070+0000 7f94b3216740 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de910ca0 0x5582de916ec0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9494012700 tx=0x7f9494012bc0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.070+0000 7f94b3216740 1 -- 172.21.15.22:0/573560323 shutdown_connections 2024-04-20T21:42:12.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.070+0000 7f94b3216740 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de919880 0x5582de917400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.071+0000 7f94b3216740 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de910ca0 0x5582de916ec0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.071+0000 7f94b3216740 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5582de929d40 0x5582de92c130 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.071+0000 7f94b3216740 1 --2- 172.21.15.22:0/573560323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910380 0x5582de916980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.071+0000 7f94b3216740 1 -- 172.21.15.22:0/573560323 >> 172.21.15.22:0/573560323 conn(0x5582de90aab0 msgr2=0x5582de90c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:12.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.071+0000 7f948f400640 0 client.16470 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:12.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.071+0000 7f94b3216740 1 -- 172.21.15.22:0/573560323 shutdown_connections 2024-04-20T21:42:12.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.071+0000 7f94b3216740 1 -- 172.21.15.22:0/573560323 wait complete. 2024-04-20T21:42:12.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.071+0000 7f94b3216740 20 client.16470 trim_cache size 0 max 16384 2024-04-20T21:42:12.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.090+0000 7f94b3216740 1 Processor -- start 2024-04-20T21:42:12.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.090+0000 7f94b3216740 1 -- start start 2024-04-20T21:42:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.091+0000 7f94b3216740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 0x5582de9190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.091+0000 7f94b3216740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910ca0 0x5582de925530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.091+0000 7f94b3216740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de919880 0x5582de909990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.091+0000 7f94b3216740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81adf70 con 0x5582de919880 2024-04-20T21:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.091+0000 7f94b3216740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b07d0 con 0x5582de910ca0 2024-04-20T21:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.091+0000 7f94b3216740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81ae5c0 con 0x5582de910380 2024-04-20T21:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.091+0000 7f94ae600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 0x5582de9190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.091+0000 7f94ae600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de919880 0x5582de909990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.091+0000 7f94ae600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 0x5582de9190b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50930/0 (socket says 172.21.15.22:50930) 2024-04-20T21:42:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.091+0000 7f94ae600640 1 -- 172.21.15.22:0/1794938011 learned_addr learned my addr 172.21.15.22:0/1794938011 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.091+0000 7f94ad200640 1 --2- 172.21.15.22:0/1794938011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910ca0 0x5582de925530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.092+0000 7f94ae600640 1 -- 172.21.15.22:0/1794938011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910ca0 msgr2=0x5582de925530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.092+0000 7f94ae600640 1 --2- 172.21.15.22:0/1794938011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910ca0 0x5582de925530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.092+0000 7f94ae600640 1 -- 172.21.15.22:0/1794938011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de919880 msgr2=0x5582de909990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.092+0000 7f94ae600640 1 --2- 172.21.15.22:0/1794938011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de919880 0x5582de909990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.092+0000 7f94ae600640 1 -- 172.21.15.22:0/1794938011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582de914ed0 con 0x5582de910380 2024-04-20T21:42:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.092+0000 7f94ae600640 1 --2- 172.21.15.22:0/1794938011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 0x5582de9190b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f94a0011d10 tx=0x7f94a0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee824c5b1ec06a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:12.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.092+0000 7f949a000640 1 -- 172.21.15.22:0/1794938011 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0019e60 con 0x5582de910380 2024-04-20T21:42:12.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.092+0000 7f949a000640 1 -- 172.21.15.22:0/1794938011 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a0019910 con 0x5582de910380 2024-04-20T21:42:12.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.092+0000 7f949a000640 1 -- 172.21.15.22:0/1794938011 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a001e040 con 0x5582de910380 2024-04-20T21:42:12.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.093+0000 7f94b3216740 1 -- 172.21.15.22:0/1794938011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 msgr2=0x5582de9190b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:12.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.093+0000 7f94b3216740 1 --2- 172.21.15.22:0/1794938011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 0x5582de9190b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f94a0011d10 tx=0x7f94a0005f90 comp rx=0 tx=0).stop 2024-04-20T21:42:12.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.093+0000 7f94b3216740 1 -- 172.21.15.22:0/1794938011 shutdown_connections 2024-04-20T21:42:12.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.093+0000 7f94b3216740 1 --2- 172.21.15.22:0/1794938011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de919880 0x5582de909990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.093+0000 7f94b3216740 1 --2- 172.21.15.22:0/1794938011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910ca0 0x5582de925530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.093+0000 7f94b3216740 1 --2- 172.21.15.22:0/1794938011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910380 0x5582de9190b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.093+0000 7f94b3216740 1 -- 172.21.15.22:0/1794938011 >> 172.21.15.22:0/1794938011 conn(0x5582de90aab0 msgr2=0x5582de909280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:12.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.093+0000 7f94b3216740 1 -- 172.21.15.22:0/1794938011 shutdown_connections 2024-04-20T21:42:12.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.093+0000 7f94b3216740 1 -- 172.21.15.22:0/1794938011 wait complete. 2024-04-20T21:42:12.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.094+0000 7f94b3216740 1 -- start start 2024-04-20T21:42:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.094+0000 7f94b3216740 1 Processor -- start 2024-04-20T21:42:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.094+0000 7f94b3216740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910380 0x5582de91d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.094+0000 7f94b3216740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910ca0 0x5582de91d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.094+0000 7f94b3216740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de919880 0x5582de91df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.094+0000 7f94b3216740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81aec90 con 0x5582de919880 2024-04-20T21:42:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.094+0000 7f94b3216740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b0100 con 0x5582de910380 2024-04-20T21:42:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.094+0000 7f94b3216740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81af360 con 0x5582de910ca0 2024-04-20T21:42:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.094+0000 7f94b3216740 10 client.0 _ll_register_callbacks cb 0x7f9494009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f94ad200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910ca0 0x5582de91d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f94ad200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910ca0 0x5582de91d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50944/0 (socket says 172.21.15.22:50944) 2024-04-20T21:42:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f94ad200640 1 -- 172.21.15.22:0/977223702 learned_addr learned my addr 172.21.15.22:0/977223702 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f94ae600640 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de919880 0x5582de91df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f94ae600640 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910380 0x5582de91d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f94ad200640 1 -- 172.21.15.22:0/977223702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910380 msgr2=0x5582de91d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f94ad200640 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910380 0x5582de91d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f94ad200640 1 -- 172.21.15.22:0/977223702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de919880 msgr2=0x5582de91df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f94ad200640 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de919880 0x5582de91df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f94ad200640 1 -- 172.21.15.22:0/977223702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582de98f300 con 0x5582de910ca0 2024-04-20T21:42:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f94ae600640 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de919880 0x5582de91df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:42:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f94ad200640 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910ca0 0x5582de91d9e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9494009850 tx=0x7f9494009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81493815d6df03a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.095+0000 7f948f400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:42:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.096+0000 7f948f400640 1 -- 172.21.15.22:0/977223702 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494031060 con 0x5582de910ca0 2024-04-20T21:42:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.096+0000 7f94b3216740 1 -- 172.21.15.22:0/977223702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a0019910 con 0x5582de910ca0 2024-04-20T21:42:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.096+0000 7f948f400640 1 -- 172.21.15.22:0/977223702 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7f94940041a0 con 0x5582de910ca0 2024-04-20T21:42:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.096+0000 7f948f400640 1 -- 172.21.15.22:0/977223702 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f949400f9a0 con 0x5582de910ca0 2024-04-20T21:42:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.096+0000 7f94b3216740 10 client.26647 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.096+0000 7f94b3216740 1 -- 172.21.15.22:0/977223702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5582de990620 con 0x5582de910ca0 2024-04-20T21:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.096+0000 7f948f400640 1 -- 172.21.15.22:0/977223702 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a0019910 con 0x5582de910ca0 2024-04-20T21:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.097+0000 7f948f400640 1 -- 172.21.15.22:0/977223702 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f94940245a0 con 0x5582de910ca0 2024-04-20T21:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.097+0000 7f94b3216740 10 client.26647 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.097+0000 7f94b3216740 10 client.26647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.097+0000 7f94b3216740 20 client.26647 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.097+0000 7f94b3216740 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5582de990d80 0x5582de9931a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.097+0000 7f94b3216740 4 client.26647 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.097+0000 7f94b3216740 1 -- 172.21.15.22:0/977223702 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582de906170 con 0x5582de990d80 2024-04-20T21:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.097+0000 7f94ae600640 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5582de990d80 0x5582de9931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f94ae600640 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5582de990d80 0x5582de9931a0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17521edec3896309 server_cookie=9d8ac9a30f4f60e8 in_seq=0 out_seq=0 2024-04-20T21:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f948f400640 10 client.26647 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f948f400640 1 -- 172.21.15.22:0/977223702 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5582de906170 con 0x5582de990d80 2024-04-20T21:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f948f400640 10 client.26647 handle_command_reply: tid=0 2024-04-20T21:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f94b3216740 1 client.26647 shutdown 2024-04-20T21:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f94b3216740 1 -- 172.21.15.22:0/977223702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910ca0 msgr2=0x5582de91d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f94b3216740 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910ca0 0x5582de91d9e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9494009850 tx=0x7f9494009f80 comp rx=0 tx=0).stop 2024-04-20T21:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f94b3216740 1 -- 172.21.15.22:0/977223702 shutdown_connections 2024-04-20T21:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f94b3216740 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582de919880 0x5582de91df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f94b3216740 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582de910ca0 0x5582de91d9e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f94b3216740 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5582de990d80 0x5582de9931a0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f94b3216740 1 --2- 172.21.15.22:0/977223702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582de910380 0x5582de91d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f948f400640 0 client.26647 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.098+0000 7f94b3216740 1 -- 172.21.15.22:0/977223702 >> 172.21.15.22:0/977223702 conn(0x5582de90aab0 msgr2=0x5582de909b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:12.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.099+0000 7f94b3216740 1 -- 172.21.15.22:0/977223702 shutdown_connections 2024-04-20T21:42:12.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.099+0000 7f94b3216740 1 -- 172.21.15.22:0/977223702 wait complete. 2024-04-20T21:42:12.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.099+0000 7f94b3216740 20 client.26647 trim_cache size 0 max 16384 2024-04-20T21:42:12.104 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:42:12.104 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:42:12.104 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:42:12.104 INFO:teuthology.orchestra.run.smithi022.stdout: "8806d031-aa85-48e6-97a9-bea3f737cfbb": { 2024-04-20T21:42:12.105 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:42:12.105 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "8806d031-aa85-48e6-97a9-bea3f737cfbb", 2024-04-20T21:42:12.105 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:42:12.105 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:42:12.105 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:42:12.105 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:42:12.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.099+0000 7f94af000640 1 -- 172.21.15.22:0/3814466885 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f94900a3da0 msgr2=0x7f94900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:12.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.099+0000 7f94af000640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f94900a3da0 0x7f94900a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x5582de90fe50 tx=0x7f949c032b00 comp rx=0 tx=0).stop 2024-04-20T21:42:12.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.100+0000 7f94af000640 1 -- 172.21.15.22:0/3814466885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94a8104280 msgr2=0x7f94a81a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:12.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.100+0000 7f94af000640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94a8104280 0x7f94a81a06a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f949c005e30 tx=0x7f949c032c90 comp rx=0 tx=0).stop 2024-04-20T21:42:12.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.100+0000 7f94af000640 1 -- 172.21.15.22:0/3814466885 shutdown_connections 2024-04-20T21:42:12.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.100+0000 7f94af000640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f94900a3da0 0x7f94900a6260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.100+0000 7f94af000640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a819a760 0x7f94a819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.100+0000 7f94af000640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94a8109350 0x7f94a81a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.100+0000 7f94af000640 1 --2- 172.21.15.22:0/3814466885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94a8104280 0x7f94a81a06a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:12.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.100+0000 7f94af000640 1 -- 172.21.15.22:0/3814466885 >> 172.21.15.22:0/3814466885 conn(0x7f94a80ffaa0 msgr2=0x7f94a8100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:12.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.100+0000 7f94af000640 1 -- 172.21.15.22:0/3814466885 shutdown_connections 2024-04-20T21:42:12.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:12.100+0000 7f94af000640 1 -- 172.21.15.22:0/3814466885 wait complete. 2024-04-20T21:42:12.118 INFO:tasks.cephfs.filesystem:scrub status for tag:8806d031-aa85-48e6-97a9-bea3f737cfbb - {'path': '/', 'tag': '8806d031-aa85-48e6-97a9-bea3f737cfbb', 'options': 'recursive,force'} 2024-04-20T21:42:12.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:12 smithi122 ceph-mon[22147]: pgmap v259: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:42:12.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:12 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:42:12.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:12 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:42:12.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:12 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:42:12.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:12 smithi033 ceph-mon[21626]: pgmap v259: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:42:12.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:12 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:42:12.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:12 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:42:12.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:12 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:42:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:12 smithi022 ceph-mon[16843]: pgmap v259: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:42:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:12 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:42:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:12 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:42:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:12 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:42:13.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:13 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:42:13.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:13 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:13.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:13 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:42:13.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:13 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:13.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:13 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:42:13.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:13 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:13.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:13 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:42:13.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:13 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:13.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:13 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:42:13.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:13 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:13.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:13 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:42:13.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:13 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:14.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:14 smithi122 ceph-mon[22147]: pgmap v260: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:42:14.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:14 smithi122 ceph-mon[22147]: osdmap e99: 12 total, 12 up, 12 in 2024-04-20T21:42:14.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:14 smithi033 ceph-mon[21626]: pgmap v260: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:42:14.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:14 smithi033 ceph-mon[21626]: osdmap e99: 12 total, 12 up, 12 in 2024-04-20T21:42:14.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:14 smithi022 ceph-mon[16843]: pgmap v260: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:42:14.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:14 smithi022 ceph-mon[16843]: osdmap e99: 12 total, 12 up, 12 in 2024-04-20T21:42:16.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:16 smithi122 ceph-mon[22147]: pgmap v262: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-20T21:42:16.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:16 smithi033 ceph-mon[21626]: pgmap v262: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-20T21:42:16.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:16 smithi022 ceph-mon[16843]: pgmap v262: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-20T21:42:17.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:17 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:42:17.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:17 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:42:17.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:17 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:42:18.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:18 smithi033 ceph-mon[21626]: pgmap v263: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 208 B/s wr, 2 op/s 2024-04-20T21:42:18.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:18 smithi022 ceph-mon[16843]: pgmap v263: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 208 B/s wr, 2 op/s 2024-04-20T21:42:18.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:18 smithi122 ceph-mon[22147]: pgmap v263: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 208 B/s wr, 2 op/s 2024-04-20T21:42:20.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:20 smithi033 ceph-mon[21626]: pgmap v264: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:42:20.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:20 smithi022 ceph-mon[16843]: pgmap v264: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:42:20.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:20 smithi122 ceph-mon[22147]: pgmap v264: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:42:22.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:22 smithi033 ceph-mon[21626]: pgmap v265: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:42:22.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:22 smithi022 ceph-mon[16843]: pgmap v265: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:42:22.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:22 smithi122 ceph-mon[22147]: pgmap v265: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:42:24.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:24 smithi033 ceph-mon[21626]: pgmap v266: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:42:24.659 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:24 smithi022 ceph-mon[16843]: pgmap v266: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:42:24.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:24 smithi122 ceph-mon[22147]: pgmap v266: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:42:26.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:26 smithi033 ceph-mon[21626]: pgmap v267: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 94 B/s wr, 3 op/s 2024-04-20T21:42:26.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:26 smithi022 ceph-mon[16843]: pgmap v267: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 94 B/s wr, 3 op/s 2024-04-20T21:42:26.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:26 smithi122 ceph-mon[22147]: pgmap v267: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 94 B/s wr, 3 op/s 2024-04-20T21:42:28.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:28 smithi022 ceph-mon[16843]: pgmap v268: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:28.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:28 smithi122 ceph-mon[22147]: pgmap v268: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:28.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:28 smithi033 ceph-mon[21626]: pgmap v268: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:30.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:30 smithi022 ceph-mon[16843]: pgmap v269: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:30.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:30 smithi122 ceph-mon[22147]: pgmap v269: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:30.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:30 smithi033 ceph-mon[21626]: pgmap v269: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:32.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:32 smithi022 ceph-mon[16843]: pgmap v270: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:42:32.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:32 smithi122 ceph-mon[22147]: pgmap v270: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:42:32.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:32 smithi033 ceph-mon[21626]: pgmap v270: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:42:34.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:34 smithi022 ceph-mon[16843]: pgmap v271: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:34.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:34 smithi122 ceph-mon[22147]: pgmap v271: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:34.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:34 smithi033 ceph-mon[21626]: pgmap v271: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:36.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:36 smithi022 ceph-mon[16843]: pgmap v272: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:42:36.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:36 smithi122 ceph-mon[22147]: pgmap v272: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:42:36.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:36 smithi033 ceph-mon[21626]: pgmap v272: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:42:38.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:38 smithi022 ceph-mon[16843]: pgmap v273: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:38.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:38 smithi122 ceph-mon[22147]: pgmap v273: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:38.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:38 smithi033 ceph-mon[21626]: pgmap v273: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:40.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:40 smithi022 ceph-mon[16843]: pgmap v274: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:40.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:40 smithi122 ceph-mon[22147]: pgmap v274: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:40.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:40 smithi033 ceph-mon[21626]: pgmap v274: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:42.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:42:42.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:42:42.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.349+0000 7fe2d1c00640 1 -- 172.21.15.22:0/4245114205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc10e6a0 msgr2=0x7fe2cc110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.349+0000 7fe2d1c00640 1 --2- 172.21.15.22:0/4245114205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc10e6a0 0x7fe2cc110a90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009ca0 tx=0x7fe2c402c180 comp rx=0 tx=0).stop 2024-04-20T21:42:42.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.349+0000 7fe2d1c00640 1 -- 172.21.15.22:0/4245114205 shutdown_connections 2024-04-20T21:42:42.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.349+0000 7fe2d1c00640 1 --2- 172.21.15.22:0/4245114205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc10e6a0 0x7fe2cc110a90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.349+0000 7fe2d1c00640 1 --2- 172.21.15.22:0/4245114205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2cc074c30 0x7fe2cc10e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.349+0000 7fe2d1c00640 1 --2- 172.21.15.22:0/4245114205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2cc074310 0x7fe2cc0746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.349+0000 7fe2d1c00640 1 -- 172.21.15.22:0/4245114205 >> 172.21.15.22:0/4245114205 conn(0x7fe2cc0fd930 msgr2=0x7fe2cc0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:42.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.349+0000 7fe2d1c00640 1 -- 172.21.15.22:0/4245114205 shutdown_connections 2024-04-20T21:42:42.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.349+0000 7fe2d1c00640 1 -- 172.21.15.22:0/4245114205 wait complete. 2024-04-20T21:42:42.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.350+0000 7fe2d1c00640 1 Processor -- start 2024-04-20T21:42:42.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.350+0000 7fe2d1c00640 1 -- start start 2024-04-20T21:42:42.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2d1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2cc074310 0x7fe2cc1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2d1c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc074c30 0x7fe2cc1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2d1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2cc10e6a0 0x7fe2cc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2d1c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc1143a0 con 0x7fe2cc10e6a0 2024-04-20T21:42:42.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2d1c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc114220 con 0x7fe2cc074c30 2024-04-20T21:42:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2d1c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc114520 con 0x7fe2cc074310 2024-04-20T21:42:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2c3400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc074c30 0x7fe2cc1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2d1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2cc10e6a0 0x7fe2cc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2c3400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc074c30 0x7fe2cc1a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:55578/0 (socket says 172.21.15.22:55578) 2024-04-20T21:42:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2c3400640 1 -- 172.21.15.22:0/622034756 learned_addr learned my addr 172.21.15.22:0/622034756 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2cbe00640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2cc074310 0x7fe2cc1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2d1200640 1 -- 172.21.15.22:0/622034756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2cc074310 msgr2=0x7fe2cc1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2d1200640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2cc074310 0x7fe2cc1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2d1200640 1 -- 172.21.15.22:0/622034756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc074c30 msgr2=0x7fe2cc1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2d1200640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc074c30 0x7fe2cc1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2d1200640 1 -- 172.21.15.22:0/622034756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc19b020 con 0x7fe2cc10e6a0 2024-04-20T21:42:42.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.351+0000 7fe2c3400640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc074c30 0x7fe2cc1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:42.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.352+0000 7fe2d1200640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2cc10e6a0 0x7fe2cc19a760 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe2c402c790 tx=0x7fe2c4035e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:42.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.352+0000 7fe2c9600640 1 -- 172.21.15.22:0/622034756 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4039040 con 0x7fe2cc10e6a0 2024-04-20T21:42:42.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.352+0000 7fe2d1c00640 1 -- 172.21.15.22:0/622034756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2cc19b2b0 con 0x7fe2cc10e6a0 2024-04-20T21:42:42.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.352+0000 7fe2c9600640 1 -- 172.21.15.22:0/622034756 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2c4044070 con 0x7fe2cc10e6a0 2024-04-20T21:42:42.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.352+0000 7fe2c9600640 1 -- 172.21.15.22:0/622034756 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40328d0 con 0x7fe2cc10e6a0 2024-04-20T21:42:42.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.352+0000 7fe2d1c00640 1 -- 172.21.15.22:0/622034756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2cc1a73f0 con 0x7fe2cc10e6a0 2024-04-20T21:42:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.355+0000 7fe2c9600640 1 -- 172.21.15.22:0/622034756 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe2c4032a70 con 0x7fe2cc10e6a0 2024-04-20T21:42:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.356+0000 7fe2d6027740 1 Processor -- start 2024-04-20T21:42:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.356+0000 7fe2d6027740 1 -- start start 2024-04-20T21:42:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.356+0000 7fe2d6027740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c801b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.356+0000 7fe2d6027740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801bca0 0x5596c8024340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.356+0000 7fe2d6027740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 0x5596c8026c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.356+0000 7fe2d6027740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5596c7f81150 con 0x5596c801b380 2024-04-20T21:42:42.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.356+0000 7fe2d6027740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5596c7f27370 con 0x5596c8024880 2024-04-20T21:42:42.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.356+0000 7fe2d6027740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5596c7f7d110 con 0x5596c801bca0 2024-04-20T21:42:42.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.356+0000 7fe2c3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c801b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.357+0000 7fe2c3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c801b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34430/0 (socket says 172.21.15.22:34430) 2024-04-20T21:42:42.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.357+0000 7fe2c3400640 1 -- 172.21.15.22:0/17249536 learned_addr learned my addr 172.21.15.22:0/17249536 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:42.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.357+0000 7fe2cbe00640 1 --2- 172.21.15.22:0/17249536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801bca0 0x5596c8024340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.357+0000 7fe2c3400640 1 --2- 172.21.15.22:0/17249536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 0x5596c8026c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.357+0000 7fe2c3400640 1 -- 172.21.15.22:0/17249536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801bca0 msgr2=0x5596c8024340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:42.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.357+0000 7fe2c3400640 1 --2- 172.21.15.22:0/17249536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801bca0 0x5596c8024340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.357+0000 7fe2c3400640 1 -- 172.21.15.22:0/17249536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 msgr2=0x5596c8026c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:42.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.357+0000 7fe2c3400640 1 --2- 172.21.15.22:0/17249536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 0x5596c8026c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.357+0000 7fe2c3400640 1 -- 172.21.15.22:0/17249536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596c7d5d7d0 con 0x5596c801b380 2024-04-20T21:42:42.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.357+0000 7fe2c3400640 1 --2- 172.21.15.22:0/17249536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c801b760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4013ca0 tx=0x7fe2b4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e804dea98f5cd9a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:42.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.357+0000 7fe2c9600640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fe2b00a3d20 0x7fe2b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.358+0000 7fe2c1600640 1 -- 172.21.15.22:0/17249536 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4013d90 con 0x5596c801b380 2024-04-20T21:42:42.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.358+0000 7fe2c9600640 1 -- 172.21.15.22:0/622034756 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7fe2c40f2bf0 con 0x7fe2cc10e6a0 2024-04-20T21:42:42.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.358+0000 7fe2c1600640 1 -- 172.21.15.22:0/17249536 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b4011c70 con 0x5596c801b380 2024-04-20T21:42:42.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.358+0000 7fe2c1600640 1 -- 172.21.15.22:0/17249536 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b400f620 con 0x5596c801b380 2024-04-20T21:42:42.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.358+0000 7fe2cbe00640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fe2b00a3d20 0x7fe2b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.358+0000 7fe2d6027740 1 -- 172.21.15.22:0/17249536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 msgr2=0x5596c801b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.358+0000 7fe2d6027740 1 --2- 172.21.15.22:0/17249536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c801b760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4013ca0 tx=0x7fe2b4009c30 comp rx=0 tx=0).stop 2024-04-20T21:42:42.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.359+0000 7fe2cbe00640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fe2b00a3d20 0x7fe2b00a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5596c801ae50 tx=0x7fe2bc005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:42.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.359+0000 7fe2d6027740 1 -- 172.21.15.22:0/17249536 shutdown_connections 2024-04-20T21:42:42.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.359+0000 7fe2d6027740 1 --2- 172.21.15.22:0/17249536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 0x5596c8026c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.359+0000 7fe2d6027740 1 --2- 172.21.15.22:0/17249536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801bca0 0x5596c8024340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.359+0000 7fe2d6027740 1 --2- 172.21.15.22:0/17249536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c801b760 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.359+0000 7fe2d6027740 1 -- 172.21.15.22:0/17249536 >> 172.21.15.22:0/17249536 conn(0x5596c8015ab0 msgr2=0x5596c8015e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:42.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.359+0000 7fe2d6027740 1 -- 172.21.15.22:0/17249536 shutdown_connections 2024-04-20T21:42:42.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.359+0000 7fe2d6027740 1 -- 172.21.15.22:0/17249536 wait complete. 2024-04-20T21:42:42.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.359+0000 7fe2d6027740 1 -- start start 2024-04-20T21:42:42.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.360+0000 7fe2d6027740 1 Processor -- start 2024-04-20T21:42:42.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.360+0000 7fe2d6027740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c8021830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.360+0000 7fe2d6027740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801bca0 0x5596c8021d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.360+0000 7fe2d6027740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 0x5596c80222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.360+0000 7fe2d6027740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5596c8027670 con 0x5596c801b380 2024-04-20T21:42:42.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.360+0000 7fe2d6027740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5596c7f3a740 con 0x5596c8024880 2024-04-20T21:42:42.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.360+0000 7fe2d6027740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5596c80277f0 con 0x5596c801bca0 2024-04-20T21:42:42.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.360+0000 7fe2d6027740 10 client.0 _ll_register_callbacks cb 0x5596c7f35b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:42.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.360+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c8021830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.360+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c8021830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34446/0 (socket says 172.21.15.22:34446) 2024-04-20T21:42:42.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.360+0000 7fe2cbe00640 1 -- 172.21.15.22:0/824903536 learned_addr learned my addr 172.21.15.22:0/824903536 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:42.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.360+0000 7fe2c3400640 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801bca0 0x5596c8021d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.361+0000 7fe2d1200640 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 0x5596c80222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.361+0000 7fe2cbe00640 1 -- 172.21.15.22:0/824903536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801bca0 msgr2=0x5596c8021d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:42.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.361+0000 7fe2cbe00640 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801bca0 0x5596c8021d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.361+0000 7fe2cbe00640 1 -- 172.21.15.22:0/824903536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 msgr2=0x5596c80222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:42.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.361+0000 7fe2cbe00640 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 0x5596c80222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.361+0000 7fe2cbe00640 1 -- 172.21.15.22:0/824903536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596c7c933f0 con 0x5596c801b380 2024-04-20T21:42:42.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.361+0000 7fe2cbe00640 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c8021830 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0045f0 tx=0x7fe2bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e6522932a21342 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:42.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.361+0000 7fe2ba000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:42:42.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.361+0000 7fe2ba000640 1 -- 172.21.15.22:0/824903536 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc01f640 con 0x5596c801b380 2024-04-20T21:42:42.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.362+0000 7fe2ba000640 1 -- 172.21.15.22:0/824903536 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7fe2bc01f7e0 con 0x5596c801b380 2024-04-20T21:42:42.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.362+0000 7fe2d6027740 1 -- 172.21.15.22:0/824903536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b4011c70 con 0x5596c801b380 2024-04-20T21:42:42.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.362+0000 7fe2ba000640 1 -- 172.21.15.22:0/824903536 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe2bc01f9e0 con 0x5596c801b380 2024-04-20T21:42:42.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.362+0000 7fe2d6027740 10 client.16488 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:42.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.362+0000 7fe2d6027740 1 -- 172.21.15.22:0/824903536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5596c7c530f0 con 0x5596c801b380 2024-04-20T21:42:42.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.362+0000 7fe2ba000640 1 -- 172.21.15.22:0/824903536 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b4011c70 con 0x5596c801b380 2024-04-20T21:42:42.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.362+0000 7fe2ba000640 1 -- 172.21.15.22:0/824903536 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fe2bc027020 con 0x5596c801b380 2024-04-20T21:42:42.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.363+0000 7fe2d6027740 10 client.16488 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:42.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.363+0000 7fe2d6027740 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:42.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.363+0000 7fe2d6027740 20 client.16488 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:42.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.363+0000 7fe2d6027740 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5596c8034c30 0x5596c8037090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.363+0000 7fe2d6027740 4 client.16488 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:42.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.363+0000 7fe2d6027740 1 -- 172.21.15.22:0/824903536 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596c7f8eb70 con 0x5596c8034c30 2024-04-20T21:42:42.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.363+0000 7fe2c3400640 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5596c8034c30 0x5596c8037090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.364+0000 7fe2c3400640 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5596c8034c30 0x5596c8037090 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6eb8baee893b668 server_cookie=478c4af478ea182a in_seq=0 out_seq=0 2024-04-20T21:42:42.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.364+0000 7fe2ba000640 10 client.16488 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:42.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.364+0000 7fe2ba000640 1 -- 172.21.15.22:0/824903536 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596c7f8eb70 con 0x5596c8034c30 2024-04-20T21:42:42.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.364+0000 7fe2ba000640 10 client.16488 handle_command_reply: tid=0 2024-04-20T21:42:42.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.364+0000 7fe2d6027740 1 client.16488 shutdown 2024-04-20T21:42:42.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.365+0000 7fe2d6027740 1 -- 172.21.15.22:0/824903536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 msgr2=0x5596c8021830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.365+0000 7fe2d6027740 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c8021830 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0045f0 tx=0x7fe2bc004730 comp rx=0 tx=0).stop 2024-04-20T21:42:42.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.365+0000 7fe2d6027740 1 -- 172.21.15.22:0/824903536 shutdown_connections 2024-04-20T21:42:42.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.365+0000 7fe2d6027740 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5596c8034c30 0x5596c8037090 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.365+0000 7fe2d6027740 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 0x5596c80222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.365+0000 7fe2d6027740 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801bca0 0x5596c8021d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.365+0000 7fe2d6027740 1 --2- 172.21.15.22:0/824903536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c8021830 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.365+0000 7fe2ba000640 0 client.16488 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:42.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.365+0000 7fe2d6027740 1 -- 172.21.15.22:0/824903536 >> 172.21.15.22:0/824903536 conn(0x5596c8015ab0 msgr2=0x5596c8017800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:42.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.365+0000 7fe2d6027740 1 -- 172.21.15.22:0/824903536 shutdown_connections 2024-04-20T21:42:42.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.365+0000 7fe2d6027740 1 -- 172.21.15.22:0/824903536 wait complete. 2024-04-20T21:42:42.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.365+0000 7fe2d6027740 20 client.16488 trim_cache size 0 max 16384 2024-04-20T21:42:42.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.384+0000 7fe2d6027740 1 Processor -- start 2024-04-20T21:42:42.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.385+0000 7fe2d6027740 1 -- start start 2024-04-20T21:42:42.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.385+0000 7fe2d6027740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c8023f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.385+0000 7fe2d6027740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c801bca0 0x5596c8030530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.385+0000 7fe2d6027740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c8024880 0x5596c8020990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.385+0000 7fe2d6027740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc1adeb0 con 0x5596c801b380 2024-04-20T21:42:42.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.385+0000 7fe2d6027740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc1b0710 con 0x5596c801bca0 2024-04-20T21:42:42.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.385+0000 7fe2d6027740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc1ae500 con 0x5596c8024880 2024-04-20T21:42:42.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.385+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c8023f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.385+0000 7fe2d1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c8024880 0x5596c8020990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.385+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c8023f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34454/0 (socket says 172.21.15.22:34454) 2024-04-20T21:42:42.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.385+0000 7fe2cbe00640 1 -- 172.21.15.22:0/3265523080 learned_addr learned my addr 172.21.15.22:0/3265523080 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:42.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.385+0000 7fe2c3400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c801bca0 0x5596c8030530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.386+0000 7fe2d1200640 1 -- 172.21.15.22:0/3265523080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c801bca0 msgr2=0x5596c8030530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:42.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.386+0000 7fe2d1200640 1 --2- 172.21.15.22:0/3265523080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c801bca0 0x5596c8030530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.386+0000 7fe2d1200640 1 -- 172.21.15.22:0/3265523080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 msgr2=0x5596c8023f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:42.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.386+0000 7fe2d1200640 1 --2- 172.21.15.22:0/3265523080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c8023f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.386+0000 7fe2d1200640 1 -- 172.21.15.22:0/3265523080 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596c801a700 con 0x5596c8024880 2024-04-20T21:42:42.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.386+0000 7fe2d1200640 1 --2- 172.21.15.22:0/3265523080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c8024880 0x5596c8020990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40f03a0 tx=0x7fe2c40df040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3da592598beaa9a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:42.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.386+0000 7fe2c0c00640 1 -- 172.21.15.22:0/3265523080 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40bf380 con 0x5596c8024880 2024-04-20T21:42:42.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.386+0000 7fe2c0c00640 1 -- 172.21.15.22:0/3265523080 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2c40bec80 con 0x5596c8024880 2024-04-20T21:42:42.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.386+0000 7fe2c0c00640 1 -- 172.21.15.22:0/3265523080 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40bb2f0 con 0x5596c8024880 2024-04-20T21:42:42.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.387+0000 7fe2d6027740 1 -- 172.21.15.22:0/3265523080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c8024880 msgr2=0x5596c8020990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.387+0000 7fe2d6027740 1 --2- 172.21.15.22:0/3265523080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c8024880 0x5596c8020990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40f03a0 tx=0x7fe2c40df040 comp rx=0 tx=0).stop 2024-04-20T21:42:42.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.387+0000 7fe2d6027740 1 -- 172.21.15.22:0/3265523080 shutdown_connections 2024-04-20T21:42:42.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.387+0000 7fe2d6027740 1 --2- 172.21.15.22:0/3265523080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c8024880 0x5596c8020990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.387+0000 7fe2d6027740 1 --2- 172.21.15.22:0/3265523080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c801bca0 0x5596c8030530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.387+0000 7fe2d6027740 1 --2- 172.21.15.22:0/3265523080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801b380 0x5596c8023f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.387+0000 7fe2d6027740 1 -- 172.21.15.22:0/3265523080 >> 172.21.15.22:0/3265523080 conn(0x5596c8015ab0 msgr2=0x5596c80217f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:42.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.387+0000 7fe2d6027740 1 -- 172.21.15.22:0/3265523080 shutdown_connections 2024-04-20T21:42:42.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.387+0000 7fe2d6027740 1 -- 172.21.15.22:0/3265523080 wait complete. 2024-04-20T21:42:42.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.388+0000 7fe2d6027740 1 -- start start 2024-04-20T21:42:42.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.388+0000 7fe2d6027740 1 Processor -- start 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.388+0000 7fe2d6027740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801b380 0x5596c801e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.388+0000 7fe2d6027740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801bca0 0x5596c801e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.388+0000 7fe2d6027740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 0x5596c801ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.388+0000 7fe2d6027740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc1aebd0 con 0x5596c801bca0 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.388+0000 7fe2d6027740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc1b0040 con 0x5596c8024880 2024-04-20T21:42:42.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.388+0000 7fe2d6027740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc1af2a0 con 0x5596c801b380 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.388+0000 7fe2d6027740 10 client.0 _ll_register_callbacks cb 0x7fe2bc0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.388+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801b380 0x5596c801e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.389+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801b380 0x5596c801e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:39866/0 (socket says 172.21.15.22:39866) 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.389+0000 7fe2cbe00640 1 -- 172.21.15.22:0/1946069264 learned_addr learned my addr 172.21.15.22:0/1946069264 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.389+0000 7fe2c3400640 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801bca0 0x5596c801e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.389+0000 7fe2d1200640 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 0x5596c801ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.389+0000 7fe2cbe00640 1 -- 172.21.15.22:0/1946069264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 msgr2=0x5596c801ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:42.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.389+0000 7fe2cbe00640 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 0x5596c801ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.389+0000 7fe2cbe00640 1 -- 172.21.15.22:0/1946069264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801bca0 msgr2=0x5596c801e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:42.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.389+0000 7fe2cbe00640 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801bca0 0x5596c801e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.389+0000 7fe2cbe00640 1 -- 172.21.15.22:0/1946069264 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596c8026c70 con 0x5596c801b380 2024-04-20T21:42:42.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.389+0000 7fe2cbe00640 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801b380 0x5596c801e390 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc01fe40 tx=0x7fe2bc01fe70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1066a0cbc1f33172 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:42.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.389+0000 7fe2ba000640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:42:42.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.389+0000 7fe2ba000640 1 -- 172.21.15.22:0/1946069264 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc040060 con 0x5596c801b380 2024-04-20T21:42:42.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.390+0000 7fe2ba000640 1 -- 172.21.15.22:0/1946069264 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7fe2bc029850 con 0x5596c801b380 2024-04-20T21:42:42.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.390+0000 7fe2d6027740 1 -- 172.21.15.22:0/1946069264 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c40bec80 con 0x5596c801b380 2024-04-20T21:42:42.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.390+0000 7fe2ba000640 1 -- 172.21.15.22:0/1946069264 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe2bc003e50 con 0x5596c801b380 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.390+0000 7fe2d6027740 10 client.26665 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.390+0000 7fe2d6027740 1 -- 172.21.15.22:0/1946069264 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5596c809b1b0 con 0x5596c801b380 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.390+0000 7fe2ba000640 1 -- 172.21.15.22:0/1946069264 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2c40bec80 con 0x5596c801b380 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.390+0000 7fe2ba000640 1 -- 172.21.15.22:0/1946069264 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fe2bc027040 con 0x5596c801b380 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.390+0000 7fe2d6027740 10 client.26665 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.390+0000 7fe2d6027740 10 client.26665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.390+0000 7fe2d6027740 20 client.26665 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:42.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.390+0000 7fe2d6027740 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5596c809b960 0x5596c809dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.391+0000 7fe2d6027740 4 client.26665 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:42:42.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.391+0000 7fe2d6027740 1 -- 172.21.15.22:0/1946069264 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5596c7f889e0 con 0x5596c809b960 2024-04-20T21:42:42.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.391+0000 7fe2c3400640 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5596c809b960 0x5596c809dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.391+0000 7fe2c3400640 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5596c809b960 0x5596c809dd80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97763566c5818bae server_cookie=182118c7693f9a92 in_seq=0 out_seq=0 2024-04-20T21:42:42.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.391+0000 7fe2ba000640 10 client.26665 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:42.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2ba000640 1 -- 172.21.15.22:0/1946069264 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5596c7f889e0 con 0x5596c809b960 2024-04-20T21:42:42.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2ba000640 10 client.26665 handle_command_reply: tid=0 2024-04-20T21:42:42.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2d6027740 1 client.26665 shutdown 2024-04-20T21:42:42.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2d6027740 1 -- 172.21.15.22:0/1946069264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801b380 msgr2=0x5596c801e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2d6027740 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801b380 0x5596c801e390 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc01fe40 tx=0x7fe2bc01fe70 comp rx=0 tx=0).stop 2024-04-20T21:42:42.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2d6027740 1 -- 172.21.15.22:0/1946069264 shutdown_connections 2024-04-20T21:42:42.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2d6027740 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5596c809b960 0x5596c809dd80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2d6027740 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596c8024880 0x5596c801ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2d6027740 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5596c801bca0 0x5596c801e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2d6027740 1 --2- 172.21.15.22:0/1946069264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596c801b380 0x5596c801e390 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2ba000640 0 client.26665 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:42.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2d6027740 1 -- 172.21.15.22:0/1946069264 >> 172.21.15.22:0/1946069264 conn(0x5596c8015ab0 msgr2=0x5596c8014b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:42.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2d6027740 1 -- 172.21.15.22:0/1946069264 shutdown_connections 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2d6027740 1 -- 172.21.15.22:0/1946069264 wait complete. 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.392+0000 7fe2d6027740 20 client.26665 trim_cache size 0 max 16384 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:42:42.399 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.393+0000 7fe2d1c00640 1 -- 172.21.15.22:0/622034756 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fe2b00a3d20 msgr2=0x7fe2b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.393+0000 7fe2d1c00640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fe2b00a3d20 0x7fe2b00a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5596c801ae50 tx=0x7fe2bc005f90 comp rx=0 tx=0).stop 2024-04-20T21:42:42.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.393+0000 7fe2d1c00640 1 -- 172.21.15.22:0/622034756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2cc10e6a0 msgr2=0x7fe2cc19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.393+0000 7fe2d1c00640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2cc10e6a0 0x7fe2cc19a760 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe2c402c790 tx=0x7fe2c4035e00 comp rx=0 tx=0).stop 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.393+0000 7fe2d1c00640 1 -- 172.21.15.22:0/622034756 shutdown_connections 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.393+0000 7fe2d1c00640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fe2b00a3d20 0x7fe2b00a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.393+0000 7fe2d1c00640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2cc10e6a0 0x7fe2cc19a760 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.393+0000 7fe2d1c00640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2cc074c30 0x7fe2cc1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.393+0000 7fe2d1c00640 1 --2- 172.21.15.22:0/622034756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2cc074310 0x7fe2cc1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.393+0000 7fe2d1c00640 1 -- 172.21.15.22:0/622034756 >> 172.21.15.22:0/622034756 conn(0x7fe2cc0fd930 msgr2=0x7fe2cc0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:42.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.393+0000 7fe2d1c00640 1 -- 172.21.15.22:0/622034756 shutdown_connections 2024-04-20T21:42:42.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.393+0000 7fe2d1c00640 1 -- 172.21.15.22:0/622034756 wait complete. 2024-04-20T21:42:42.409 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:42:42.409 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:42.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:42 smithi022 ceph-mon[16843]: pgmap v275: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:42:42.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.612+0000 7fa707200640 1 -- 172.21.15.22:0/2145670194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa708101ff0 msgr2=0x7fa7081023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.612+0000 7fa707200640 1 --2- 172.21.15.22:0/2145670194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa708101ff0 0x7fa7081023d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009900 tx=0x7fa6f802bc30 comp rx=0 tx=0).stop 2024-04-20T21:42:42.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.613+0000 7fa707200640 1 -- 172.21.15.22:0/2145670194 shutdown_connections 2024-04-20T21:42:42.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.613+0000 7fa707200640 1 --2- 172.21.15.22:0/2145670194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7081070c0 0x7fa70810ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.613+0000 7fa707200640 1 --2- 172.21.15.22:0/2145670194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7081029a0 0x7fa7081069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.613+0000 7fa707200640 1 --2- 172.21.15.22:0/2145670194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa708101ff0 0x7fa7081023d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.613+0000 7fa707200640 1 -- 172.21.15.22:0/2145670194 >> 172.21.15.22:0/2145670194 conn(0x7fa7080fd890 msgr2=0x7fa7080ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:42.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.613+0000 7fa707200640 1 -- 172.21.15.22:0/2145670194 shutdown_connections 2024-04-20T21:42:42.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.613+0000 7fa707200640 1 -- 172.21.15.22:0/2145670194 wait complete. 2024-04-20T21:42:42.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.614+0000 7fa707200640 1 Processor -- start 2024-04-20T21:42:42.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.614+0000 7fa707200640 1 -- start start 2024-04-20T21:42:42.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.614+0000 7fa707200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708101ff0 0x7fa7081a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.614+0000 7fa707200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7081070c0 0x7fa7081a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.614+0000 7fa707200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa70819a6b0 0x7fa70819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.614+0000 7fa707200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa7081121a0 con 0x7fa7081070c0 2024-04-20T21:42:42.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.614+0000 7fa707200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa708112020 con 0x7fa708101ff0 2024-04-20T21:42:42.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.614+0000 7fa707200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa708112320 con 0x7fa70819a6b0 2024-04-20T21:42:42.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.615+0000 7fa705e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708101ff0 0x7fa7081a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.615+0000 7fa705e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708101ff0 0x7fa7081a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:55630/0 (socket says 172.21.15.22:55630) 2024-04-20T21:42:42.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.615+0000 7fa705e00640 1 -- 172.21.15.22:0/2040089950 learned_addr learned my addr 172.21.15.22:0/2040089950 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:42.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.615+0000 7fa705400640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7081070c0 0x7fa7081a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.615+0000 7fa706800640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa70819a6b0 0x7fa70819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.615+0000 7fa705e00640 1 -- 172.21.15.22:0/2040089950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa70819a6b0 msgr2=0x7fa70819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.615+0000 7fa705e00640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa70819a6b0 0x7fa70819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.615+0000 7fa705e00640 1 -- 172.21.15.22:0/2040089950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7081070c0 msgr2=0x7fa7081a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.615+0000 7fa705e00640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7081070c0 0x7fa7081a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.615+0000 7fa705e00640 1 -- 172.21.15.22:0/2040089950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70819b2b0 con 0x7fa708101ff0 2024-04-20T21:42:42.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.616+0000 7fa705e00640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708101ff0 0x7fa7081a05f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa6f802c240 tx=0x7fa6f80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:42.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.616+0000 7fa6fea00640 1 -- 172.21.15.22:0/2040089950 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8043020 con 0x7fa708101ff0 2024-04-20T21:42:42.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.616+0000 7fa707200640 1 -- 172.21.15.22:0/2040089950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70819b540 con 0x7fa708101ff0 2024-04-20T21:42:42.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.616+0000 7fa6fea00640 1 -- 172.21.15.22:0/2040089950 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f80054e0 con 0x7fa708101ff0 2024-04-20T21:42:42.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.616+0000 7fa6fea00640 1 -- 172.21.15.22:0/2040089950 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8003b20 con 0x7fa708101ff0 2024-04-20T21:42:42.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.616+0000 7fa707200640 1 -- 172.21.15.22:0/2040089950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7081a7730 con 0x7fa708101ff0 2024-04-20T21:42:42.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.617+0000 7fa707200640 1 -- 172.21.15.22:0/2040089950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7081034c0 con 0x7fa708101ff0 2024-04-20T21:42:42.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.620+0000 7fa6fea00640 1 -- 172.21.15.22:0/2040089950 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa6f8046020 con 0x7fa708101ff0 2024-04-20T21:42:42.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.621+0000 7fa6fea00640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa6e80a3da0 0x7fa6e80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:42.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.621+0000 7fa6fea00640 1 -- 172.21.15.22:0/2040089950 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7fa6f80f2a50 con 0x7fa708101ff0 2024-04-20T21:42:42.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.621+0000 7fa705400640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa6e80a3da0 0x7fa6e80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:42.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.622+0000 7fa705400640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa6e80a3da0 0x7fa6e80a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00099a0 tx=0x7fa6f0009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:42.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.623+0000 7fa6fea00640 1 -- 172.21.15.22:0/2040089950 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa6f804d020 con 0x7fa708101ff0 2024-04-20T21:42:42.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:42 smithi122 ceph-mon[22147]: pgmap v275: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:42:42.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:42 smithi033 ceph-mon[21626]: pgmap v275: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:42:42.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.871+0000 7fa707200640 1 -- 172.21.15.22:0/2040089950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7081029d0 con 0x7fa708101ff0 2024-04-20T21:42:42.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.872+0000 7fa6fea00640 1 -- 172.21.15.22:0/2040089950 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7fa6f80bf150 con 0x7fa708101ff0 2024-04-20T21:42:42.872 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:42:42.872 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:42:42.872 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:42:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.875+0000 7fa707200640 1 -- 172.21.15.22:0/2040089950 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa6e80a3da0 msgr2=0x7fa6e80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.875+0000 7fa707200640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa6e80a3da0 0x7fa6e80a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00099a0 tx=0x7fa6f0009500 comp rx=0 tx=0).stop 2024-04-20T21:42:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.875+0000 7fa707200640 1 -- 172.21.15.22:0/2040089950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708101ff0 msgr2=0x7fa7081a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.875+0000 7fa707200640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708101ff0 0x7fa7081a05f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa6f802c240 tx=0x7fa6f80042f0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.875+0000 7fa707200640 1 -- 172.21.15.22:0/2040089950 shutdown_connections 2024-04-20T21:42:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.875+0000 7fa707200640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa6e80a3da0 0x7fa6e80a6260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.875+0000 7fa707200640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa70819a6b0 0x7fa70819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.875+0000 7fa707200640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7081070c0 0x7fa7081a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.875+0000 7fa707200640 1 --2- 172.21.15.22:0/2040089950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708101ff0 0x7fa7081a05f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.875+0000 7fa707200640 1 -- 172.21.15.22:0/2040089950 >> 172.21.15.22:0/2040089950 conn(0x7fa7080fd890 msgr2=0x7fa7080ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:42.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.875+0000 7fa707200640 1 -- 172.21.15.22:0/2040089950 shutdown_connections 2024-04-20T21:42:42.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:42.875+0000 7fa707200640 1 -- 172.21.15.22:0/2040089950 wait complete. 2024-04-20T21:42:42.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:42:43.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.105+0000 7f43eec00640 1 -- 172.21.15.22:0/345186312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43e8109340 msgr2=0x7f43e8110f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:43.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.105+0000 7f43eec00640 1 --2- 172.21.15.22:0/345186312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43e8109340 0x7f43e8110f10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009ca0 tx=0x7f43e002c100 comp rx=0 tx=0).stop 2024-04-20T21:42:43.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.105+0000 7f43eec00640 1 -- 172.21.15.22:0/345186312 shutdown_connections 2024-04-20T21:42:43.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.105+0000 7f43eec00640 1 --2- 172.21.15.22:0/345186312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43e8109340 0x7f43e8110f10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.105+0000 7f43eec00640 1 --2- 172.21.15.22:0/345186312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43e8104c20 0x7f43e8108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.105+0000 7f43eec00640 1 --2- 172.21.15.22:0/345186312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e8104270 0x7f43e8104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.105+0000 7f43eec00640 1 -- 172.21.15.22:0/345186312 >> 172.21.15.22:0/345186312 conn(0x7f43e80ffa70 msgr2=0x7f43e8101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:43.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.105+0000 7f43eec00640 1 -- 172.21.15.22:0/345186312 shutdown_connections 2024-04-20T21:42:43.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.105+0000 7f43eec00640 1 -- 172.21.15.22:0/345186312 wait complete. 2024-04-20T21:42:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.106+0000 7f43eec00640 1 Processor -- start 2024-04-20T21:42:43.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.106+0000 7f43eec00640 1 -- start start 2024-04-20T21:42:43.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.107+0000 7f43eec00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43e8104270 0x7f43e81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.107+0000 7f43eec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43e8109340 0x7f43e81a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.107+0000 7f43eec00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e819a6a0 0x7f43e819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.107+0000 7f43eec00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8114400 con 0x7f43e8104270 2024-04-20T21:42:43.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.107+0000 7f43eec00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8114280 con 0x7f43e819a6a0 2024-04-20T21:42:43.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.107+0000 7f43eec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8114580 con 0x7f43e8109340 2024-04-20T21:42:43.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.107+0000 7f43ed800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43e8104270 0x7f43e81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.107+0000 7f43ed800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43e8104270 0x7f43e81a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34506/0 (socket says 172.21.15.22:34506) 2024-04-20T21:42:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.107+0000 7f43ed800640 1 -- 172.21.15.22:0/3737630746 learned_addr learned my addr 172.21.15.22:0/3737630746 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.107+0000 7f43ece00640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43e8109340 0x7f43e81a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.107+0000 7f43ee200640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e819a6a0 0x7f43e819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.108+0000 7f43ed800640 1 -- 172.21.15.22:0/3737630746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43e8109340 msgr2=0x7f43e81a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.108+0000 7f43ed800640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43e8109340 0x7f43e81a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.108+0000 7f43ed800640 1 -- 172.21.15.22:0/3737630746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e819a6a0 msgr2=0x7f43e819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.108+0000 7f43ed800640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e819a6a0 0x7f43e819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.108+0000 7f43ed800640 1 -- 172.21.15.22:0/3737630746 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e819b2a0 con 0x7f43e8104270 2024-04-20T21:42:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.108+0000 7f43ed800640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43e8104270 0x7f43e81a05e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00e9f0 tx=0x7f43dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.108+0000 7f43e6000640 1 -- 172.21.15.22:0/3737630746 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00cd80 con 0x7f43e8104270 2024-04-20T21:42:43.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.108+0000 7f43e6000640 1 -- 172.21.15.22:0/3737630746 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43dc010910 con 0x7f43e8104270 2024-04-20T21:42:43.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.108+0000 7f43e6000640 1 -- 172.21.15.22:0/3737630746 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc018990 con 0x7f43e8104270 2024-04-20T21:42:43.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.108+0000 7f43eec00640 1 -- 172.21.15.22:0/3737630746 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e81a7380 con 0x7f43e8104270 2024-04-20T21:42:43.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.108+0000 7f43eec00640 1 -- 172.21.15.22:0/3737630746 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e81a77e0 con 0x7f43e8104270 2024-04-20T21:42:43.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.111+0000 7f43f2fa3740 1 Processor -- start 2024-04-20T21:42:43.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43f2fa3740 1 -- start start 2024-04-20T21:42:43.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43f2fa3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108a170 0x55e61108a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43f2fa3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e61108aa90 0x55e611093130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43e6000640 1 -- 172.21.15.22:0/3737630746 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f43dc010ac0 con 0x7f43e8104270 2024-04-20T21:42:43.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43f2fa3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e611093670 0x55e611095a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43f2fa3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e610fef050 con 0x55e61108aa90 2024-04-20T21:42:43.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43f2fa3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e610f95490 con 0x55e611093670 2024-04-20T21:42:43.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43f2fa3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e610feb010 con 0x55e61108a170 2024-04-20T21:42:43.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43ece00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e611093670 0x55e611095a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43ed800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e61108aa90 0x55e611093130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43ee200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108a170 0x55e61108a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43ece00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e611093670 0x55e611095a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:55652/0 (socket says 172.21.15.22:55652) 2024-04-20T21:42:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.112+0000 7f43ece00640 1 -- 172.21.15.22:0/3105365887 learned_addr learned my addr 172.21.15.22:0/3105365887 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.113+0000 7f43ece00640 1 -- 172.21.15.22:0/3105365887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108a170 msgr2=0x55e61108a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.113+0000 7f43ece00640 1 --2- 172.21.15.22:0/3105365887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108a170 0x55e61108a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.113+0000 7f43ece00640 1 -- 172.21.15.22:0/3105365887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e61108aa90 msgr2=0x55e611093130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.113+0000 7f43ece00640 1 --2- 172.21.15.22:0/3105365887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e61108aa90 0x55e611093130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.113+0000 7f43ece00640 1 -- 172.21.15.22:0/3105365887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e610f9d070 con 0x55e611093670 2024-04-20T21:42:43.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.113+0000 7f43ee200640 1 --2- 172.21.15.22:0/3105365887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108a170 0x55e61108a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:43.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.113+0000 7f43ece00640 1 --2- 172.21.15.22:0/3105365887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e611093670 0x55e611095a60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009c00 tx=0x7f43d4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11b35bebe5e2aa07 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:43.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.113+0000 7f43daa00640 1 -- 172.21.15.22:0/3105365887 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4004230 con 0x55e611093670 2024-04-20T21:42:43.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.114+0000 7f43e6000640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43d00a3da0 0x7f43d00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.114+0000 7f43daa00640 1 -- 172.21.15.22:0/3105365887 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d40043d0 con 0x55e611093670 2024-04-20T21:42:43.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.114+0000 7f43daa00640 1 -- 172.21.15.22:0/3105365887 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4019710 con 0x55e611093670 2024-04-20T21:42:43.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.114+0000 7f43e6000640 1 -- 172.21.15.22:0/3737630746 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7f43dc0d0ad0 con 0x7f43e8104270 2024-04-20T21:42:43.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.114+0000 7f43ee200640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43d00a3da0 0x7f43d00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.114+0000 7f43f2fa3740 1 -- 172.21.15.22:0/3105365887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e611093670 msgr2=0x55e611095a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:43.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.114+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/3105365887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e611093670 0x55e611095a60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009c00 tx=0x7f43d4011810 comp rx=0 tx=0).stop 2024-04-20T21:42:43.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.115+0000 7f43f2fa3740 1 -- 172.21.15.22:0/3105365887 shutdown_connections 2024-04-20T21:42:43.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.115+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/3105365887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e611093670 0x55e611095a60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.115+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/3105365887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e61108aa90 0x55e611093130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.115+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/3105365887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108a170 0x55e61108a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.115+0000 7f43f2fa3740 1 -- 172.21.15.22:0/3105365887 >> 172.21.15.22:0/3105365887 conn(0x55e6110848a0 msgr2=0x55e611084c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:43.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.115+0000 7f43f2fa3740 1 -- 172.21.15.22:0/3105365887 shutdown_connections 2024-04-20T21:42:43.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.115+0000 7f43f2fa3740 1 -- 172.21.15.22:0/3105365887 wait complete. 2024-04-20T21:42:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.115+0000 7f43ee200640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43d00a3da0 0x7f43d00a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f43e0005e30 tx=0x7f43e0005d80 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.115+0000 7f43f2fa3740 1 -- start start 2024-04-20T21:42:43.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.115+0000 7f43f2fa3740 1 Processor -- start 2024-04-20T21:42:43.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.116+0000 7f43f2fa3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108a170 0x55e6110905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.116+0000 7f43f2fa3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108aa90 0x55e611090ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.116+0000 7f43f2fa3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e611093670 0x55e611091020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.116+0000 7f43f2fa3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e611096460 con 0x55e611093670 2024-04-20T21:42:43.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.116+0000 7f43f2fa3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e610fa8860 con 0x55e61108aa90 2024-04-20T21:42:43.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.116+0000 7f43f2fa3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e6110965e0 con 0x55e61108a170 2024-04-20T21:42:43.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.116+0000 7f43f2fa3740 10 client.0 _ll_register_callbacks cb 0x55e610f76fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:43.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.116+0000 7f43ece00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108aa90 0x55e611090ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.116+0000 7f43ee200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e611093670 0x55e611091020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.116+0000 7f43ece00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108aa90 0x55e611090ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:55660/0 (socket says 172.21.15.22:55660) 2024-04-20T21:42:43.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.116+0000 7f43ece00640 1 -- 172.21.15.22:0/4102855228 learned_addr learned my addr 172.21.15.22:0/4102855228 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:43.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.116+0000 7f43ed800640 1 --2- 172.21.15.22:0/4102855228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108a170 0x55e6110905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.117+0000 7f43ed800640 1 -- 172.21.15.22:0/4102855228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108aa90 msgr2=0x55e611090ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:43.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.117+0000 7f43ed800640 1 --2- 172.21.15.22:0/4102855228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108aa90 0x55e611090ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.117+0000 7f43ed800640 1 -- 172.21.15.22:0/4102855228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e611093670 msgr2=0x55e611091020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.117+0000 7f43ed800640 1 --2- 172.21.15.22:0/4102855228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e611093670 0x55e611091020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.117+0000 7f43ed800640 1 -- 172.21.15.22:0/4102855228 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e610fac950 con 0x55e61108a170 2024-04-20T21:42:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.117+0000 7f43ed800640 1 --2- 172.21.15.22:0/4102855228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108a170 0x55e6110905a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0ce1d0 tx=0x7f43dc0d4210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25b20768ff418951 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.117+0000 7f43cf400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:42:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.117+0000 7f43cf400640 1 -- 172.21.15.22:0/4102855228 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc09c610 con 0x55e61108a170 2024-04-20T21:42:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.117+0000 7f43cf400640 1 -- 172.21.15.22:0/4102855228 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7f43dc0bbde0 con 0x55e61108a170 2024-04-20T21:42:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.117+0000 7f43f2fa3740 1 -- 172.21.15.22:0/4102855228 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d40043d0 con 0x55e61108a170 2024-04-20T21:42:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.118+0000 7f43cf400640 1 -- 172.21.15.22:0/4102855228 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f43dc09c430 con 0x55e61108a170 2024-04-20T21:42:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.118+0000 7f43f2fa3740 10 client.26683 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.118+0000 7f43f2fa3740 1 -- 172.21.15.22:0/4102855228 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e610f7a820 con 0x55e61108a170 2024-04-20T21:42:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.118+0000 7f43cf400640 1 -- 172.21.15.22:0/4102855228 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d40043d0 con 0x55e61108a170 2024-04-20T21:42:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.118+0000 7f43cf400640 1 -- 172.21.15.22:0/4102855228 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f43dc09c040 con 0x55e61108a170 2024-04-20T21:42:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.118+0000 7f43f2fa3740 10 client.26683 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.118+0000 7f43f2fa3740 10 client.26683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.118+0000 7f43f2fa3740 20 client.26683 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.118+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/4102855228 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e6110a3530 0x55e6110a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.118+0000 7f43f2fa3740 4 client.26683 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.118+0000 7f43f2fa3740 1 -- 172.21.15.22:0/4102855228 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e610ffca70 con 0x55e6110a3530 2024-04-20T21:42:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.119+0000 7f43ece00640 1 --2- 172.21.15.22:0/4102855228 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e6110a3530 0x55e6110a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.119+0000 7f43ece00640 1 --2- 172.21.15.22:0/4102855228 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e6110a3530 0x55e6110a59b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d2648220434525 server_cookie=f9e7d3fd5e81d241 in_seq=0 out_seq=0 2024-04-20T21:42:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.119+0000 7f43cf400640 10 client.26683 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.120+0000 7f43cf400640 1 -- 172.21.15.22:0/4102855228 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e610ffca70 con 0x55e6110a3530 2024-04-20T21:42:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.120+0000 7f43cf400640 10 client.26683 handle_command_reply: tid=0 2024-04-20T21:42:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.120+0000 7f43f2fa3740 1 client.26683 shutdown 2024-04-20T21:42:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.120+0000 7f43f2fa3740 1 -- 172.21.15.22:0/4102855228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108a170 msgr2=0x55e6110905a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.120+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/4102855228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108a170 0x55e6110905a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0ce1d0 tx=0x7f43dc0d4210 comp rx=0 tx=0).stop 2024-04-20T21:42:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.120+0000 7f43f2fa3740 1 -- 172.21.15.22:0/4102855228 shutdown_connections 2024-04-20T21:42:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.120+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/4102855228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e611093670 0x55e611091020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.120+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/4102855228 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e6110a3530 0x55e6110a59b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.120+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/4102855228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108aa90 0x55e611090ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.120+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/4102855228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108a170 0x55e6110905a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.120+0000 7f43cf400640 0 client.26683 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.120+0000 7f43f2fa3740 1 -- 172.21.15.22:0/4102855228 >> 172.21.15.22:0/4102855228 conn(0x55e6110848a0 msgr2=0x55e6110865f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.121+0000 7f43f2fa3740 1 -- 172.21.15.22:0/4102855228 shutdown_connections 2024-04-20T21:42:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.121+0000 7f43f2fa3740 1 -- 172.21.15.22:0/4102855228 wait complete. 2024-04-20T21:42:43.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.121+0000 7f43f2fa3740 20 client.26683 trim_cache size 0 max 16384 2024-04-20T21:42:43.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.136+0000 7f43f2fa3740 1 Processor -- start 2024-04-20T21:42:43.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.136+0000 7f43f2fa3740 1 -- start start 2024-04-20T21:42:43.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.136+0000 7f43f2fa3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108a170 0x55e61109f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.136+0000 7f43f2fa3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108aa90 0x55e611092c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.136+0000 7f43f2fa3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e611093670 0x55e6110ab280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.136+0000 7f43f2fa3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81add90 con 0x55e611093670 2024-04-20T21:42:43.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.136+0000 7f43f2fa3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81b05f0 con 0x55e61108a170 2024-04-20T21:42:43.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.136+0000 7f43f2fa3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81ae3e0 con 0x55e61108aa90 2024-04-20T21:42:43.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.136+0000 7f43ed800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108a170 0x55e61109f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.136+0000 7f43ee200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e611093670 0x55e6110ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.136+0000 7f43ed800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108a170 0x55e61109f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:55674/0 (socket says 172.21.15.22:55674) 2024-04-20T21:42:43.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.136+0000 7f43ed800640 1 -- 172.21.15.22:0/393589216 learned_addr learned my addr 172.21.15.22:0/393589216 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:43.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.137+0000 7f43ece00640 1 --2- 172.21.15.22:0/393589216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108aa90 0x55e611092c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.137+0000 7f43ed800640 1 -- 172.21.15.22:0/393589216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108aa90 msgr2=0x55e611092c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:43.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.137+0000 7f43ed800640 1 --2- 172.21.15.22:0/393589216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108aa90 0x55e611092c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.137+0000 7f43ed800640 1 -- 172.21.15.22:0/393589216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e611093670 msgr2=0x55e6110ab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:43.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.137+0000 7f43ed800640 1 --2- 172.21.15.22:0/393589216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e611093670 0x55e6110ab280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.137+0000 7f43ed800640 1 -- 172.21.15.22:0/393589216 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61108c1c0 con 0x55e61108a170 2024-04-20T21:42:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.137+0000 7f43ed800640 1 --2- 172.21.15.22:0/393589216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108a170 0x55e61109f350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0097b0 tx=0x7f43dc0bd1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a230532f9c9fccc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.137+0000 7f43da000640 1 -- 172.21.15.22:0/393589216 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0bbde0 con 0x55e61108a170 2024-04-20T21:42:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.138+0000 7f43da000640 1 -- 172.21.15.22:0/393589216 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43dc0a0dd0 con 0x55e61108a170 2024-04-20T21:42:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.138+0000 7f43da000640 1 -- 172.21.15.22:0/393589216 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc09c120 con 0x55e61108a170 2024-04-20T21:42:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.138+0000 7f43f2fa3740 1 -- 172.21.15.22:0/393589216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108a170 msgr2=0x55e61109f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.138+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/393589216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108a170 0x55e61109f350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0097b0 tx=0x7f43dc0bd1c0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.138+0000 7f43f2fa3740 1 -- 172.21.15.22:0/393589216 shutdown_connections 2024-04-20T21:42:43.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.138+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/393589216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e611093670 0x55e6110ab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.138+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/393589216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e61108aa90 0x55e611092c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.138+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/393589216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108a170 0x55e61109f350 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.138+0000 7f43f2fa3740 1 -- 172.21.15.22:0/393589216 >> 172.21.15.22:0/393589216 conn(0x55e6110848a0 msgr2=0x55e611090df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.138+0000 7f43f2fa3740 1 -- 172.21.15.22:0/393589216 shutdown_connections 2024-04-20T21:42:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.138+0000 7f43f2fa3740 1 -- 172.21.15.22:0/393589216 wait complete. 2024-04-20T21:42:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.139+0000 7f43f2fa3740 1 -- start start 2024-04-20T21:42:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.139+0000 7f43f2fa3740 1 Processor -- start 2024-04-20T21:42:43.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.139+0000 7f43f2fa3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108a170 0x55e61108d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.139+0000 7f43f2fa3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e61108aa90 0x55e61108da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.139+0000 7f43f2fa3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e611093670 0x55e611111300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.139+0000 7f43f2fa3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81aeab0 con 0x55e61108aa90 2024-04-20T21:42:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.139+0000 7f43f2fa3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81aff20 con 0x55e61108a170 2024-04-20T21:42:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.139+0000 7f43f2fa3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81af180 con 0x55e611093670 2024-04-20T21:42:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.140+0000 7f43f2fa3740 10 client.0 _ll_register_callbacks cb 0x7f43dc009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.140+0000 7f43ed800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108a170 0x55e61108d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.140+0000 7f43ee200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e611093670 0x55e611111300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.140+0000 7f43ee200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e611093670 0x55e611111300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:39906/0 (socket says 172.21.15.22:39906) 2024-04-20T21:42:43.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.140+0000 7f43ee200640 1 -- 172.21.15.22:0/1744008638 learned_addr learned my addr 172.21.15.22:0/1744008638 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:43.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.140+0000 7f43ece00640 1 --2- 172.21.15.22:0/1744008638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e61108aa90 0x55e61108da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.140+0000 7f43ee200640 1 -- 172.21.15.22:0/1744008638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108a170 msgr2=0x55e61108d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:43.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.140+0000 7f43ee200640 1 --2- 172.21.15.22:0/1744008638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108a170 0x55e61108d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.140+0000 7f43ee200640 1 -- 172.21.15.22:0/1744008638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e61108aa90 msgr2=0x55e61108da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:43.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.140+0000 7f43ee200640 1 --2- 172.21.15.22:0/1744008638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e61108aa90 0x55e61108da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.140+0000 7f43ee200640 1 -- 172.21.15.22:0/1744008638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61108a8c0 con 0x55e611093670 2024-04-20T21:42:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.141+0000 7f43ee200640 1 --2- 172.21.15.22:0/1744008638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e611093670 0x55e611111300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009ca0 tx=0x7f43e00095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc1d187e83f54256 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.141+0000 7f43cf400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:42:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.141+0000 7f43cf400640 1 -- 172.21.15.22:0/1744008638 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e0039040 con 0x55e611093670 2024-04-20T21:42:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.141+0000 7f43f2fa3740 1 -- 172.21.15.22:0/1744008638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43dc0a0dd0 con 0x55e611093670 2024-04-20T21:42:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.141+0000 7f43cf400640 1 -- 172.21.15.22:0/1744008638 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7f43e0004450 con 0x55e611093670 2024-04-20T21:42:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.141+0000 7f43cf400640 1 -- 172.21.15.22:0/1744008638 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f43e0032e50 con 0x55e611093670 2024-04-20T21:42:43.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.141+0000 7f43f2fa3740 10 client.26689 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.141+0000 7f43f2fa3740 1 -- 172.21.15.22:0/1744008638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e610d013f0 con 0x55e611093670 2024-04-20T21:42:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.141+0000 7f43cf400640 1 -- 172.21.15.22:0/1744008638 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43dc0a0dd0 con 0x55e611093670 2024-04-20T21:42:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.141+0000 7f43cf400640 1 -- 172.21.15.22:0/1744008638 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f43e0040430 con 0x55e611093670 2024-04-20T21:42:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.142+0000 7f43f2fa3740 10 client.26689 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.142+0000 7f43f2fa3740 10 client.26689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.142+0000 7f43f2fa3740 20 client.26689 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.142+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/1744008638 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e611112c20 0x55e611115040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:43.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.142+0000 7f43f2fa3740 4 client.26689 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:42:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.142+0000 7f43f2fa3740 1 -- 172.21.15.22:0/1744008638 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e610ff68e0 con 0x55e611112c20 2024-04-20T21:42:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.142+0000 7f43ed800640 1 --2- 172.21.15.22:0/1744008638 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e611112c20 0x55e611115040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.142+0000 7f43ed800640 1 --2- 172.21.15.22:0/1744008638 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e611112c20 0x55e611115040 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc5b03cba55f0dd server_cookie=42ee3474bdb4126d in_seq=0 out_seq=0 2024-04-20T21:42:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.142+0000 7f43cf400640 10 client.26689 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43cf400640 1 -- 172.21.15.22:0/1744008638 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e610ff68e0 con 0x55e611112c20 2024-04-20T21:42:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43cf400640 10 client.26689 handle_command_reply: tid=0 2024-04-20T21:42:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43f2fa3740 1 client.26689 shutdown 2024-04-20T21:42:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43f2fa3740 1 -- 172.21.15.22:0/1744008638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e611093670 msgr2=0x55e611111300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/1744008638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e611093670 0x55e611111300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009ca0 tx=0x7f43e00095c0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43f2fa3740 1 -- 172.21.15.22:0/1744008638 shutdown_connections 2024-04-20T21:42:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/1744008638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e611093670 0x55e611111300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/1744008638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e61108aa90 0x55e61108da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/1744008638 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e611112c20 0x55e611115040 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43f2fa3740 1 --2- 172.21.15.22:0/1744008638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61108a170 0x55e61108d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43cf400640 0 client.26689 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:43.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43f2fa3740 1 -- 172.21.15.22:0/1744008638 >> 172.21.15.22:0/1744008638 conn(0x55e6110848a0 msgr2=0x55e6110830d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:43.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43f2fa3740 1 -- 172.21.15.22:0/1744008638 shutdown_connections 2024-04-20T21:42:43.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.143+0000 7f43f2fa3740 1 -- 172.21.15.22:0/1744008638 wait complete. 2024-04-20T21:42:43.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.144+0000 7f43f2fa3740 20 client.26689 trim_cache size 0 max 16384 2024-04-20T21:42:43.151 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:42:43.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.144+0000 7f43eec00640 1 -- 172.21.15.22:0/3737630746 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43d00a3da0 msgr2=0x7f43d00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:43.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.144+0000 7f43eec00640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43d00a3da0 0x7f43d00a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f43e0005e30 tx=0x7f43e0005d80 comp rx=0 tx=0).stop 2024-04-20T21:42:43.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.144+0000 7f43eec00640 1 -- 172.21.15.22:0/3737630746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43e8104270 msgr2=0x7f43e81a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:43.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.144+0000 7f43eec00640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43e8104270 0x7f43e81a05e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00e9f0 tx=0x7f43dc00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.144+0000 7f43eec00640 1 -- 172.21.15.22:0/3737630746 shutdown_connections 2024-04-20T21:42:43.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.144+0000 7f43eec00640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43d00a3da0 0x7f43d00a6260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.144+0000 7f43eec00640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e819a6a0 0x7f43e819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.144+0000 7f43eec00640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43e8109340 0x7f43e81a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.144+0000 7f43eec00640 1 --2- 172.21.15.22:0/3737630746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43e8104270 0x7f43e81a05e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:43.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.144+0000 7f43eec00640 1 -- 172.21.15.22:0/3737630746 >> 172.21.15.22:0/3737630746 conn(0x7f43e80ffa70 msgr2=0x7f43e8100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:43.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.145+0000 7f43eec00640 1 -- 172.21.15.22:0/3737630746 shutdown_connections 2024-04-20T21:42:43.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:43.145+0000 7f43eec00640 1 -- 172.21.15.22:0/3737630746 wait complete. 2024-04-20T21:42:43.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:43 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2040089950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:42:43.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:43 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2040089950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:42:43.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:43 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2040089950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:42:44.165 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:42:44.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:42:44.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:42:44.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.393+0000 7f07d2c00640 1 -- 172.21.15.22:0/489540350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07cc100f60 msgr2=0x7f07cc101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.393+0000 7f07d2c00640 1 --2- 172.21.15.22:0/489540350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07cc100f60 0x7f07cc101340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009900 tx=0x7f07c002bc50 comp rx=0 tx=0).stop 2024-04-20T21:42:44.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.394+0000 7f07d2c00640 1 -- 172.21.15.22:0/489540350 shutdown_connections 2024-04-20T21:42:44.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.394+0000 7f07d2c00640 1 --2- 172.21.15.22:0/489540350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07cc106030 0x7f07cc10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.394+0000 7f07d2c00640 1 --2- 172.21.15.22:0/489540350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07cc101910 0x7f07cc105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.394+0000 7f07d2c00640 1 --2- 172.21.15.22:0/489540350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07cc100f60 0x7f07cc101340 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.394+0000 7f07d2c00640 1 -- 172.21.15.22:0/489540350 >> 172.21.15.22:0/489540350 conn(0x7f07cc0fc7e0 msgr2=0x7f07cc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.394+0000 7f07d2c00640 1 -- 172.21.15.22:0/489540350 shutdown_connections 2024-04-20T21:42:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.394+0000 7f07d2c00640 1 -- 172.21.15.22:0/489540350 wait complete. 2024-04-20T21:42:44.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.395+0000 7f07d2c00640 1 Processor -- start 2024-04-20T21:42:44.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.395+0000 7f07d2c00640 1 -- start start 2024-04-20T21:42:44.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.395+0000 7f07d2c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07cc100f60 0x7f07cc1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.395+0000 7f07d2c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07cc106030 0x7f07cc1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.395+0000 7f07d2c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07cc19a7e0 0x7f07cc19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.395+0000 7f07d2c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1110f0 con 0x7f07cc106030 2024-04-20T21:42:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.395+0000 7f07d2c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc110f70 con 0x7f07cc19a7e0 2024-04-20T21:42:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.395+0000 7f07d2c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc111270 con 0x7f07cc100f60 2024-04-20T21:42:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.396+0000 7f07d0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07cc106030 0x7f07cc1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.396+0000 7f07d1800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07cc100f60 0x7f07cc1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.396+0000 7f07d0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07cc106030 0x7f07cc1a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34558/0 (socket says 172.21.15.22:34558) 2024-04-20T21:42:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.396+0000 7f07d0e00640 1 -- 172.21.15.22:0/2814803088 learned_addr learned my addr 172.21.15.22:0/2814803088 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.396+0000 7f07d2200640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07cc19a7e0 0x7f07cc19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.396+0000 7f07d0e00640 1 -- 172.21.15.22:0/2814803088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07cc100f60 msgr2=0x7f07cc1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.396+0000 7f07d0e00640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07cc100f60 0x7f07cc1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.396+0000 7f07d0e00640 1 -- 172.21.15.22:0/2814803088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07cc19a7e0 msgr2=0x7f07cc19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.396+0000 7f07d0e00640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07cc19a7e0 0x7f07cc19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.396+0000 7f07d0e00640 1 -- 172.21.15.22:0/2814803088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc19b370 con 0x7f07cc106030 2024-04-20T21:42:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.396+0000 7f07d2200640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07cc19a7e0 0x7f07cc19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.397+0000 7f07d0e00640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07cc106030 0x7f07cc1a0bb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e9f0 tx=0x7f07b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.397+0000 7f07ca000640 1 -- 172.21.15.22:0/2814803088 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b800ccc0 con 0x7f07cc106030 2024-04-20T21:42:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.397+0000 7f07ca000640 1 -- 172.21.15.22:0/2814803088 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b8005500 con 0x7f07cc106030 2024-04-20T21:42:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.397+0000 7f07d2c00640 1 -- 172.21.15.22:0/2814803088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07cc19b660 con 0x7f07cc106030 2024-04-20T21:42:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.397+0000 7f07ca000640 1 -- 172.21.15.22:0/2814803088 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8010690 con 0x7f07cc106030 2024-04-20T21:42:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.397+0000 7f07d2c00640 1 -- 172.21.15.22:0/2814803088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07cc1a77f0 con 0x7f07cc106030 2024-04-20T21:42:44.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.400+0000 7f07d7030740 1 Processor -- start 2024-04-20T21:42:44.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.400+0000 7f07d7030740 1 -- start start 2024-04-20T21:42:44.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.401+0000 7f07ca000640 1 -- 172.21.15.22:0/2814803088 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f07b80046c0 con 0x7f07cc106030 2024-04-20T21:42:44.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.401+0000 7f07d7030740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c30f0 0x55dbb74c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.401+0000 7f07d7030740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c3a10 0x55dbb74cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.401+0000 7f07d7030740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74cc5f0 0x55dbb74ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.401+0000 7f07d7030740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbb7427d20 con 0x55dbb74c3a10 2024-04-20T21:42:44.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.401+0000 7f07d7030740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dbb73ce3e0 con 0x55dbb74c30f0 2024-04-20T21:42:44.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.401+0000 7f07d7030740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dbb7423ce0 con 0x55dbb74cc5f0 2024-04-20T21:42:44.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.401+0000 7f07d1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c3a10 0x55dbb74cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.401+0000 7f07d1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c3a10 0x55dbb74cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34560/0 (socket says 172.21.15.22:34560) 2024-04-20T21:42:44.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.401+0000 7f07d1800640 1 -- 172.21.15.22:0/3982331793 learned_addr learned my addr 172.21.15.22:0/3982331793 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:44.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.401+0000 7f07d2200640 1 --2- 172.21.15.22:0/3982331793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c30f0 0x55dbb74c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.401+0000 7f07d0e00640 1 --2- 172.21.15.22:0/3982331793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74cc5f0 0x55dbb74ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.402+0000 7f07d1800640 1 -- 172.21.15.22:0/3982331793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74cc5f0 msgr2=0x55dbb74ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.402+0000 7f07d1800640 1 --2- 172.21.15.22:0/3982331793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74cc5f0 0x55dbb74ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.402+0000 7f07d1800640 1 -- 172.21.15.22:0/3982331793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c30f0 msgr2=0x55dbb74c34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.402+0000 7f07d1800640 1 --2- 172.21.15.22:0/3982331793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c30f0 0x55dbb74c34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.402+0000 7f07d1800640 1 -- 172.21.15.22:0/3982331793 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbb73ed980 con 0x55dbb74c3a10 2024-04-20T21:42:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.402+0000 7f07d2200640 1 --2- 172.21.15.22:0/3982331793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c30f0 0x55dbb74c34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.402+0000 7f07d1800640 1 --2- 172.21.15.22:0/3982331793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c3a10 0x55dbb74cc0b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f07c002c190 tx=0x7f07c0009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799c876e408e766e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.402+0000 7f07bea00640 1 -- 172.21.15.22:0/3982331793 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c002caa0 con 0x55dbb74c3a10 2024-04-20T21:42:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.402+0000 7f07bea00640 1 -- 172.21.15.22:0/3982331793 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c002c6b0 con 0x55dbb74c3a10 2024-04-20T21:42:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.402+0000 7f07bea00640 1 -- 172.21.15.22:0/3982331793 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0004ee0 con 0x55dbb74c3a10 2024-04-20T21:42:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.403+0000 7f07ca000640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07b40a3d20 0x7f07b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.403+0000 7f07ca000640 1 -- 172.21.15.22:0/2814803088 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7f07b80d0c00 con 0x7f07cc106030 2024-04-20T21:42:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.403+0000 7f07d2200640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07b40a3d20 0x7f07b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.403+0000 7f07d7030740 1 -- 172.21.15.22:0/3982331793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c3a10 msgr2=0x55dbb74cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.403+0000 7f07d7030740 1 --2- 172.21.15.22:0/3982331793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c3a10 0x55dbb74cc0b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f07c002c190 tx=0x7f07c0009610 comp rx=0 tx=0).stop 2024-04-20T21:42:44.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.403+0000 7f07d7030740 1 -- 172.21.15.22:0/3982331793 shutdown_connections 2024-04-20T21:42:44.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.403+0000 7f07d7030740 1 --2- 172.21.15.22:0/3982331793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74cc5f0 0x55dbb74ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.403+0000 7f07d7030740 1 --2- 172.21.15.22:0/3982331793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c3a10 0x55dbb74cc0b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.403+0000 7f07d7030740 1 --2- 172.21.15.22:0/3982331793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c30f0 0x55dbb74c34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.403+0000 7f07d7030740 1 -- 172.21.15.22:0/3982331793 >> 172.21.15.22:0/3982331793 conn(0x55dbb74bd820 msgr2=0x55dbb74bdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.404+0000 7f07d7030740 1 -- 172.21.15.22:0/3982331793 shutdown_connections 2024-04-20T21:42:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.404+0000 7f07d2200640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07b40a3d20 0x7f07b40a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009fd0 tx=0x7f07c4009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.404+0000 7f07d7030740 1 -- 172.21.15.22:0/3982331793 wait complete. 2024-04-20T21:42:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.404+0000 7f07d7030740 1 -- start start 2024-04-20T21:42:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.404+0000 7f07d7030740 1 Processor -- start 2024-04-20T21:42:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.405+0000 7f07d7030740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c30f0 0x55dbb74c9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.405+0000 7f07d7030740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74c3a10 0x55dbb74c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.405+0000 7f07d7030740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74cc5f0 0x55dbb74ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.405+0000 7f07d7030740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbb74cf380 con 0x55dbb74cc5f0 2024-04-20T21:42:44.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.405+0000 7f07d7030740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dbb73e17a0 con 0x55dbb74c30f0 2024-04-20T21:42:44.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.405+0000 7f07d7030740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dbb74cf500 con 0x55dbb74c3a10 2024-04-20T21:42:44.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.405+0000 7f07d7030740 10 client.0 _ll_register_callbacks cb 0x55dbb73e3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:44.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.405+0000 7f07d2200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74cc5f0 0x55dbb74ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.405+0000 7f07d2200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74cc5f0 0x55dbb74ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34574/0 (socket says 172.21.15.22:34574) 2024-04-20T21:42:44.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.405+0000 7f07d2200640 1 -- 172.21.15.22:0/1057521441 learned_addr learned my addr 172.21.15.22:0/1057521441 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:44.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.405+0000 7f07d1800640 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c30f0 0x55dbb74c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.405+0000 7f07d0e00640 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74c3a10 0x55dbb74c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.406+0000 7f07d2200640 1 -- 172.21.15.22:0/1057521441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74c3a10 msgr2=0x55dbb74c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.406+0000 7f07d2200640 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74c3a10 0x55dbb74c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.406+0000 7f07d2200640 1 -- 172.21.15.22:0/1057521441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c30f0 msgr2=0x55dbb74c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.406+0000 7f07d2200640 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c30f0 0x55dbb74c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.406+0000 7f07d2200640 1 -- 172.21.15.22:0/1057521441 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbb72027e0 con 0x55dbb74cc5f0 2024-04-20T21:42:44.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.406+0000 7f07d1800640 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c30f0 0x55dbb74c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:44.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.406+0000 7f07d2200640 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74cc5f0 0x55dbb74ca0c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f07c40120c0 tx=0x7f07c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778e6f21720c0710 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.406+0000 7f07b3400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:42:44.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.406+0000 7f07b3400640 1 -- 172.21.15.22:0/1057521441 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4010840 con 0x55dbb74cc5f0 2024-04-20T21:42:44.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.406+0000 7f07b3400640 1 -- 172.21.15.22:0/1057521441 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7f07c40109e0 con 0x55dbb74cc5f0 2024-04-20T21:42:44.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.406+0000 7f07d7030740 1 -- 172.21.15.22:0/1057521441 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dbb7435740 con 0x55dbb74cc5f0 2024-04-20T21:42:44.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.407+0000 7f07b3400640 1 -- 172.21.15.22:0/1057521441 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f07c4010be0 con 0x55dbb74cc5f0 2024-04-20T21:42:44.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.407+0000 7f07d7030740 10 client.16530 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:44.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.407+0000 7f07d7030740 1 -- 172.21.15.22:0/1057521441 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dbb7262f70 con 0x55dbb74cc5f0 2024-04-20T21:42:44.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.407+0000 7f07b3400640 1 -- 172.21.15.22:0/1057521441 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dbb7435740 con 0x55dbb74cc5f0 2024-04-20T21:42:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.407+0000 7f07b3400640 1 -- 172.21.15.22:0/1057521441 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f07c4021410 con 0x55dbb74cc5f0 2024-04-20T21:42:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.407+0000 7f07d7030740 10 client.16530 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.407+0000 7f07d7030740 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.407+0000 7f07d7030740 20 client.16530 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.408+0000 7f07d7030740 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dbb74dc9e0 0x55dbb74dee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.408+0000 7f07d7030740 4 client.16530 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.408+0000 7f07d7030740 1 -- 172.21.15.22:0/1057521441 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbb742f5b0 con 0x55dbb74dc9e0 2024-04-20T21:42:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.408+0000 7f07d1800640 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dbb74dc9e0 0x55dbb74dee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.409+0000 7f07d1800640 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dbb74dc9e0 0x55dbb74dee10 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e6cb183ca5faa3 server_cookie=2a7248bb404165e4 in_seq=0 out_seq=0 2024-04-20T21:42:44.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.409+0000 7f07b3400640 10 client.16530 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07b3400640 1 -- 172.21.15.22:0/1057521441 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbb742f5b0 con 0x55dbb74dc9e0 2024-04-20T21:42:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07b3400640 10 client.16530 handle_command_reply: tid=0 2024-04-20T21:42:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07d7030740 1 client.16530 shutdown 2024-04-20T21:42:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07d7030740 1 -- 172.21.15.22:0/1057521441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74cc5f0 msgr2=0x55dbb74ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07d7030740 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74cc5f0 0x55dbb74ca0c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f07c40120c0 tx=0x7f07c4004ab0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07d7030740 1 -- 172.21.15.22:0/1057521441 shutdown_connections 2024-04-20T21:42:44.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07d7030740 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74cc5f0 0x55dbb74ca0c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07d7030740 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74c3a10 0x55dbb74c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07d7030740 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dbb74dc9e0 0x55dbb74dee10 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07d7030740 1 --2- 172.21.15.22:0/1057521441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c30f0 0x55dbb74c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07d7030740 1 -- 172.21.15.22:0/1057521441 >> 172.21.15.22:0/1057521441 conn(0x55dbb74bd820 msgr2=0x55dbb74bf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07b3400640 0 client.16530 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07d7030740 1 -- 172.21.15.22:0/1057521441 shutdown_connections 2024-04-20T21:42:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07d7030740 1 -- 172.21.15.22:0/1057521441 wait complete. 2024-04-20T21:42:44.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.410+0000 7f07d7030740 20 client.16530 trim_cache size 0 max 16384 2024-04-20T21:42:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.430+0000 7f07d7030740 1 Processor -- start 2024-04-20T21:42:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.430+0000 7f07d7030740 1 -- start start 2024-04-20T21:42:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.430+0000 7f07d7030740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74bd820 0x55dbb74d8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.430+0000 7f07d7030740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c30f0 0x55dbb74c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.430+0000 7f07d7030740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c3a10 0x55dbb74e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.430+0000 7f07d7030740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1adea0 con 0x55dbb74c30f0 2024-04-20T21:42:44.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.430+0000 7f07d7030740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1b0700 con 0x55dbb74c3a10 2024-04-20T21:42:44.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.430+0000 7f07d7030740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1ae4f0 con 0x55dbb74bd820 2024-04-20T21:42:44.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.431+0000 7f07d0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c30f0 0x55dbb74c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.431+0000 7f07d0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c30f0 0x55dbb74c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34580/0 (socket says 172.21.15.22:34580) 2024-04-20T21:42:44.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.431+0000 7f07d0e00640 1 -- 172.21.15.22:0/2438685967 learned_addr learned my addr 172.21.15.22:0/2438685967 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:44.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.431+0000 7f07d2200640 1 --2- 172.21.15.22:0/2438685967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c3a10 0x55dbb74e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.431+0000 7f07d1800640 1 --2- 172.21.15.22:0/2438685967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74bd820 0x55dbb74d8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.431+0000 7f07d0e00640 1 -- 172.21.15.22:0/2438685967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74bd820 msgr2=0x55dbb74d8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.431+0000 7f07d0e00640 1 --2- 172.21.15.22:0/2438685967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74bd820 0x55dbb74d8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.431+0000 7f07d0e00640 1 -- 172.21.15.22:0/2438685967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c3a10 msgr2=0x55dbb74e46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.431+0000 7f07d0e00640 1 --2- 172.21.15.22:0/2438685967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c3a10 0x55dbb74e46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.431+0000 7f07d0e00640 1 -- 172.21.15.22:0/2438685967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbb70fa170 con 0x55dbb74c30f0 2024-04-20T21:42:44.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.431+0000 7f07d2200640 1 --2- 172.21.15.22:0/2438685967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c3a10 0x55dbb74e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:42:44.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.431+0000 7f07d0e00640 1 --2- 172.21.15.22:0/2438685967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c30f0 0x55dbb74c9d10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f07b800cf10 tx=0x7f07b8004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea46f89809f50d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.432+0000 7f07be000640 1 -- 172.21.15.22:0/2438685967 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b809c450 con 0x55dbb74c30f0 2024-04-20T21:42:44.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.432+0000 7f07be000640 1 -- 172.21.15.22:0/2438685967 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b809d1f0 con 0x55dbb74c30f0 2024-04-20T21:42:44.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.432+0000 7f07be000640 1 -- 172.21.15.22:0/2438685967 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80056b0 con 0x55dbb74c30f0 2024-04-20T21:42:44.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.432+0000 7f07d7030740 1 -- 172.21.15.22:0/2438685967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c30f0 msgr2=0x55dbb74c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.432+0000 7f07d7030740 1 --2- 172.21.15.22:0/2438685967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c30f0 0x55dbb74c9d10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f07b800cf10 tx=0x7f07b8004ba0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.432+0000 7f07d7030740 1 -- 172.21.15.22:0/2438685967 shutdown_connections 2024-04-20T21:42:44.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.432+0000 7f07d7030740 1 --2- 172.21.15.22:0/2438685967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c3a10 0x55dbb74e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.432+0000 7f07d7030740 1 --2- 172.21.15.22:0/2438685967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74c30f0 0x55dbb74c9d10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.432+0000 7f07d7030740 1 --2- 172.21.15.22:0/2438685967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74bd820 0x55dbb74d8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.432+0000 7f07d7030740 1 -- 172.21.15.22:0/2438685967 >> 172.21.15.22:0/2438685967 conn(0x55dbb74c8280 msgr2=0x55dbb74c9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.433+0000 7f07d7030740 1 -- 172.21.15.22:0/2438685967 shutdown_connections 2024-04-20T21:42:44.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.433+0000 7f07d7030740 1 -- 172.21.15.22:0/2438685967 wait complete. 2024-04-20T21:42:44.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.433+0000 7f07d7030740 1 -- start start 2024-04-20T21:42:44.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.433+0000 7f07d7030740 1 Processor -- start 2024-04-20T21:42:44.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d7030740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74bd820 0x55dbb74c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d7030740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74c30f0 0x55dbb74c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d7030740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c3a10 0x55dbb74c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d7030740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1aebc0 con 0x55dbb74bd820 2024-04-20T21:42:44.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d7030740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1b0030 con 0x55dbb74c3a10 2024-04-20T21:42:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d7030740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc1af290 con 0x55dbb74c30f0 2024-04-20T21:42:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d7030740 10 client.0 _ll_register_callbacks cb 0x7f07c4012840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74bd820 0x55dbb74c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d2200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c3a10 0x55dbb74c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74bd820 0x55dbb74c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34590/0 (socket says 172.21.15.22:34590) 2024-04-20T21:42:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d1800640 1 -- 172.21.15.22:0/1107849504 learned_addr learned my addr 172.21.15.22:0/1107849504 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d0e00640 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74c30f0 0x55dbb74c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d2200640 1 -- 172.21.15.22:0/1107849504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74c30f0 msgr2=0x55dbb74c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d2200640 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74c30f0 0x55dbb74c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d2200640 1 -- 172.21.15.22:0/1107849504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74bd820 msgr2=0x55dbb74c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d2200640 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74bd820 0x55dbb74c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d2200640 1 -- 172.21.15.22:0/1107849504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbb7542100 con 0x55dbb74c3a10 2024-04-20T21:42:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d1800640 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74bd820 0x55dbb74c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.434+0000 7f07d0e00640 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74c30f0 0x55dbb74c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.435+0000 7f07d2200640 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c3a10 0x55dbb74c6f40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f07c4021880 tx=0x7f07c40125d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f10b71fa8b0b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.435+0000 7f07b3400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:42:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.435+0000 7f07b3400640 1 -- 172.21.15.22:0/1107849504 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c403f060 con 0x55dbb74c3a10 2024-04-20T21:42:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.435+0000 7f07d7030740 1 -- 172.21.15.22:0/1107849504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b809d1f0 con 0x55dbb74c3a10 2024-04-20T21:42:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.435+0000 7f07b3400640 1 -- 172.21.15.22:0/1107849504 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7f07c4021af0 con 0x55dbb74c3a10 2024-04-20T21:42:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.435+0000 7f07b3400640 1 -- 172.21.15.22:0/1107849504 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f07c4036040 con 0x55dbb74c3a10 2024-04-20T21:42:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.435+0000 7f07d7030740 10 client.26681 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.435+0000 7f07d7030740 1 -- 172.21.15.22:0/1107849504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dbb7543480 con 0x55dbb74c3a10 2024-04-20T21:42:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.436+0000 7f07b3400640 1 -- 172.21.15.22:0/1107849504 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b809d1f0 con 0x55dbb74c3a10 2024-04-20T21:42:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.436+0000 7f07b3400640 1 -- 172.21.15.22:0/1107849504 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f07c4039030 con 0x55dbb74c3a10 2024-04-20T21:42:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.436+0000 7f07d7030740 10 client.26681 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.436+0000 7f07d7030740 10 client.26681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.436+0000 7f07d7030740 20 client.26681 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.436+0000 7f07d7030740 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dbb74cc5f0 0x55dbb7545c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.436+0000 7f07d7030740 4 client.26681 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:42:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.436+0000 7f07d7030740 1 -- 172.21.15.22:0/1107849504 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dbb73abb10 con 0x55dbb74cc5f0 2024-04-20T21:42:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.436+0000 7f07d1800640 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dbb74cc5f0 0x55dbb7545c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.437+0000 7f07d1800640 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dbb74cc5f0 0x55dbb7545c10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c9e334e99fd5561 server_cookie=e2349697cc46c875 in_seq=0 out_seq=0 2024-04-20T21:42:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.437+0000 7f07b3400640 10 client.26681 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.438+0000 7f07b3400640 1 -- 172.21.15.22:0/1107849504 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dbb73abb10 con 0x55dbb74cc5f0 2024-04-20T21:42:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.438+0000 7f07b3400640 10 client.26681 handle_command_reply: tid=0 2024-04-20T21:42:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.438+0000 7f07d7030740 1 client.26681 shutdown 2024-04-20T21:42:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.438+0000 7f07d7030740 1 -- 172.21.15.22:0/1107849504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c3a10 msgr2=0x55dbb74c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.438+0000 7f07d7030740 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c3a10 0x55dbb74c6f40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f07c4021880 tx=0x7f07c40125d0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.439+0000 7f07d7030740 1 -- 172.21.15.22:0/1107849504 shutdown_connections 2024-04-20T21:42:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.439+0000 7f07d7030740 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55dbb74cc5f0 0x55dbb7545c10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.439+0000 7f07d7030740 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbb74c3a10 0x55dbb74c6f40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.439+0000 7f07d7030740 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb74c30f0 0x55dbb74c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.439+0000 7f07d7030740 1 --2- 172.21.15.22:0/1107849504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb74bd820 0x55dbb74c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.439+0000 7f07d7030740 1 -- 172.21.15.22:0/1107849504 >> 172.21.15.22:0/1107849504 conn(0x55dbb74c8280 msgr2=0x55dbb74c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.439+0000 7f07b3400640 0 client.26681 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.439+0000 7f07d7030740 1 -- 172.21.15.22:0/1107849504 shutdown_connections 2024-04-20T21:42:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.439+0000 7f07d7030740 1 -- 172.21.15.22:0/1107849504 wait complete. 2024-04-20T21:42:44.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.439+0000 7f07d7030740 20 client.26681 trim_cache size 0 max 16384 2024-04-20T21:42:44.444 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:42:44.445 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:42:44.445 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "bba2e851-f414-441d-af2a-15f18515560e", 2024-04-20T21:42:44.445 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:42:44.445 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:42:44.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.439+0000 7f07d2c00640 1 -- 172.21.15.22:0/2814803088 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07b40a3d20 msgr2=0x7f07b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.439+0000 7f07d2c00640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07b40a3d20 0x7f07b40a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009fd0 tx=0x7f07c4009500 comp rx=0 tx=0).stop 2024-04-20T21:42:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.440+0000 7f07d2c00640 1 -- 172.21.15.22:0/2814803088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07cc106030 msgr2=0x7f07cc1a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.440+0000 7f07d2c00640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07cc106030 0x7f07cc1a0bb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e9f0 tx=0x7f07b800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.440+0000 7f07d2c00640 1 -- 172.21.15.22:0/2814803088 shutdown_connections 2024-04-20T21:42:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.440+0000 7f07d2c00640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f07b40a3d20 0x7f07b40a61e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.440+0000 7f07d2c00640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07cc19a7e0 0x7f07cc19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.440+0000 7f07d2c00640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07cc106030 0x7f07cc1a0bb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.440+0000 7f07d2c00640 1 --2- 172.21.15.22:0/2814803088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07cc100f60 0x7f07cc1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.440+0000 7f07d2c00640 1 -- 172.21.15.22:0/2814803088 >> 172.21.15.22:0/2814803088 conn(0x7f07cc0fc7e0 msgr2=0x7f07cc0699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.440+0000 7f07d2c00640 1 -- 172.21.15.22:0/2814803088 shutdown_connections 2024-04-20T21:42:44.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.440+0000 7f07d2c00640 1 -- 172.21.15.22:0/2814803088 wait complete. 2024-04-20T21:42:44.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:42:44.459 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:42:44.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:44 smithi022 ceph-mon[16843]: pgmap v276: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.669+0000 7fa253e00640 1 -- 172.21.15.22:0/1655969524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa25410e780 msgr2=0x7fa254110b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.669+0000 7fa253e00640 1 --2- 172.21.15.22:0/1655969524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa25410e780 0x7fa254110b70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009ca0 tx=0x7fa24c02c100 comp rx=0 tx=0).stop 2024-04-20T21:42:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.669+0000 7fa253e00640 1 -- 172.21.15.22:0/1655969524 shutdown_connections 2024-04-20T21:42:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.669+0000 7fa253e00640 1 --2- 172.21.15.22:0/1655969524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa25410e780 0x7fa254110b70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.669+0000 7fa253e00640 1 --2- 172.21.15.22:0/1655969524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa254101580 0x7fa25410e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.669+0000 7fa253e00640 1 --2- 172.21.15.22:0/1655969524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa254100bd0 0x7fa254100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.669+0000 7fa253e00640 1 -- 172.21.15.22:0/1655969524 >> 172.21.15.22:0/1655969524 conn(0x7fa2540fc7e0 msgr2=0x7fa2540fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.669+0000 7fa253e00640 1 -- 172.21.15.22:0/1655969524 shutdown_connections 2024-04-20T21:42:44.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.670+0000 7fa253e00640 1 -- 172.21.15.22:0/1655969524 wait complete. 2024-04-20T21:42:44.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.670+0000 7fa253e00640 1 Processor -- start 2024-04-20T21:42:44.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.671+0000 7fa253e00640 1 -- start start 2024-04-20T21:42:44.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.671+0000 7fa253e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa254100bd0 0x7fa2541a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.671+0000 7fa253e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa254101580 0x7fa2541a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.671+0000 7fa253e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25410e780 0x7fa25419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.671+0000 7fa253e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541143b0 con 0x7fa254100bd0 2024-04-20T21:42:44.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.671+0000 7fa253e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa254114230 con 0x7fa25410e780 2024-04-20T21:42:44.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.671+0000 7fa253e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa254114530 con 0x7fa254101580 2024-04-20T21:42:44.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.671+0000 7fa252a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa254100bd0 0x7fa2541a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.671+0000 7fa253400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25410e780 0x7fa25419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.671+0000 7fa252a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa254100bd0 0x7fa2541a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34618/0 (socket says 172.21.15.22:34618) 2024-04-20T21:42:44.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.671+0000 7fa252a00640 1 -- 172.21.15.22:0/367482970 learned_addr learned my addr 172.21.15.22:0/367482970 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:44.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.671+0000 7fa252000640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa254101580 0x7fa2541a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.672+0000 7fa253400640 1 -- 172.21.15.22:0/367482970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa254101580 msgr2=0x7fa2541a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.672+0000 7fa253400640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa254101580 0x7fa2541a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.672+0000 7fa253400640 1 -- 172.21.15.22:0/367482970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa254100bd0 msgr2=0x7fa2541a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.672+0000 7fa253400640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa254100bd0 0x7fa2541a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.672+0000 7fa253400640 1 -- 172.21.15.22:0/367482970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25419af90 con 0x7fa25410e780 2024-04-20T21:42:44.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.672+0000 7fa252a00640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa254100bd0 0x7fa2541a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:44.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.672+0000 7fa253400640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25410e780 0x7fa25419a6d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa24c02c710 tx=0x7fa24c032f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.672+0000 7fa24b400640 1 -- 172.21.15.22:0/367482970 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c039040 con 0x7fa25410e780 2024-04-20T21:42:44.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.672+0000 7fa253e00640 1 -- 172.21.15.22:0/367482970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25419b220 con 0x7fa25410e780 2024-04-20T21:42:44.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.672+0000 7fa24b400640 1 -- 172.21.15.22:0/367482970 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24c044070 con 0x7fa25410e780 2024-04-20T21:42:44.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.673+0000 7fa24b400640 1 -- 172.21.15.22:0/367482970 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c0042a0 con 0x7fa25410e780 2024-04-20T21:42:44.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.673+0000 7fa253e00640 1 -- 172.21.15.22:0/367482970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2541a73b0 con 0x7fa25410e780 2024-04-20T21:42:44.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.676+0000 7fa25c3e1740 1 Processor -- start 2024-04-20T21:42:44.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.676+0000 7fa24b400640 1 -- 172.21.15.22:0/367482970 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa24c032590 con 0x7fa25410e780 2024-04-20T21:42:44.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.676+0000 7fa25c3e1740 1 -- start start 2024-04-20T21:42:44.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.677+0000 7fa25c3e1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135cb380 0x55a0135cb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.677+0000 7fa25c3e1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cbca0 0x55a0135d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.677+0000 7fa25c3e1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135d4880 0x55a0135d6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.677+0000 7fa25c3e1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a013531150 con 0x55a0135d4880 2024-04-20T21:42:44.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.677+0000 7fa25c3e1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a0134d7370 con 0x55a0135cb380 2024-04-20T21:42:44.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.677+0000 7fa25c3e1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a01352d110 con 0x55a0135cbca0 2024-04-20T21:42:44.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.677+0000 7fa252000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135d4880 0x55a0135d6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.677+0000 7fa252a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cbca0 0x55a0135d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.677+0000 7fa252a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cbca0 0x55a0135d4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:39982/0 (socket says 172.21.15.22:39982) 2024-04-20T21:42:44.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.677+0000 7fa252a00640 1 -- 172.21.15.22:0/514635750 learned_addr learned my addr 172.21.15.22:0/514635750 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:44.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.677+0000 7fa252000640 1 --2- 172.21.15.22:0/514635750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135cb380 0x55a0135cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.678+0000 7fa252a00640 1 -- 172.21.15.22:0/514635750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135cb380 msgr2=0x55a0135cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.678+0000 7fa252a00640 1 --2- 172.21.15.22:0/514635750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135cb380 0x55a0135cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.678+0000 7fa252a00640 1 -- 172.21.15.22:0/514635750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135d4880 msgr2=0x55a0135d6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.678+0000 7fa252a00640 1 --2- 172.21.15.22:0/514635750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135d4880 0x55a0135d6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.678+0000 7fa252a00640 1 -- 172.21.15.22:0/514635750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a01330d7d0 con 0x55a0135cbca0 2024-04-20T21:42:44.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.678+0000 7fa252000640 1 --2- 172.21.15.22:0/514635750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135d4880 0x55a0135d6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:44.679 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:44 smithi122 ceph-mon[22147]: pgmap v276: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:44.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.678+0000 7fa252a00640 1 --2- 172.21.15.22:0/514635750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cbca0 0x55a0135d4340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa244009fd0 tx=0x7fa244011ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e03faff76980f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.678+0000 7fa23f400640 1 -- 172.21.15.22:0/514635750 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24401e070 con 0x55a0135cbca0 2024-04-20T21:42:44.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.678+0000 7fa24b400640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa2380a3da0 0x7fa2380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.678+0000 7fa23f400640 1 -- 172.21.15.22:0/514635750 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24400fbd0 con 0x55a0135cbca0 2024-04-20T21:42:44.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.678+0000 7fa23f400640 1 -- 172.21.15.22:0/514635750 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244019c80 con 0x55a0135cbca0 2024-04-20T21:42:44.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.679+0000 7fa24b400640 1 -- 172.21.15.22:0/367482970 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7fa24c0f2c20 con 0x7fa25410e780 2024-04-20T21:42:44.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.679+0000 7fa252000640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa2380a3da0 0x7fa2380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.679+0000 7fa25c3e1740 1 -- 172.21.15.22:0/514635750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cbca0 msgr2=0x55a0135d4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.679+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/514635750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cbca0 0x55a0135d4340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa244009fd0 tx=0x7fa244011ae0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.679+0000 7fa25c3e1740 1 -- 172.21.15.22:0/514635750 shutdown_connections 2024-04-20T21:42:44.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.679+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/514635750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135d4880 0x55a0135d6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.679+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/514635750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cbca0 0x55a0135d4340 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.679+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/514635750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135cb380 0x55a0135cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.679+0000 7fa25c3e1740 1 -- 172.21.15.22:0/514635750 >> 172.21.15.22:0/514635750 conn(0x55a0135c5ab0 msgr2=0x55a0135c5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.680+0000 7fa25c3e1740 1 -- 172.21.15.22:0/514635750 shutdown_connections 2024-04-20T21:42:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.680+0000 7fa252000640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa2380a3da0 0x7fa2380a6260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55a0135c6c40 tx=0x7fa240004730 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.680+0000 7fa25c3e1740 1 -- 172.21.15.22:0/514635750 wait complete. 2024-04-20T21:42:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.680+0000 7fa25c3e1740 1 -- start start 2024-04-20T21:42:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.680+0000 7fa25c3e1740 1 Processor -- start 2024-04-20T21:42:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa25c3e1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 0x55a0135d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa25c3e1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a0135d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa25c3e1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135d4880 0x55a0135d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa25c3e1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a0135d7670 con 0x55a0135cbca0 2024-04-20T21:42:44.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa25c3e1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a0134ea740 con 0x55a0135d4880 2024-04-20T21:42:44.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa25c3e1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a0135d77f0 con 0x55a0135cb380 2024-04-20T21:42:44.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa25c3e1740 10 client.0 _ll_register_callbacks cb 0x55a0134e5b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:44.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa252a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 0x55a0135d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa252000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a0135d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa252a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 0x55a0135d1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:39992/0 (socket says 172.21.15.22:39992) 2024-04-20T21:42:44.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa252000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a0135d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34636/0 (socket says 172.21.15.22:34636) 2024-04-20T21:42:44.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa252a00640 1 -- 172.21.15.22:0/291583946 learned_addr learned my addr 172.21.15.22:0/291583946 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:44.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa253400640 1 --2- 172.21.15.22:0/291583946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135d4880 0x55a0135d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa252a00640 1 -- 172.21.15.22:0/291583946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135d4880 msgr2=0x55a0135d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa252a00640 1 --2- 172.21.15.22:0/291583946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135d4880 0x55a0135d2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa252a00640 1 -- 172.21.15.22:0/291583946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 msgr2=0x55a0135d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.681+0000 7fa252a00640 1 --2- 172.21.15.22:0/291583946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a0135d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.682+0000 7fa252a00640 1 -- 172.21.15.22:0/291583946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0132433f0 con 0x55a0135cb380 2024-04-20T21:42:44.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.682+0000 7fa252a00640 1 --2- 172.21.15.22:0/291583946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 0x55a0135d1910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa244000c00 tx=0x7fa24400fb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9471b67719400735 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.682+0000 7fa233e00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:42:44.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.682+0000 7fa233e00640 1 -- 172.21.15.22:0/291583946 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24401e040 con 0x55a0135cb380 2024-04-20T21:42:44.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.682+0000 7fa233e00640 1 -- 172.21.15.22:0/291583946 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7fa244004040 con 0x55a0135cb380 2024-04-20T21:42:44.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.682+0000 7fa25c3e1740 1 -- 172.21.15.22:0/291583946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa24400fbd0 con 0x55a0135cb380 2024-04-20T21:42:44.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.682+0000 7fa233e00640 1 -- 172.21.15.22:0/291583946 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa24402f050 con 0x55a0135cb380 2024-04-20T21:42:44.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.682+0000 7fa25c3e1740 10 client.26719 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:44.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.683+0000 7fa25c3e1740 1 -- 172.21.15.22:0/291583946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a0132030f0 con 0x55a0135cb380 2024-04-20T21:42:44.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.683+0000 7fa233e00640 1 -- 172.21.15.22:0/291583946 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24400fbd0 con 0x55a0135cb380 2024-04-20T21:42:44.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.683+0000 7fa233e00640 1 -- 172.21.15.22:0/291583946 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fa24402a030 con 0x55a0135cb380 2024-04-20T21:42:44.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.683+0000 7fa25c3e1740 10 client.26719 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:44.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.683+0000 7fa25c3e1740 10 client.26719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:44.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.683+0000 7fa25c3e1740 20 client.26719 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:44.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.683+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/291583946 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0135e4c00 0x55a0135e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.683+0000 7fa25c3e1740 4 client.26719 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:44.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.683+0000 7fa25c3e1740 1 -- 172.21.15.22:0/291583946 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a01353eb70 con 0x55a0135e4c00 2024-04-20T21:42:44.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.683+0000 7fa252000640 1 --2- 172.21.15.22:0/291583946 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0135e4c00 0x55a0135e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.684+0000 7fa252000640 1 --2- 172.21.15.22:0/291583946 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0135e4c00 0x55a0135e6ff0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea479074d79f6345 server_cookie=c8ec30aaf130f104 in_seq=0 out_seq=0 2024-04-20T21:42:44.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.684+0000 7fa233e00640 10 client.26719 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:44.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa233e00640 1 -- 172.21.15.22:0/291583946 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a01353eb70 con 0x55a0135e4c00 2024-04-20T21:42:44.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa233e00640 10 client.26719 handle_command_reply: tid=0 2024-04-20T21:42:44.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa25c3e1740 1 client.26719 shutdown 2024-04-20T21:42:44.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa25c3e1740 1 -- 172.21.15.22:0/291583946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 msgr2=0x55a0135d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/291583946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 0x55a0135d1910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa244000c00 tx=0x7fa24400fb20 comp rx=0 tx=0).stop 2024-04-20T21:42:44.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa25c3e1740 1 -- 172.21.15.22:0/291583946 shutdown_connections 2024-04-20T21:42:44.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/291583946 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0135e4c00 0x55a0135e6ff0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/291583946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135d4880 0x55a0135d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/291583946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a0135d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa233e00640 0 client.26719 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:44.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/291583946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 0x55a0135d1910 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa25c3e1740 1 -- 172.21.15.22:0/291583946 >> 172.21.15.22:0/291583946 conn(0x55a0135c5ab0 msgr2=0x55a0135c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa25c3e1740 1 -- 172.21.15.22:0/291583946 shutdown_connections 2024-04-20T21:42:44.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa25c3e1740 1 -- 172.21.15.22:0/291583946 wait complete. 2024-04-20T21:42:44.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.685+0000 7fa25c3e1740 20 client.26719 trim_cache size 0 max 16384 2024-04-20T21:42:44.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.705+0000 7fa25c3e1740 1 Processor -- start 2024-04-20T21:42:44.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.705+0000 7fa25c3e1740 1 -- start start 2024-04-20T21:42:44.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.705+0000 7fa25c3e1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135c5ab0 0x55a0135d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.705+0000 7fa25c3e1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 0x55a0135d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.705+0000 7fa25c3e1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a0135e0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.705+0000 7fa25c3e1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541adda0 con 0x55a0135cbca0 2024-04-20T21:42:44.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.705+0000 7fa25c3e1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b0600 con 0x55a0135c5ab0 2024-04-20T21:42:44.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.705+0000 7fa25c3e1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541ae3f0 con 0x55a0135cb380 2024-04-20T21:42:44.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.706+0000 7fa252000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 0x55a0135d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.706+0000 7fa252a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135c5ab0 0x55a0135d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.706+0000 7fa252000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 0x55a0135d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:40008/0 (socket says 172.21.15.22:40008) 2024-04-20T21:42:44.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.706+0000 7fa252000640 1 -- 172.21.15.22:0/1902143653 learned_addr learned my addr 172.21.15.22:0/1902143653 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:44.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.706+0000 7fa252000640 1 -- 172.21.15.22:0/1902143653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135c5ab0 msgr2=0x55a0135d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.706+0000 7fa252000640 1 --2- 172.21.15.22:0/1902143653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135c5ab0 0x55a0135d4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.706+0000 7fa252000640 1 -- 172.21.15.22:0/1902143653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 msgr2=0x55a0135e0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.706+0000 7fa253400640 1 --2- 172.21.15.22:0/1902143653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a0135e0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.706+0000 7fa252000640 1 --2- 172.21.15.22:0/1902143653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a0135e0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.706+0000 7fa252000640 1 -- 172.21.15.22:0/1902143653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0135d7550 con 0x55a0135cb380 2024-04-20T21:42:44.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.706+0000 7fa253400640 1 --2- 172.21.15.22:0/1902143653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a0135e0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:42:44.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.706+0000 7fa252000640 1 --2- 172.21.15.22:0/1902143653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 0x55a0135d0850 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa240009d30 tx=0x7fa240013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=751c6416616e3260 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.707+0000 7fa23ea00640 1 -- 172.21.15.22:0/1902143653 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240020d40 con 0x55a0135cb380 2024-04-20T21:42:44.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.707+0000 7fa23ea00640 1 -- 172.21.15.22:0/1902143653 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa240033070 con 0x55a0135cb380 2024-04-20T21:42:44.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.707+0000 7fa23ea00640 1 -- 172.21.15.22:0/1902143653 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24002c470 con 0x55a0135cb380 2024-04-20T21:42:44.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.707+0000 7fa25c3e1740 1 -- 172.21.15.22:0/1902143653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 msgr2=0x55a0135d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.707+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/1902143653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 0x55a0135d0850 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa240009d30 tx=0x7fa240013040 comp rx=0 tx=0).stop 2024-04-20T21:42:44.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.708+0000 7fa25c3e1740 1 -- 172.21.15.22:0/1902143653 shutdown_connections 2024-04-20T21:42:44.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.708+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/1902143653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a0135e0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.708+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/1902143653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135cb380 0x55a0135d0850 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.708+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/1902143653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135c5ab0 0x55a0135d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.708+0000 7fa25c3e1740 1 -- 172.21.15.22:0/1902143653 >> 172.21.15.22:0/1902143653 conn(0x55a0135cd2e0 msgr2=0x55a0135c4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.708+0000 7fa25c3e1740 1 -- 172.21.15.22:0/1902143653 shutdown_connections 2024-04-20T21:42:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.708+0000 7fa25c3e1740 1 -- 172.21.15.22:0/1902143653 wait complete. 2024-04-20T21:42:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.708+0000 7fa25c3e1740 1 -- start start 2024-04-20T21:42:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.708+0000 7fa25c3e1740 1 Processor -- start 2024-04-20T21:42:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa25c3e1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135c5ab0 0x55a0135d87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa25c3e1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135cb380 0x55a0135d8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa25c3e1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a01364a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa25c3e1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541aeac0 con 0x55a0135cbca0 2024-04-20T21:42:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa25c3e1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541aff30 con 0x55a0135cb380 2024-04-20T21:42:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa25c3e1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541af190 con 0x55a0135c5ab0 2024-04-20T21:42:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa25c3e1740 10 client.0 _ll_register_callbacks cb 0x7fa24401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa253400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a01364a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa253400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a01364a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34650/0 (socket says 172.21.15.22:34650) 2024-04-20T21:42:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa253400640 1 -- 172.21.15.22:0/3214923103 learned_addr learned my addr 172.21.15.22:0/3214923103 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:42:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa252a00640 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135c5ab0 0x55a0135d87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa252000640 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135cb380 0x55a0135d8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa252a00640 1 -- 172.21.15.22:0/3214923103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135cb380 msgr2=0x55a0135d8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa252a00640 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135cb380 0x55a0135d8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa252a00640 1 -- 172.21.15.22:0/3214923103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 msgr2=0x55a01364a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:44.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa252a00640 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a01364a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa252a00640 1 -- 172.21.15.22:0/3214923103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0135d95e0 con 0x55a0135c5ab0 2024-04-20T21:42:44.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa252000640 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135cb380 0x55a0135d8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:44.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.709+0000 7fa253400640 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a01364a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:44.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.710+0000 7fa252a00640 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135c5ab0 0x55a0135d87a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa244005140 tx=0x7fa2440197c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c612cd6ae0ac2fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:44.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.710+0000 7fa233e00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:42:44.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.710+0000 7fa233e00640 1 -- 172.21.15.22:0/3214923103 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24401e040 con 0x55a0135c5ab0 2024-04-20T21:42:44.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.710+0000 7fa233e00640 1 -- 172.21.15.22:0/3214923103 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8600+0+0 (secure 0 0 0) 0x7fa244005540 con 0x55a0135c5ab0 2024-04-20T21:42:44.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.710+0000 7fa25c3e1740 1 -- 172.21.15.22:0/3214923103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa240033070 con 0x55a0135c5ab0 2024-04-20T21:42:44.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.710+0000 7fa233e00640 1 -- 172.21.15.22:0/3214923103 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa24400f3f0 con 0x55a0135c5ab0 2024-04-20T21:42:44.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.710+0000 7fa25c3e1740 10 client.26725 fetch_fsmap learned FSMap version 20 2024-04-20T21:42:44.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.710+0000 7fa25c3e1740 1 -- 172.21.15.22:0/3214923103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a01364b5e0 con 0x55a0135c5ab0 2024-04-20T21:42:44.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.711+0000 7fa233e00640 1 -- 172.21.15.22:0/3214923103 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa240033070 con 0x55a0135c5ab0 2024-04-20T21:42:44.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.711+0000 7fa233e00640 1 -- 172.21.15.22:0/3214923103 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7fa24402a040 con 0x55a0135c5ab0 2024-04-20T21:42:44.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.711+0000 7fa25c3e1740 10 client.26725 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:42:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.711+0000 7fa25c3e1740 10 client.26725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.711+0000 7fa25c3e1740 20 client.26725 populate_metadata read hostname 'smithi022' 2024-04-20T21:42:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.711+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0135d4880 0x55a01364dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.711+0000 7fa25c3e1740 4 client.26725 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:42:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.711+0000 7fa25c3e1740 1 -- 172.21.15.22:0/3214923103 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0135389e0 con 0x55a0135d4880 2024-04-20T21:42:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.711+0000 7fa252000640 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0135d4880 0x55a01364dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:44.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa252000640 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0135d4880 0x55a01364dca0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b18dd2f965f76bf0 server_cookie=db7a0dafe64c767e in_seq=0 out_seq=0 2024-04-20T21:42:44.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa233e00640 10 client.26725 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:44.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa233e00640 1 -- 172.21.15.22:0/3214923103 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a0135389e0 con 0x55a0135d4880 2024-04-20T21:42:44.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa233e00640 10 client.26725 handle_command_reply: tid=0 2024-04-20T21:42:44.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa25c3e1740 1 client.26725 shutdown 2024-04-20T21:42:44.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa25c3e1740 1 -- 172.21.15.22:0/3214923103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135c5ab0 msgr2=0x55a0135d87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135c5ab0 0x55a0135d87a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa244005140 tx=0x7fa2440197c0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa25c3e1740 1 -- 172.21.15.22:0/3214923103 shutdown_connections 2024-04-20T21:42:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a0135cbca0 0x55a01364a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55a0135d4880 0x55a01364dca0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a0135cb380 0x55a0135d8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa25c3e1740 1 --2- 172.21.15.22:0/3214923103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a0135c5ab0 0x55a0135d87a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa233e00640 0 client.26725 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:42:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.712+0000 7fa25c3e1740 1 -- 172.21.15.22:0/3214923103 >> 172.21.15.22:0/3214923103 conn(0x55a0135cd2e0 msgr2=0x55a0135d18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.713+0000 7fa25c3e1740 1 -- 172.21.15.22:0/3214923103 shutdown_connections 2024-04-20T21:42:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.713+0000 7fa25c3e1740 1 -- 172.21.15.22:0/3214923103 wait complete. 2024-04-20T21:42:44.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.713+0000 7fa25c3e1740 20 client.26725 trim_cache size 0 max 16384 2024-04-20T21:42:44.719 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:42:44.719 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:42:44.719 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:42:44.720 INFO:teuthology.orchestra.run.smithi022.stdout: "bba2e851-f414-441d-af2a-15f18515560e": { 2024-04-20T21:42:44.720 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:42:44.720 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "bba2e851-f414-441d-af2a-15f18515560e", 2024-04-20T21:42:44.720 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:42:44.720 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:42:44.720 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:42:44.720 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:42:44.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.713+0000 7fa253e00640 1 -- 172.21.15.22:0/367482970 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa2380a3da0 msgr2=0x7fa2380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.713+0000 7fa253e00640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa2380a3da0 0x7fa2380a6260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55a0135c6c40 tx=0x7fa240004730 comp rx=0 tx=0).stop 2024-04-20T21:42:44.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.713+0000 7fa253e00640 1 -- 172.21.15.22:0/367482970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25410e780 msgr2=0x7fa25419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:44.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.713+0000 7fa253e00640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25410e780 0x7fa25419a6d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa24c02c710 tx=0x7fa24c032f30 comp rx=0 tx=0).stop 2024-04-20T21:42:44.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.714+0000 7fa253e00640 1 -- 172.21.15.22:0/367482970 shutdown_connections 2024-04-20T21:42:44.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.714+0000 7fa253e00640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fa2380a3da0 0x7fa2380a6260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.714+0000 7fa253e00640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25410e780 0x7fa25419a6d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.714+0000 7fa253e00640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa254101580 0x7fa2541a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.714+0000 7fa253e00640 1 --2- 172.21.15.22:0/367482970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa254100bd0 0x7fa2541a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:44.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.714+0000 7fa253e00640 1 -- 172.21.15.22:0/367482970 >> 172.21.15.22:0/367482970 conn(0x7fa2540fc7e0 msgr2=0x7fa25410fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:44.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.714+0000 7fa253e00640 1 -- 172.21.15.22:0/367482970 shutdown_connections 2024-04-20T21:42:44.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:42:44.714+0000 7fa253e00640 1 -- 172.21.15.22:0/367482970 wait complete. 2024-04-20T21:42:44.730 INFO:tasks.cephfs.filesystem:scrub status for tag:bba2e851-f414-441d-af2a-15f18515560e - {'path': '/', 'tag': 'bba2e851-f414-441d-af2a-15f18515560e', 'options': 'recursive,force'} 2024-04-20T21:42:44.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:44 smithi033 ceph-mon[21626]: pgmap v276: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:45.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:45 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:42:45.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:45 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:45.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:45 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:42:45.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:45 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:45.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:45 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:42:45.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:45 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:45.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:45 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:42:45.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:45 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:45.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:45 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:42:45.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:45 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:45.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:45 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:42:45.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:45 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:46 smithi022 ceph-mon[16843]: pgmap v277: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:42:46.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:46 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:42:46.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:46 smithi122 ceph-mon[22147]: pgmap v277: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:42:46.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:46 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:42:46.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:46 smithi033 ceph-mon[21626]: pgmap v277: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:42:46.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:46 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:42:48.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:48 smithi022 ceph-mon[16843]: pgmap v278: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:42:48.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:48 smithi122 ceph-mon[22147]: pgmap v278: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:42:48.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:48 smithi033 ceph-mon[21626]: pgmap v278: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:42:50.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:50 smithi022 ceph-mon[16843]: pgmap v279: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:42:50.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:50 smithi122 ceph-mon[22147]: pgmap v279: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:42:50.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:50 smithi033 ceph-mon[21626]: pgmap v279: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:42:52.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:52 smithi022 ceph-mon[16843]: pgmap v280: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:42:52.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:52 smithi122 ceph-mon[22147]: pgmap v280: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:42:52.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:52 smithi033 ceph-mon[21626]: pgmap v280: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:42:54.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:54 smithi022 ceph-mon[16843]: pgmap v281: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:42:54.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:54 smithi122 ceph-mon[22147]: pgmap v281: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:42:54.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:54 smithi033 ceph-mon[21626]: pgmap v281: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:42:56.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:56 smithi022 ceph-mon[16843]: pgmap v282: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:42:56.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:56 smithi122 ceph-mon[22147]: pgmap v282: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:42:56.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:56 smithi033 ceph-mon[21626]: pgmap v282: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:42:58.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:42:58 smithi022 ceph-mon[16843]: pgmap v283: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:58.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:42:58 smithi122 ceph-mon[22147]: pgmap v283: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:42:58.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:42:58 smithi033 ceph-mon[21626]: pgmap v283: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:00.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:00 smithi022 ceph-mon[16843]: pgmap v284: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:00.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:00 smithi122 ceph-mon[22147]: pgmap v284: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:00.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:00 smithi033 ceph-mon[21626]: pgmap v284: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:02.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:02 smithi022 ceph-mon[16843]: pgmap v285: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:43:02.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:02 smithi122 ceph-mon[22147]: pgmap v285: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:43:02.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:02 smithi033 ceph-mon[21626]: pgmap v285: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:43:04.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:04 smithi122 ceph-mon[22147]: pgmap v286: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:04.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:04 smithi033 ceph-mon[21626]: pgmap v286: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:04.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:04 smithi022 ceph-mon[16843]: pgmap v286: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:06.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:06 smithi122 ceph-mon[22147]: pgmap v287: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-20T21:43:06.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:06 smithi033 ceph-mon[21626]: pgmap v287: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-20T21:43:06.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:06 smithi022 ceph-mon[16843]: pgmap v287: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-20T21:43:07.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:07 smithi122 ceph-mon[22147]: osdmap e100: 12 total, 12 up, 12 in 2024-04-20T21:43:07.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:07 smithi033 ceph-mon[21626]: osdmap e100: 12 total, 12 up, 12 in 2024-04-20T21:43:07.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:07 smithi022 ceph-mon[16843]: osdmap e100: 12 total, 12 up, 12 in 2024-04-20T21:43:08.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:08 smithi122 ceph-mon[22147]: pgmap v289: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:43:08.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:08 smithi033 ceph-mon[21626]: pgmap v289: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:43:08.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:08 smithi022 ceph-mon[16843]: pgmap v289: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:43:10.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:10 smithi122 ceph-mon[22147]: pgmap v290: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:43:10.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:10 smithi033 ceph-mon[21626]: pgmap v290: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:43:10.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:10 smithi022 ceph-mon[16843]: pgmap v290: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:43:12.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:12 smithi122 ceph-mon[22147]: pgmap v291: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:43:12.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:12 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:43:12.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:12 smithi033 ceph-mon[21626]: pgmap v291: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:43:12.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:12 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:43:12.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:12 smithi022 ceph-mon[16843]: pgmap v291: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:43:12.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:12 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:43:14.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:43:14.731 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:43:14.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:14 smithi122 ceph-mon[22147]: pgmap v292: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:43:14.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:14 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:43:14.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:14 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:43:14.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:14 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:43:14.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:14 smithi122 ceph-mon[22147]: osdmap e101: 12 total, 12 up, 12 in 2024-04-20T21:43:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:14 smithi033 ceph-mon[21626]: pgmap v292: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:43:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:14 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:43:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:14 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:43:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:14 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:43:14.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:14 smithi033 ceph-mon[21626]: osdmap e101: 12 total, 12 up, 12 in 2024-04-20T21:43:14.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:14 smithi022 ceph-mon[16843]: pgmap v292: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:43:14.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:14 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:43:14.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:14 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:43:14.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:14 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:43:14.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:14 smithi022 ceph-mon[16843]: osdmap e101: 12 total, 12 up, 12 in 2024-04-20T21:43:14.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.958+0000 7f846a800640 1 -- 172.21.15.22:0/498213573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8464102ba0 msgr2=0x7f8464106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:14.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.958+0000 7f846a800640 1 --2- 172.21.15.22:0/498213573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8464102ba0 0x7f8464106be0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8450009860 tx=0x7f845002bb70 comp rx=0 tx=0).stop 2024-04-20T21:43:14.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.958+0000 7f846a800640 1 -- 172.21.15.22:0/498213573 shutdown_connections 2024-04-20T21:43:14.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.958+0000 7f846a800640 1 --2- 172.21.15.22:0/498213573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84641072c0 0x7f846410ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.958+0000 7f846a800640 1 --2- 172.21.15.22:0/498213573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8464102ba0 0x7f8464106be0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.958+0000 7f846a800640 1 --2- 172.21.15.22:0/498213573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84641021f0 0x7f84641025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.958+0000 7f846a800640 1 -- 172.21.15.22:0/498213573 >> 172.21.15.22:0/498213573 conn(0x7f84640fda70 msgr2=0x7f84640ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:14.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.959+0000 7f846a800640 1 -- 172.21.15.22:0/498213573 shutdown_connections 2024-04-20T21:43:14.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.959+0000 7f846a800640 1 -- 172.21.15.22:0/498213573 wait complete. 2024-04-20T21:43:14.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.960+0000 7f846a800640 1 Processor -- start 2024-04-20T21:43:14.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.960+0000 7f846a800640 1 -- start start 2024-04-20T21:43:14.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.960+0000 7f846a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84641021f0 0x7f84641a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.960+0000 7f846a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84641072c0 0x7f84641a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.960+0000 7f846a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f846419a950 0x7f846419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.960+0000 7f846a800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8464112330 con 0x7f84641072c0 2024-04-20T21:43:14.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.960+0000 7f846a800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84641121b0 con 0x7f84641021f0 2024-04-20T21:43:14.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.960+0000 7f846a800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84641124b0 con 0x7f846419a950 2024-04-20T21:43:14.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.960+0000 7f8462a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84641072c0 0x7f84641a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.961+0000 7f8462a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84641072c0 0x7f84641a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53944/0 (socket says 172.21.15.22:53944) 2024-04-20T21:43:14.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.961+0000 7f8462a00640 1 -- 172.21.15.22:0/3719125317 learned_addr learned my addr 172.21.15.22:0/3719125317 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:14.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.961+0000 7f8463e00640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f846419a950 0x7f846419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.961+0000 7f8463400640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84641021f0 0x7f84641a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.961+0000 7f8462a00640 1 -- 172.21.15.22:0/3719125317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f846419a950 msgr2=0x7f846419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:14.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.961+0000 7f8462a00640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f846419a950 0x7f846419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.961+0000 7f8462a00640 1 -- 172.21.15.22:0/3719125317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84641021f0 msgr2=0x7f84641a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:14.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.961+0000 7f8462a00640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84641021f0 0x7f84641a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.961+0000 7f8462a00640 1 -- 172.21.15.22:0/3719125317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846419b470 con 0x7f84641072c0 2024-04-20T21:43:14.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.961+0000 7f8462a00640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84641072c0 0x7f84641a0ce0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8450037040 tx=0x7f84500045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:14.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.962+0000 7f845be00640 1 -- 172.21.15.22:0/3719125317 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450041020 con 0x7f84641072c0 2024-04-20T21:43:14.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.962+0000 7f845be00640 1 -- 172.21.15.22:0/3719125317 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f845002c9f0 con 0x7f84641072c0 2024-04-20T21:43:14.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.962+0000 7f846a800640 1 -- 172.21.15.22:0/3719125317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846419b700 con 0x7f84641072c0 2024-04-20T21:43:14.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.962+0000 7f845be00640 1 -- 172.21.15.22:0/3719125317 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450039040 con 0x7f84641072c0 2024-04-20T21:43:14.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.962+0000 7f846a800640 1 -- 172.21.15.22:0/3719125317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84641a78f0 con 0x7f84641072c0 2024-04-20T21:43:14.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.966+0000 7f846d413740 1 Processor -- start 2024-04-20T21:43:14.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.966+0000 7f846d413740 1 -- start start 2024-04-20T21:43:14.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.967+0000 7f846d413740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e931a0 0x556e00e93580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.967+0000 7f846d413740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e9c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.967+0000 7f846d413740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e9c6a0 0x556e00e9ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.967+0000 7f846d413740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556e00df9150 con 0x556e00e93ac0 2024-04-20T21:43:14.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.967+0000 7f846d413740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556e00d9f370 con 0x556e00e9c6a0 2024-04-20T21:43:14.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.967+0000 7f846d413740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556e00df5110 con 0x556e00e931a0 2024-04-20T21:43:14.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.967+0000 7f8463400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e9c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.967+0000 7f8463e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e931a0 0x556e00e93580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.967+0000 7f8463e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e931a0 0x556e00e93580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:45638/0 (socket says 172.21.15.22:45638) 2024-04-20T21:43:14.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.967+0000 7f8463e00640 1 -- 172.21.15.22:0/2694594243 learned_addr learned my addr 172.21.15.22:0/2694594243 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:14.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.967+0000 7f845be00640 1 -- 172.21.15.22:0/3719125317 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8450044020 con 0x7f84641072c0 2024-04-20T21:43:14.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.968+0000 7f8463e00640 1 -- 172.21.15.22:0/2694594243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e9c6a0 msgr2=0x556e00e9ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:14.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.968+0000 7f8463e00640 1 --2- 172.21.15.22:0/2694594243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e9c6a0 0x556e00e9ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.968+0000 7f8463e00640 1 -- 172.21.15.22:0/2694594243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 msgr2=0x556e00e9c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:14.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.968+0000 7f8463e00640 1 --2- 172.21.15.22:0/2694594243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e9c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.968+0000 7f8463e00640 1 -- 172.21.15.22:0/2694594243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e00bd57d0 con 0x556e00e931a0 2024-04-20T21:43:14.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.968+0000 7f8463400640 1 --2- 172.21.15.22:0/2694594243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e9c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:14.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.968+0000 7f8463e00640 1 --2- 172.21.15.22:0/2694594243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e931a0 0x556e00e93580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f845c009f60 tx=0x7f845c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ace0ae8af25c618 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:14.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.968+0000 7f8458c00640 1 -- 172.21.15.22:0/2694594243 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c01e070 con 0x556e00e931a0 2024-04-20T21:43:14.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.968+0000 7f8458c00640 1 -- 172.21.15.22:0/2694594243 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f845c003c30 con 0x556e00e931a0 2024-04-20T21:43:14.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.968+0000 7f8458c00640 1 -- 172.21.15.22:0/2694594243 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c00f9c0 con 0x556e00e931a0 2024-04-20T21:43:14.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.969+0000 7f845be00640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f84480a3d20 0x7f84480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.969+0000 7f846d413740 1 -- 172.21.15.22:0/2694594243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e931a0 msgr2=0x556e00e93580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:14.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.969+0000 7f846d413740 1 --2- 172.21.15.22:0/2694594243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e931a0 0x556e00e93580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f845c009f60 tx=0x7f845c009f90 comp rx=0 tx=0).stop 2024-04-20T21:43:14.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.969+0000 7f845be00640 1 -- 172.21.15.22:0/3719125317 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f84500f2040 con 0x7f84641072c0 2024-04-20T21:43:14.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.969+0000 7f846d413740 1 -- 172.21.15.22:0/2694594243 shutdown_connections 2024-04-20T21:43:14.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.969+0000 7f846d413740 1 --2- 172.21.15.22:0/2694594243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e9c6a0 0x556e00e9ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.969+0000 7f846d413740 1 --2- 172.21.15.22:0/2694594243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e9c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.969+0000 7f846d413740 1 --2- 172.21.15.22:0/2694594243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e931a0 0x556e00e93580 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.969+0000 7f846d413740 1 -- 172.21.15.22:0/2694594243 >> 172.21.15.22:0/2694594243 conn(0x556e00e8d8d0 msgr2=0x556e00e8dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:14.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.969+0000 7f846d413740 1 -- 172.21.15.22:0/2694594243 shutdown_connections 2024-04-20T21:43:14.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.969+0000 7f8463400640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f84480a3d20 0x7f84480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.969+0000 7f846d413740 1 -- 172.21.15.22:0/2694594243 wait complete. 2024-04-20T21:43:14.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.970+0000 7f846d413740 1 -- start start 2024-04-20T21:43:14.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.970+0000 7f846d413740 1 Processor -- start 2024-04-20T21:43:14.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.970+0000 7f8463400640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f84480a3d20 0x7f84480a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x556e00e92c70 tx=0x7f8454009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:14.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f846d413740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e931a0 0x556e00e99790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f846d413740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e99cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f846d413740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e9c6a0 0x556e00e9a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f846d413740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556e00e9f490 con 0x556e00e93ac0 2024-04-20T21:43:14.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f846d413740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556e00db2740 con 0x556e00e9c6a0 2024-04-20T21:43:14.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f846d413740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556e00e9f610 con 0x556e00e931a0 2024-04-20T21:43:14.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f846d413740 10 client.0 _ll_register_callbacks cb 0x556e00dadb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:14.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f8462a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e99cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f8462a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e99cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53970/0 (socket says 172.21.15.22:53970) 2024-04-20T21:43:14.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f8463400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e931a0 0x556e00e99790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f8462a00640 1 -- 172.21.15.22:0/213959795 learned_addr learned my addr 172.21.15.22:0/213959795 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:14.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f8463e00640 1 --2- 172.21.15.22:0/213959795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e9c6a0 0x556e00e9a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f8462a00640 1 -- 172.21.15.22:0/213959795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e931a0 msgr2=0x556e00e99790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:14.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f8462a00640 1 --2- 172.21.15.22:0/213959795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e931a0 0x556e00e99790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f8462a00640 1 -- 172.21.15.22:0/213959795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e9c6a0 msgr2=0x556e00e9a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:14.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f8462a00640 1 --2- 172.21.15.22:0/213959795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e9c6a0 0x556e00e9a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.971+0000 7f8462a00640 1 -- 172.21.15.22:0/213959795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e00b0b3f0 con 0x556e00e93ac0 2024-04-20T21:43:14.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.972+0000 7f8462a00640 1 --2- 172.21.15.22:0/213959795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e99cd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8450000c00 tx=0x7f84500ba3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93db575afb1d3483 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:14.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.972+0000 7f844d600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:43:14.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.972+0000 7f844d600640 1 -- 172.21.15.22:0/213959795 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84500be7a0 con 0x556e00e93ac0 2024-04-20T21:43:14.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.972+0000 7f844d600640 1 -- 172.21.15.22:0/213959795 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f84500ba710 con 0x556e00e93ac0 2024-04-20T21:43:14.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.972+0000 7f846d413740 1 -- 172.21.15.22:0/213959795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f845c003c30 con 0x556e00e93ac0 2024-04-20T21:43:14.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.972+0000 7f844d600640 1 -- 172.21.15.22:0/213959795 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f84500be0a0 con 0x556e00e93ac0 2024-04-20T21:43:14.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.972+0000 7f846d413740 10 client.16563 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:14.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.972+0000 7f846d413740 1 -- 172.21.15.22:0/213959795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556e00acb0f0 con 0x556e00e93ac0 2024-04-20T21:43:14.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.972+0000 7f844d600640 1 -- 172.21.15.22:0/213959795 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f845c003c30 con 0x556e00e93ac0 2024-04-20T21:43:14.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.973+0000 7f844d600640 1 -- 172.21.15.22:0/213959795 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f845003c030 con 0x556e00e93ac0 2024-04-20T21:43:14.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.973+0000 7f846d413740 10 client.16563 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:14.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.973+0000 7f846d413740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:14.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.973+0000 7f846d413740 20 client.16563 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:14.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.973+0000 7f846d413740 1 --2- 172.21.15.22:0/213959795 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e00eac9a0 0x556e00eaedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.973+0000 7f846d413740 4 client.16563 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:14.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.973+0000 7f846d413740 1 -- 172.21.15.22:0/213959795 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e00e06b70 con 0x556e00eac9a0 2024-04-20T21:43:14.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.973+0000 7f8463400640 1 --2- 172.21.15.22:0/213959795 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e00eac9a0 0x556e00eaedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.974+0000 7f8463400640 1 --2- 172.21.15.22:0/213959795 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e00eac9a0 0x556e00eaedb0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b4e4374d17b5a server_cookie=dde4b58ead081c6f in_seq=0 out_seq=0 2024-04-20T21:43:14.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.974+0000 7f844d600640 10 client.16563 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:14.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f844d600640 1 -- 172.21.15.22:0/213959795 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e00e06b70 con 0x556e00eac9a0 2024-04-20T21:43:14.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f844d600640 10 client.16563 handle_command_reply: tid=0 2024-04-20T21:43:14.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f846d413740 1 client.16563 shutdown 2024-04-20T21:43:14.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f846d413740 1 -- 172.21.15.22:0/213959795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 msgr2=0x556e00e99cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:14.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f846d413740 1 --2- 172.21.15.22:0/213959795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e99cd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8450000c00 tx=0x7f84500ba3c0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f846d413740 1 -- 172.21.15.22:0/213959795 shutdown_connections 2024-04-20T21:43:14.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f846d413740 1 --2- 172.21.15.22:0/213959795 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e00eac9a0 0x556e00eaedb0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f846d413740 1 --2- 172.21.15.22:0/213959795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e9c6a0 0x556e00e9a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f846d413740 1 --2- 172.21.15.22:0/213959795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e99cd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f846d413740 1 --2- 172.21.15.22:0/213959795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e931a0 0x556e00e99790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f844d600640 0 client.16563 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:14.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f846d413740 1 -- 172.21.15.22:0/213959795 >> 172.21.15.22:0/213959795 conn(0x556e00e8d8d0 msgr2=0x556e00e8f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:14.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f846d413740 1 -- 172.21.15.22:0/213959795 shutdown_connections 2024-04-20T21:43:14.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.975+0000 7f846d413740 1 -- 172.21.15.22:0/213959795 wait complete. 2024-04-20T21:43:14.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.976+0000 7f846d413740 20 client.16563 trim_cache size 0 max 16384 2024-04-20T21:43:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.995+0000 7f846d413740 1 Processor -- start 2024-04-20T21:43:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.995+0000 7f846d413740 1 -- start start 2024-04-20T21:43:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.995+0000 7f846d413740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e931a0 0x556e00e9bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.995+0000 7f846d413740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e93ac0 0x556e00e98770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.995+0000 7f846d413740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e9c6a0 0x556e00ea8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:14.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.995+0000 7f846d413740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f84641ae010 con 0x556e00e931a0 2024-04-20T21:43:14.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.995+0000 7f846d413740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84641b0870 con 0x556e00e93ac0 2024-04-20T21:43:14.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.995+0000 7f846d413740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84641ae660 con 0x556e00e9c6a0 2024-04-20T21:43:14.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.996+0000 7f8463400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e931a0 0x556e00e9bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.996+0000 7f8463400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e931a0 0x556e00e9bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53984/0 (socket says 172.21.15.22:53984) 2024-04-20T21:43:14.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.996+0000 7f8463400640 1 -- 172.21.15.22:0/653106314 learned_addr learned my addr 172.21.15.22:0/653106314 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:14.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.996+0000 7f8462a00640 1 --2- 172.21.15.22:0/653106314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e93ac0 0x556e00e98770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.996+0000 7f8463e00640 1 --2- 172.21.15.22:0/653106314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e9c6a0 0x556e00ea8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:14.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.996+0000 7f8463400640 1 -- 172.21.15.22:0/653106314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e9c6a0 msgr2=0x556e00ea8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:14.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.996+0000 7f8463400640 1 --2- 172.21.15.22:0/653106314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e9c6a0 0x556e00ea8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.996+0000 7f8463400640 1 -- 172.21.15.22:0/653106314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e93ac0 msgr2=0x556e00e98770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:14.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.996+0000 7f8463400640 1 --2- 172.21.15.22:0/653106314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e93ac0 0x556e00e98770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:14.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.996+0000 7f8463400640 1 -- 172.21.15.22:0/653106314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e00e97d80 con 0x556e00e931a0 2024-04-20T21:43:14.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.996+0000 7f8462a00640 1 --2- 172.21.15.22:0/653106314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e93ac0 0x556e00e98770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:14.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.997+0000 7f8463400640 1 --2- 172.21.15.22:0/653106314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e931a0 0x556e00e9bec0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x556e00eaf640 tx=0x7f845401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6c57e3a84e867f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.997+0000 7f844fe00640 1 -- 172.21.15.22:0/653106314 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454030070 con 0x556e00e931a0 2024-04-20T21:43:15.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.997+0000 7f844fe00640 1 -- 172.21.15.22:0/653106314 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f845402a910 con 0x556e00e931a0 2024-04-20T21:43:15.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.997+0000 7f844fe00640 1 -- 172.21.15.22:0/653106314 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454028dd0 con 0x556e00e931a0 2024-04-20T21:43:15.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.998+0000 7f846d413740 1 -- 172.21.15.22:0/653106314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e931a0 msgr2=0x556e00e9bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.998+0000 7f846d413740 1 --2- 172.21.15.22:0/653106314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e931a0 0x556e00e9bec0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x556e00eaf640 tx=0x7f845401a000 comp rx=0 tx=0).stop 2024-04-20T21:43:15.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.998+0000 7f846d413740 1 -- 172.21.15.22:0/653106314 shutdown_connections 2024-04-20T21:43:15.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.998+0000 7f846d413740 1 --2- 172.21.15.22:0/653106314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e9c6a0 0x556e00ea8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.998+0000 7f846d413740 1 --2- 172.21.15.22:0/653106314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e93ac0 0x556e00e98770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.998+0000 7f846d413740 1 --2- 172.21.15.22:0/653106314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e931a0 0x556e00e9bec0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.998+0000 7f846d413740 1 -- 172.21.15.22:0/653106314 >> 172.21.15.22:0/653106314 conn(0x556e00e8d8d0 msgr2=0x556e00e8f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.998+0000 7f846d413740 1 -- 172.21.15.22:0/653106314 shutdown_connections 2024-04-20T21:43:15.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.998+0000 7f846d413740 1 -- 172.21.15.22:0/653106314 wait complete. 2024-04-20T21:43:15.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.998+0000 7f846d413740 1 -- start start 2024-04-20T21:43:15.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f846d413740 1 Processor -- start 2024-04-20T21:43:15.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f846d413740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e931a0 0x556e00e96710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f846d413740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e96c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f846d413740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e9c6a0 0x556e00e97190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f846d413740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f84641aed30 con 0x556e00e93ac0 2024-04-20T21:43:15.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f846d413740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84641b01a0 con 0x556e00e931a0 2024-04-20T21:43:15.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f846d413740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84641af400 con 0x556e00e9c6a0 2024-04-20T21:43:15.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f846d413740 10 client.0 _ll_register_callbacks cb 0x7f84500097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:15.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f8462a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e96c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f8463400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e931a0 0x556e00e96710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f8463e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e9c6a0 0x556e00e97190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f8463400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e931a0 0x556e00e96710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:40434/0 (socket says 172.21.15.22:40434) 2024-04-20T21:43:15.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f8463400640 1 -- 172.21.15.22:0/657924354 learned_addr learned my addr 172.21.15.22:0/657924354 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:15.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:14.999+0000 7f8462a00640 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e96c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53988/0 (socket says 172.21.15.22:53988) 2024-04-20T21:43:15.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.000+0000 7f8463400640 1 -- 172.21.15.22:0/657924354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e9c6a0 msgr2=0x556e00e97190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:15.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.000+0000 7f8463400640 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e9c6a0 0x556e00e97190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.000+0000 7f8463400640 1 -- 172.21.15.22:0/657924354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 msgr2=0x556e00e96c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:15.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.000+0000 7f8463400640 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e96c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.000+0000 7f8463400640 1 -- 172.21.15.22:0/657924354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e00f12110 con 0x556e00e931a0 2024-04-20T21:43:15.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.000+0000 7f8462a00640 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e96c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:15.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.000+0000 7f8463400640 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e931a0 0x556e00e96710 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8454009530 tx=0x7f845401c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a6a049be303517 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.000+0000 7f844d600640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:43:15.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.000+0000 7f844d600640 1 -- 172.21.15.22:0/657924354 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454030040 con 0x556e00e931a0 2024-04-20T21:43:15.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.000+0000 7f844d600640 1 -- 172.21.15.22:0/657924354 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f8454028640 con 0x556e00e931a0 2024-04-20T21:43:15.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.000+0000 7f846d413740 1 -- 172.21.15.22:0/657924354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f845402a910 con 0x556e00e931a0 2024-04-20T21:43:15.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.001+0000 7f844d600640 1 -- 172.21.15.22:0/657924354 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f845402a430 con 0x556e00e931a0 2024-04-20T21:43:15.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.001+0000 7f846d413740 10 client.26714 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:15.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.001+0000 7f846d413740 1 -- 172.21.15.22:0/657924354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556e00f133e0 con 0x556e00e931a0 2024-04-20T21:43:15.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.001+0000 7f844d600640 1 -- 172.21.15.22:0/657924354 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f845402a910 con 0x556e00e931a0 2024-04-20T21:43:15.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.001+0000 7f844d600640 1 -- 172.21.15.22:0/657924354 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f84540046c0 con 0x556e00e931a0 2024-04-20T21:43:15.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.001+0000 7f846d413740 10 client.26714 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:15.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.001+0000 7f846d413740 10 client.26714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:15.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.001+0000 7f846d413740 20 client.26714 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:15.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.001+0000 7f846d413740 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e00f13af0 0x556e00f15f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.001+0000 7f846d413740 4 client.26714 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:43:15.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.001+0000 7f846d413740 1 -- 172.21.15.22:0/657924354 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e00e009e0 con 0x556e00f13af0 2024-04-20T21:43:15.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.002+0000 7f8462a00640 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e00f13af0 0x556e00f15f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.002+0000 7f8462a00640 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e00f13af0 0x556e00f15f10 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576f0d831a055126 server_cookie=b5534b3e0997411b in_seq=0 out_seq=0 2024-04-20T21:43:15.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.002+0000 7f844d600640 10 client.26714 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:15.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.002+0000 7f844d600640 1 -- 172.21.15.22:0/657924354 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556e00e009e0 con 0x556e00f13af0 2024-04-20T21:43:15.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.002+0000 7f844d600640 10 client.26714 handle_command_reply: tid=0 2024-04-20T21:43:15.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.002+0000 7f846d413740 1 client.26714 shutdown 2024-04-20T21:43:15.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f846d413740 1 -- 172.21.15.22:0/657924354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e931a0 msgr2=0x556e00e96710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f846d413740 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e931a0 0x556e00e96710 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8454009530 tx=0x7f845401c650 comp rx=0 tx=0).stop 2024-04-20T21:43:15.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f846d413740 1 -- 172.21.15.22:0/657924354 shutdown_connections 2024-04-20T21:43:15.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f846d413740 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e00e9c6a0 0x556e00e97190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f846d413740 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e00e93ac0 0x556e00e96c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f846d413740 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x556e00f13af0 0x556e00f15f10 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f846d413740 1 --2- 172.21.15.22:0/657924354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e00e931a0 0x556e00e96710 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f846d413740 1 -- 172.21.15.22:0/657924354 >> 172.21.15.22:0/657924354 conn(0x556e00e8d8d0 msgr2=0x556e00e8c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f844d600640 0 client.26714 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:15.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f846d413740 1 -- 172.21.15.22:0/657924354 shutdown_connections 2024-04-20T21:43:15.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f846d413740 1 -- 172.21.15.22:0/657924354 wait complete. 2024-04-20T21:43:15.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f846d413740 20 client.26714 trim_cache size 0 max 16384 2024-04-20T21:43:15.008 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:43:15.008 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:43:15.009 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:43:15.009 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:43:15.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.003+0000 7f846a800640 1 -- 172.21.15.22:0/3719125317 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f84480a3d20 msgr2=0x7f84480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.004+0000 7f846a800640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f84480a3d20 0x7f84480a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x556e00e92c70 tx=0x7f8454009260 comp rx=0 tx=0).stop 2024-04-20T21:43:15.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.004+0000 7f846a800640 1 -- 172.21.15.22:0/3719125317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84641072c0 msgr2=0x7f84641a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.004+0000 7f846a800640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84641072c0 0x7f84641a0ce0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8450037040 tx=0x7f84500045b0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.004+0000 7f846a800640 1 -- 172.21.15.22:0/3719125317 shutdown_connections 2024-04-20T21:43:15.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.004+0000 7f846a800640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f84480a3d20 0x7f84480a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.004+0000 7f846a800640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f846419a950 0x7f846419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.004+0000 7f846a800640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84641072c0 0x7f84641a0ce0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.004+0000 7f846a800640 1 --2- 172.21.15.22:0/3719125317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84641021f0 0x7f84641a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.004+0000 7f846a800640 1 -- 172.21.15.22:0/3719125317 >> 172.21.15.22:0/3719125317 conn(0x7f84640fda70 msgr2=0x7f84640fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.004+0000 7f846a800640 1 -- 172.21.15.22:0/3719125317 shutdown_connections 2024-04-20T21:43:15.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.004+0000 7f846a800640 1 -- 172.21.15.22:0/3719125317 wait complete. 2024-04-20T21:43:15.021 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:43:15.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:15.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.228+0000 7f3c3be00640 1 -- 172.21.15.22:0/2570793539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c100de0 msgr2=0x7f3c3c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.228+0000 7f3c3be00640 1 --2- 172.21.15.22:0/2570793539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c100de0 0x7f3c3c10be60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009900 tx=0x7f3c2402bcc0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.228+0000 7f3c3be00640 1 -- 172.21.15.22:0/2570793539 shutdown_connections 2024-04-20T21:43:15.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.228+0000 7f3c3be00640 1 --2- 172.21.15.22:0/2570793539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c10c3a0 0x7f3c3c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.228+0000 7f3c3be00640 1 --2- 172.21.15.22:0/2570793539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c100de0 0x7f3c3c10be60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.228+0000 7f3c3be00640 1 --2- 172.21.15.22:0/2570793539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c1004c0 0x7f3c3c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.228+0000 7f3c3be00640 1 -- 172.21.15.22:0/2570793539 >> 172.21.15.22:0/2570793539 conn(0x7f3c3c076be0 msgr2=0x7f3c3c076ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.228+0000 7f3c3be00640 1 -- 172.21.15.22:0/2570793539 shutdown_connections 2024-04-20T21:43:15.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.228+0000 7f3c3be00640 1 -- 172.21.15.22:0/2570793539 wait complete. 2024-04-20T21:43:15.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.229+0000 7f3c3be00640 1 Processor -- start 2024-04-20T21:43:15.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.230+0000 7f3c3be00640 1 -- start start 2024-04-20T21:43:15.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.230+0000 7f3c3be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c1004c0 0x7f3c3c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.230+0000 7f3c3be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c100de0 0x7f3c3c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.230+0000 7f3c3be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c10c3a0 0x7f3c3c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.230+0000 7f3c3be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c112150 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.230+0000 7f3c3be00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c111fd0 con 0x7f3c3c1004c0 2024-04-20T21:43:15.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.230+0000 7f3c3be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c1122d0 con 0x7f3c3c100de0 2024-04-20T21:43:15.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.230+0000 7f3c3b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c10c3a0 0x7f3c3c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.230+0000 7f3c3b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c10c3a0 0x7f3c3c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54006/0 (socket says 172.21.15.22:54006) 2024-04-20T21:43:15.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.230+0000 7f3c3b400640 1 -- 172.21.15.22:0/2193324222 learned_addr learned my addr 172.21.15.22:0/2193324222 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:15.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.230+0000 7f3c33e00640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c100de0 0x7f3c3c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.230+0000 7f3c3aa00640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c1004c0 0x7f3c3c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.231+0000 7f3c3b400640 1 -- 172.21.15.22:0/2193324222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c100de0 msgr2=0x7f3c3c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.231+0000 7f3c3b400640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c100de0 0x7f3c3c198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.231+0000 7f3c3b400640 1 -- 172.21.15.22:0/2193324222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c1004c0 msgr2=0x7f3c3c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.231+0000 7f3c3b400640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c1004c0 0x7f3c3c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.231+0000 7f3c3b400640 1 -- 172.21.15.22:0/2193324222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3c19d5b0 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.231+0000 7f3c3b400640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c10c3a0 0x7f3c3c19ce80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400eb10 tx=0x7f3c3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.231+0000 7f3c33400640 1 -- 172.21.15.22:0/2193324222 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34004b20 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.231+0000 7f3c3be00640 1 -- 172.21.15.22:0/2193324222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3c19d8a0 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.231+0000 7f3c33400640 1 -- 172.21.15.22:0/2193324222 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c34004cc0 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.231+0000 7f3c33400640 1 -- 172.21.15.22:0/2193324222 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34010760 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.232+0000 7f3c3be00640 1 -- 172.21.15.22:0/2193324222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c3c1a5130 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.233+0000 7f3c3be00640 1 -- 172.21.15.22:0/2193324222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c3c0690f0 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.235+0000 7f3c33400640 1 -- 172.21.15.22:0/2193324222 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3c34005020 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.237+0000 7f3c33400640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3c200a3da0 0x7f3c200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.238+0000 7f3c33400640 1 -- 172.21.15.22:0/2193324222 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f3c340d0db0 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.238+0000 7f3c3aa00640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3c200a3da0 0x7f3c200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.239+0000 7f3c3aa00640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3c200a3da0 0x7f3c200a6260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00bbe0 tx=0x7f3c2c00b390 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.242+0000 7f3c33400640 1 -- 172.21.15.22:0/2193324222 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3c340d5070 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.494+0000 7f3c3be00640 1 -- 172.21.15.22:0/2193324222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c3c1a5410 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.496+0000 7f3c33400640 1 -- 172.21.15.22:0/2193324222 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f3c3409d3a0 con 0x7f3c3c10c3a0 2024-04-20T21:43:15.495 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:43:15.495 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:43:15.495 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:43:15.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.499+0000 7f3c3be00640 1 -- 172.21.15.22:0/2193324222 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3c200a3da0 msgr2=0x7f3c200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.499+0000 7f3c3be00640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3c200a3da0 0x7f3c200a6260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00bbe0 tx=0x7f3c2c00b390 comp rx=0 tx=0).stop 2024-04-20T21:43:15.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.499+0000 7f3c3be00640 1 -- 172.21.15.22:0/2193324222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c10c3a0 msgr2=0x7f3c3c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.499+0000 7f3c3be00640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c10c3a0 0x7f3c3c19ce80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400eb10 tx=0x7f3c3400c2d0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.499+0000 7f3c3be00640 1 -- 172.21.15.22:0/2193324222 shutdown_connections 2024-04-20T21:43:15.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.499+0000 7f3c3be00640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3c200a3da0 0x7f3c200a6260 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.499+0000 7f3c3be00640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c10c3a0 0x7f3c3c19ce80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.499+0000 7f3c3be00640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c100de0 0x7f3c3c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.499+0000 7f3c3be00640 1 --2- 172.21.15.22:0/2193324222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c1004c0 0x7f3c3c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.499+0000 7f3c3be00640 1 -- 172.21.15.22:0/2193324222 >> 172.21.15.22:0/2193324222 conn(0x7f3c3c076be0 msgr2=0x7f3c3c0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.499+0000 7f3c3be00640 1 -- 172.21.15.22:0/2193324222 shutdown_connections 2024-04-20T21:43:15.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.499+0000 7f3c3be00640 1 -- 172.21.15.22:0/2193324222 wait complete. 2024-04-20T21:43:15.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:43:15.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.735+0000 7f4957200640 1 -- 172.21.15.22:0/1203902057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4958102fc0 msgr2=0x7f49581033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.735+0000 7f4957200640 1 --2- 172.21.15.22:0/1203902057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4958102fc0 0x7f49581033a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4948009860 tx=0x7f494802bc10 comp rx=0 tx=0).stop 2024-04-20T21:43:15.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.735+0000 7f4957200640 1 -- 172.21.15.22:0/1203902057 shutdown_connections 2024-04-20T21:43:15.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.735+0000 7f4957200640 1 --2- 172.21.15.22:0/1203902057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f495810c3e0 0x7f495810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.735+0000 7f4957200640 1 --2- 172.21.15.22:0/1203902057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49581038e0 0x7f495810bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.735+0000 7f4957200640 1 --2- 172.21.15.22:0/1203902057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4958102fc0 0x7f49581033a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.735+0000 7f4957200640 1 -- 172.21.15.22:0/1203902057 >> 172.21.15.22:0/1203902057 conn(0x7f49580fc7e0 msgr2=0x7f49580fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.735+0000 7f4957200640 1 -- 172.21.15.22:0/1203902057 shutdown_connections 2024-04-20T21:43:15.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.736+0000 7f4957200640 1 -- 172.21.15.22:0/1203902057 wait complete. 2024-04-20T21:43:15.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.737+0000 7f4957200640 1 Processor -- start 2024-04-20T21:43:15.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.737+0000 7f4957200640 1 -- start start 2024-04-20T21:43:15.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.737+0000 7f4957200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4958102fc0 0x7f49581a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.737+0000 7f4957200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49581038e0 0x7f49581a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.737+0000 7f4957200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f495810c3e0 0x7f495819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.737+0000 7f4957200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49581121c0 con 0x7f4958102fc0 2024-04-20T21:43:15.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.737+0000 7f4957200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4958112040 con 0x7f49581038e0 2024-04-20T21:43:15.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.737+0000 7f4957200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4958112340 con 0x7f495810c3e0 2024-04-20T21:43:15.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.737+0000 7f4955e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4958102fc0 0x7f49581a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.738+0000 7f4955e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4958102fc0 0x7f49581a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54030/0 (socket says 172.21.15.22:54030) 2024-04-20T21:43:15.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.738+0000 7f4955400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49581038e0 0x7f49581a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.738+0000 7f4955e00640 1 -- 172.21.15.22:0/4016439440 learned_addr learned my addr 172.21.15.22:0/4016439440 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:15.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.738+0000 7f4956800640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f495810c3e0 0x7f495819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.738+0000 7f4955e00640 1 -- 172.21.15.22:0/4016439440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f495810c3e0 msgr2=0x7f495819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.738+0000 7f4955e00640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f495810c3e0 0x7f495819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.738+0000 7f4955e00640 1 -- 172.21.15.22:0/4016439440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49581038e0 msgr2=0x7f49581a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.738+0000 7f4955e00640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49581038e0 0x7f49581a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.738+0000 7f4955e00640 1 -- 172.21.15.22:0/4016439440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495819afc0 con 0x7f4958102fc0 2024-04-20T21:43:15.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.738+0000 7f4955e00640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4958102fc0 0x7f49581a0640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4948037040 tx=0x7f49480346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.739+0000 7f494ea00640 1 -- 172.21.15.22:0/4016439440 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948041020 con 0x7f4958102fc0 2024-04-20T21:43:15.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.739+0000 7f4957200640 1 -- 172.21.15.22:0/4016439440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495819b250 con 0x7f4958102fc0 2024-04-20T21:43:15.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.739+0000 7f494ea00640 1 -- 172.21.15.22:0/4016439440 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49480044d0 con 0x7f4958102fc0 2024-04-20T21:43:15.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.739+0000 7f494ea00640 1 -- 172.21.15.22:0/4016439440 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948039040 con 0x7f4958102fc0 2024-04-20T21:43:15.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.739+0000 7f4957200640 1 -- 172.21.15.22:0/4016439440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49581a75c0 con 0x7f4958102fc0 2024-04-20T21:43:15.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.742+0000 7f495fa62740 1 Processor -- start 2024-04-20T21:43:15.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.742+0000 7f494ea00640 1 -- 172.21.15.22:0/4016439440 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4948044020 con 0x7f4958102fc0 2024-04-20T21:43:15.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.743+0000 7f495fa62740 1 -- start start 2024-04-20T21:43:15.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.743+0000 7f495fa62740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430170 0x55664e430550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.743+0000 7f495fa62740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430a90 0x55664e439130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.743+0000 7f495fa62740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e439670 0x55664e43ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.743+0000 7f495fa62740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55664e395050 con 0x55664e430a90 2024-04-20T21:43:15.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.743+0000 7f495fa62740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55664e33b490 con 0x55664e439670 2024-04-20T21:43:15.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.743+0000 7f495fa62740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55664e391010 con 0x55664e430170 2024-04-20T21:43:15.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.743+0000 7f4955e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430a90 0x55664e439130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.743+0000 7f4955e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430a90 0x55664e439130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54034/0 (socket says 172.21.15.22:54034) 2024-04-20T21:43:15.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.743+0000 7f4955e00640 1 -- 172.21.15.22:0/2041048033 learned_addr learned my addr 172.21.15.22:0/2041048033 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:15.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.743+0000 7f4955400640 1 --2- 172.21.15.22:0/2041048033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e439670 0x55664e43ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.743+0000 7f4956800640 1 --2- 172.21.15.22:0/2041048033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430170 0x55664e430550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.744+0000 7f4955e00640 1 -- 172.21.15.22:0/2041048033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430170 msgr2=0x55664e430550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:15.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.744+0000 7f4955e00640 1 --2- 172.21.15.22:0/2041048033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430170 0x55664e430550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.744+0000 7f4955e00640 1 -- 172.21.15.22:0/2041048033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e439670 msgr2=0x55664e43ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:15.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.744+0000 7f4955e00640 1 --2- 172.21.15.22:0/2041048033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e439670 0x55664e43ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.744+0000 7f4955e00640 1 -- 172.21.15.22:0/2041048033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664e343070 con 0x55664e430a90 2024-04-20T21:43:15.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.744+0000 7f4955e00640 1 --2- 172.21.15.22:0/2041048033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430a90 0x55664e439130 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f49480ef690 tx=0x7f49480f5f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9d7478d82108c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.744+0000 7f494ea00640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f493c0a3d20 0x7f493c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.745+0000 7f4947400640 1 -- 172.21.15.22:0/2041048033 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49480ba210 con 0x55664e430a90 2024-04-20T21:43:15.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.745+0000 7f4956800640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f493c0a3d20 0x7f493c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.745+0000 7f494ea00640 1 -- 172.21.15.22:0/4016439440 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f49480f1f10 con 0x7f4958102fc0 2024-04-20T21:43:15.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.745+0000 7f4947400640 1 -- 172.21.15.22:0/2041048033 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49480ff070 con 0x55664e430a90 2024-04-20T21:43:15.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.745+0000 7f4947400640 1 -- 172.21.15.22:0/2041048033 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49480bd3c0 con 0x55664e430a90 2024-04-20T21:43:15.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.746+0000 7f495fa62740 1 -- 172.21.15.22:0/2041048033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430a90 msgr2=0x55664e439130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.746+0000 7f4956800640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f493c0a3d20 0x7f493c0a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4950005fd0 tx=0x7f4950009600 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.746+0000 7f495fa62740 1 --2- 172.21.15.22:0/2041048033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430a90 0x55664e439130 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f49480ef690 tx=0x7f49480f5f00 comp rx=0 tx=0).stop 2024-04-20T21:43:15.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.746+0000 7f495fa62740 1 -- 172.21.15.22:0/2041048033 shutdown_connections 2024-04-20T21:43:15.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.746+0000 7f495fa62740 1 --2- 172.21.15.22:0/2041048033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e439670 0x55664e43ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.746+0000 7f495fa62740 1 --2- 172.21.15.22:0/2041048033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430a90 0x55664e439130 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.746+0000 7f495fa62740 1 --2- 172.21.15.22:0/2041048033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430170 0x55664e430550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.746+0000 7f495fa62740 1 -- 172.21.15.22:0/2041048033 >> 172.21.15.22:0/2041048033 conn(0x55664e42a8a0 msgr2=0x55664e42ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.746+0000 7f495fa62740 1 -- 172.21.15.22:0/2041048033 shutdown_connections 2024-04-20T21:43:15.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.746+0000 7f495fa62740 1 -- 172.21.15.22:0/2041048033 wait complete. 2024-04-20T21:43:15.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.746+0000 7f495fa62740 1 -- start start 2024-04-20T21:43:15.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.746+0000 7f495fa62740 1 Processor -- start 2024-04-20T21:43:15.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.747+0000 7f495fa62740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430170 0x55664e436600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.747+0000 7f495fa62740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430a90 0x55664e436b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.747+0000 7f495fa62740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e439670 0x55664e437080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.747+0000 7f495fa62740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55664e43c220 con 0x55664e439670 2024-04-20T21:43:15.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.747+0000 7f495fa62740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55664e34e860 con 0x55664e430a90 2024-04-20T21:43:15.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.747+0000 7f495fa62740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55664e43c3a0 con 0x55664e430170 2024-04-20T21:43:15.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.747+0000 7f495fa62740 10 client.0 _ll_register_callbacks cb 0x55664e31cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:15.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.747+0000 7f4956800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e439670 0x55664e437080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.747+0000 7f4955e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430170 0x55664e436600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.747+0000 7f4955400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430a90 0x55664e436b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.747+0000 7f4955e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430170 0x55664e436600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:45690/0 (socket says 172.21.15.22:45690) 2024-04-20T21:43:15.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.747+0000 7f4955e00640 1 -- 172.21.15.22:0/550666312 learned_addr learned my addr 172.21.15.22:0/550666312 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:15.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.748+0000 7f4955e00640 1 -- 172.21.15.22:0/550666312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430a90 msgr2=0x55664e436b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:15.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.748+0000 7f4955e00640 1 --2- 172.21.15.22:0/550666312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430a90 0x55664e436b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.748+0000 7f4955e00640 1 -- 172.21.15.22:0/550666312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e439670 msgr2=0x55664e437080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:15.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.748+0000 7f4955e00640 1 --2- 172.21.15.22:0/550666312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e439670 0x55664e437080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.748+0000 7f4955e00640 1 -- 172.21.15.22:0/550666312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664e352950 con 0x55664e430170 2024-04-20T21:43:15.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.748+0000 7f4955400640 1 --2- 172.21.15.22:0/550666312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430a90 0x55664e436b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:15.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.748+0000 7f4955e00640 1 --2- 172.21.15.22:0/550666312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430170 0x55664e436600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f49480ef7d0 tx=0x7f494804df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcc903496ecda6e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.748+0000 7f493be00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:43:15.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.748+0000 7f493be00640 1 -- 172.21.15.22:0/550666312 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49480bd6f0 con 0x55664e430170 2024-04-20T21:43:15.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.748+0000 7f495fa62740 1 -- 172.21.15.22:0/550666312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49480ff070 con 0x55664e430170 2024-04-20T21:43:15.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.748+0000 7f493be00640 1 -- 172.21.15.22:0/550666312 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f49480ba5e0 con 0x55664e430170 2024-04-20T21:43:15.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.749+0000 7f493be00640 1 -- 172.21.15.22:0/550666312 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f49480bd8d0 con 0x55664e430170 2024-04-20T21:43:15.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.749+0000 7f495fa62740 10 client.26761 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:15.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.749+0000 7f495fa62740 1 -- 172.21.15.22:0/550666312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55664e320820 con 0x55664e430170 2024-04-20T21:43:15.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.749+0000 7f493be00640 1 -- 172.21.15.22:0/550666312 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49480ff070 con 0x55664e430170 2024-04-20T21:43:15.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.749+0000 7f493be00640 1 -- 172.21.15.22:0/550666312 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f494803c030 con 0x55664e430170 2024-04-20T21:43:15.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.749+0000 7f495fa62740 10 client.26761 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:15.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.749+0000 7f495fa62740 10 client.26761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:15.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.749+0000 7f495fa62740 20 client.26761 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:15.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.749+0000 7f495fa62740 1 --2- 172.21.15.22:0/550666312 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55664e449180 0x55664e44b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.749+0000 7f495fa62740 4 client.26761 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:15.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.749+0000 7f495fa62740 1 -- 172.21.15.22:0/550666312 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55664e3a2a70 con 0x55664e449180 2024-04-20T21:43:15.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.750+0000 7f4955400640 1 --2- 172.21.15.22:0/550666312 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55664e449180 0x55664e44b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.750+0000 7f4955400640 1 --2- 172.21.15.22:0/550666312 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55664e449180 0x55664e44b570 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=610777b45e6b61cd server_cookie=af551352b77d8e62 in_seq=0 out_seq=0 2024-04-20T21:43:15.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.750+0000 7f493be00640 10 client.26761 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:15.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.751+0000 7f493be00640 1 -- 172.21.15.22:0/550666312 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55664e3a2a70 con 0x55664e449180 2024-04-20T21:43:15.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.751+0000 7f493be00640 10 client.26761 handle_command_reply: tid=0 2024-04-20T21:43:15.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.751+0000 7f495fa62740 1 client.26761 shutdown 2024-04-20T21:43:15.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.752+0000 7f495fa62740 1 -- 172.21.15.22:0/550666312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430170 msgr2=0x55664e436600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.752+0000 7f495fa62740 1 --2- 172.21.15.22:0/550666312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430170 0x55664e436600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f49480ef7d0 tx=0x7f494804df90 comp rx=0 tx=0).stop 2024-04-20T21:43:15.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.752+0000 7f495fa62740 1 -- 172.21.15.22:0/550666312 shutdown_connections 2024-04-20T21:43:15.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.752+0000 7f495fa62740 1 --2- 172.21.15.22:0/550666312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e439670 0x55664e437080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.752+0000 7f495fa62740 1 --2- 172.21.15.22:0/550666312 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55664e449180 0x55664e44b570 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.752+0000 7f495fa62740 1 --2- 172.21.15.22:0/550666312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430a90 0x55664e436b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.752+0000 7f493be00640 0 client.26761 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:15.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.752+0000 7f495fa62740 1 --2- 172.21.15.22:0/550666312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430170 0x55664e436600 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.752+0000 7f495fa62740 1 -- 172.21.15.22:0/550666312 >> 172.21.15.22:0/550666312 conn(0x55664e42a8a0 msgr2=0x55664e42c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.752+0000 7f495fa62740 1 -- 172.21.15.22:0/550666312 shutdown_connections 2024-04-20T21:43:15.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.752+0000 7f495fa62740 1 -- 172.21.15.22:0/550666312 wait complete. 2024-04-20T21:43:15.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.752+0000 7f495fa62740 20 client.26761 trim_cache size 0 max 16384 2024-04-20T21:43:15.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.767+0000 7f495fa62740 1 Processor -- start 2024-04-20T21:43:15.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.767+0000 7f495fa62740 1 -- start start 2024-04-20T21:43:15.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.767+0000 7f495fa62740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430170 0x55664e438d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.767+0000 7f495fa62740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430a90 0x55664e445000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.767+0000 7f495fa62740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e439670 0x55664e435760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.767+0000 7f495fa62740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49581adef0 con 0x55664e430170 2024-04-20T21:43:15.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.767+0000 7f495fa62740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49581b0750 con 0x55664e430a90 2024-04-20T21:43:15.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.767+0000 7f495fa62740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49581ae540 con 0x55664e439670 2024-04-20T21:43:15.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.768+0000 7f4955e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430170 0x55664e438d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.768+0000 7f4955400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430a90 0x55664e445000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.768+0000 7f4955e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430170 0x55664e438d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54058/0 (socket says 172.21.15.22:54058) 2024-04-20T21:43:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.768+0000 7f4955e00640 1 -- 172.21.15.22:0/596660098 learned_addr learned my addr 172.21.15.22:0/596660098 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.768+0000 7f4956800640 1 --2- 172.21.15.22:0/596660098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e439670 0x55664e435760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.768+0000 7f4955400640 1 -- 172.21.15.22:0/596660098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e439670 msgr2=0x55664e435760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.768+0000 7f4955400640 1 --2- 172.21.15.22:0/596660098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e439670 0x55664e435760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.768+0000 7f4955400640 1 -- 172.21.15.22:0/596660098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430170 msgr2=0x55664e438d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.768+0000 7f4955400640 1 --2- 172.21.15.22:0/596660098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430170 0x55664e438d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.768+0000 7f4955400640 1 -- 172.21.15.22:0/596660098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664e4308c0 con 0x55664e430a90 2024-04-20T21:43:15.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.768+0000 7f4955e00640 1 --2- 172.21.15.22:0/596660098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430170 0x55664e438d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:15.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.768+0000 7f4955400640 1 --2- 172.21.15.22:0/596660098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430a90 0x55664e445000 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55664e435e60 tx=0x7f4940009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afbd54ec2f3211ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.769+0000 7f4946a00640 1 -- 172.21.15.22:0/596660098 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940026070 con 0x55664e430a90 2024-04-20T21:43:15.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.769+0000 7f4946a00640 1 -- 172.21.15.22:0/596660098 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4940020890 con 0x55664e430a90 2024-04-20T21:43:15.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.769+0000 7f4946a00640 1 -- 172.21.15.22:0/596660098 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940010dd0 con 0x55664e430a90 2024-04-20T21:43:15.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.769+0000 7f495fa62740 1 -- 172.21.15.22:0/596660098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430a90 msgr2=0x55664e445000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.769+0000 7f495fa62740 1 --2- 172.21.15.22:0/596660098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430a90 0x55664e445000 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55664e435e60 tx=0x7f4940009260 comp rx=0 tx=0).stop 2024-04-20T21:43:15.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.769+0000 7f495fa62740 1 -- 172.21.15.22:0/596660098 shutdown_connections 2024-04-20T21:43:15.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.770+0000 7f495fa62740 1 --2- 172.21.15.22:0/596660098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e439670 0x55664e435760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.770+0000 7f495fa62740 1 --2- 172.21.15.22:0/596660098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430a90 0x55664e445000 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.770+0000 7f495fa62740 1 --2- 172.21.15.22:0/596660098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e430170 0x55664e438d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.770+0000 7f495fa62740 1 -- 172.21.15.22:0/596660098 >> 172.21.15.22:0/596660098 conn(0x55664e42a8a0 msgr2=0x55664e430e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.770+0000 7f495fa62740 1 -- 172.21.15.22:0/596660098 shutdown_connections 2024-04-20T21:43:15.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.770+0000 7f495fa62740 1 -- 172.21.15.22:0/596660098 wait complete. 2024-04-20T21:43:15.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.770+0000 7f495fa62740 1 -- start start 2024-04-20T21:43:15.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.770+0000 7f495fa62740 1 Processor -- start 2024-04-20T21:43:15.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f495fa62740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430170 0x55664e4a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f495fa62740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430a90 0x55664e4a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f495fa62740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e439670 0x55664e43c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f495fa62740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49581aec10 con 0x55664e439670 2024-04-20T21:43:15.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f495fa62740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49581b0080 con 0x55664e430170 2024-04-20T21:43:15.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f495fa62740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49581af2e0 con 0x55664e430a90 2024-04-20T21:43:15.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f495fa62740 10 client.0 _ll_register_callbacks cb 0x7f4948005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:15.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f4956800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e439670 0x55664e43c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f4955e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430170 0x55664e4a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f4956800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e439670 0x55664e43c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54062/0 (socket says 172.21.15.22:54062) 2024-04-20T21:43:15.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f4956800640 1 -- 172.21.15.22:0/1647852061 learned_addr learned my addr 172.21.15.22:0/1647852061 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:15.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f4955e00640 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430170 0x55664e4a7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:40522/0 (socket says 172.21.15.22:40522) 2024-04-20T21:43:15.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f4955400640 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430a90 0x55664e4a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f4955e00640 1 -- 172.21.15.22:0/1647852061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430a90 msgr2=0x55664e4a7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:15.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f4955e00640 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430a90 0x55664e4a7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f4955e00640 1 -- 172.21.15.22:0/1647852061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e439670 msgr2=0x55664e43c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:15.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f4955e00640 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e439670 0x55664e43c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f4955e00640 1 -- 172.21.15.22:0/1647852061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664e0a73f0 con 0x55664e430170 2024-04-20T21:43:15.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.771+0000 7f4956800640 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e439670 0x55664e43c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:15.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.772+0000 7f4955e00640 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430170 0x55664e4a7920 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4948009830 tx=0x7f49480f5f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45297be3b86487cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:15.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.772+0000 7f493be00640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:43:15.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.772+0000 7f493be00640 1 -- 172.21.15.22:0/1647852061 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49480bdab0 con 0x55664e430170 2024-04-20T21:43:15.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.772+0000 7f495fa62740 1 -- 172.21.15.22:0/1647852061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4940020890 con 0x55664e430170 2024-04-20T21:43:15.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.772+0000 7f493be00640 1 -- 172.21.15.22:0/1647852061 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f49480be3d0 con 0x55664e430170 2024-04-20T21:43:15.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.772+0000 7f493be00640 1 -- 172.21.15.22:0/1647852061 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f49480db050 con 0x55664e430170 2024-04-20T21:43:15.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.772+0000 7f495fa62740 10 client.26738 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:15.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.772+0000 7f495fa62740 1 -- 172.21.15.22:0/1647852061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55664e1717d0 con 0x55664e430170 2024-04-20T21:43:15.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.772+0000 7f493be00640 1 -- 172.21.15.22:0/1647852061 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4940020890 con 0x55664e430170 2024-04-20T21:43:15.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.773+0000 7f493be00640 1 -- 172.21.15.22:0/1647852061 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f49480c6750 con 0x55664e430170 2024-04-20T21:43:15.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.773+0000 7f495fa62740 10 client.26738 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:15.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.773+0000 7f495fa62740 10 client.26738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:15.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.773+0000 7f495fa62740 20 client.26738 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:15.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.773+0000 7f495fa62740 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55664e4af010 0x55664e4b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:15.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.773+0000 7f495fa62740 4 client.26738 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:43:15.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.773+0000 7f495fa62740 1 -- 172.21.15.22:0/1647852061 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55664e39c8e0 con 0x55664e4af010 2024-04-20T21:43:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.773+0000 7f4955400640 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55664e4af010 0x55664e4b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.773+0000 7f4955400640 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55664e4af010 0x55664e4b1400 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e86d1dc3096411 server_cookie=64fe739fe6b9ba71 in_seq=0 out_seq=0 2024-04-20T21:43:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.773+0000 7f493be00640 10 client.26738 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f493be00640 1 -- 172.21.15.22:0/1647852061 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55664e39c8e0 con 0x55664e4af010 2024-04-20T21:43:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f493be00640 10 client.26738 handle_command_reply: tid=0 2024-04-20T21:43:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f495fa62740 1 client.26738 shutdown 2024-04-20T21:43:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f495fa62740 1 -- 172.21.15.22:0/1647852061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430170 msgr2=0x55664e4a7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f495fa62740 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430170 0x55664e4a7920 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4948009830 tx=0x7f49480f5f00 comp rx=0 tx=0).stop 2024-04-20T21:43:15.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f495fa62740 1 -- 172.21.15.22:0/1647852061 shutdown_connections 2024-04-20T21:43:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f495fa62740 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55664e439670 0x55664e43c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f495fa62740 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664e430a90 0x55664e4a7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f495fa62740 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55664e4af010 0x55664e4b1400 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f495fa62740 1 --2- 172.21.15.22:0/1647852061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55664e430170 0x55664e4a7920 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f493be00640 0 client.26738 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f495fa62740 1 -- 172.21.15.22:0/1647852061 >> 172.21.15.22:0/1647852061 conn(0x55664e42a8a0 msgr2=0x55664e4365a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f495fa62740 1 -- 172.21.15.22:0/1647852061 shutdown_connections 2024-04-20T21:43:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f495fa62740 1 -- 172.21.15.22:0/1647852061 wait complete. 2024-04-20T21:43:15.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.774+0000 7f495fa62740 20 client.26738 trim_cache size 0 max 16384 2024-04-20T21:43:15.783 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:43:15.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.775+0000 7f4957200640 1 -- 172.21.15.22:0/4016439440 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f493c0a3d20 msgr2=0x7f493c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.775+0000 7f4957200640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f493c0a3d20 0x7f493c0a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4950005fd0 tx=0x7f4950009600 comp rx=0 tx=0).stop 2024-04-20T21:43:15.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.775+0000 7f4957200640 1 -- 172.21.15.22:0/4016439440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4958102fc0 msgr2=0x7f49581a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:15.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.775+0000 7f4957200640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4958102fc0 0x7f49581a0640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4948037040 tx=0x7f49480346f0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.775+0000 7f4957200640 1 -- 172.21.15.22:0/4016439440 shutdown_connections 2024-04-20T21:43:15.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.775+0000 7f4957200640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f493c0a3d20 0x7f493c0a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.775+0000 7f4957200640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f495810c3e0 0x7f495819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.775+0000 7f4957200640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49581038e0 0x7f49581a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.775+0000 7f4957200640 1 --2- 172.21.15.22:0/4016439440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4958102fc0 0x7f49581a0640 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:15.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.775+0000 7f4957200640 1 -- 172.21.15.22:0/4016439440 >> 172.21.15.22:0/4016439440 conn(0x7f49580fc7e0 msgr2=0x7f4958100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:15.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.775+0000 7f4957200640 1 -- 172.21.15.22:0/4016439440 shutdown_connections 2024-04-20T21:43:15.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:15.775+0000 7f4957200640 1 -- 172.21.15.22:0/4016439440 wait complete. 2024-04-20T21:43:15.785 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:15 smithi122 ceph-mon[22147]: pgmap v294: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 236 B/s wr, 2 op/s 2024-04-20T21:43:15.789 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:15 smithi022 ceph-mon[16843]: pgmap v294: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 236 B/s wr, 2 op/s 2024-04-20T21:43:15.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:15 smithi033 ceph-mon[21626]: pgmap v294: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 236 B/s wr, 2 op/s 2024-04-20T21:43:16.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:16 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2193324222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:43:16.794 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:43:16.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:43:16.795 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:43:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:16 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2193324222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:43:16.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:16 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2193324222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:43:17.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.023+0000 7f77eb200640 1 -- 172.21.15.22:0/1195387563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec104c20 msgr2=0x7f77ec108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.023+0000 7f77eb200640 1 --2- 172.21.15.22:0/1195387563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec104c20 0x7f77ec108c60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009900 tx=0x7f77d402bc30 comp rx=0 tx=0).stop 2024-04-20T21:43:17.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.023+0000 7f77eb200640 1 -- 172.21.15.22:0/1195387563 shutdown_connections 2024-04-20T21:43:17.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.023+0000 7f77eb200640 1 --2- 172.21.15.22:0/1195387563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77ec109340 0x7f77ec110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.023+0000 7f77eb200640 1 --2- 172.21.15.22:0/1195387563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec104c20 0x7f77ec108c60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.023+0000 7f77eb200640 1 --2- 172.21.15.22:0/1195387563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77ec104270 0x7f77ec104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.023+0000 7f77eb200640 1 -- 172.21.15.22:0/1195387563 >> 172.21.15.22:0/1195387563 conn(0x7f77ec0ffad0 msgr2=0x7f77ec101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.024+0000 7f77eb200640 1 -- 172.21.15.22:0/1195387563 shutdown_connections 2024-04-20T21:43:17.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.024+0000 7f77eb200640 1 -- 172.21.15.22:0/1195387563 wait complete. 2024-04-20T21:43:17.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.025+0000 7f77eb200640 1 Processor -- start 2024-04-20T21:43:17.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.025+0000 7f77eb200640 1 -- start start 2024-04-20T21:43:17.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.025+0000 7f77eb200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77ec104270 0x7f77ec1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.025+0000 7f77eb200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec109340 0x7f77ec1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.025+0000 7f77eb200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77ec19a7a0 0x7f77ec19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.025+0000 7f77eb200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec114420 con 0x7f77ec104270 2024-04-20T21:43:17.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.025+0000 7f77eb200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1142a0 con 0x7f77ec109340 2024-04-20T21:43:17.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.025+0000 7f77eb200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1145a0 con 0x7f77ec19a7a0 2024-04-20T21:43:17.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.025+0000 7f77e9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77ec104270 0x7f77ec1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.026+0000 7f77e9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77ec104270 0x7f77ec1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54070/0 (socket says 172.21.15.22:54070) 2024-04-20T21:43:17.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.026+0000 7f77e9e00640 1 -- 172.21.15.22:0/380834614 learned_addr learned my addr 172.21.15.22:0/380834614 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:17.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.026+0000 7f77e3e00640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec109340 0x7f77ec1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.026+0000 7f77ea800640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77ec19a7a0 0x7f77ec19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.026+0000 7f77e9e00640 1 -- 172.21.15.22:0/380834614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77ec19a7a0 msgr2=0x7f77ec19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.026+0000 7f77e9e00640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77ec19a7a0 0x7f77ec19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.026+0000 7f77e9e00640 1 -- 172.21.15.22:0/380834614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec109340 msgr2=0x7f77ec1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.026+0000 7f77e9e00640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec109340 0x7f77ec1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.026+0000 7f77e9e00640 1 -- 172.21.15.22:0/380834614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec19b330 con 0x7f77ec104270 2024-04-20T21:43:17.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.026+0000 7f77e9e00640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77ec104270 0x7f77ec1a0630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00e9f0 tx=0x7f77dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.027+0000 7f77e2a00640 1 -- 172.21.15.22:0/380834614 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc00cd80 con 0x7f77ec104270 2024-04-20T21:43:17.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.027+0000 7f77e2a00640 1 -- 172.21.15.22:0/380834614 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77dc005500 con 0x7f77ec104270 2024-04-20T21:43:17.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.027+0000 7f77eb200640 1 -- 172.21.15.22:0/380834614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ec19b620 con 0x7f77ec104270 2024-04-20T21:43:17.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.027+0000 7f77e2a00640 1 -- 172.21.15.22:0/380834614 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc010690 con 0x7f77ec104270 2024-04-20T21:43:17.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.027+0000 7f77eb200640 1 -- 172.21.15.22:0/380834614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ec1a77d0 con 0x7f77ec104270 2024-04-20T21:43:17.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.030+0000 7f77e2a00640 1 -- 172.21.15.22:0/380834614 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f77dc0046c0 con 0x7f77ec104270 2024-04-20T21:43:17.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.030+0000 7f77f3be5740 1 Processor -- start 2024-04-20T21:43:17.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.031+0000 7f77f3be5740 1 -- start start 2024-04-20T21:43:17.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.031+0000 7f77f3be5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 0x5603289364d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.031+0000 7f77f3be5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560328936a10 0x56032893f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.031+0000 7f77f3be5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x5603289419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.031+0000 7f77f3be5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56032889ad20 con 0x5603289360f0 2024-04-20T21:43:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.031+0000 7f77f3be5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5603288413e0 con 0x560328936a10 2024-04-20T21:43:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.031+0000 7f77f3be5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560328896ce0 con 0x56032893f5f0 2024-04-20T21:43:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.031+0000 7f77ea800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 0x5603289364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.031+0000 7f77ea800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 0x5603289364d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54084/0 (socket says 172.21.15.22:54084) 2024-04-20T21:43:17.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.031+0000 7f77ea800640 1 -- 172.21.15.22:0/916715174 learned_addr learned my addr 172.21.15.22:0/916715174 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:17.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.031+0000 7f77e9e00640 1 --2- 172.21.15.22:0/916715174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560328936a10 0x56032893f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.031+0000 7f77e3e00640 1 --2- 172.21.15.22:0/916715174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x5603289419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.032+0000 7f77ea800640 1 -- 172.21.15.22:0/916715174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 msgr2=0x5603289419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.032+0000 7f77ea800640 1 --2- 172.21.15.22:0/916715174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x5603289419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.032+0000 7f77ea800640 1 -- 172.21.15.22:0/916715174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560328936a10 msgr2=0x56032893f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.032+0000 7f77ea800640 1 --2- 172.21.15.22:0/916715174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560328936a10 0x56032893f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.032+0000 7f77ea800640 1 -- 172.21.15.22:0/916715174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560328860980 con 0x5603289360f0 2024-04-20T21:43:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.032+0000 7f77e3e00640 1 --2- 172.21.15.22:0/916715174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x5603289419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.032+0000 7f77ea800640 1 --2- 172.21.15.22:0/916715174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 0x5603289364d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009f50 tx=0x7f77e40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ec5f72192daf9f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.032+0000 7f77e2a00640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f77d00a3e70 0x7f77d00a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.032+0000 7f77db400640 1 -- 172.21.15.22:0/916715174 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4004d20 con 0x5603289360f0 2024-04-20T21:43:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.033+0000 7f77e2a00640 1 -- 172.21.15.22:0/380834614 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f77dc0d0b60 con 0x7f77ec104270 2024-04-20T21:43:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.033+0000 7f77e9e00640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f77d00a3e70 0x7f77d00a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.033+0000 7f77db400640 1 -- 172.21.15.22:0/916715174 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e4004050 con 0x5603289360f0 2024-04-20T21:43:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.033+0000 7f77db400640 1 -- 172.21.15.22:0/916715174 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4005280 con 0x5603289360f0 2024-04-20T21:43:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.033+0000 7f77f3be5740 1 -- 172.21.15.22:0/916715174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 msgr2=0x5603289364d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.033+0000 7f77e9e00640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f77d00a3e70 0x7f77d00a6330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0ce310 tx=0x7f77dc0d46b0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.033+0000 7f77f3be5740 1 --2- 172.21.15.22:0/916715174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 0x5603289364d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009f50 tx=0x7f77e40118d0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.034+0000 7f77f3be5740 1 -- 172.21.15.22:0/916715174 shutdown_connections 2024-04-20T21:43:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.034+0000 7f77f3be5740 1 --2- 172.21.15.22:0/916715174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x5603289419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.034+0000 7f77f3be5740 1 --2- 172.21.15.22:0/916715174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560328936a10 0x56032893f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.034+0000 7f77f3be5740 1 --2- 172.21.15.22:0/916715174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 0x5603289364d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.034+0000 7f77f3be5740 1 -- 172.21.15.22:0/916715174 >> 172.21.15.22:0/916715174 conn(0x560328930820 msgr2=0x560328930c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.034+0000 7f77f3be5740 1 -- 172.21.15.22:0/916715174 shutdown_connections 2024-04-20T21:43:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.034+0000 7f77f3be5740 1 -- 172.21.15.22:0/916715174 wait complete. 2024-04-20T21:43:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.034+0000 7f77f3be5740 1 -- start start 2024-04-20T21:43:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.034+0000 7f77f3be5740 1 Processor -- start 2024-04-20T21:43:17.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.035+0000 7f77f3be5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603289360f0 0x56032893c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.035+0000 7f77f3be5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 0x56032893cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.035+0000 7f77f3be5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x56032893d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.035+0000 7f77f3be5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5603289423e0 con 0x560328936a10 2024-04-20T21:43:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.035+0000 7f77f3be5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5603288547a0 con 0x5603289360f0 2024-04-20T21:43:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.035+0000 7f77f3be5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560328942560 con 0x56032893f5f0 2024-04-20T21:43:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.035+0000 7f77f3be5740 10 client.0 _ll_register_callbacks cb 0x560328856b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.035+0000 7f77e3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 0x56032893cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.035+0000 7f77e3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 0x56032893cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54088/0 (socket says 172.21.15.22:54088) 2024-04-20T21:43:17.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.035+0000 7f77e3e00640 1 -- 172.21.15.22:0/1408192275 learned_addr learned my addr 172.21.15.22:0/1408192275 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:17.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.035+0000 7f77ea800640 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603289360f0 0x56032893c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.035+0000 7f77ea800640 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x56032893d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.036+0000 7f77e3e00640 1 -- 172.21.15.22:0/1408192275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 msgr2=0x56032893d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.036+0000 7f77e3e00640 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x56032893d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.036+0000 7f77e3e00640 1 -- 172.21.15.22:0/1408192275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603289360f0 msgr2=0x56032893c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.036+0000 7f77e3e00640 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603289360f0 0x56032893c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.036+0000 7f77e3e00640 1 -- 172.21.15.22:0/1408192275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603286757e0 con 0x560328936a10 2024-04-20T21:43:17.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.036+0000 7f77e3e00640 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 0x56032893cb70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009900 tx=0x7f77d4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0217d654cdc95d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.036+0000 7f77cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:43:17.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.036+0000 7f77cfe00640 1 -- 172.21.15.22:0/1408192275 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d402cb50 con 0x560328936a10 2024-04-20T21:43:17.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.037+0000 7f77f3be5740 1 -- 172.21.15.22:0/1408192275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77e4004050 con 0x560328936a10 2024-04-20T21:43:17.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.037+0000 7f77cfe00640 1 -- 172.21.15.22:0/1408192275 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f77d40046c0 con 0x560328936a10 2024-04-20T21:43:17.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.037+0000 7f77cfe00640 1 -- 172.21.15.22:0/1408192275 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f77d402ccf0 con 0x560328936a10 2024-04-20T21:43:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.037+0000 7f77f3be5740 10 client.16605 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.037+0000 7f77f3be5740 1 -- 172.21.15.22:0/1408192275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5603286d5f70 con 0x560328936a10 2024-04-20T21:43:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.037+0000 7f77cfe00640 1 -- 172.21.15.22:0/1408192275 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e4004050 con 0x560328936a10 2024-04-20T21:43:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.037+0000 7f77cfe00640 1 -- 172.21.15.22:0/1408192275 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f77d40035f0 con 0x560328936a10 2024-04-20T21:43:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.037+0000 7f77f3be5740 10 client.16605 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.037+0000 7f77f3be5740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.038+0000 7f77f3be5740 20 client.16605 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.038+0000 7f77f3be5740 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56032894f930 0x560328951d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.038+0000 7f77f3be5740 4 client.16605 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.038+0000 7f77f3be5740 1 -- 172.21.15.22:0/1408192275 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603288a8740 con 0x56032894f930 2024-04-20T21:43:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.038+0000 7f77ea800640 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56032894f930 0x560328951d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.038+0000 7f77ea800640 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56032894f930 0x560328951d20 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9498116cc7a7c6 server_cookie=467e9f342c12eff0 in_seq=0 out_seq=0 2024-04-20T21:43:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.038+0000 7f77cfe00640 10 client.16605 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.039+0000 7f77cfe00640 1 -- 172.21.15.22:0/1408192275 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603288a8740 con 0x56032894f930 2024-04-20T21:43:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.039+0000 7f77cfe00640 10 client.16605 handle_command_reply: tid=0 2024-04-20T21:43:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77f3be5740 1 client.16605 shutdown 2024-04-20T21:43:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77f3be5740 1 -- 172.21.15.22:0/1408192275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 msgr2=0x56032893cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77f3be5740 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 0x56032893cb70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009900 tx=0x7f77d4005dd0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77f3be5740 1 -- 172.21.15.22:0/1408192275 shutdown_connections 2024-04-20T21:43:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77f3be5740 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x56032893d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77f3be5740 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 0x56032893cb70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77f3be5740 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56032894f930 0x560328951d20 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77f3be5740 1 --2- 172.21.15.22:0/1408192275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603289360f0 0x56032893c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77cfe00640 0 client.16605 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77f3be5740 1 -- 172.21.15.22:0/1408192275 >> 172.21.15.22:0/1408192275 conn(0x560328930820 msgr2=0x560328932570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77f3be5740 1 -- 172.21.15.22:0/1408192275 shutdown_connections 2024-04-20T21:43:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77f3be5740 1 -- 172.21.15.22:0/1408192275 wait complete. 2024-04-20T21:43:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.040+0000 7f77f3be5740 20 client.16605 trim_cache size 0 max 16384 2024-04-20T21:43:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.060+0000 7f77f3be5740 1 Processor -- start 2024-04-20T21:43:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.060+0000 7f77f3be5740 1 -- start start 2024-04-20T21:43:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.060+0000 7f77f3be5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603289360f0 0x56032893ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.060+0000 7f77f3be5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 0x56032894b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.060+0000 7f77f3be5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x5603289575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.060+0000 7f77f3be5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1ade80 con 0x560328936a10 2024-04-20T21:43:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.060+0000 7f77f3be5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1b06e0 con 0x5603289360f0 2024-04-20T21:43:17.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.060+0000 7f77f3be5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1ae4d0 con 0x56032893f5f0 2024-04-20T21:43:17.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.060+0000 7f77e3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 0x56032894b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.060+0000 7f77e3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 0x56032894b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54102/0 (socket says 172.21.15.22:54102) 2024-04-20T21:43:17.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.060+0000 7f77e3e00640 1 -- 172.21.15.22:0/2952134135 learned_addr learned my addr 172.21.15.22:0/2952134135 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:17.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.061+0000 7f77ea800640 1 --2- 172.21.15.22:0/2952134135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603289360f0 0x56032893ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.061+0000 7f77ea800640 1 --2- 172.21.15.22:0/2952134135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x5603289575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.061+0000 7f77e3e00640 1 -- 172.21.15.22:0/2952134135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 msgr2=0x5603289575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.061+0000 7f77e3e00640 1 --2- 172.21.15.22:0/2952134135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x5603289575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.061+0000 7f77e3e00640 1 -- 172.21.15.22:0/2952134135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603289360f0 msgr2=0x56032893ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.061+0000 7f77e3e00640 1 --2- 172.21.15.22:0/2952134135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603289360f0 0x56032893ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.061+0000 7f77e3e00640 1 -- 172.21.15.22:0/2952134135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560328936840 con 0x560328936a10 2024-04-20T21:43:17.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.061+0000 7f77e3e00640 1 --2- 172.21.15.22:0/2952134135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 0x56032894b2a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f77d4006010 tx=0x7f77d402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30824eecb436cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.062+0000 7f77daa00640 1 -- 172.21.15.22:0/2952134135 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4056070 con 0x560328936a10 2024-04-20T21:43:17.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.062+0000 7f77daa00640 1 -- 172.21.15.22:0/2952134135 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d4009520 con 0x560328936a10 2024-04-20T21:43:17.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.062+0000 7f77daa00640 1 -- 172.21.15.22:0/2952134135 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4000e60 con 0x560328936a10 2024-04-20T21:43:17.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.062+0000 7f77f3be5740 1 -- 172.21.15.22:0/2952134135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 msgr2=0x56032894b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.062+0000 7f77f3be5740 1 --2- 172.21.15.22:0/2952134135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 0x56032894b2a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f77d4006010 tx=0x7f77d402bc30 comp rx=0 tx=0).stop 2024-04-20T21:43:17.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.062+0000 7f77f3be5740 1 -- 172.21.15.22:0/2952134135 shutdown_connections 2024-04-20T21:43:17.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.062+0000 7f77f3be5740 1 --2- 172.21.15.22:0/2952134135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56032893f5f0 0x5603289575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.062+0000 7f77f3be5740 1 --2- 172.21.15.22:0/2952134135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560328936a10 0x56032894b2a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.062+0000 7f77f3be5740 1 --2- 172.21.15.22:0/2952134135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603289360f0 0x56032893ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.062+0000 7f77f3be5740 1 -- 172.21.15.22:0/2952134135 >> 172.21.15.22:0/2952134135 conn(0x560328930820 msgr2=0x56032892fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.062+0000 7f77f3be5740 1 -- 172.21.15.22:0/2952134135 shutdown_connections 2024-04-20T21:43:17.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.063+0000 7f77f3be5740 1 -- 172.21.15.22:0/2952134135 wait complete. 2024-04-20T21:43:17.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.063+0000 7f77f3be5740 1 -- start start 2024-04-20T21:43:17.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.063+0000 7f77f3be5740 1 Processor -- start 2024-04-20T21:43:17.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77f3be5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 0x560328943ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77f3be5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560328936a10 0x560328943ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77f3be5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56032893f5f0 0x5603289aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77f3be5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1aeba0 con 0x5603289360f0 2024-04-20T21:43:17.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77f3be5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1b0010 con 0x56032893f5f0 2024-04-20T21:43:17.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77f3be5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1af270 con 0x560328936a10 2024-04-20T21:43:17.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77f3be5740 10 client.0 _ll_register_callbacks cb 0x7f77d4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:17.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77ea800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 0x560328943ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77ea800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 0x560328943ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54116/0 (socket says 172.21.15.22:54116) 2024-04-20T21:43:17.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77ea800640 1 -- 172.21.15.22:0/2817446843 learned_addr learned my addr 172.21.15.22:0/2817446843 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:17.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77e3e00640 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560328936a10 0x560328943ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77ea800640 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56032893f5f0 0x5603289aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77e3e00640 1 -- 172.21.15.22:0/2817446843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56032893f5f0 msgr2=0x5603289aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77e3e00640 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56032893f5f0 0x5603289aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77e3e00640 1 -- 172.21.15.22:0/2817446843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 msgr2=0x560328943ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.064+0000 7f77e3e00640 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 0x560328943ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.065+0000 7f77e3e00640 1 -- 172.21.15.22:0/2817446843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56032856d170 con 0x560328936a10 2024-04-20T21:43:17.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.065+0000 7f77ea800640 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 0x560328943ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:17.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.065+0000 7f77e3e00640 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560328936a10 0x560328943ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x56032893c200 tx=0x7f77d4004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d534e18cb0b7c4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.065+0000 7f77cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:43:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.065+0000 7f77cfe00640 1 -- 172.21.15.22:0/2817446843 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4056040 con 0x560328936a10 2024-04-20T21:43:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.065+0000 7f77f3be5740 1 -- 172.21.15.22:0/2817446843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77d4009520 con 0x560328936a10 2024-04-20T21:43:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.065+0000 7f77cfe00640 1 -- 172.21.15.22:0/2817446843 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f77d40050e0 con 0x560328936a10 2024-04-20T21:43:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.065+0000 7f77cfe00640 1 -- 172.21.15.22:0/2817446843 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f77d4000e40 con 0x560328936a10 2024-04-20T21:43:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.065+0000 7f77f3be5740 10 client.26785 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.066+0000 7f77f3be5740 1 -- 172.21.15.22:0/2817446843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5603289b6670 con 0x560328936a10 2024-04-20T21:43:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.066+0000 7f77cfe00640 1 -- 172.21.15.22:0/2817446843 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d4009520 con 0x560328936a10 2024-04-20T21:43:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.066+0000 7f77cfe00640 1 -- 172.21.15.22:0/2817446843 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f77d4063450 con 0x560328936a10 2024-04-20T21:43:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.066+0000 7f77f3be5740 10 client.26785 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.066+0000 7f77f3be5740 10 client.26785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.066+0000 7f77f3be5740 20 client.26785 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.066+0000 7f77f3be5740 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5603289b6ce0 0x5603289b9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.066+0000 7f77f3be5740 4 client.26785 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:43:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.066+0000 7f77f3be5740 1 -- 172.21.15.22:0/2817446843 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5603288a25b0 con 0x5603289b6ce0 2024-04-20T21:43:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.067+0000 7f77ea800640 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5603289b6ce0 0x5603289b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.067+0000 7f77ea800640 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5603289b6ce0 0x5603289b9100 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd73f8199b7edba2 server_cookie=592b5c6576d207e7 in_seq=0 out_seq=0 2024-04-20T21:43:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.067+0000 7f77cfe00640 10 client.26785 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.068+0000 7f77cfe00640 1 -- 172.21.15.22:0/2817446843 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5603288a25b0 con 0x5603289b6ce0 2024-04-20T21:43:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.068+0000 7f77cfe00640 10 client.26785 handle_command_reply: tid=0 2024-04-20T21:43:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.068+0000 7f77f3be5740 1 client.26785 shutdown 2024-04-20T21:43:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.068+0000 7f77f3be5740 1 -- 172.21.15.22:0/2817446843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560328936a10 msgr2=0x560328943ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.068+0000 7f77f3be5740 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560328936a10 0x560328943ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x56032893c200 tx=0x7f77d4004970 comp rx=0 tx=0).stop 2024-04-20T21:43:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.069+0000 7f77f3be5740 1 -- 172.21.15.22:0/2817446843 shutdown_connections 2024-04-20T21:43:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.069+0000 7f77f3be5740 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5603289b6ce0 0x5603289b9100 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.069+0000 7f77f3be5740 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56032893f5f0 0x5603289aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.069+0000 7f77f3be5740 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560328936a10 0x560328943ff0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.069+0000 7f77f3be5740 1 --2- 172.21.15.22:0/2817446843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603289360f0 0x560328943ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.069+0000 7f77cfe00640 0 client.26785 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:17.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.069+0000 7f77f3be5740 1 -- 172.21.15.22:0/2817446843 >> 172.21.15.22:0/2817446843 conn(0x560328930820 msgr2=0x56032892f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.069+0000 7f77f3be5740 1 -- 172.21.15.22:0/2817446843 shutdown_connections 2024-04-20T21:43:17.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.069+0000 7f77f3be5740 1 -- 172.21.15.22:0/2817446843 wait complete. 2024-04-20T21:43:17.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.069+0000 7f77f3be5740 20 client.26785 trim_cache size 0 max 16384 2024-04-20T21:43:17.077 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:43:17.077 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:43:17.077 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "86294b5a-2e8b-4d3f-bd8f-e2654d1b5ab5", 2024-04-20T21:43:17.077 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:43:17.077 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:43:17.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.070+0000 7f77eb200640 1 -- 172.21.15.22:0/380834614 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f77d00a3e70 msgr2=0x7f77d00a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.070+0000 7f77eb200640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f77d00a3e70 0x7f77d00a6330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0ce310 tx=0x7f77dc0d46b0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.070+0000 7f77eb200640 1 -- 172.21.15.22:0/380834614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77ec104270 msgr2=0x7f77ec1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.070+0000 7f77eb200640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77ec104270 0x7f77ec1a0630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00e9f0 tx=0x7f77dc00edf0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.070+0000 7f77eb200640 1 -- 172.21.15.22:0/380834614 shutdown_connections 2024-04-20T21:43:17.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.070+0000 7f77eb200640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f77d00a3e70 0x7f77d00a6330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.070+0000 7f77eb200640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77ec19a7a0 0x7f77ec19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.070+0000 7f77eb200640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec109340 0x7f77ec1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.070+0000 7f77eb200640 1 --2- 172.21.15.22:0/380834614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77ec104270 0x7f77ec1a0630 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.070+0000 7f77eb200640 1 -- 172.21.15.22:0/380834614 >> 172.21.15.22:0/380834614 conn(0x7f77ec0ffad0 msgr2=0x7f77ec100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.070+0000 7f77eb200640 1 -- 172.21.15.22:0/380834614 shutdown_connections 2024-04-20T21:43:17.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.070+0000 7f77eb200640 1 -- 172.21.15.22:0/380834614 wait complete. 2024-04-20T21:43:17.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:43:17.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:43:17.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.301+0000 7f7860c00640 1 -- 172.21.15.22:0/1243088008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c112f30 msgr2=0x7f785c115320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.301+0000 7f7860c00640 1 --2- 172.21.15.22:0/1243088008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c112f30 0x7f785c115320 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7854009ca0 tx=0x7f785402c180 comp rx=0 tx=0).stop 2024-04-20T21:43:17.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.301+0000 7f7860c00640 1 -- 172.21.15.22:0/1243088008 shutdown_connections 2024-04-20T21:43:17.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.301+0000 7f7860c00640 1 --2- 172.21.15.22:0/1243088008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c112f30 0x7f785c115320 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.301+0000 7f7860c00640 1 --2- 172.21.15.22:0/1243088008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f785c074c20 0x7f785c1129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.301+0000 7f7860c00640 1 --2- 172.21.15.22:0/1243088008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f785c074300 0x7f785c0746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.301+0000 7f7860c00640 1 -- 172.21.15.22:0/1243088008 >> 172.21.15.22:0/1243088008 conn(0x7f785c0fd9d0 msgr2=0x7f785c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.301+0000 7f7860c00640 1 -- 172.21.15.22:0/1243088008 shutdown_connections 2024-04-20T21:43:17.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.301+0000 7f7860c00640 1 -- 172.21.15.22:0/1243088008 wait complete. 2024-04-20T21:43:17.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.303+0000 7f7860c00640 1 Processor -- start 2024-04-20T21:43:17.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.303+0000 7f7860c00640 1 -- start start 2024-04-20T21:43:17.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.303+0000 7f7860c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f785c074300 0x7f785c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.303+0000 7f7860c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c074c20 0x7f785c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.303+0000 7f7860c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f785c112f30 0x7f785c1a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.303+0000 7f7860c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f785c1188f0 con 0x7f785c112f30 2024-04-20T21:43:17.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.303+0000 7f7860c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f785c118770 con 0x7f785c074c20 2024-04-20T21:43:17.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.303+0000 7f7860c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f785c118a70 con 0x7f785c074300 2024-04-20T21:43:17.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.304+0000 7f7859e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f785c074300 0x7f785c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.304+0000 7f785a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f785c112f30 0x7f785c1a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.304+0000 7f7859e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f785c074300 0x7f785c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:45792/0 (socket says 172.21.15.22:45792) 2024-04-20T21:43:17.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.304+0000 7f7859e00640 1 -- 172.21.15.22:0/1100337964 learned_addr learned my addr 172.21.15.22:0/1100337964 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:17.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.304+0000 7f7851400640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c074c20 0x7f785c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.304+0000 7f7859e00640 1 -- 172.21.15.22:0/1100337964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c074c20 msgr2=0x7f785c19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.304+0000 7f7859e00640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c074c20 0x7f785c19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.304+0000 7f7859e00640 1 -- 172.21.15.22:0/1100337964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f785c112f30 msgr2=0x7f785c1a3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.304+0000 7f7859e00640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f785c112f30 0x7f785c1a3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.304+0000 7f7859e00640 1 -- 172.21.15.22:0/1100337964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785c1a3e50 con 0x7f785c074300 2024-04-20T21:43:17.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.305+0000 7f7859e00640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f785c074300 0x7f785c19eeb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f784c00e9f0 tx=0x7f784c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.305+0000 7f7853400640 1 -- 172.21.15.22:0/1100337964 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c00cd80 con 0x7f785c074300 2024-04-20T21:43:17.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.305+0000 7f7853400640 1 -- 172.21.15.22:0/1100337964 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f784c005500 con 0x7f785c074300 2024-04-20T21:43:17.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.305+0000 7f7860c00640 1 -- 172.21.15.22:0/1100337964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785c1a4140 con 0x7f785c074300 2024-04-20T21:43:17.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.305+0000 7f7853400640 1 -- 172.21.15.22:0/1100337964 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c010690 con 0x7f785c074300 2024-04-20T21:43:17.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.305+0000 7f7860c00640 1 -- 172.21.15.22:0/1100337964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785c1ab9d0 con 0x7f785c074300 2024-04-20T21:43:17.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.308+0000 7f7863861740 1 Processor -- start 2024-04-20T21:43:17.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.308+0000 7f7863861740 1 -- start start 2024-04-20T21:43:17.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7863861740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7853400640 1 -- 172.21.15.22:0/1100337964 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f784c0046c0 con 0x7f785c074300 2024-04-20T21:43:17.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7863861740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5fa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7863861740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5fca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7863861740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f3aa557150 con 0x55f3aa5f11a0 2024-04-20T21:43:17.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7863861740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f3aa4fd370 con 0x55f3aa5fa6a0 2024-04-20T21:43:17.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7863861740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f3aa553110 con 0x55f3aa5f1ac0 2024-04-20T21:43:17.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f785a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7851400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7851400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5fca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:40604/0 (socket says 172.21.15.22:40604) 2024-04-20T21:43:17.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7851400640 1 -- 172.21.15.22:0/1398654326 learned_addr learned my addr 172.21.15.22:0/1398654326 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:17.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7859e00640 1 --2- 172.21.15.22:0/1398654326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5fa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7851400640 1 -- 172.21.15.22:0/1398654326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f1ac0 msgr2=0x55f3aa5fa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7851400640 1 --2- 172.21.15.22:0/1398654326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5fa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7851400640 1 -- 172.21.15.22:0/1398654326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f11a0 msgr2=0x55f3aa5f1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7851400640 1 --2- 172.21.15.22:0/1398654326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.309+0000 7f7851400640 1 -- 172.21.15.22:0/1398654326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3aa3337d0 con 0x55f3aa5fa6a0 2024-04-20T21:43:17.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.310+0000 7f7851400640 1 --2- 172.21.15.22:0/1398654326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5fca90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f784400bc10 tx=0x7f78440107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af66075de96ae9b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.310+0000 7f784b000640 1 -- 172.21.15.22:0/1398654326 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844005020 con 0x55f3aa5fa6a0 2024-04-20T21:43:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.310+0000 7f784b000640 1 -- 172.21.15.22:0/1398654326 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78440047a0 con 0x55f3aa5fa6a0 2024-04-20T21:43:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.310+0000 7f784b000640 1 -- 172.21.15.22:0/1398654326 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844011730 con 0x55f3aa5fa6a0 2024-04-20T21:43:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.310+0000 7f7853400640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f78400a3d20 0x7f78400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.311+0000 7f7853400640 1 -- 172.21.15.22:0/1100337964 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f784c0d0c40 con 0x7f785c074300 2024-04-20T21:43:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.311+0000 7f785a800640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f78400a3d20 0x7f78400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.311+0000 7f7863861740 1 -- 172.21.15.22:0/1398654326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 msgr2=0x55f3aa5fca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.311+0000 7f7863861740 1 --2- 172.21.15.22:0/1398654326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5fca90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f784400bc10 tx=0x7f78440107a0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.311+0000 7f7863861740 1 -- 172.21.15.22:0/1398654326 shutdown_connections 2024-04-20T21:43:17.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.311+0000 7f7863861740 1 --2- 172.21.15.22:0/1398654326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5fca90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.311+0000 7f7863861740 1 --2- 172.21.15.22:0/1398654326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.311+0000 7f7863861740 1 --2- 172.21.15.22:0/1398654326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.311+0000 7f7863861740 1 -- 172.21.15.22:0/1398654326 >> 172.21.15.22:0/1398654326 conn(0x55f3aa5eb8d0 msgr2=0x55f3aa5ebcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.311+0000 7f7863861740 1 -- 172.21.15.22:0/1398654326 shutdown_connections 2024-04-20T21:43:17.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.311+0000 7f7863861740 1 -- 172.21.15.22:0/1398654326 wait complete. 2024-04-20T21:43:17.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.312+0000 7f785a800640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f78400a3d20 0x7f78400a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f785402c6c0 tx=0x7f7854005d80 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.312+0000 7f7863861740 1 -- start start 2024-04-20T21:43:17.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.312+0000 7f7863861740 1 Processor -- start 2024-04-20T21:43:17.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.312+0000 7f7863861740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.312+0000 7f7863861740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.312+0000 7f7863861740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.312+0000 7f7863861740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f3aa5fd490 con 0x55f3aa5fa6a0 2024-04-20T21:43:17.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.312+0000 7f7863861740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f3aa510740 con 0x55f3aa5f1ac0 2024-04-20T21:43:17.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.312+0000 7f7863861740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f3aa5fd610 con 0x55f3aa5f11a0 2024-04-20T21:43:17.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.313+0000 7f7863861740 10 client.0 _ll_register_callbacks cb 0x55f3aa50bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:17.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.313+0000 7f7859e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.313+0000 7f785a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.313+0000 7f7859e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:45808/0 (socket says 172.21.15.22:45808) 2024-04-20T21:43:17.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.313+0000 7f7851400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.313+0000 7f785a800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54150/0 (socket says 172.21.15.22:54150) 2024-04-20T21:43:17.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.313+0000 7f7859e00640 1 -- 172.21.15.22:0/2893669340 learned_addr learned my addr 172.21.15.22:0/2893669340 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:17.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.313+0000 7f7859e00640 1 -- 172.21.15.22:0/2893669340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5f1ac0 msgr2=0x55f3aa5f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.313+0000 7f7859e00640 1 --2- 172.21.15.22:0/2893669340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5f7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.313+0000 7f7859e00640 1 -- 172.21.15.22:0/2893669340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5fa6a0 msgr2=0x55f3aa5f81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.313+0000 7f7859e00640 1 --2- 172.21.15.22:0/2893669340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5f81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.313+0000 7f7859e00640 1 -- 172.21.15.22:0/2893669340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3aa2693f0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.314+0000 7f7859e00640 1 --2- 172.21.15.22:0/2893669340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f7740 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f784c00c6a0 tx=0x7f784c0d4380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=435d203620683d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.314+0000 7f783f400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:43:17.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.314+0000 7f783f400640 1 -- 172.21.15.22:0/2893669340 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c09c6d0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.314+0000 7f783f400640 1 -- 172.21.15.22:0/2893669340 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f784c0a5370 con 0x55f3aa5f11a0 2024-04-20T21:43:17.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.314+0000 7f7863861740 1 -- 172.21.15.22:0/2893669340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78440047a0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.314+0000 7f783f400640 1 -- 172.21.15.22:0/2893669340 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f784c09c4f0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.314+0000 7f7863861740 10 client.26797 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:17.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.314+0000 7f7863861740 1 -- 172.21.15.22:0/2893669340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f3aa2290f0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.315+0000 7f783f400640 1 -- 172.21.15.22:0/2893669340 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78440047a0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.315+0000 7f783f400640 1 -- 172.21.15.22:0/2893669340 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f784c09bfe0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.315+0000 7f7863861740 10 client.26797 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:17.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.315+0000 7f7863861740 10 client.26797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:17.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.315+0000 7f7863861740 20 client.26797 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:17.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.315+0000 7f7863861740 1 --2- 172.21.15.22:0/2893669340 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55f3aa60aa30 0x55f3aa60ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.315+0000 7f7863861740 4 client.26797 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:17.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.315+0000 7f7863861740 1 -- 172.21.15.22:0/2893669340 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3aa564b70 con 0x55f3aa60aa30 2024-04-20T21:43:17.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.315+0000 7f7851400640 1 --2- 172.21.15.22:0/2893669340 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55f3aa60aa30 0x55f3aa60ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.316+0000 7f7851400640 1 --2- 172.21.15.22:0/2893669340 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55f3aa60aa30 0x55f3aa60ce80 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d6a076ca2750776 server_cookie=1933d770503b3386 in_seq=0 out_seq=0 2024-04-20T21:43:17.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.316+0000 7f783f400640 10 client.26797 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:17.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f783f400640 1 -- 172.21.15.22:0/2893669340 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3aa564b70 con 0x55f3aa60aa30 2024-04-20T21:43:17.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f783f400640 10 client.26797 handle_command_reply: tid=0 2024-04-20T21:43:17.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f7863861740 1 client.26797 shutdown 2024-04-20T21:43:17.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f7863861740 1 -- 172.21.15.22:0/2893669340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 msgr2=0x55f3aa5f7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f7863861740 1 --2- 172.21.15.22:0/2893669340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f7740 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f784c00c6a0 tx=0x7f784c0d4380 comp rx=0 tx=0).stop 2024-04-20T21:43:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f7863861740 1 -- 172.21.15.22:0/2893669340 shutdown_connections 2024-04-20T21:43:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f7863861740 1 --2- 172.21.15.22:0/2893669340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5f81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f7863861740 1 --2- 172.21.15.22:0/2893669340 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55f3aa60aa30 0x55f3aa60ce80 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f7863861740 1 --2- 172.21.15.22:0/2893669340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f7863861740 1 --2- 172.21.15.22:0/2893669340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f7740 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f783f400640 0 client.26797 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f7863861740 1 -- 172.21.15.22:0/2893669340 >> 172.21.15.22:0/2893669340 conn(0x55f3aa5eb8d0 msgr2=0x55f3aa5ed620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f7863861740 1 -- 172.21.15.22:0/2893669340 shutdown_connections 2024-04-20T21:43:17.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f7863861740 1 -- 172.21.15.22:0/2893669340 wait complete. 2024-04-20T21:43:17.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.317+0000 7f7863861740 20 client.26797 trim_cache size 0 max 16384 2024-04-20T21:43:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.336+0000 7f7863861740 1 Processor -- start 2024-04-20T21:43:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.337+0000 7f7863861740 1 -- start start 2024-04-20T21:43:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.337+0000 7f7863861740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.337+0000 7f7863861740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa606350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.337+0000 7f7863861740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5ea740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.337+0000 7f7863861740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f785c1b24a0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.337+0000 7f7863861740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f785c1b4d00 con 0x55f3aa5f1ac0 2024-04-20T21:43:17.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.337+0000 7f7863861740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f785c1b2af0 con 0x55f3aa5fa6a0 2024-04-20T21:43:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.337+0000 7f7859e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.337+0000 7f7859e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54154/0 (socket says 172.21.15.22:54154) 2024-04-20T21:43:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.337+0000 7f7859e00640 1 -- 172.21.15.22:0/1379778264 learned_addr learned my addr 172.21.15.22:0/1379778264 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.337+0000 7f7851400640 1 --2- 172.21.15.22:0/1379778264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa606350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.337+0000 7f785a800640 1 --2- 172.21.15.22:0/1379778264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5ea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.338+0000 7f7859e00640 1 -- 172.21.15.22:0/1379778264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5fa6a0 msgr2=0x55f3aa5ea740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.338+0000 7f7859e00640 1 --2- 172.21.15.22:0/1379778264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5ea740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.338+0000 7f7859e00640 1 -- 172.21.15.22:0/1379778264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5f1ac0 msgr2=0x55f3aa606350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.338+0000 7f7859e00640 1 --2- 172.21.15.22:0/1379778264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa606350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.338+0000 7f7859e00640 1 -- 172.21.15.22:0/1379778264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3aa5f0940 con 0x55f3aa5f11a0 2024-04-20T21:43:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.338+0000 7f7859e00640 1 --2- 172.21.15.22:0/1379778264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f9e00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f784c0ce940 tx=0x7f784c09c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5b0b0c68fc5bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.338+0000 7f784a600640 1 -- 172.21.15.22:0/1379778264 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c09c6d0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.338+0000 7f784a600640 1 -- 172.21.15.22:0/1379778264 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f784c09d290 con 0x55f3aa5f11a0 2024-04-20T21:43:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.338+0000 7f784a600640 1 -- 172.21.15.22:0/1379778264 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c099200 con 0x55f3aa5f11a0 2024-04-20T21:43:17.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.339+0000 7f7863861740 1 -- 172.21.15.22:0/1379778264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f11a0 msgr2=0x55f3aa5f9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.339+0000 7f7863861740 1 --2- 172.21.15.22:0/1379778264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f9e00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f784c0ce940 tx=0x7f784c09c440 comp rx=0 tx=0).stop 2024-04-20T21:43:17.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.339+0000 7f7863861740 1 -- 172.21.15.22:0/1379778264 shutdown_connections 2024-04-20T21:43:17.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.339+0000 7f7863861740 1 --2- 172.21.15.22:0/1379778264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5ea740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.339+0000 7f7863861740 1 --2- 172.21.15.22:0/1379778264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa606350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.339+0000 7f7863861740 1 --2- 172.21.15.22:0/1379778264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f11a0 0x55f3aa5f9e00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.339+0000 7f7863861740 1 -- 172.21.15.22:0/1379778264 >> 172.21.15.22:0/1379778264 conn(0x55f3aa5eb8d0 msgr2=0x55f3aa5ea030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.339+0000 7f7863861740 1 -- 172.21.15.22:0/1379778264 shutdown_connections 2024-04-20T21:43:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.339+0000 7f7863861740 1 -- 172.21.15.22:0/1379778264 wait complete. 2024-04-20T21:43:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.339+0000 7f7863861740 1 -- start start 2024-04-20T21:43:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f7863861740 1 Processor -- start 2024-04-20T21:43:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f7863861740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 0x55f3aa602de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f7863861740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f7863861740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5fe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f7863861740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f785c1b31c0 con 0x55f3aa5f1ac0 2024-04-20T21:43:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f7863861740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f785c1b4630 con 0x55f3aa5fa6a0 2024-04-20T21:43:17.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f7863861740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f785c1b3890 con 0x55f3aa5f11a0 2024-04-20T21:43:17.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f7863861740 10 client.0 _ll_register_callbacks cb 0x7f784c0d4480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f7859e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 0x55f3aa602de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f7851400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f785a800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5fe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f7859e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 0x55f3aa602de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:45822/0 (socket says 172.21.15.22:45822) 2024-04-20T21:43:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.340+0000 7f7859e00640 1 -- 172.21.15.22:0/4089999498 learned_addr learned my addr 172.21.15.22:0/4089999498 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.341+0000 7f7859e00640 1 -- 172.21.15.22:0/4089999498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 msgr2=0x55f3aa5fe890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.341+0000 7f7859e00640 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5fe890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.341+0000 7f7859e00640 1 -- 172.21.15.22:0/4089999498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f1ac0 msgr2=0x55f3aa5fe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:17.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.341+0000 7f7859e00640 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5fe150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.341+0000 7f7859e00640 1 -- 172.21.15.22:0/4089999498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3aa66ffe0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.341+0000 7f785a800640 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5fe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:17.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.341+0000 7f7851400640 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:17.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.341+0000 7f7859e00640 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 0x55f3aa602de0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f784c09ae40 tx=0x7f784c0994b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8946b69ddde5fc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:17.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.341+0000 7f783f400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:43:17.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.341+0000 7f783f400640 1 -- 172.21.15.22:0/4089999498 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c09c6d0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.341+0000 7f783f400640 1 -- 172.21.15.22:0/4089999498 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f784c0988e0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.341+0000 7f7863861740 1 -- 172.21.15.22:0/4089999498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f784c09d290 con 0x55f3aa5f11a0 2024-04-20T21:43:17.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.342+0000 7f783f400640 1 -- 172.21.15.22:0/4089999498 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f784c0a5800 con 0x55f3aa5f11a0 2024-04-20T21:43:17.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.342+0000 7f7863861740 10 client.26803 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:17.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.342+0000 7f7863861740 1 -- 172.21.15.22:0/4089999498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f3aa6711b0 con 0x55f3aa5f11a0 2024-04-20T21:43:17.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.342+0000 7f783f400640 1 -- 172.21.15.22:0/4089999498 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f784c09d290 con 0x55f3aa5f11a0 2024-04-20T21:43:17.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.342+0000 7f783f400640 1 -- 172.21.15.22:0/4089999498 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f784c0b1200 con 0x55f3aa5f11a0 2024-04-20T21:43:17.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.342+0000 7f7863861740 10 client.26803 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:17.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.342+0000 7f7863861740 10 client.26803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:17.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.342+0000 7f7863861740 20 client.26803 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:17.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.342+0000 7f7863861740 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55f3aa671640 0x55f3aa673a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:17.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.342+0000 7f7863861740 4 client.26803 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:43:17.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.342+0000 7f7863861740 1 -- 172.21.15.22:0/4089999498 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3aa55e9e0 con 0x55f3aa671640 2024-04-20T21:43:17.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.342+0000 7f7851400640 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55f3aa671640 0x55f3aa673a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:17.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.343+0000 7f7851400640 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55f3aa671640 0x55f3aa673a30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c97d6aca6be9dd34 server_cookie=a683967ef80f35b4 in_seq=0 out_seq=0 2024-04-20T21:43:17.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.343+0000 7f783f400640 10 client.26803 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:17.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.343+0000 7f783f400640 1 -- 172.21.15.22:0/4089999498 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f3aa55e9e0 con 0x55f3aa671640 2024-04-20T21:43:17.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.343+0000 7f783f400640 10 client.26803 handle_command_reply: tid=0 2024-04-20T21:43:17.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.343+0000 7f7863861740 1 client.26803 shutdown 2024-04-20T21:43:17.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.343+0000 7f7863861740 1 -- 172.21.15.22:0/4089999498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 msgr2=0x55f3aa602de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.343+0000 7f7863861740 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 0x55f3aa602de0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f784c09ae40 tx=0x7f784c0994b0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.343+0000 7f7863861740 1 -- 172.21.15.22:0/4089999498 shutdown_connections 2024-04-20T21:43:17.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f7863861740 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55f3aa671640 0x55f3aa673a30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f7863861740 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3aa5fa6a0 0x55f3aa5fe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f7863861740 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3aa5f1ac0 0x55f3aa5fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f7863861740 1 --2- 172.21.15.22:0/4089999498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aa5f11a0 0x55f3aa602de0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f7863861740 1 -- 172.21.15.22:0/4089999498 >> 172.21.15.22:0/4089999498 conn(0x55f3aa5eb8d0 msgr2=0x55f3aa5ea8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f783f400640 0 client.26803 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:17.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f7863861740 1 -- 172.21.15.22:0/4089999498 shutdown_connections 2024-04-20T21:43:17.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f7863861740 1 -- 172.21.15.22:0/4089999498 wait complete. 2024-04-20T21:43:17.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f7863861740 20 client.26803 trim_cache size 0 max 16384 2024-04-20T21:43:17.352 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:43:17.352 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:43:17.352 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:43:17.352 INFO:teuthology.orchestra.run.smithi022.stdout: "86294b5a-2e8b-4d3f-bd8f-e2654d1b5ab5": { 2024-04-20T21:43:17.352 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:43:17.352 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "86294b5a-2e8b-4d3f-bd8f-e2654d1b5ab5", 2024-04-20T21:43:17.352 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:43:17.352 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:43:17.353 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:43:17.353 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:43:17.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f7860c00640 1 -- 172.21.15.22:0/1100337964 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f78400a3d20 msgr2=0x7f78400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f7860c00640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f78400a3d20 0x7f78400a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f785402c6c0 tx=0x7f7854005d80 comp rx=0 tx=0).stop 2024-04-20T21:43:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f7860c00640 1 -- 172.21.15.22:0/1100337964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f785c074300 msgr2=0x7f785c19eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.344+0000 7f7860c00640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f785c074300 0x7f785c19eeb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f784c00e9f0 tx=0x7f784c00edf0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.345+0000 7f7860c00640 1 -- 172.21.15.22:0/1100337964 shutdown_connections 2024-04-20T21:43:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.345+0000 7f7860c00640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f78400a3d20 0x7f78400a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.345+0000 7f7860c00640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f785c112f30 0x7f785c1a3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.345+0000 7f7860c00640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c074c20 0x7f785c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.345+0000 7f7860c00640 1 --2- 172.21.15.22:0/1100337964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f785c074300 0x7f785c19eeb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:17.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.345+0000 7f7860c00640 1 -- 172.21.15.22:0/1100337964 >> 172.21.15.22:0/1100337964 conn(0x7f785c0fd9d0 msgr2=0x7f785c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:17.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.345+0000 7f7860c00640 1 -- 172.21.15.22:0/1100337964 shutdown_connections 2024-04-20T21:43:17.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:17.345+0000 7f7860c00640 1 -- 172.21.15.22:0/1100337964 wait complete. 2024-04-20T21:43:17.362 INFO:tasks.cephfs.filesystem:scrub status for tag:86294b5a-2e8b-4d3f-bd8f-e2654d1b5ab5 - {'path': '/', 'tag': '86294b5a-2e8b-4d3f-bd8f-e2654d1b5ab5', 'options': 'recursive,force'} 2024-04-20T21:43:17.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:17 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:43:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:17 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:17 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:43:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:17 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:17 smithi122 ceph-mon[22147]: pgmap v295: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:43:17.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:17 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:43:17.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:17 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:17.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:17 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:43:17.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:17 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:17.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:17 smithi033 ceph-mon[21626]: pgmap v295: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:43:17.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:17 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:43:17.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:17 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:17.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:17 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:43:17.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:17 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:17.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:17 smithi022 ceph-mon[16843]: pgmap v295: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:43:20.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:20 smithi022 ceph-mon[16843]: pgmap v296: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:43:20.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:20 smithi122 ceph-mon[22147]: pgmap v296: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:43:20.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:20 smithi033 ceph-mon[21626]: pgmap v296: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:43:22.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:22 smithi122 ceph-mon[22147]: pgmap v297: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:43:22.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:22 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:43:22.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:22 smithi033 ceph-mon[21626]: pgmap v297: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:43:22.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:22 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:43:22.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:22 smithi022 ceph-mon[16843]: pgmap v297: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:43:22.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:22 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:43:24.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:24 smithi122 ceph-mon[22147]: pgmap v298: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:43:24.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:24 smithi033 ceph-mon[21626]: pgmap v298: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:43:24.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:24 smithi022 ceph-mon[16843]: pgmap v298: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:43:26.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:26 smithi122 ceph-mon[22147]: pgmap v299: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 94 B/s wr, 3 op/s 2024-04-20T21:43:26.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:26 smithi033 ceph-mon[21626]: pgmap v299: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 94 B/s wr, 3 op/s 2024-04-20T21:43:26.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:26 smithi022 ceph-mon[16843]: pgmap v299: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 94 B/s wr, 3 op/s 2024-04-20T21:43:28.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:28 smithi122 ceph-mon[22147]: pgmap v300: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:43:28.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:28 smithi033 ceph-mon[21626]: pgmap v300: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:43:28.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:28 smithi022 ceph-mon[16843]: pgmap v300: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:43:30.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:30 smithi122 ceph-mon[22147]: pgmap v301: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:43:30.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:30 smithi033 ceph-mon[21626]: pgmap v301: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:43:30.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:30 smithi022 ceph-mon[16843]: pgmap v301: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:43:32.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:32 smithi122 ceph-mon[22147]: pgmap v302: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:43:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:32 smithi033 ceph-mon[21626]: pgmap v302: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:43:32.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:32 smithi022 ceph-mon[16843]: pgmap v302: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:43:34.459 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:34 smithi022 ceph-mon[16843]: pgmap v303: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:34.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:34 smithi122 ceph-mon[22147]: pgmap v303: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:34 smithi033 ceph-mon[21626]: pgmap v303: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:36.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:36 smithi122 ceph-mon[22147]: pgmap v304: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:43:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:36 smithi033 ceph-mon[21626]: pgmap v304: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:43:36.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:36 smithi022 ceph-mon[16843]: pgmap v304: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:43:38.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:38 smithi122 ceph-mon[22147]: pgmap v305: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:38 smithi033 ceph-mon[21626]: pgmap v305: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:38.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:38 smithi022 ceph-mon[16843]: pgmap v305: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:40.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:40 smithi122 ceph-mon[22147]: pgmap v306: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:40 smithi033 ceph-mon[21626]: pgmap v306: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:40.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:40 smithi022 ceph-mon[16843]: pgmap v306: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:42.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:42 smithi122 ceph-mon[22147]: pgmap v307: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:43:42.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:42 smithi033 ceph-mon[21626]: pgmap v307: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:43:42.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:42 smithi022 ceph-mon[16843]: pgmap v307: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:43:44.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:44 smithi122 ceph-mon[22147]: pgmap v308: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:44.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:44 smithi033 ceph-mon[21626]: pgmap v308: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:44.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:44 smithi022 ceph-mon[16843]: pgmap v308: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:46.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:46 smithi122 ceph-mon[22147]: pgmap v309: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:43:46.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:46 smithi033 ceph-mon[21626]: pgmap v309: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:43:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:46 smithi022 ceph-mon[16843]: pgmap v309: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:43:47.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:43:47.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:43:47.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.588+0000 7efecd400640 1 -- 172.21.15.22:0/3296751041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec8104c30 msgr2=0x7efec8108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.588+0000 7efecd400640 1 --2- 172.21.15.22:0/3296751041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec8104c30 0x7efec8108c70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009ca0 tx=0x7efeb002c100 comp rx=0 tx=0).stop 2024-04-20T21:43:47.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.588+0000 7efecd400640 1 -- 172.21.15.22:0/3296751041 shutdown_connections 2024-04-20T21:43:47.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.588+0000 7efecd400640 1 --2- 172.21.15.22:0/3296751041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec8109350 0x7efec8110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.588+0000 7efecd400640 1 --2- 172.21.15.22:0/3296751041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec8104c30 0x7efec8108c70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.588+0000 7efecd400640 1 --2- 172.21.15.22:0/3296751041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efec8104280 0x7efec8104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.588+0000 7efecd400640 1 -- 172.21.15.22:0/3296751041 >> 172.21.15.22:0/3296751041 conn(0x7efec80ffaa0 msgr2=0x7efec8101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:47.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.588+0000 7efecd400640 1 -- 172.21.15.22:0/3296751041 shutdown_connections 2024-04-20T21:43:47.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.588+0000 7efecd400640 1 -- 172.21.15.22:0/3296751041 wait complete. 2024-04-20T21:43:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.589+0000 7efecd400640 1 Processor -- start 2024-04-20T21:43:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.589+0000 7efecd400640 1 -- start start 2024-04-20T21:43:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.589+0000 7efecd400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efec8104280 0x7efec81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.589+0000 7efecd400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec8109350 0x7efec81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.589+0000 7efecd400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec819a800 0x7efec819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.589+0000 7efecd400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efec81143c0 con 0x7efec8109350 2024-04-20T21:43:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.589+0000 7efecd400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efec8114240 con 0x7efec819a800 2024-04-20T21:43:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.589+0000 7efecd400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efec8114540 con 0x7efec8104280 2024-04-20T21:43:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efecca00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec819a800 0x7efec819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efec7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efec8104280 0x7efec81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efec7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec8109350 0x7efec81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efecca00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec819a800 0x7efec819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34128/0 (socket says 172.21.15.22:34128) 2024-04-20T21:43:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efecca00640 1 -- 172.21.15.22:0/2716137814 learned_addr learned my addr 172.21.15.22:0/2716137814 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efec7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efec8104280 0x7efec81a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50630/0 (socket says 172.21.15.22:50630) 2024-04-20T21:43:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efecca00640 1 -- 172.21.15.22:0/2716137814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efec8104280 msgr2=0x7efec81a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efecca00640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efec8104280 0x7efec81a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efecca00640 1 -- 172.21.15.22:0/2716137814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec8109350 msgr2=0x7efec81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efecca00640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec8109350 0x7efec81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efecca00640 1 -- 172.21.15.22:0/2716137814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec819b390 con 0x7efec819a800 2024-04-20T21:43:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efec7400640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec8109350 0x7efec81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efec7e00640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efec8104280 0x7efec81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.590+0000 7efecca00640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec819a800 0x7efec819acb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efeb800d950 tx=0x7efeb800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.591+0000 7efec4c00640 1 -- 172.21.15.22:0/2716137814 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8017030 con 0x7efec819a800 2024-04-20T21:43:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.591+0000 7efec4c00640 1 -- 172.21.15.22:0/2716137814 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb8004020 con 0x7efec819a800 2024-04-20T21:43:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.591+0000 7efec4c00640 1 -- 172.21.15.22:0/2716137814 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8003c00 con 0x7efec819a800 2024-04-20T21:43:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.591+0000 7efecd400640 1 -- 172.21.15.22:0/2716137814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec819b680 con 0x7efec819a800 2024-04-20T21:43:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.591+0000 7efecd400640 1 -- 172.21.15.22:0/2716137814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec81a7790 con 0x7efec819a800 2024-04-20T21:43:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.594+0000 7efec4c00640 1 -- 172.21.15.22:0/2716137814 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7efeb801a070 con 0x7efec819a800 2024-04-20T21:43:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.594+0000 7efed17a2740 1 Processor -- start 2024-04-20T21:43:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.594+0000 7efed17a2740 1 -- start start 2024-04-20T21:43:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.595+0000 7efed17a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 0x55e465901760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.595+0000 7efed17a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e465901ca0 0x55e46590a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.595+0000 7efed17a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e46590a880 0x55e46590cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.595+0000 7efed17a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e465867150 con 0x55e465901ca0 2024-04-20T21:43:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.595+0000 7efed17a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e46580d370 con 0x55e46590a880 2024-04-20T21:43:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.595+0000 7efed17a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e465863110 con 0x55e465901380 2024-04-20T21:43:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.595+0000 7efec7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e46590a880 0x55e46590cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.595+0000 7efec7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e465901ca0 0x55e46590a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.595+0000 7efec7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e46590a880 0x55e46590cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34138/0 (socket says 172.21.15.22:34138) 2024-04-20T21:43:47.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.595+0000 7efec7400640 1 -- 172.21.15.22:0/1330629519 learned_addr learned my addr 172.21.15.22:0/1330629519 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:47.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.595+0000 7efec7400640 1 --2- 172.21.15.22:0/1330629519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 0x55e465901760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.596+0000 7efec7e00640 1 -- 172.21.15.22:0/1330629519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 msgr2=0x55e465901760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:47.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.596+0000 7efec7e00640 1 --2- 172.21.15.22:0/1330629519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 0x55e465901760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.596+0000 7efec7e00640 1 -- 172.21.15.22:0/1330629519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e46590a880 msgr2=0x55e46590cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:47.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.596+0000 7efec7e00640 1 --2- 172.21.15.22:0/1330629519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e46590a880 0x55e46590cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.596+0000 7efec7e00640 1 -- 172.21.15.22:0/1330629519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4656437d0 con 0x55e465901ca0 2024-04-20T21:43:47.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.596+0000 7efec4c00640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efeac0a3d20 0x7efeac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.596+0000 7efec7e00640 1 --2- 172.21.15.22:0/1330629519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e465901ca0 0x55e46590a340 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efec00103d0 tx=0x7efec0010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc214b2deb1e68f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:47.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.596+0000 7efec4c00640 1 -- 172.21.15.22:0/2716137814 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7efeb80d0070 con 0x7efec819a800 2024-04-20T21:43:47.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.596+0000 7efec7400640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efeac0a3d20 0x7efeac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.597+0000 7efebd200640 1 -- 172.21.15.22:0/1330629519 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0004220 con 0x55e465901ca0 2024-04-20T21:43:47.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.597+0000 7efebd200640 1 -- 172.21.15.22:0/1330629519 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec00043c0 con 0x55e465901ca0 2024-04-20T21:43:47.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.597+0000 7efebd200640 1 -- 172.21.15.22:0/1330629519 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0011a80 con 0x55e465901ca0 2024-04-20T21:43:47.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.597+0000 7efed17a2740 1 -- 172.21.15.22:0/1330629519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e465901ca0 msgr2=0x55e46590a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.597+0000 7efec7400640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efeac0a3d20 0x7efeac0a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55e4658fcc40 tx=0x7efeb002e040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:47.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.597+0000 7efed17a2740 1 --2- 172.21.15.22:0/1330629519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e465901ca0 0x55e46590a340 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efec00103d0 tx=0x7efec0010890 comp rx=0 tx=0).stop 2024-04-20T21:43:47.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.598+0000 7efed17a2740 1 -- 172.21.15.22:0/1330629519 shutdown_connections 2024-04-20T21:43:47.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.598+0000 7efed17a2740 1 --2- 172.21.15.22:0/1330629519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e46590a880 0x55e46590cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.598+0000 7efed17a2740 1 --2- 172.21.15.22:0/1330629519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e465901ca0 0x55e46590a340 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.598+0000 7efed17a2740 1 --2- 172.21.15.22:0/1330629519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 0x55e465901760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.598+0000 7efed17a2740 1 -- 172.21.15.22:0/1330629519 >> 172.21.15.22:0/1330629519 conn(0x55e4658fbab0 msgr2=0x55e4658fbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:47.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.598+0000 7efed17a2740 1 -- 172.21.15.22:0/1330629519 shutdown_connections 2024-04-20T21:43:47.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.598+0000 7efed17a2740 1 -- 172.21.15.22:0/1330629519 wait complete. 2024-04-20T21:43:47.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.598+0000 7efed17a2740 1 -- start start 2024-04-20T21:43:47.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.598+0000 7efed17a2740 1 Processor -- start 2024-04-20T21:43:47.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.599+0000 7efed17a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901380 0x55e4659078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.599+0000 7efed17a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901ca0 0x55e465907de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.599+0000 7efed17a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e465908320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.599+0000 7efed17a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e46590d670 con 0x55e46590a880 2024-04-20T21:43:47.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.599+0000 7efed17a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e465820740 con 0x55e465901380 2024-04-20T21:43:47.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.599+0000 7efed17a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e46590d7f0 con 0x55e465901ca0 2024-04-20T21:43:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.599+0000 7efed17a2740 10 client.0 _ll_register_callbacks cb 0x55e46581bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.599+0000 7efecca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e465908320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.599+0000 7efecca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e465908320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57046/0 (socket says 172.21.15.22:57046) 2024-04-20T21:43:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.599+0000 7efecca00640 1 -- 172.21.15.22:0/294161950 learned_addr learned my addr 172.21.15.22:0/294161950 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.599+0000 7efec7400640 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901ca0 0x55e465907de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.599+0000 7efec7e00640 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901380 0x55e4659078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efecca00640 1 -- 172.21.15.22:0/294161950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901ca0 msgr2=0x55e465907de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efecca00640 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901ca0 0x55e465907de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efecca00640 1 -- 172.21.15.22:0/294161950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901380 msgr2=0x55e4659078a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efecca00640 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901380 0x55e4659078a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efecca00640 1 -- 172.21.15.22:0/294161950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4655793f0 con 0x55e46590a880 2024-04-20T21:43:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efecca00640 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e465908320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efeb8098930 tx=0x7efeb809c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2fcf0c30c6a653 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efeb5600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:43:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efeb5600640 1 -- 172.21.15.22:0/294161950 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb80cde60 con 0x55e46590a880 2024-04-20T21:43:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efeb5600640 1 -- 172.21.15.22:0/294161950 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7efeb80985d0 con 0x55e46590a880 2024-04-20T21:43:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efed17a2740 1 -- 172.21.15.22:0/294161950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec00043c0 con 0x55e46590a880 2024-04-20T21:43:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efeb5600640 1 -- 172.21.15.22:0/294161950 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7efeb80a4bd0 con 0x55e46590a880 2024-04-20T21:43:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efed17a2740 10 client.16641 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.600+0000 7efed17a2740 1 -- 172.21.15.22:0/294161950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e4655390f0 con 0x55e46590a880 2024-04-20T21:43:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.601+0000 7efeb5600640 1 -- 172.21.15.22:0/294161950 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec00043c0 con 0x55e46590a880 2024-04-20T21:43:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.601+0000 7efeb5600640 1 -- 172.21.15.22:0/294161950 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7efeb809b3b0 con 0x55e46590a880 2024-04-20T21:43:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.601+0000 7efed17a2740 10 client.16641 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.601+0000 7efed17a2740 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.601+0000 7efed17a2740 20 client.16641 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.601+0000 7efed17a2740 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e46591abe0 0x55e46591d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.601+0000 7efed17a2740 4 client.16641 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.601+0000 7efed17a2740 1 -- 172.21.15.22:0/294161950 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e465874b70 con 0x55e46591abe0 2024-04-20T21:43:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.601+0000 7efec7e00640 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e46591abe0 0x55e46591d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.602+0000 7efec7e00640 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e46591abe0 0x55e46591d040 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6e953c3c5535210 server_cookie=c4213ae432c10b93 in_seq=0 out_seq=0 2024-04-20T21:43:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.602+0000 7efeb5600640 10 client.16641 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efeb5600640 1 -- 172.21.15.22:0/294161950 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e465874b70 con 0x55e46591abe0 2024-04-20T21:43:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efeb5600640 10 client.16641 handle_command_reply: tid=0 2024-04-20T21:43:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efed17a2740 1 client.16641 shutdown 2024-04-20T21:43:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efed17a2740 1 -- 172.21.15.22:0/294161950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 msgr2=0x55e465908320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efed17a2740 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e465908320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efeb8098930 tx=0x7efeb809c660 comp rx=0 tx=0).stop 2024-04-20T21:43:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efed17a2740 1 -- 172.21.15.22:0/294161950 shutdown_connections 2024-04-20T21:43:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efed17a2740 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e465908320 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efed17a2740 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901ca0 0x55e465907de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efed17a2740 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e46591abe0 0x55e46591d040 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efed17a2740 1 --2- 172.21.15.22:0/294161950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901380 0x55e4659078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efed17a2740 1 -- 172.21.15.22:0/294161950 >> 172.21.15.22:0/294161950 conn(0x55e4658fbab0 msgr2=0x55e4658fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:47.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efeb5600640 0 client.16641 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:47.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efed17a2740 1 -- 172.21.15.22:0/294161950 shutdown_connections 2024-04-20T21:43:47.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.603+0000 7efed17a2740 1 -- 172.21.15.22:0/294161950 wait complete. 2024-04-20T21:43:47.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.604+0000 7efed17a2740 20 client.16641 trim_cache size 0 max 16384 2024-04-20T21:43:47.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.623+0000 7efed17a2740 1 Processor -- start 2024-04-20T21:43:47.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.623+0000 7efed17a2740 1 -- start start 2024-04-20T21:43:47.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.623+0000 7efed17a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 0x55e465909fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.623+0000 7efed17a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901ca0 0x55e4659067e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.623+0000 7efed17a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e465916530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.623+0000 7efed17a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efec81ade70 con 0x55e46590a880 2024-04-20T21:43:47.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.623+0000 7efed17a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efec81b06d0 con 0x55e465901ca0 2024-04-20T21:43:47.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.623+0000 7efed17a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efec81ae4c0 con 0x55e465901380 2024-04-20T21:43:47.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.623+0000 7efecca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e465916530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.623+0000 7efecca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e465916530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57050/0 (socket says 172.21.15.22:57050) 2024-04-20T21:43:47.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.623+0000 7efecca00640 1 -- 172.21.15.22:0/2687927833 learned_addr learned my addr 172.21.15.22:0/2687927833 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:47.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.623+0000 7efec7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901ca0 0x55e4659067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.624+0000 7efec7e00640 1 --2- 172.21.15.22:0/2687927833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 0x55e465909fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.624+0000 7efecca00640 1 -- 172.21.15.22:0/2687927833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 msgr2=0x55e465909fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:47.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.624+0000 7efecca00640 1 --2- 172.21.15.22:0/2687927833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 0x55e465909fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.624+0000 7efecca00640 1 -- 172.21.15.22:0/2687927833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901ca0 msgr2=0x55e4659067e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:47.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.624+0000 7efecca00640 1 --2- 172.21.15.22:0/2687927833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901ca0 0x55e4659067e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.624+0000 7efecca00640 1 -- 172.21.15.22:0/2687927833 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e46590cc70 con 0x55e46590a880 2024-04-20T21:43:47.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.624+0000 7efec7400640 1 --2- 172.21.15.22:0/2687927833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901ca0 0x55e4659067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:47.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.624+0000 7efecca00640 1 --2- 172.21.15.22:0/2687927833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e465916530 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efeb8005fd0 tx=0x7efeb809a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a29cd30544b47d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:47.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.624+0000 7efeb7e00640 1 -- 172.21.15.22:0/2687927833 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb80cde60 con 0x55e46590a880 2024-04-20T21:43:47.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.624+0000 7efeb7e00640 1 -- 172.21.15.22:0/2687927833 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb80cd990 con 0x55e46590a880 2024-04-20T21:43:47.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.624+0000 7efeb7e00640 1 -- 172.21.15.22:0/2687927833 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb80041d0 con 0x55e46590a880 2024-04-20T21:43:47.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.625+0000 7efed17a2740 1 -- 172.21.15.22:0/2687927833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 msgr2=0x55e465916530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.625+0000 7efed17a2740 1 --2- 172.21.15.22:0/2687927833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e465916530 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efeb8005fd0 tx=0x7efeb809a470 comp rx=0 tx=0).stop 2024-04-20T21:43:47.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.625+0000 7efed17a2740 1 -- 172.21.15.22:0/2687927833 shutdown_connections 2024-04-20T21:43:47.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.625+0000 7efed17a2740 1 --2- 172.21.15.22:0/2687927833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e465916530 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.625+0000 7efed17a2740 1 --2- 172.21.15.22:0/2687927833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901ca0 0x55e4659067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.625+0000 7efed17a2740 1 --2- 172.21.15.22:0/2687927833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 0x55e465909fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.625+0000 7efed17a2740 1 -- 172.21.15.22:0/2687927833 >> 172.21.15.22:0/2687927833 conn(0x55e4658fbab0 msgr2=0x55e4658fab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:47.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.625+0000 7efed17a2740 1 -- 172.21.15.22:0/2687927833 shutdown_connections 2024-04-20T21:43:47.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.625+0000 7efed17a2740 1 -- 172.21.15.22:0/2687927833 wait complete. 2024-04-20T21:43:47.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.626+0000 7efed17a2740 1 -- start start 2024-04-20T21:43:47.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.626+0000 7efed17a2740 1 Processor -- start 2024-04-20T21:43:47.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.626+0000 7efed17a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 0x55e465912a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.626+0000 7efed17a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901ca0 0x55e465912f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.626+0000 7efed17a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e46590fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.626+0000 7efed17a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efec81aeb90 con 0x55e46590a880 2024-04-20T21:43:47.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.626+0000 7efed17a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efec81b0000 con 0x55e465901ca0 2024-04-20T21:43:47.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.626+0000 7efed17a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efec81af260 con 0x55e465901380 2024-04-20T21:43:47.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.626+0000 7efed17a2740 10 client.0 _ll_register_callbacks cb 0x7efeb8005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:47.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.627+0000 7efecca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e46590fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.627+0000 7efecca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e46590fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57056/0 (socket says 172.21.15.22:57056) 2024-04-20T21:43:47.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.627+0000 7efecca00640 1 -- 172.21.15.22:0/2151188515 learned_addr learned my addr 172.21.15.22:0/2151188515 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:47.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.627+0000 7efec7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 0x55e465912a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.627+0000 7efec7400640 1 --2- 172.21.15.22:0/2151188515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901ca0 0x55e465912f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.627+0000 7efecca00640 1 -- 172.21.15.22:0/2151188515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 msgr2=0x55e465912a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:47.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.627+0000 7efecca00640 1 --2- 172.21.15.22:0/2151188515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 0x55e465912a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.627+0000 7efecca00640 1 -- 172.21.15.22:0/2151188515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901ca0 msgr2=0x55e465912f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:47.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.627+0000 7efecca00640 1 --2- 172.21.15.22:0/2151188515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901ca0 0x55e465912f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.627+0000 7efecca00640 1 -- 172.21.15.22:0/2151188515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e465910330 con 0x55e46590a880 2024-04-20T21:43:47.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.627+0000 7efeb5600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:43:47.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.627+0000 7efecca00640 1 --2- 172.21.15.22:0/2151188515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e46590fb00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efeb80a4d90 tx=0x7efeb8004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b5cae3596183bb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:47.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efeb5600640 1 -- 172.21.15.22:0/2151188515 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb80cde60 con 0x55e46590a880 2024-04-20T21:43:47.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efeb5600640 1 -- 172.21.15.22:0/2151188515 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7efeb80041d0 con 0x55e46590a880 2024-04-20T21:43:47.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efed17a2740 1 -- 172.21.15.22:0/2151188515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb80cd990 con 0x55e46590a880 2024-04-20T21:43:47.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efeb5600640 1 -- 172.21.15.22:0/2151188515 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7efeb80b54d0 con 0x55e46590a880 2024-04-20T21:43:47.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efed17a2740 10 client.16647 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:47.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efed17a2740 1 -- 172.21.15.22:0/2151188515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e465981dd0 con 0x55e46590a880 2024-04-20T21:43:47.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efeb5600640 1 -- 172.21.15.22:0/2151188515 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb80cd990 con 0x55e46590a880 2024-04-20T21:43:47.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efeb5600640 1 -- 172.21.15.22:0/2151188515 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7efeb8009b30 con 0x55e46590a880 2024-04-20T21:43:47.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efed17a2740 10 client.16647 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:47.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efed17a2740 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:47.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efed17a2740 20 client.16647 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:47.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efed17a2740 1 --2- 172.21.15.22:0/2151188515 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e465982420 0x55e465984810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efed17a2740 4 client.16647 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:43:47.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.628+0000 7efed17a2740 1 -- 172.21.15.22:0/2151188515 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e46586e9e0 con 0x55e465982420 2024-04-20T21:43:47.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.629+0000 7efec7e00640 1 --2- 172.21.15.22:0/2151188515 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e465982420 0x55e465984810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.629+0000 7efec7e00640 1 --2- 172.21.15.22:0/2151188515 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e465982420 0x55e465984810 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6c4fb7181a0058 server_cookie=6480d808fbccbcf0 in_seq=0 out_seq=0 2024-04-20T21:43:47.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.629+0000 7efeb5600640 10 client.16647 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:47.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.629+0000 7efeb5600640 1 -- 172.21.15.22:0/2151188515 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e46586e9e0 con 0x55e465982420 2024-04-20T21:43:47.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.629+0000 7efeb5600640 10 client.16647 handle_command_reply: tid=0 2024-04-20T21:43:47.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.629+0000 7efed17a2740 1 client.16647 shutdown 2024-04-20T21:43:47.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.630+0000 7efed17a2740 1 -- 172.21.15.22:0/2151188515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 msgr2=0x55e46590fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.630+0000 7efed17a2740 1 --2- 172.21.15.22:0/2151188515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e46590fb00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efeb80a4d90 tx=0x7efeb8004540 comp rx=0 tx=0).stop 2024-04-20T21:43:47.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.630+0000 7efed17a2740 1 -- 172.21.15.22:0/2151188515 shutdown_connections 2024-04-20T21:43:47.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.630+0000 7efed17a2740 1 --2- 172.21.15.22:0/2151188515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e46590a880 0x55e46590fb00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.630+0000 7efed17a2740 1 --2- 172.21.15.22:0/2151188515 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e465982420 0x55e465984810 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.630+0000 7efed17a2740 1 --2- 172.21.15.22:0/2151188515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e465901ca0 0x55e465912f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.630+0000 7efed17a2740 1 --2- 172.21.15.22:0/2151188515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e465901380 0x55e465912a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.630+0000 7efed17a2740 1 -- 172.21.15.22:0/2151188515 >> 172.21.15.22:0/2151188515 conn(0x55e4658fbab0 msgr2=0x55e4658fa280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:47.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.630+0000 7efeb5600640 0 client.16647 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:47.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.630+0000 7efed17a2740 1 -- 172.21.15.22:0/2151188515 shutdown_connections 2024-04-20T21:43:47.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.630+0000 7efed17a2740 1 -- 172.21.15.22:0/2151188515 wait complete. 2024-04-20T21:43:47.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.630+0000 7efed17a2740 20 client.16647 trim_cache size 0 max 16384 2024-04-20T21:43:47.644 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:43:47.645 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:43:47.645 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:43:47.645 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:43:47.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.631+0000 7efecd400640 1 -- 172.21.15.22:0/2716137814 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efeac0a3d20 msgr2=0x7efeac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.631+0000 7efecd400640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efeac0a3d20 0x7efeac0a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55e4658fcc40 tx=0x7efeb002e040 comp rx=0 tx=0).stop 2024-04-20T21:43:47.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.631+0000 7efecd400640 1 -- 172.21.15.22:0/2716137814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec819a800 msgr2=0x7efec819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.631+0000 7efecd400640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec819a800 0x7efec819acb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efeb800d950 tx=0x7efeb800de10 comp rx=0 tx=0).stop 2024-04-20T21:43:47.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.631+0000 7efecd400640 1 -- 172.21.15.22:0/2716137814 shutdown_connections 2024-04-20T21:43:47.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.631+0000 7efecd400640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7efeac0a3d20 0x7efeac0a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.631+0000 7efecd400640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec819a800 0x7efec819acb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.631+0000 7efecd400640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efec8109350 0x7efec81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.631+0000 7efecd400640 1 --2- 172.21.15.22:0/2716137814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efec8104280 0x7efec81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.631+0000 7efecd400640 1 -- 172.21.15.22:0/2716137814 >> 172.21.15.22:0/2716137814 conn(0x7efec80ffaa0 msgr2=0x7efec8100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:47.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.631+0000 7efecd400640 1 -- 172.21.15.22:0/2716137814 shutdown_connections 2024-04-20T21:43:47.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.631+0000 7efecd400640 1 -- 172.21.15.22:0/2716137814 wait complete. 2024-04-20T21:43:47.651 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:43:47.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:47.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.853+0000 7f6471400640 1 -- 172.21.15.22:0/3113437400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c10e470 msgr2=0x7f646c10e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.853+0000 7f6471400640 1 --2- 172.21.15.22:0/3113437400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c10e470 0x7f646c10e850 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f645c009900 tx=0x7f645c02bcd0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.853+0000 7f6471400640 1 -- 172.21.15.22:0/3113437400 shutdown_connections 2024-04-20T21:43:47.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.853+0000 7f6471400640 1 --2- 172.21.15.22:0/3113437400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f646c108da0 0x7f646c115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.853+0000 7f6471400640 1 --2- 172.21.15.22:0/3113437400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f646c108400 0x7f646c108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.853+0000 7f6471400640 1 --2- 172.21.15.22:0/3113437400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c10e470 0x7f646c10e850 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.853+0000 7f6471400640 1 -- 172.21.15.22:0/3113437400 >> 172.21.15.22:0/3113437400 conn(0x7f646c077ae0 msgr2=0x7f646c079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:47.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.853+0000 7f6471400640 1 -- 172.21.15.22:0/3113437400 shutdown_connections 2024-04-20T21:43:47.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.853+0000 7f6471400640 1 -- 172.21.15.22:0/3113437400 wait complete. 2024-04-20T21:43:47.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.854+0000 7f6471400640 1 Processor -- start 2024-04-20T21:43:47.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.854+0000 7f6471400640 1 -- start start 2024-04-20T21:43:47.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.855+0000 7f6471400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f646c108400 0x7f646c1a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.855+0000 7f6471400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f646c108da0 0x7f646c1a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.855+0000 7f6471400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c10e470 0x7f646c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.855+0000 7f6471400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f646c118730 con 0x7f646c10e470 2024-04-20T21:43:47.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.855+0000 7f6471400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f646c1185b0 con 0x7f646c108da0 2024-04-20T21:43:47.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.855+0000 7f6471400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f646c1188b0 con 0x7f646c108400 2024-04-20T21:43:47.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.855+0000 7f6470a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c10e470 0x7f646c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.855+0000 7f6470a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c10e470 0x7f646c19e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57082/0 (socket says 172.21.15.22:57082) 2024-04-20T21:43:47.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.855+0000 7f6470a00640 1 -- 172.21.15.22:0/1441780567 learned_addr learned my addr 172.21.15.22:0/1441780567 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:47.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.855+0000 7f646be00640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f646c108400 0x7f646c1a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.855+0000 7f6463400640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f646c108da0 0x7f646c1a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.856+0000 7f6470a00640 1 -- 172.21.15.22:0/1441780567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f646c108400 msgr2=0x7f646c1a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.856+0000 7f6470a00640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f646c108400 0x7f646c1a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.856+0000 7f6470a00640 1 -- 172.21.15.22:0/1441780567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f646c108da0 msgr2=0x7f646c1a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.856+0000 7f6470a00640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f646c108da0 0x7f646c1a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.856+0000 7f6470a00640 1 -- 172.21.15.22:0/1441780567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c19f150 con 0x7f646c10e470 2024-04-20T21:43:47.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.856+0000 7f646be00640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f646c108400 0x7f646c1a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:47.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.856+0000 7f6470a00640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c10e470 0x7f646c19e9d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f646400eb10 tx=0x7f646400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:47.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.856+0000 7f6469600640 1 -- 172.21.15.22:0/1441780567 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464004b20 con 0x7f646c10e470 2024-04-20T21:43:47.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.856+0000 7f6469600640 1 -- 172.21.15.22:0/1441780567 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6464004cc0 con 0x7f646c10e470 2024-04-20T21:43:47.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.856+0000 7f6469600640 1 -- 172.21.15.22:0/1441780567 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464010760 con 0x7f646c10e470 2024-04-20T21:43:47.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.857+0000 7f6471400640 1 -- 172.21.15.22:0/1441780567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646c19f440 con 0x7f646c10e470 2024-04-20T21:43:47.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.857+0000 7f6471400640 1 -- 172.21.15.22:0/1441780567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646c1ab6e0 con 0x7f646c10e470 2024-04-20T21:43:47.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.858+0000 7f6471400640 1 -- 172.21.15.22:0/1441780567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f646c0690f0 con 0x7f646c10e470 2024-04-20T21:43:47.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.860+0000 7f6469600640 1 -- 172.21.15.22:0/1441780567 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f6464005020 con 0x7f646c10e470 2024-04-20T21:43:47.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.861+0000 7f6469600640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f64500a3da0 0x7f64500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.862+0000 7f6469600640 1 -- 172.21.15.22:0/1441780567 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f64640d0db0 con 0x7f646c10e470 2024-04-20T21:43:47.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.862+0000 7f646be00640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f64500a3da0 0x7f64500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.863+0000 7f646be00640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f64500a3da0 0x7f64500a6260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f645c02efd0 tx=0x7f645c02cf90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:47.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:47.866+0000 7f6469600640 1 -- 172.21.15.22:0/1441780567 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f64640d5070 con 0x7f646c10e470 2024-04-20T21:43:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.115+0000 7f6471400640 1 -- 172.21.15.22:0/1441780567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f646c02d5d0 con 0x7f646c10e470 2024-04-20T21:43:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.117+0000 7f6469600640 1 -- 172.21.15.22:0/1441780567 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f646409d3a0 con 0x7f646c10e470 2024-04-20T21:43:48.116 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:43:48.116 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:43:48.116 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.119+0000 7f6471400640 1 -- 172.21.15.22:0/1441780567 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f64500a3da0 msgr2=0x7f64500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.119+0000 7f6471400640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f64500a3da0 0x7f64500a6260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f645c02efd0 tx=0x7f645c02cf90 comp rx=0 tx=0).stop 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.119+0000 7f6471400640 1 -- 172.21.15.22:0/1441780567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c10e470 msgr2=0x7f646c19e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.119+0000 7f6471400640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c10e470 0x7f646c19e9d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f646400eb10 tx=0x7f646400c2d0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.119+0000 7f6471400640 1 -- 172.21.15.22:0/1441780567 shutdown_connections 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.119+0000 7f6471400640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f64500a3da0 0x7f64500a6260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.119+0000 7f6471400640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c10e470 0x7f646c19e9d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.119+0000 7f6471400640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f646c108da0 0x7f646c1a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.120+0000 7f6471400640 1 --2- 172.21.15.22:0/1441780567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f646c108400 0x7f646c1a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.120+0000 7f6471400640 1 -- 172.21.15.22:0/1441780567 >> 172.21.15.22:0/1441780567 conn(0x7f646c077ae0 msgr2=0x7f646c079340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:48.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.120+0000 7f6471400640 1 -- 172.21.15.22:0/1441780567 shutdown_connections 2024-04-20T21:43:48.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.120+0000 7f6471400640 1 -- 172.21.15.22:0/1441780567 wait complete. 2024-04-20T21:43:48.139 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.357+0000 7f9f52600640 1 -- 172.21.15.22:0/2609135522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f4c10d3d0 msgr2=0x7f9f4c10d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.357+0000 7f9f52600640 1 --2- 172.21.15.22:0/2609135522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f4c10d3d0 0x7f9f4c10d7b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009900 tx=0x7f9f4002bcd0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.357+0000 7f9f52600640 1 -- 172.21.15.22:0/2609135522 shutdown_connections 2024-04-20T21:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.357+0000 7f9f52600640 1 --2- 172.21.15.22:0/2609135522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c108ea0 0x7f9f4c1152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.357+0000 7f9f52600640 1 --2- 172.21.15.22:0/2609135522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f4c108500 0x7f9f4c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.357+0000 7f9f52600640 1 --2- 172.21.15.22:0/2609135522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f4c10d3d0 0x7f9f4c10d7b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.357+0000 7f9f52600640 1 -- 172.21.15.22:0/2609135522 >> 172.21.15.22:0/2609135522 conn(0x7f9f4c077d80 msgr2=0x7f9f4c106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.357+0000 7f9f52600640 1 -- 172.21.15.22:0/2609135522 shutdown_connections 2024-04-20T21:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.357+0000 7f9f52600640 1 -- 172.21.15.22:0/2609135522 wait complete. 2024-04-20T21:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.359+0000 7f9f52600640 1 Processor -- start 2024-04-20T21:43:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.359+0000 7f9f52600640 1 -- start start 2024-04-20T21:43:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.359+0000 7f9f52600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c108500 0x7f9f4c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.359+0000 7f9f52600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f4c108ea0 0x7f9f4c19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.359+0000 7f9f52600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f4c10d3d0 0x7f9f4c1a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.359+0000 7f9f52600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1187f0 con 0x7f9f4c10d3d0 2024-04-20T21:43:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.359+0000 7f9f52600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c118670 con 0x7f9f4c108ea0 2024-04-20T21:43:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.359+0000 7f9f52600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c118970 con 0x7f9f4c108500 2024-04-20T21:43:48.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.360+0000 7f9f51200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c108500 0x7f9f4c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.360+0000 7f9f51200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c108500 0x7f9f4c19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50678/0 (socket says 172.21.15.22:50678) 2024-04-20T21:43:48.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.360+0000 7f9f51200640 1 -- 172.21.15.22:0/371174722 learned_addr learned my addr 172.21.15.22:0/371174722 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:48.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.360+0000 7f9f4be00640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f4c108ea0 0x7f9f4c19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.360+0000 7f9f51c00640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f4c10d3d0 0x7f9f4c1a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.360+0000 7f9f51200640 1 -- 172.21.15.22:0/371174722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f4c108ea0 msgr2=0x7f9f4c19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.360+0000 7f9f51200640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f4c108ea0 0x7f9f4c19f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.360+0000 7f9f51200640 1 -- 172.21.15.22:0/371174722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f4c10d3d0 msgr2=0x7f9f4c1a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.360+0000 7f9f51200640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f4c10d3d0 0x7f9f4c1a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.360+0000 7f9f51200640 1 -- 172.21.15.22:0/371174722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c1a3e20 con 0x7f9f4c108500 2024-04-20T21:43:48.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.360+0000 7f9f51200640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c108500 0x7f9f4c19ee30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9f40005940 tx=0x7f9f4002cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:48.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.361+0000 7f9f49600640 1 -- 172.21.15.22:0/371174722 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40041070 con 0x7f9f4c108500 2024-04-20T21:43:48.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.361+0000 7f9f52600640 1 -- 172.21.15.22:0/371174722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4c1a40b0 con 0x7f9f4c108500 2024-04-20T21:43:48.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.361+0000 7f9f49600640 1 -- 172.21.15.22:0/371174722 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f40004450 con 0x7f9f4c108500 2024-04-20T21:43:48.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.361+0000 7f9f49600640 1 -- 172.21.15.22:0/371174722 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40003d80 con 0x7f9f4c108500 2024-04-20T21:43:48.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.361+0000 7f9f52600640 1 -- 172.21.15.22:0/371174722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4c1ab9a0 con 0x7f9f4c108500 2024-04-20T21:43:48.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.365+0000 7f9f56975740 1 Processor -- start 2024-04-20T21:43:48.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.366+0000 7f9f56975740 1 -- start start 2024-04-20T21:43:48.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.366+0000 7f9f56975740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 0x561952534550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.366+0000 7f9f56975740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534a90 0x56195253d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.366+0000 7f9f56975740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 0x56195253fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.366+0000 7f9f56975740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561952499050 con 0x561952534170 2024-04-20T21:43:48.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.366+0000 7f9f56975740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56195243f490 con 0x561952534a90 2024-04-20T21:43:48.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.366+0000 7f9f56975740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561952495010 con 0x56195253d670 2024-04-20T21:43:48.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.366+0000 7f9f49600640 1 -- 172.21.15.22:0/371174722 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9f40044070 con 0x7f9f4c108500 2024-04-20T21:43:48.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.366+0000 7f9f51c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 0x561952534550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.366+0000 7f9f51c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 0x561952534550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57098/0 (socket says 172.21.15.22:57098) 2024-04-20T21:43:48.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.366+0000 7f9f51c00640 1 -- 172.21.15.22:0/1210190599 learned_addr learned my addr 172.21.15.22:0/1210190599 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:48.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.367+0000 7f9f4be00640 1 --2- 172.21.15.22:0/1210190599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 0x56195253fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.367+0000 7f9f51200640 1 --2- 172.21.15.22:0/1210190599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534a90 0x56195253d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.367+0000 7f9f51c00640 1 -- 172.21.15.22:0/1210190599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 msgr2=0x56195253fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:48.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.367+0000 7f9f51c00640 1 --2- 172.21.15.22:0/1210190599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 0x56195253fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.367+0000 7f9f51c00640 1 -- 172.21.15.22:0/1210190599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534a90 msgr2=0x56195253d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:48.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.367+0000 7f9f51c00640 1 --2- 172.21.15.22:0/1210190599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534a90 0x56195253d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.367+0000 7f9f51c00640 1 -- 172.21.15.22:0/1210190599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561952447070 con 0x561952534170 2024-04-20T21:43:48.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.367+0000 7f9f51c00640 1 --2- 172.21.15.22:0/1210190599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 0x561952534550 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009f40 tx=0x7f9f440118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95ca5e75db02655 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.367+0000 7f9f3a000640 1 -- 172.21.15.22:0/1210190599 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4401e070 con 0x561952534170 2024-04-20T21:43:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.368+0000 7f9f3a000640 1 -- 172.21.15.22:0/1210190599 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f44003c30 con 0x561952534170 2024-04-20T21:43:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.368+0000 7f9f3a000640 1 -- 172.21.15.22:0/1210190599 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400f990 con 0x561952534170 2024-04-20T21:43:48.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.368+0000 7f9f49600640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9f340a3d20 0x7f9f340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.368+0000 7f9f56975740 1 -- 172.21.15.22:0/1210190599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 msgr2=0x561952534550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.368+0000 7f9f56975740 1 --2- 172.21.15.22:0/1210190599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 0x561952534550 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009f40 tx=0x7f9f440118d0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.368+0000 7f9f49600640 1 -- 172.21.15.22:0/371174722 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f9f400f2150 con 0x7f9f4c108500 2024-04-20T21:43:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.368+0000 7f9f51200640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9f340a3d20 0x7f9f340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.368+0000 7f9f56975740 1 -- 172.21.15.22:0/1210190599 shutdown_connections 2024-04-20T21:43:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.369+0000 7f9f56975740 1 --2- 172.21.15.22:0/1210190599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 0x56195253fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.369+0000 7f9f56975740 1 --2- 172.21.15.22:0/1210190599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534a90 0x56195253d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.369+0000 7f9f56975740 1 --2- 172.21.15.22:0/1210190599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 0x561952534550 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.369+0000 7f9f56975740 1 -- 172.21.15.22:0/1210190599 >> 172.21.15.22:0/1210190599 conn(0x56195252e8a0 msgr2=0x56195252ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:48.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.369+0000 7f9f56975740 1 -- 172.21.15.22:0/1210190599 shutdown_connections 2024-04-20T21:43:48.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.369+0000 7f9f56975740 1 -- 172.21.15.22:0/1210190599 wait complete. 2024-04-20T21:43:48.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.369+0000 7f9f51200640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9f340a3d20 0x7f9f340a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9f400efab0 tx=0x7f9f4002bcd0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:48.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.369+0000 7f9f56975740 1 -- start start 2024-04-20T21:43:48.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.369+0000 7f9f56975740 1 Processor -- start 2024-04-20T21:43:48.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.370+0000 7f9f56975740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 0x56195253a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.370+0000 7f9f56975740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534a90 0x56195253ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.370+0000 7f9f56975740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 0x56195253b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.370+0000 7f9f56975740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561952540460 con 0x561952534170 2024-04-20T21:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.370+0000 7f9f56975740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561952452860 con 0x561952534a90 2024-04-20T21:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.370+0000 7f9f56975740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5619525405e0 con 0x56195253d670 2024-04-20T21:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.370+0000 7f9f56975740 10 client.0 _ll_register_callbacks cb 0x561952420fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.370+0000 7f9f51c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 0x56195253a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.370+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534a90 0x56195253ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.370+0000 7f9f51c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 0x56195253a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57112/0 (socket says 172.21.15.22:57112) 2024-04-20T21:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.370+0000 7f9f51c00640 1 -- 172.21.15.22:0/398482785 learned_addr learned my addr 172.21.15.22:0/398482785 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.371+0000 7f9f51c00640 1 --2- 172.21.15.22:0/398482785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 0x56195253b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.371+0000 7f9f51c00640 1 -- 172.21.15.22:0/398482785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 msgr2=0x56195253b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.371+0000 7f9f51c00640 1 --2- 172.21.15.22:0/398482785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 0x56195253b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.371+0000 7f9f51c00640 1 -- 172.21.15.22:0/398482785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534a90 msgr2=0x56195253ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.371+0000 7f9f51c00640 1 --2- 172.21.15.22:0/398482785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534a90 0x56195253ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.371+0000 7f9f51c00640 1 -- 172.21.15.22:0/398482785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561952456950 con 0x561952534170 2024-04-20T21:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.371+0000 7f9f51c00640 1 --2- 172.21.15.22:0/398482785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 0x56195253a6c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9f44006010 tx=0x7f9f44004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80468115a8719fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.371+0000 7f9f2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.371+0000 7f9f2ea00640 1 -- 172.21.15.22:0/398482785 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4402d060 con 0x561952534170 2024-04-20T21:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.371+0000 7f9f2ea00640 1 -- 172.21.15.22:0/398482785 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f9f44004460 con 0x561952534170 2024-04-20T21:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.372+0000 7f9f56975740 1 -- 172.21.15.22:0/398482785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f44003c30 con 0x561952534170 2024-04-20T21:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.372+0000 7f9f2ea00640 1 -- 172.21.15.22:0/398482785 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9f4401e070 con 0x561952534170 2024-04-20T21:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.372+0000 7f9f56975740 10 client.16665 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.372+0000 7f9f56975740 1 -- 172.21.15.22:0/398482785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561952424820 con 0x561952534170 2024-04-20T21:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.372+0000 7f9f2ea00640 1 -- 172.21.15.22:0/398482785 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f44003c30 con 0x561952534170 2024-04-20T21:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.372+0000 7f9f2ea00640 1 -- 172.21.15.22:0/398482785 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f9f44025070 con 0x561952534170 2024-04-20T21:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.372+0000 7f9f56975740 10 client.16665 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.372+0000 7f9f56975740 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.372+0000 7f9f56975740 20 client.16665 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.373+0000 7f9f56975740 1 --2- 172.21.15.22:0/398482785 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56195254d5a0 0x56195254f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.373+0000 7f9f56975740 4 client.16665 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.373+0000 7f9f56975740 1 -- 172.21.15.22:0/398482785 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619524a6a70 con 0x56195254d5a0 2024-04-20T21:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.373+0000 7f9f4be00640 1 --2- 172.21.15.22:0/398482785 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56195254d5a0 0x56195254f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.373+0000 7f9f4be00640 1 --2- 172.21.15.22:0/398482785 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56195254d5a0 0x56195254f990 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67dcf0d0951a008d server_cookie=ce8835ae66db3448 in_seq=0 out_seq=0 2024-04-20T21:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.373+0000 7f9f2ea00640 10 client.16665 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.374+0000 7f9f2ea00640 1 -- 172.21.15.22:0/398482785 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619524a6a70 con 0x56195254d5a0 2024-04-20T21:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.374+0000 7f9f2ea00640 10 client.16665 handle_command_reply: tid=0 2024-04-20T21:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f56975740 1 client.16665 shutdown 2024-04-20T21:43:48.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f56975740 1 -- 172.21.15.22:0/398482785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 msgr2=0x56195253a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f56975740 1 --2- 172.21.15.22:0/398482785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 0x56195253a6c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9f44006010 tx=0x7f9f44004740 comp rx=0 tx=0).stop 2024-04-20T21:43:48.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f56975740 1 -- 172.21.15.22:0/398482785 shutdown_connections 2024-04-20T21:43:48.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f56975740 1 --2- 172.21.15.22:0/398482785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 0x56195253b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f56975740 1 --2- 172.21.15.22:0/398482785 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x56195254d5a0 0x56195254f990 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f56975740 1 --2- 172.21.15.22:0/398482785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534a90 0x56195253ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f56975740 1 --2- 172.21.15.22:0/398482785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534170 0x56195253a6c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f56975740 1 -- 172.21.15.22:0/398482785 >> 172.21.15.22:0/398482785 conn(0x56195252e8a0 msgr2=0x5619525305f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f2ea00640 0 client.16665 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f56975740 1 -- 172.21.15.22:0/398482785 shutdown_connections 2024-04-20T21:43:48.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f56975740 1 -- 172.21.15.22:0/398482785 wait complete. 2024-04-20T21:43:48.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.375+0000 7f9f56975740 20 client.16665 trim_cache size 0 max 16384 2024-04-20T21:43:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.390+0000 7f9f56975740 1 Processor -- start 2024-04-20T21:43:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.390+0000 7f9f56975740 1 -- start start 2024-04-20T21:43:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f56975740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534170 0x56195253cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f56975740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 0x561952549350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f56975740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 0x561952539820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f56975740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1b2230 con 0x561952534a90 2024-04-20T21:43:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f56975740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1b4a90 con 0x561952534170 2024-04-20T21:43:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f56975740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1b2880 con 0x56195253d670 2024-04-20T21:43:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 0x561952549350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 0x561952549350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57128/0 (socket says 172.21.15.22:57128) 2024-04-20T21:43:48.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f4be00640 1 -- 172.21.15.22:0/1277287407 learned_addr learned my addr 172.21.15.22:0/1277287407 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f51c00640 1 --2- 172.21.15.22:0/1277287407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534170 0x56195253cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f51c00640 1 --2- 172.21.15.22:0/1277287407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 0x561952539820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f4be00640 1 -- 172.21.15.22:0/1277287407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 msgr2=0x561952539820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f4be00640 1 --2- 172.21.15.22:0/1277287407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 0x561952539820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f4be00640 1 -- 172.21.15.22:0/1277287407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534170 msgr2=0x56195253cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f4be00640 1 --2- 172.21.15.22:0/1277287407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534170 0x56195253cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f4be00640 1 -- 172.21.15.22:0/1277287407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619521ab3f0 con 0x561952534a90 2024-04-20T21:43:48.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.391+0000 7f9f51c00640 1 --2- 172.21.15.22:0/1277287407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534170 0x56195253cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.392+0000 7f9f4be00640 1 --2- 172.21.15.22:0/1277287407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 0x561952549350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009dd0 tx=0x7f9f3c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195fb76750855619 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.392+0000 7f9f39600640 1 -- 172.21.15.22:0/1277287407 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c023020 con 0x561952534a90 2024-04-20T21:43:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.392+0000 7f9f39600640 1 -- 172.21.15.22:0/1277287407 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c003e40 con 0x561952534a90 2024-04-20T21:43:48.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.392+0000 7f9f39600640 1 -- 172.21.15.22:0/1277287407 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c01f050 con 0x561952534a90 2024-04-20T21:43:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.393+0000 7f9f56975740 1 -- 172.21.15.22:0/1277287407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 msgr2=0x561952549350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.393+0000 7f9f56975740 1 --2- 172.21.15.22:0/1277287407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 0x561952549350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009dd0 tx=0x7f9f3c005f90 comp rx=0 tx=0).stop 2024-04-20T21:43:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.393+0000 7f9f56975740 1 -- 172.21.15.22:0/1277287407 shutdown_connections 2024-04-20T21:43:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.393+0000 7f9f56975740 1 --2- 172.21.15.22:0/1277287407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195253d670 0x561952539820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.393+0000 7f9f56975740 1 --2- 172.21.15.22:0/1277287407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 0x561952549350 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.393+0000 7f9f56975740 1 --2- 172.21.15.22:0/1277287407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561952534170 0x56195253cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.393+0000 7f9f56975740 1 -- 172.21.15.22:0/1277287407 >> 172.21.15.22:0/1277287407 conn(0x56195252e8a0 msgr2=0x56195253a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:48.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.393+0000 7f9f56975740 1 -- 172.21.15.22:0/1277287407 shutdown_connections 2024-04-20T21:43:48.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.393+0000 7f9f56975740 1 -- 172.21.15.22:0/1277287407 wait complete. 2024-04-20T21:43:48.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.393+0000 7f9f56975740 1 -- start start 2024-04-20T21:43:48.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.393+0000 7f9f56975740 1 Processor -- start 2024-04-20T21:43:48.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.394+0000 7f9f56975740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561952534170 0x561952545860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.394+0000 7f9f56975740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 0x561952545da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.394+0000 7f9f56975740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56195253d670 0x561952536ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.394+0000 7f9f56975740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1b2f50 con 0x561952534a90 2024-04-20T21:43:48.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.394+0000 7f9f56975740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1b43c0 con 0x56195253d670 2024-04-20T21:43:48.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.394+0000 7f9f56975740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1b3620 con 0x561952534170 2024-04-20T21:43:48.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.394+0000 7f9f56975740 10 client.0 _ll_register_callbacks cb 0x7f9f44009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:48.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.394+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 0x561952545da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.394+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 0x561952545da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57140/0 (socket says 172.21.15.22:57140) 2024-04-20T21:43:48.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.394+0000 7f9f4be00640 1 -- 172.21.15.22:0/409488399 learned_addr learned my addr 172.21.15.22:0/409488399 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:48.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.394+0000 7f9f51c00640 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561952534170 0x561952545860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.394+0000 7f9f51c00640 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56195253d670 0x561952536ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.395+0000 7f9f4be00640 1 -- 172.21.15.22:0/409488399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561952534170 msgr2=0x561952545860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.395+0000 7f9f4be00640 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561952534170 0x561952545860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.395+0000 7f9f4be00640 1 -- 172.21.15.22:0/409488399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56195253d670 msgr2=0x561952536ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.395+0000 7f9f4be00640 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56195253d670 0x561952536ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.395+0000 7f9f4be00640 1 -- 172.21.15.22:0/409488399 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619522757d0 con 0x561952534a90 2024-04-20T21:43:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.395+0000 7f9f51c00640 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561952534170 0x561952545860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:43:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.395+0000 7f9f4be00640 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 0x561952545da0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c003cd0 tx=0x7f9f3c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f42537719f77ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.395+0000 7f9f2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:43:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.395+0000 7f9f2ea00640 1 -- 172.21.15.22:0/409488399 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c02d030 con 0x561952534a90 2024-04-20T21:43:48.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.395+0000 7f9f2ea00640 1 -- 172.21.15.22:0/409488399 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f9f3c004910 con 0x561952534a90 2024-04-20T21:43:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.395+0000 7f9f56975740 1 -- 172.21.15.22:0/409488399 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619524a08e0 con 0x561952534a90 2024-04-20T21:43:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.396+0000 7f9f2ea00640 1 -- 172.21.15.22:0/409488399 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9f3c023070 con 0x561952534a90 2024-04-20T21:43:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.396+0000 7f9f56975740 10 client.16671 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.396+0000 7f9f56975740 1 -- 172.21.15.22:0/409488399 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5619525b3170 con 0x561952534a90 2024-04-20T21:43:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.396+0000 7f9f2ea00640 1 -- 172.21.15.22:0/409488399 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5619524a08e0 con 0x561952534a90 2024-04-20T21:43:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.396+0000 7f9f2ea00640 1 -- 172.21.15.22:0/409488399 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f9f3c01f5b0 con 0x561952534a90 2024-04-20T21:43:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.396+0000 7f9f56975740 10 client.16671 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.396+0000 7f9f56975740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.396+0000 7f9f56975740 20 client.16671 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.396+0000 7f9f56975740 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5619525b38c0 0x5619525b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.396+0000 7f9f56975740 4 client.16671 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:43:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.396+0000 7f9f56975740 1 -- 172.21.15.22:0/409488399 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56195241c5a0 con 0x5619525b38c0 2024-04-20T21:43:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.397+0000 7f9f51c00640 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5619525b38c0 0x5619525b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.397+0000 7f9f51c00640 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5619525b38c0 0x5619525b5ce0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=864f046142173c59 server_cookie=24e9250ff95c8fc9 in_seq=0 out_seq=0 2024-04-20T21:43:48.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.397+0000 7f9f2ea00640 10 client.16671 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.397+0000 7f9f2ea00640 1 -- 172.21.15.22:0/409488399 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56195241c5a0 con 0x5619525b38c0 2024-04-20T21:43:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.397+0000 7f9f2ea00640 10 client.16671 handle_command_reply: tid=0 2024-04-20T21:43:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.397+0000 7f9f56975740 1 client.16671 shutdown 2024-04-20T21:43:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.398+0000 7f9f56975740 1 -- 172.21.15.22:0/409488399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 msgr2=0x561952545da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.398+0000 7f9f56975740 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 0x561952545da0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c003cd0 tx=0x7f9f3c003d00 comp rx=0 tx=0).stop 2024-04-20T21:43:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.398+0000 7f9f56975740 1 -- 172.21.15.22:0/409488399 shutdown_connections 2024-04-20T21:43:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.398+0000 7f9f56975740 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x5619525b38c0 0x5619525b5ce0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.398+0000 7f9f56975740 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56195253d670 0x561952536ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.398+0000 7f9f56975740 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561952534a90 0x561952545da0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.398+0000 7f9f56975740 1 --2- 172.21.15.22:0/409488399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561952534170 0x561952545860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.398+0000 7f9f2ea00640 0 client.16671 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:48.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.398+0000 7f9f56975740 1 -- 172.21.15.22:0/409488399 >> 172.21.15.22:0/409488399 conn(0x56195252e8a0 msgr2=0x56195252d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:48.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.398+0000 7f9f56975740 1 -- 172.21.15.22:0/409488399 shutdown_connections 2024-04-20T21:43:48.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.398+0000 7f9f56975740 1 -- 172.21.15.22:0/409488399 wait complete. 2024-04-20T21:43:48.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.398+0000 7f9f56975740 20 client.16671 trim_cache size 0 max 16384 2024-04-20T21:43:48.405 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:43:48.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.399+0000 7f9f52600640 1 -- 172.21.15.22:0/371174722 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9f340a3d20 msgr2=0x7f9f340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.399+0000 7f9f52600640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9f340a3d20 0x7f9f340a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9f400efab0 tx=0x7f9f4002bcd0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.399+0000 7f9f52600640 1 -- 172.21.15.22:0/371174722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c108500 msgr2=0x7f9f4c19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.399+0000 7f9f52600640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c108500 0x7f9f4c19ee30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9f40005940 tx=0x7f9f4002cbd0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.399+0000 7f9f52600640 1 -- 172.21.15.22:0/371174722 shutdown_connections 2024-04-20T21:43:48.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.399+0000 7f9f52600640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f9f340a3d20 0x7f9f340a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.399+0000 7f9f52600640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f4c10d3d0 0x7f9f4c1a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.399+0000 7f9f52600640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f4c108ea0 0x7f9f4c19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.399+0000 7f9f52600640 1 --2- 172.21.15.22:0/371174722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c108500 0x7f9f4c19ee30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.399+0000 7f9f52600640 1 -- 172.21.15.22:0/371174722 >> 172.21.15.22:0/371174722 conn(0x7f9f4c077d80 msgr2=0x7f9f4c078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:48.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.399+0000 7f9f52600640 1 -- 172.21.15.22:0/371174722 shutdown_connections 2024-04-20T21:43:48.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:48.399+0000 7f9f52600640 1 -- 172.21.15.22:0/371174722 wait complete. 2024-04-20T21:43:48.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:48 smithi022 ceph-mon[16843]: pgmap v310: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:48.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:48 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1441780567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:43:48.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:48 smithi122 ceph-mon[22147]: pgmap v310: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:48.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:48 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1441780567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:43:48.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:48 smithi033 ceph-mon[21626]: pgmap v310: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:48.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:48 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1441780567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:43:49.419 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:43:49.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:43:49.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:43:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.648+0000 7f5d2ec00640 1 -- 172.21.15.22:0/1196400417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d28108500 msgr2=0x7f5d28108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.648+0000 7f5d2ec00640 1 --2- 172.21.15.22:0/1196400417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d28108500 0x7f5d28108960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009900 tx=0x7f5d1402bcc0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.648+0000 7f5d2ec00640 1 -- 172.21.15.22:0/1196400417 shutdown_connections 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.648+0000 7f5d2ec00640 1 --2- 172.21.15.22:0/1196400417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d28108ea0 0x7f5d281152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.648+0000 7f5d2ec00640 1 --2- 172.21.15.22:0/1196400417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d28108500 0x7f5d28108960 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.648+0000 7f5d2ec00640 1 --2- 172.21.15.22:0/1196400417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2810d3d0 0x7f5d2810d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.648+0000 7f5d2ec00640 1 -- 172.21.15.22:0/1196400417 >> 172.21.15.22:0/1196400417 conn(0x7f5d28077d80 msgr2=0x7f5d28106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.648+0000 7f5d2ec00640 1 -- 172.21.15.22:0/1196400417 shutdown_connections 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.648+0000 7f5d2ec00640 1 -- 172.21.15.22:0/1196400417 wait complete. 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.649+0000 7f5d2ec00640 1 Processor -- start 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.649+0000 7f5d2ec00640 1 -- start start 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.650+0000 7f5d2ec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d28108500 0x7f5d281a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.650+0000 7f5d2ec00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d28108ea0 0x7f5d281a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.650+0000 7f5d2ec00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2810d3d0 0x7f5d2819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.650+0000 7f5d2ec00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5d28118840 con 0x7f5d28108ea0 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.650+0000 7f5d2ec00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281186c0 con 0x7f5d2810d3d0 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.650+0000 7f5d2ec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281189c0 con 0x7f5d28108500 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.650+0000 7f5d2ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d28108ea0 0x7f5d281a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.650+0000 7f5d2d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d28108500 0x7f5d281a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.650+0000 7f5d2d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d28108500 0x7f5d281a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50752/0 (socket says 172.21.15.22:50752) 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.650+0000 7f5d2ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d28108ea0 0x7f5d281a4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57166/0 (socket says 172.21.15.22:57166) 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.650+0000 7f5d2d800640 1 -- 172.21.15.22:0/11819515 learned_addr learned my addr 172.21.15.22:0/11819515 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.650+0000 7f5d2e200640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2810d3d0 0x7f5d2819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.651+0000 7f5d2d800640 1 -- 172.21.15.22:0/11819515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2810d3d0 msgr2=0x7f5d2819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.651+0000 7f5d2d800640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2810d3d0 0x7f5d2819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.651+0000 7f5d2d800640 1 -- 172.21.15.22:0/11819515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d28108ea0 msgr2=0x7f5d281a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.651+0000 7f5d2d800640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d28108ea0 0x7f5d281a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.651+0000 7f5d2d800640 1 -- 172.21.15.22:0/11819515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2819f280 con 0x7f5d28108500 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.651+0000 7f5d2d800640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d28108500 0x7f5d281a4950 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00e9f0 tx=0x7f5d1c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.651+0000 7f5d26000640 1 -- 172.21.15.22:0/11819515 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00cd80 con 0x7f5d28108500 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.651+0000 7f5d26000640 1 -- 172.21.15.22:0/11819515 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c010910 con 0x7f5d28108500 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.651+0000 7f5d26000640 1 -- 172.21.15.22:0/11819515 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c018990 con 0x7f5d28108500 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.651+0000 7f5d2ec00640 1 -- 172.21.15.22:0/11819515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2819f570 con 0x7f5d28108500 2024-04-20T21:43:49.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.652+0000 7f5d2ec00640 1 -- 172.21.15.22:0/11819515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d281ab810 con 0x7f5d28108500 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.655+0000 7f5d32e72740 1 Processor -- start 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.655+0000 7f5d26000640 1 -- 172.21.15.22:0/11819515 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5d1c010ac0 con 0x7f5d28108500 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.655+0000 7f5d32e72740 1 -- start start 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.655+0000 7f5d32e72740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e78320f0 0x55b5e78324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.655+0000 7f5d32e72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e7832a10 0x55b5e783b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.655+0000 7f5d32e72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e783d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.655+0000 7f5d32e72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b5e7796d20 con 0x55b5e783b5f0 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.655+0000 7f5d32e72740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b5e773d3e0 con 0x55b5e78320f0 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.655+0000 7f5d32e72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b5e7792ce0 con 0x55b5e7832a10 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.656+0000 7f5d2ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e783d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.656+0000 7f5d2e200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e78320f0 0x55b5e78324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.656+0000 7f5d2e200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e78320f0 0x55b5e78324d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34286/0 (socket says 172.21.15.22:34286) 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.656+0000 7f5d2e200640 1 -- 172.21.15.22:0/800986831 learned_addr learned my addr 172.21.15.22:0/800986831 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.656+0000 7f5d2d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e7832a10 0x55b5e783b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.656+0000 7f5d2e200640 1 -- 172.21.15.22:0/800986831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e7832a10 msgr2=0x55b5e783b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.656+0000 7f5d2e200640 1 --2- 172.21.15.22:0/800986831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e7832a10 0x55b5e783b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.656+0000 7f5d2e200640 1 -- 172.21.15.22:0/800986831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 msgr2=0x55b5e783d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.656+0000 7f5d2e200640 1 --2- 172.21.15.22:0/800986831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e783d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.656+0000 7f5d2e200640 1 -- 172.21.15.22:0/800986831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e775c980 con 0x55b5e78320f0 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.656+0000 7f5d2e200640 1 --2- 172.21.15.22:0/800986831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e78320f0 0x55b5e78324d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009fc0 tx=0x7f5d200118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7265c700a642853b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.657+0000 7f5d26000640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5d100a3d20 0x7f5d100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.657+0000 7f5d1aa00640 1 -- 172.21.15.22:0/800986831 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20004d20 con 0x55b5e78320f0 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.657+0000 7f5d1aa00640 1 -- 172.21.15.22:0/800986831 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d20004050 con 0x55b5e78320f0 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.657+0000 7f5d1aa00640 1 -- 172.21.15.22:0/800986831 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20005260 con 0x55b5e78320f0 2024-04-20T21:43:49.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.657+0000 7f5d26000640 1 -- 172.21.15.22:0/11819515 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f5d1c0d0b30 con 0x7f5d28108500 2024-04-20T21:43:49.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.657+0000 7f5d2d800640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5d100a3d20 0x7f5d100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.657+0000 7f5d32e72740 1 -- 172.21.15.22:0/800986831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e78320f0 msgr2=0x55b5e78324d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.657+0000 7f5d32e72740 1 --2- 172.21.15.22:0/800986831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e78320f0 0x55b5e78324d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009fc0 tx=0x7f5d200118d0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.658+0000 7f5d32e72740 1 -- 172.21.15.22:0/800986831 shutdown_connections 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.658+0000 7f5d32e72740 1 --2- 172.21.15.22:0/800986831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e783d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.658+0000 7f5d32e72740 1 --2- 172.21.15.22:0/800986831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e7832a10 0x55b5e783b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.658+0000 7f5d32e72740 1 --2- 172.21.15.22:0/800986831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e78320f0 0x55b5e78324d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.658+0000 7f5d32e72740 1 -- 172.21.15.22:0/800986831 >> 172.21.15.22:0/800986831 conn(0x55b5e782c820 msgr2=0x55b5e782cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.658+0000 7f5d32e72740 1 -- 172.21.15.22:0/800986831 shutdown_connections 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.658+0000 7f5d32e72740 1 -- 172.21.15.22:0/800986831 wait complete. 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.658+0000 7f5d2d800640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5d100a3d20 0x7f5d100a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55b5e7831bc0 tx=0x7f5d1c0d45e0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.658+0000 7f5d32e72740 1 -- start start 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.659+0000 7f5d32e72740 1 Processor -- start 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.659+0000 7f5d32e72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e7838650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.659+0000 7f5d32e72740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e7838b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.659+0000 7f5d32e72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e78390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.659+0000 7f5d32e72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b5e783e3e0 con 0x55b5e783b5f0 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.659+0000 7f5d32e72740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b5e77507a0 con 0x55b5e7832a10 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.659+0000 7f5d32e72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b5e783e560 con 0x55b5e78320f0 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.659+0000 7f5d32e72740 10 client.0 _ll_register_callbacks cb 0x55b5e7752b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.659+0000 7f5d2e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e7838650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.660+0000 7f5d2e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e7838650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50772/0 (socket says 172.21.15.22:50772) 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.660+0000 7f5d2ce00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e7838b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.660+0000 7f5d2e200640 1 -- 172.21.15.22:0/2058995818 learned_addr learned my addr 172.21.15.22:0/2058995818 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.660+0000 7f5d2e200640 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e78390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.660+0000 7f5d2ce00640 1 -- 172.21.15.22:0/2058995818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 msgr2=0x55b5e7838650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.660+0000 7f5d2ce00640 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e7838650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.660+0000 7f5d2ce00640 1 -- 172.21.15.22:0/2058995818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 msgr2=0x55b5e78390d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.660+0000 7f5d2ce00640 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e78390d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.660+0000 7f5d2ce00640 1 -- 172.21.15.22:0/2058995818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e75717e0 con 0x55b5e7832a10 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.660+0000 7f5d2e200640 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e7838650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.660+0000 7f5d2ce00640 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e7838b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5d14036040 tx=0x7f5d14005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0229d12a0a1eab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.660+0000 7f5d0f400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.661+0000 7f5d0f400640 1 -- 172.21.15.22:0/2058995818 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14044070 con 0x55b5e7832a10 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.661+0000 7f5d0f400640 1 -- 172.21.15.22:0/2058995818 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f5d1402cbe0 con 0x55b5e7832a10 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.661+0000 7f5d32e72740 1 -- 172.21.15.22:0/2058995818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d20004050 con 0x55b5e7832a10 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.661+0000 7f5d0f400640 1 -- 172.21.15.22:0/2058995818 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5d1402cde0 con 0x55b5e7832a10 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.661+0000 7f5d32e72740 10 client.26828 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.661+0000 7f5d32e72740 1 -- 172.21.15.22:0/2058995818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b5e75d1f70 con 0x55b5e7832a10 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.661+0000 7f5d0f400640 1 -- 172.21.15.22:0/2058995818 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d20004050 con 0x55b5e7832a10 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.662+0000 7f5d0f400640 1 -- 172.21.15.22:0/2058995818 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f5d140053c0 con 0x55b5e7832a10 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.662+0000 7f5d32e72740 10 client.26828 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.662+0000 7f5d32e72740 10 client.26828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.662+0000 7f5d32e72740 20 client.26828 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.662+0000 7f5d32e72740 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b5e784b9a0 0x55b5e784ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.662+0000 7f5d32e72740 4 client.26828 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.662+0000 7f5d32e72740 1 -- 172.21.15.22:0/2058995818 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5e77a4740 con 0x55b5e784b9a0 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.662+0000 7f5d2e200640 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b5e784b9a0 0x55b5e784ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.663+0000 7f5d2e200640 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b5e784b9a0 0x55b5e784ddd0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=419b4d77ef89d4ef server_cookie=a992aa5b9194cc14 in_seq=0 out_seq=0 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.663+0000 7f5d0f400640 10 client.26828 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d0f400640 1 -- 172.21.15.22:0/2058995818 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b5e77a4740 con 0x55b5e784b9a0 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d0f400640 10 client.26828 handle_command_reply: tid=0 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d32e72740 1 client.26828 shutdown 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d32e72740 1 -- 172.21.15.22:0/2058995818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 msgr2=0x55b5e7838b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d32e72740 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e7838b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5d14036040 tx=0x7f5d14005dd0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d32e72740 1 -- 172.21.15.22:0/2058995818 shutdown_connections 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d32e72740 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e78390d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d32e72740 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b5e784b9a0 0x55b5e784ddd0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d32e72740 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e7838b90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d32e72740 1 --2- 172.21.15.22:0/2058995818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e7838650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d32e72740 1 -- 172.21.15.22:0/2058995818 >> 172.21.15.22:0/2058995818 conn(0x55b5e782c820 msgr2=0x55b5e782e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d0f400640 0 client.26828 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:49.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d32e72740 1 -- 172.21.15.22:0/2058995818 shutdown_connections 2024-04-20T21:43:49.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d32e72740 1 -- 172.21.15.22:0/2058995818 wait complete. 2024-04-20T21:43:49.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.664+0000 7f5d32e72740 20 client.26828 trim_cache size 0 max 16384 2024-04-20T21:43:49.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.684+0000 7f5d32e72740 1 Processor -- start 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.684+0000 7f5d32e72740 1 -- start start 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.685+0000 7f5d32e72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e783ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.685+0000 7f5d32e72740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e78472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.685+0000 7f5d32e72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e78536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.685+0000 7f5d32e72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281b22f0 con 0x55b5e783b5f0 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.685+0000 7f5d32e72740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281b4b50 con 0x55b5e7832a10 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.685+0000 7f5d32e72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281b2940 con 0x55b5e78320f0 2024-04-20T21:43:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.685+0000 7f5d2e200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e78536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.685+0000 7f5d2ce00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e78472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.685+0000 7f5d2e200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e78536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57176/0 (socket says 172.21.15.22:57176) 2024-04-20T21:43:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.685+0000 7f5d2e200640 1 -- 172.21.15.22:0/2510147982 learned_addr learned my addr 172.21.15.22:0/2510147982 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.685+0000 7f5d2ce00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e78472a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34302/0 (socket says 172.21.15.22:34302) 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.685+0000 7f5d2e200640 1 --2- 172.21.15.22:0/2510147982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e783ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.686+0000 7f5d2ce00640 1 -- 172.21.15.22:0/2510147982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 msgr2=0x55b5e783ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.686+0000 7f5d2ce00640 1 --2- 172.21.15.22:0/2510147982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e783ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.686+0000 7f5d2ce00640 1 -- 172.21.15.22:0/2510147982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 msgr2=0x55b5e78536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.686+0000 7f5d2ce00640 1 --2- 172.21.15.22:0/2510147982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e78536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.686+0000 7f5d2ce00640 1 -- 172.21.15.22:0/2510147982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e7469170 con 0x55b5e7832a10 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.686+0000 7f5d2e200640 1 --2- 172.21.15.22:0/2510147982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e78536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.686+0000 7f5d2ce00640 1 --2- 172.21.15.22:0/2510147982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e78472a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5d14005ab0 tx=0x7f5d1402df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e751b03a2ace5e2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.686+0000 7f5d1a000640 1 -- 172.21.15.22:0/2510147982 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14054070 con 0x55b5e7832a10 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.686+0000 7f5d1a000640 1 -- 172.21.15.22:0/2510147982 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d140057c0 con 0x55b5e7832a10 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.686+0000 7f5d1a000640 1 -- 172.21.15.22:0/2510147982 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14044040 con 0x55b5e7832a10 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.687+0000 7f5d32e72740 1 -- 172.21.15.22:0/2510147982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 msgr2=0x55b5e78472a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.687+0000 7f5d32e72740 1 --2- 172.21.15.22:0/2510147982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e78472a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5d14005ab0 tx=0x7f5d1402df90 comp rx=0 tx=0).stop 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.687+0000 7f5d32e72740 1 -- 172.21.15.22:0/2510147982 shutdown_connections 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.687+0000 7f5d32e72740 1 --2- 172.21.15.22:0/2510147982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e78536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.687+0000 7f5d32e72740 1 --2- 172.21.15.22:0/2510147982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e78472a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.687+0000 7f5d32e72740 1 --2- 172.21.15.22:0/2510147982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e783ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.687+0000 7f5d32e72740 1 -- 172.21.15.22:0/2510147982 >> 172.21.15.22:0/2510147982 conn(0x55b5e782c820 msgr2=0x55b5e7832df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:49.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.687+0000 7f5d32e72740 1 -- 172.21.15.22:0/2510147982 shutdown_connections 2024-04-20T21:43:49.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.687+0000 7f5d32e72740 1 -- 172.21.15.22:0/2510147982 wait complete. 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.688+0000 7f5d32e72740 1 -- start start 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.688+0000 7f5d32e72740 1 Processor -- start 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.688+0000 7f5d32e72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e783f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.688+0000 7f5d32e72740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e783f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.688+0000 7f5d32e72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e783fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.688+0000 7f5d32e72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281b3010 con 0x55b5e783b5f0 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.688+0000 7f5d32e72740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281b4480 con 0x55b5e7832a10 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.688+0000 7f5d32e72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281b36e0 con 0x55b5e78320f0 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d32e72740 10 client.0 _ll_register_callbacks cb 0x7f5d14009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d2e200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e783fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d2e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e783f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d2e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e783f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50788/0 (socket says 172.21.15.22:50788) 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d2e200640 1 -- 172.21.15.22:0/922358189 learned_addr learned my addr 172.21.15.22:0/922358189 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d2ce00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e783f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d2e200640 1 -- 172.21.15.22:0/922358189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 msgr2=0x55b5e783f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d2e200640 1 --2- 172.21.15.22:0/922358189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e783f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d2e200640 1 -- 172.21.15.22:0/922358189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 msgr2=0x55b5e783fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d2e200640 1 --2- 172.21.15.22:0/922358189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e783fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d2e200640 1 -- 172.21.15.22:0/922358189 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e78b10e0 con 0x55b5e78320f0 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d2e200640 1 --2- 172.21.15.22:0/922358189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e783f360 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5d20003f50 tx=0x7f5d20009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a94c3171ed565bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.689+0000 7f5d0f400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d0f400640 1 -- 172.21.15.22:0/922358189 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20004350 con 0x55b5e78320f0 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d0f400640 1 -- 172.21.15.22:0/922358189 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f5d200044f0 con 0x55b5e78320f0 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d32e72740 1 -- 172.21.15.22:0/922358189 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d140057c0 con 0x55b5e78320f0 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d0f400640 1 -- 172.21.15.22:0/922358189 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5d200046f0 con 0x55b5e78320f0 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d32e72740 10 client.26863 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d32e72740 1 -- 172.21.15.22:0/922358189 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b5e78b23e0 con 0x55b5e78320f0 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d0f400640 1 -- 172.21.15.22:0/922358189 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d140057c0 con 0x55b5e78320f0 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d0f400640 1 -- 172.21.15.22:0/922358189 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f5d20025040 con 0x55b5e78320f0 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d32e72740 10 client.26863 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d32e72740 10 client.26863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d32e72740 20 client.26863 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d32e72740 1 --2- 172.21.15.22:0/922358189 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b5e78b29a0 0x55b5e78b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d32e72740 4 client.26863 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.690+0000 7f5d32e72740 1 -- 172.21.15.22:0/922358189 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b5e779e5b0 con 0x55b5e78b29a0 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.691+0000 7f5d2ce00640 1 --2- 172.21.15.22:0/922358189 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b5e78b29a0 0x55b5e78b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.691+0000 7f5d2ce00640 1 --2- 172.21.15.22:0/922358189 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b5e78b29a0 0x55b5e78b4dc0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce0b0658b37282eb server_cookie=6230696527eed328 in_seq=0 out_seq=0 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.691+0000 7f5d0f400640 10 client.26863 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.692+0000 7f5d0f400640 1 -- 172.21.15.22:0/922358189 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b5e779e5b0 con 0x55b5e78b29a0 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.692+0000 7f5d0f400640 10 client.26863 handle_command_reply: tid=0 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.692+0000 7f5d32e72740 1 client.26863 shutdown 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.693+0000 7f5d32e72740 1 -- 172.21.15.22:0/922358189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 msgr2=0x55b5e783f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.693+0000 7f5d32e72740 1 --2- 172.21.15.22:0/922358189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e783f360 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5d20003f50 tx=0x7f5d20009500 comp rx=0 tx=0).stop 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.693+0000 7f5d32e72740 1 -- 172.21.15.22:0/922358189 shutdown_connections 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.693+0000 7f5d32e72740 1 --2- 172.21.15.22:0/922358189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b5e783b5f0 0x55b5e783fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.693+0000 7f5d32e72740 1 --2- 172.21.15.22:0/922358189 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b5e78b29a0 0x55b5e78b4dc0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.693+0000 7f5d32e72740 1 --2- 172.21.15.22:0/922358189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b5e7832a10 0x55b5e783f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.693+0000 7f5d32e72740 1 --2- 172.21.15.22:0/922358189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5e78320f0 0x55b5e783f360 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.693+0000 7f5d0f400640 0 client.26863 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.693+0000 7f5d32e72740 1 -- 172.21.15.22:0/922358189 >> 172.21.15.22:0/922358189 conn(0x55b5e782c820 msgr2=0x55b5e782b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.693+0000 7f5d32e72740 1 -- 172.21.15.22:0/922358189 shutdown_connections 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.693+0000 7f5d32e72740 1 -- 172.21.15.22:0/922358189 wait complete. 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.693+0000 7f5d32e72740 20 client.26863 trim_cache size 0 max 16384 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-04-20T21:43:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "aab35fa0-e712-496b-a78e-5b87ecc80c50", 2024-04-20T21:43:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-04-20T21:43:49.699 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.694+0000 7f5d2ec00640 1 -- 172.21.15.22:0/11819515 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5d100a3d20 msgr2=0x7f5d100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.694+0000 7f5d2ec00640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5d100a3d20 0x7f5d100a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55b5e7831bc0 tx=0x7f5d1c0d45e0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.694+0000 7f5d2ec00640 1 -- 172.21.15.22:0/11819515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d28108500 msgr2=0x7f5d281a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.694+0000 7f5d2ec00640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d28108500 0x7f5d281a4950 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00e9f0 tx=0x7f5d1c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.694+0000 7f5d2ec00640 1 -- 172.21.15.22:0/11819515 shutdown_connections 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.694+0000 7f5d2ec00640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5d100a3d20 0x7f5d100a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.694+0000 7f5d2ec00640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2810d3d0 0x7f5d2819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.694+0000 7f5d2ec00640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d28108ea0 0x7f5d281a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.694+0000 7f5d2ec00640 1 --2- 172.21.15.22:0/11819515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d28108500 0x7f5d281a4950 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.694+0000 7f5d2ec00640 1 -- 172.21.15.22:0/11819515 >> 172.21.15.22:0/11819515 conn(0x7f5d28077d80 msgr2=0x7f5d28078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.694+0000 7f5d2ec00640 1 -- 172.21.15.22:0/11819515 shutdown_connections 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.694+0000 7f5d2ec00640 1 -- 172.21.15.22:0/11819515 wait complete. 2024-04-20T21:43:49.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:43:49.714 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:43:49.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.932+0000 7f685ac00640 1 -- 172.21.15.22:0/1820821382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6854102fc0 msgr2=0x7f68541033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.932+0000 7f685ac00640 1 --2- 172.21.15.22:0/1820821382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6854102fc0 0x7f68541033a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6848009ca0 tx=0x7f684802c180 comp rx=0 tx=0).stop 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.932+0000 7f685ac00640 1 -- 172.21.15.22:0/1820821382 shutdown_connections 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.932+0000 7f685ac00640 1 --2- 172.21.15.22:0/1820821382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f685410c3e0 0x7f685410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.932+0000 7f685ac00640 1 --2- 172.21.15.22:0/1820821382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68541038e0 0x7f685410bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.932+0000 7f685ac00640 1 --2- 172.21.15.22:0/1820821382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6854102fc0 0x7f68541033a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.932+0000 7f685ac00640 1 -- 172.21.15.22:0/1820821382 >> 172.21.15.22:0/1820821382 conn(0x7f68540fc7e0 msgr2=0x7f68540fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.932+0000 7f685ac00640 1 -- 172.21.15.22:0/1820821382 shutdown_connections 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.933+0000 7f685ac00640 1 -- 172.21.15.22:0/1820821382 wait complete. 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.933+0000 7f685ac00640 1 Processor -- start 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.934+0000 7f685ac00640 1 -- start start 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.934+0000 7f685ac00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6854102fc0 0x7f68541a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.934+0000 7f685ac00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68541038e0 0x7f68541a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.934+0000 7f685ac00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f685410c3e0 0x7f685419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.934+0000 7f685ac00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f68541121c0 con 0x7f685410c3e0 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.934+0000 7f685ac00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6854112040 con 0x7f6854102fc0 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.934+0000 7f685ac00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6854112340 con 0x7f68541038e0 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.934+0000 7f6858e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68541038e0 0x7f68541a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.935+0000 7f6858e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68541038e0 0x7f68541a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:50810/0 (socket says 172.21.15.22:50810) 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.935+0000 7f685a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f685410c3e0 0x7f685419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.935+0000 7f6858e00640 1 -- 172.21.15.22:0/1538625702 learned_addr learned my addr 172.21.15.22:0/1538625702 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.935+0000 7f6859800640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6854102fc0 0x7f68541a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.935+0000 7f6858e00640 1 -- 172.21.15.22:0/1538625702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6854102fc0 msgr2=0x7f68541a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.935+0000 7f6858e00640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6854102fc0 0x7f68541a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.935+0000 7f6858e00640 1 -- 172.21.15.22:0/1538625702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f685410c3e0 msgr2=0x7f685419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.935+0000 7f6858e00640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f685410c3e0 0x7f685419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.935+0000 7f6858e00640 1 -- 172.21.15.22:0/1538625702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685419b050 con 0x7f68541038e0 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.935+0000 7f6859800640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6854102fc0 0x7f68541a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.935+0000 7f6858e00640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68541038e0 0x7f68541a0bc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f684000d750 tx=0x7f684000dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.936+0000 7f6852000640 1 -- 172.21.15.22:0/1538625702 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840017070 con 0x7f68541038e0 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.936+0000 7f6852000640 1 -- 172.21.15.22:0/1538625702 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6840005020 con 0x7f68541038e0 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.936+0000 7f6852000640 1 -- 172.21.15.22:0/1538625702 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840004d00 con 0x7f68541038e0 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.936+0000 7f685ac00640 1 -- 172.21.15.22:0/1538625702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685419b340 con 0x7f68541038e0 2024-04-20T21:43:49.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.936+0000 7f685ac00640 1 -- 172.21.15.22:0/1538625702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68541a7600 con 0x7f68541038e0 2024-04-20T21:43:49.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.939+0000 7f685ee25740 1 Processor -- start 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.939+0000 7f685ee25740 1 -- start start 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f685ee25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f685ee25740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5bca0 0x55b902b64340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f685ee25740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b66c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f685ee25740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b902ac1150 con 0x55b902b64880 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f685ee25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b902a67370 con 0x55b902b5b380 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f685ee25740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b902abd110 con 0x55b902b5bca0 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f6852000640 1 -- 172.21.15.22:0/1538625702 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f6840009a30 con 0x7f68541038e0 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f685a200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f685a200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b5b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34334/0 (socket says 172.21.15.22:34334) 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f685a200640 1 -- 172.21.15.22:0/1577645877 learned_addr learned my addr 172.21.15.22:0/1577645877 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f6859800640 1 --2- 172.21.15.22:0/1577645877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5bca0 0x55b902b64340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f6858e00640 1 --2- 172.21.15.22:0/1577645877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b66c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f685a200640 1 -- 172.21.15.22:0/1577645877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5bca0 msgr2=0x55b902b64340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f685a200640 1 --2- 172.21.15.22:0/1577645877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5bca0 0x55b902b64340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.940+0000 7f685a200640 1 -- 172.21.15.22:0/1577645877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 msgr2=0x55b902b66c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.941+0000 7f685a200640 1 --2- 172.21.15.22:0/1577645877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b66c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.941+0000 7f685a200640 1 -- 172.21.15.22:0/1577645877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b90289d7d0 con 0x55b902b5b380 2024-04-20T21:43:49.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.941+0000 7f6858e00640 1 --2- 172.21.15.22:0/1577645877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b66c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:43:49.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.941+0000 7f685a200640 1 --2- 172.21.15.22:0/1577645877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b5b760 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f684c00b6b0 tx=0x7f684c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e43d1c5d3f44c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:49.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.941+0000 7f6846a00640 1 -- 172.21.15.22:0/1577645877 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c003b20 con 0x55b902b5b380 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.941+0000 7f6846a00640 1 -- 172.21.15.22:0/1577645877 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684c015070 con 0x55b902b5b380 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.941+0000 7f6846a00640 1 -- 172.21.15.22:0/1577645877 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c0106a0 con 0x55b902b5b380 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.941+0000 7f6852000640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f683c0a3d20 0x7f683c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.942+0000 7f6852000640 1 -- 172.21.15.22:0/1538625702 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f68400d0190 con 0x7f68541038e0 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.942+0000 7f6859800640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f683c0a3d20 0x7f683c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.942+0000 7f685ee25740 1 -- 172.21.15.22:0/1577645877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 msgr2=0x55b902b5b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.942+0000 7f685ee25740 1 --2- 172.21.15.22:0/1577645877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b5b760 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f684c00b6b0 tx=0x7f684c004610 comp rx=0 tx=0).stop 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.942+0000 7f685ee25740 1 -- 172.21.15.22:0/1577645877 shutdown_connections 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.942+0000 7f685ee25740 1 --2- 172.21.15.22:0/1577645877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b66c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.942+0000 7f685ee25740 1 --2- 172.21.15.22:0/1577645877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5bca0 0x55b902b64340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.942+0000 7f685ee25740 1 --2- 172.21.15.22:0/1577645877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b5b760 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.942+0000 7f685ee25740 1 -- 172.21.15.22:0/1577645877 >> 172.21.15.22:0/1577645877 conn(0x55b902b55ab0 msgr2=0x55b902b55e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.942+0000 7f685ee25740 1 -- 172.21.15.22:0/1577645877 shutdown_connections 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.942+0000 7f685ee25740 1 -- 172.21.15.22:0/1577645877 wait complete. 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.943+0000 7f6859800640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f683c0a3d20 0x7f683c0a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6848004af0 tx=0x7f6848005c50 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.943+0000 7f685ee25740 1 -- start start 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.943+0000 7f685ee25740 1 Processor -- start 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f685ee25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b61900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f685ee25740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b5bca0 0x55b902b61e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f685ee25740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b64880 0x55b902b62380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f685ee25740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b902b67670 con 0x55b902b5bca0 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f685ee25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b902a7a740 con 0x55b902b5b380 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f685ee25740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b902b677f0 con 0x55b902b64880 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f685ee25740 10 client.0 _ll_register_callbacks cb 0x55b902a75b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f6858e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b5bca0 0x55b902b61e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f6859800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b61900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f6859800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b61900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34338/0 (socket says 172.21.15.22:34338) 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f6859800640 1 -- 172.21.15.22:0/1804214263 learned_addr learned my addr 172.21.15.22:0/1804214263 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f685a200640 1 --2- 172.21.15.22:0/1804214263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b64880 0x55b902b62380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f6859800640 1 -- 172.21.15.22:0/1804214263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b64880 msgr2=0x55b902b62380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f6859800640 1 --2- 172.21.15.22:0/1804214263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b64880 0x55b902b62380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f6859800640 1 -- 172.21.15.22:0/1804214263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b5bca0 msgr2=0x55b902b61e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.944+0000 7f6859800640 1 --2- 172.21.15.22:0/1804214263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b5bca0 0x55b902b61e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.945+0000 7f6859800640 1 -- 172.21.15.22:0/1804214263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9027d33f0 con 0x55b902b5b380 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.945+0000 7f6859800640 1 --2- 172.21.15.22:0/1804214263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b61900 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6848003e90 tx=0x7f68480095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73dafb468be374f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.945+0000 7f683b400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.945+0000 7f683b400640 1 -- 172.21.15.22:0/1804214263 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848051020 con 0x55b902b5b380 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.945+0000 7f685ee25740 1 -- 172.21.15.22:0/1804214263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f684c015070 con 0x55b902b5b380 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.945+0000 7f683b400640 1 -- 172.21.15.22:0/1804214263 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f6848032870 con 0x55b902b5b380 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.945+0000 7f683b400640 1 -- 172.21.15.22:0/1804214263 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f684805c050 con 0x55b902b5b380 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.945+0000 7f685ee25740 10 client.26846 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.946+0000 7f685ee25740 1 -- 172.21.15.22:0/1804214263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b9027930f0 con 0x55b902b5b380 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.946+0000 7f683b400640 1 -- 172.21.15.22:0/1804214263 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684c015070 con 0x55b902b5b380 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.946+0000 7f683b400640 1 -- 172.21.15.22:0/1804214263 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f68480046d0 con 0x55b902b5b380 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.946+0000 7f685ee25740 10 client.26846 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.946+0000 7f685ee25740 10 client.26846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.946+0000 7f685ee25740 20 client.26846 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.946+0000 7f685ee25740 1 --2- 172.21.15.22:0/1804214263 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b902b74bb0 0x55b902b77000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.946+0000 7f685ee25740 4 client.26846 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.946+0000 7f685ee25740 1 -- 172.21.15.22:0/1804214263 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b902aceb70 con 0x55b902b74bb0 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.947+0000 7f6858e00640 1 --2- 172.21.15.22:0/1804214263 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b902b74bb0 0x55b902b77000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.947+0000 7f6858e00640 1 --2- 172.21.15.22:0/1804214263 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b902b74bb0 0x55b902b77000 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb5b1388638f92f server_cookie=350897963a0e8eb4 in_seq=0 out_seq=0 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.947+0000 7f683b400640 10 client.26846 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.948+0000 7f683b400640 1 -- 172.21.15.22:0/1804214263 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b902aceb70 con 0x55b902b74bb0 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.948+0000 7f683b400640 10 client.26846 handle_command_reply: tid=0 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.948+0000 7f685ee25740 1 client.26846 shutdown 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.948+0000 7f685ee25740 1 -- 172.21.15.22:0/1804214263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 msgr2=0x55b902b61900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.948+0000 7f685ee25740 1 --2- 172.21.15.22:0/1804214263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b61900 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6848003e90 tx=0x7f68480095c0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.948+0000 7f685ee25740 1 -- 172.21.15.22:0/1804214263 shutdown_connections 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.948+0000 7f685ee25740 1 --2- 172.21.15.22:0/1804214263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b64880 0x55b902b62380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.948+0000 7f685ee25740 1 --2- 172.21.15.22:0/1804214263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b5bca0 0x55b902b61e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.948+0000 7f685ee25740 1 --2- 172.21.15.22:0/1804214263 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b902b74bb0 0x55b902b77000 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.948+0000 7f685ee25740 1 --2- 172.21.15.22:0/1804214263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b61900 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.948+0000 7f683b400640 0 client.26846 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.948+0000 7f685ee25740 1 -- 172.21.15.22:0/1804214263 >> 172.21.15.22:0/1804214263 conn(0x55b902b55ab0 msgr2=0x55b902b57800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.949+0000 7f685ee25740 1 -- 172.21.15.22:0/1804214263 shutdown_connections 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.949+0000 7f685ee25740 1 -- 172.21.15.22:0/1804214263 wait complete. 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.949+0000 7f685ee25740 20 client.26846 trim_cache size 0 max 16384 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.969+0000 7f685ee25740 1 Processor -- start 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.969+0000 7f685ee25740 1 -- start start 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.969+0000 7f685ee25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b64030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.969+0000 7f685ee25740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5bca0 0x55b902b60860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.969+0000 7f685ee25740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b70530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.969+0000 7f685ee25740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f68541ade90 con 0x55b902b64880 2024-04-20T21:43:49.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.969+0000 7f685ee25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68541b06f0 con 0x55b902b5b380 2024-04-20T21:43:49.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.969+0000 7f685ee25740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68541ae4e0 con 0x55b902b5bca0 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f6859800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b64030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f685a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b70530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f685a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b70530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57216/0 (socket says 172.21.15.22:57216) 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f685a200640 1 -- 172.21.15.22:0/459029272 learned_addr learned my addr 172.21.15.22:0/459029272 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f6859800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b64030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34342/0 (socket says 172.21.15.22:34342) 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f6858e00640 1 --2- 172.21.15.22:0/459029272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5bca0 0x55b902b60860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f6859800640 1 -- 172.21.15.22:0/459029272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5bca0 msgr2=0x55b902b60860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f6859800640 1 --2- 172.21.15.22:0/459029272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5bca0 0x55b902b60860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f6859800640 1 -- 172.21.15.22:0/459029272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 msgr2=0x55b902b70530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f6859800640 1 --2- 172.21.15.22:0/459029272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b70530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f6859800640 1 -- 172.21.15.22:0/459029272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b902b55220 con 0x55b902b5b380 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f685a200640 1 --2- 172.21.15.22:0/459029272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b70530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.970+0000 7f6859800640 1 --2- 172.21.15.22:0/459029272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b64030 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f684804d520 tx=0x7f6848034ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=762496465f7b454f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.971+0000 7f6846000640 1 -- 172.21.15.22:0/459029272 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848051050 con 0x55b902b5b380 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.971+0000 7f6846000640 1 -- 172.21.15.22:0/459029272 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6848034e30 con 0x55b902b5b380 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.971+0000 7f6846000640 1 -- 172.21.15.22:0/459029272 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848039040 con 0x55b902b5b380 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.971+0000 7f685ee25740 1 -- 172.21.15.22:0/459029272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 msgr2=0x55b902b64030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.971+0000 7f685ee25740 1 --2- 172.21.15.22:0/459029272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b64030 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f684804d520 tx=0x7f6848034ce0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.971+0000 7f685ee25740 1 -- 172.21.15.22:0/459029272 shutdown_connections 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.971+0000 7f685ee25740 1 --2- 172.21.15.22:0/459029272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b70530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.971+0000 7f685ee25740 1 --2- 172.21.15.22:0/459029272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5bca0 0x55b902b60860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.971+0000 7f685ee25740 1 --2- 172.21.15.22:0/459029272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5b380 0x55b902b64030 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.971+0000 7f685ee25740 1 -- 172.21.15.22:0/459029272 >> 172.21.15.22:0/459029272 conn(0x55b902b55ab0 msgr2=0x55b902b54b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.972+0000 7f685ee25740 1 -- 172.21.15.22:0/459029272 shutdown_connections 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.972+0000 7f685ee25740 1 -- 172.21.15.22:0/459029272 wait complete. 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.972+0000 7f685ee25740 1 -- start start 2024-04-20T21:43:49.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.972+0000 7f685ee25740 1 Processor -- start 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f685ee25740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5b380 0x55b902b5e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f685ee25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5bca0 0x55b902b5ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f685ee25740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b5f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f685ee25740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f68541aebb0 con 0x55b902b64880 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f685ee25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68541b0020 con 0x55b902b5bca0 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f685ee25740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68541af280 con 0x55b902b5b380 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f685ee25740 10 client.0 _ll_register_callbacks cb 0x7f6848044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f685a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b5f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f6858e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5bca0 0x55b902b5ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f6858e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5bca0 0x55b902b5ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:34356/0 (socket says 172.21.15.22:34356) 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f685a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b5f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57218/0 (socket says 172.21.15.22:57218) 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f6858e00640 1 -- 172.21.15.22:0/3125234895 learned_addr learned my addr 172.21.15.22:0/3125234895 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.973+0000 7f6859800640 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5b380 0x55b902b5e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f6858e00640 1 -- 172.21.15.22:0/3125234895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5b380 msgr2=0x55b902b5e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f6858e00640 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5b380 0x55b902b5e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f6858e00640 1 -- 172.21.15.22:0/3125234895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 msgr2=0x55b902b5f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f6858e00640 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b5f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f6858e00640 1 -- 172.21.15.22:0/3125234895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b902bda340 con 0x55b902b5bca0 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f6859800640 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5b380 0x55b902b5e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f685a200640 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b5f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f6858e00640 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5bca0 0x55b902b5ed80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f68400cd910 tx=0x7f6840098540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4666e7c48259db1d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f683b400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f683b400640 1 -- 172.21.15.22:0/3125234895 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840020040 con 0x55b902b5bca0 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f683b400640 1 -- 172.21.15.22:0/3125234895 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f68400051d0 con 0x55b902b5bca0 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f685ee25740 1 -- 172.21.15.22:0/3125234895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6848034e30 con 0x55b902b5bca0 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f683b400640 1 -- 172.21.15.22:0/3125234895 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f68400a0170 con 0x55b902b5bca0 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f685ee25740 10 client.26852 fetch_fsmap learned FSMap version 20 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.974+0000 7f685ee25740 1 -- 172.21.15.22:0/3125234895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b902bdb630 con 0x55b902b5bca0 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.975+0000 7f683b400640 1 -- 172.21.15.22:0/3125234895 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6848034e30 con 0x55b902b5bca0 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.975+0000 7f683b400640 1 -- 172.21.15.22:0/3125234895 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f68400cdb30 con 0x55b902b5bca0 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.975+0000 7f685ee25740 10 client.26852 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.975+0000 7f685ee25740 10 client.26852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.975+0000 7f685ee25740 20 client.26852 populate_metadata read hostname 'smithi022' 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.975+0000 7f685ee25740 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b902bdbdb0 0x55b902bde1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.975+0000 7f685ee25740 4 client.26852 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.975+0000 7f685ee25740 1 -- 172.21.15.22:0/3125234895 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b902ac89e0 con 0x55b902bdbdb0 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.975+0000 7f6859800640 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b902bdbdb0 0x55b902bde1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f6859800640 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b902bdbdb0 0x55b902bde1d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2170686fafefe454 server_cookie=4f81550129e36d70 in_seq=0 out_seq=0 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f683b400640 10 client.26852 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f683b400640 1 -- 172.21.15.22:0/3125234895 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b902ac89e0 con 0x55b902bdbdb0 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f683b400640 10 client.26852 handle_command_reply: tid=0 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f685ee25740 1 client.26852 shutdown 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f685ee25740 1 -- 172.21.15.22:0/3125234895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5bca0 msgr2=0x55b902b5ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f685ee25740 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5bca0 0x55b902b5ed80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f68400cd910 tx=0x7f6840098540 comp rx=0 tx=0).stop 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f685ee25740 1 -- 172.21.15.22:0/3125234895 shutdown_connections 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f685ee25740 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b902b64880 0x55b902b5f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f685ee25740 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55b902bdbdb0 0x55b902bde1d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f685ee25740 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b902b5bca0 0x55b902b5ed80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f685ee25740 1 --2- 172.21.15.22:0/3125234895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b902b5b380 0x55b902b5e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f683b400640 0 client.26852 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f685ee25740 1 -- 172.21.15.22:0/3125234895 >> 172.21.15.22:0/3125234895 conn(0x55b902b55ab0 msgr2=0x55b902b56db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f685ee25740 1 -- 172.21.15.22:0/3125234895 shutdown_connections 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f685ee25740 1 -- 172.21.15.22:0/3125234895 wait complete. 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.976+0000 7f685ee25740 20 client.26852 trim_cache size 0 max 16384 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "aab35fa0-e712-496b-a78e-5b87ecc80c50": { 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "aab35fa0-e712-496b-a78e-5b87ecc80c50", 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.977+0000 7f685ac00640 1 -- 172.21.15.22:0/1538625702 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f683c0a3d20 msgr2=0x7f683c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.977+0000 7f685ac00640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f683c0a3d20 0x7f683c0a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6848004af0 tx=0x7f6848005c50 comp rx=0 tx=0).stop 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.977+0000 7f685ac00640 1 -- 172.21.15.22:0/1538625702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68541038e0 msgr2=0x7f68541a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.977+0000 7f685ac00640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68541038e0 0x7f68541a0bc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f684000d750 tx=0x7f684000dc10 comp rx=0 tx=0).stop 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.977+0000 7f685ac00640 1 -- 172.21.15.22:0/1538625702 shutdown_connections 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.977+0000 7f685ac00640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f683c0a3d20 0x7f683c0a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.977+0000 7f685ac00640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f685410c3e0 0x7f685419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.977+0000 7f685ac00640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68541038e0 0x7f68541a0bc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.977+0000 7f685ac00640 1 --2- 172.21.15.22:0/1538625702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6854102fc0 0x7f68541a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.977+0000 7f685ac00640 1 -- 172.21.15.22:0/1538625702 >> 172.21.15.22:0/1538625702 conn(0x7f68540fc7e0 msgr2=0x7f6854100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.977+0000 7f685ac00640 1 -- 172.21.15.22:0/1538625702 shutdown_connections 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:49.977+0000 7f685ac00640 1 -- 172.21.15.22:0/1538625702 wait complete. 2024-04-20T21:43:49.994 INFO:tasks.cephfs.filesystem:scrub status for tag:aab35fa0-e712-496b-a78e-5b87ecc80c50 - {'path': '/', 'tag': 'aab35fa0-e712-496b-a78e-5b87ecc80c50', 'options': 'recursive,force'} 2024-04-20T21:43:50.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:50 smithi122 ceph-mon[22147]: pgmap v311: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:50.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:50 smithi033 ceph-mon[21626]: pgmap v311: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:50.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:50 smithi022 ceph-mon[16843]: pgmap v311: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:43:51.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:51 smithi122 ceph-mon[22147]: scrub queued for path: / 2024-04-20T21:43:51.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:51 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:51.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:51 smithi122 ceph-mon[22147]: scrub summary: active paths [/] 2024-04-20T21:43:51.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:51 smithi122 ceph-mon[22147]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:51.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:51 smithi033 ceph-mon[21626]: scrub queued for path: / 2024-04-20T21:43:51.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:51 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:51.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:51 smithi033 ceph-mon[21626]: scrub summary: active paths [/] 2024-04-20T21:43:51.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:51 smithi033 ceph-mon[21626]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:51.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:51 smithi022 ceph-mon[16843]: scrub queued for path: / 2024-04-20T21:43:51.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:51 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:51.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:51 smithi022 ceph-mon[16843]: scrub summary: active paths [/] 2024-04-20T21:43:51.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:51 smithi022 ceph-mon[16843]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:52.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:52 smithi122 ceph-mon[22147]: pgmap v312: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:43:52.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:52 smithi122 ceph-mon[22147]: scrub summary: idle 2024-04-20T21:43:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:52 smithi033 ceph-mon[21626]: pgmap v312: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:43:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:52 smithi033 ceph-mon[21626]: scrub summary: idle 2024-04-20T21:43:52.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:52 smithi022 ceph-mon[16843]: pgmap v312: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:43:52.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:52 smithi022 ceph-mon[16843]: scrub summary: idle 2024-04-20T21:43:54.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:54 smithi122 ceph-mon[22147]: pgmap v313: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:43:54.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:54 smithi033 ceph-mon[21626]: pgmap v313: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:43:54.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:54 smithi022 ceph-mon[16843]: pgmap v313: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:43:56.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:56 smithi122 ceph-mon[22147]: pgmap v314: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:43:56.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:56 smithi033 ceph-mon[21626]: pgmap v314: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:43:56.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:56 smithi022 ceph-mon[16843]: pgmap v314: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T21:43:58.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-20T21:43:58.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.490+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1958828289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed81070b0 msgr2=0x7f4ed810ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:58.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.490+0000 7f4ed7e00640 1 --2- 172.21.15.22:0/1958828289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed81070b0 0x7f4ed810ec80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009ca0 tx=0x7f4ed002c100 comp rx=0 tx=0).stop 2024-04-20T21:43:58.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.490+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1958828289 shutdown_connections 2024-04-20T21:43:58.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.490+0000 7f4ed7e00640 1 --2- 172.21.15.22:0/1958828289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed81070b0 0x7f4ed810ec80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.490+0000 7f4ed7e00640 1 --2- 172.21.15.22:0/1958828289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ed8102990 0x7f4ed81069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.490+0000 7f4ed7e00640 1 --2- 172.21.15.22:0/1958828289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ed8101fe0 0x7f4ed81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.490+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1958828289 >> 172.21.15.22:0/1958828289 conn(0x7f4ed80fd8a0 msgr2=0x7f4ed80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:58.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.490+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1958828289 shutdown_connections 2024-04-20T21:43:58.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.490+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1958828289 wait complete. 2024-04-20T21:43:58.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.491+0000 7f4ed7e00640 1 Processor -- start 2024-04-20T21:43:58.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.491+0000 7f4ed7e00640 1 -- start start 2024-04-20T21:43:58.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.492+0000 7f4ed7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ed8101fe0 0x7f4ed81a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:58.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.492+0000 7f4ed7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ed81070b0 0x7f4ed81a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:58.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.492+0000 7f4ed7e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed819a6f0 0x7f4ed819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:58.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.492+0000 7f4ed7e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed8112150 con 0x7f4ed8101fe0 2024-04-20T21:43:58.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.492+0000 7f4ed7e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed8111fd0 con 0x7f4ed819a6f0 2024-04-20T21:43:58.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.492+0000 7f4ed7e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed81122d0 con 0x7f4ed81070b0 2024-04-20T21:43:58.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.492+0000 7f4ed6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ed8101fe0 0x7f4ed81a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:58.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.492+0000 7f4ece000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ed81070b0 0x7f4ed81a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:58.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.492+0000 7f4ed6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ed8101fe0 0x7f4ed81a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42248/0 (socket says 172.21.15.22:42248) 2024-04-20T21:43:58.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.492+0000 7f4ed7400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed819a6f0 0x7f4ed819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:58.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.492+0000 7f4ed6a00640 1 -- 172.21.15.22:0/1367077145 learned_addr learned my addr 172.21.15.22:0/1367077145 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:58.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.493+0000 7f4ece000640 1 -- 172.21.15.22:0/1367077145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed819a6f0 msgr2=0x7f4ed819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:58.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.493+0000 7f4ece000640 1 --2- 172.21.15.22:0/1367077145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed819a6f0 0x7f4ed819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.493+0000 7f4ece000640 1 -- 172.21.15.22:0/1367077145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ed8101fe0 msgr2=0x7f4ed81a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:58.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.493+0000 7f4ece000640 1 --2- 172.21.15.22:0/1367077145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ed8101fe0 0x7f4ed81a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.493+0000 7f4ece000640 1 -- 172.21.15.22:0/1367077145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed819b2f0 con 0x7f4ed81070b0 2024-04-20T21:43:58.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.493+0000 7f4ece000640 1 --2- 172.21.15.22:0/1367077145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ed81070b0 0x7f4ed81a0b70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000e9c0 tx=0x7f4ec000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:58.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.493+0000 7f4ecfe00640 1 -- 172.21.15.22:0/1367077145 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000ccc0 con 0x7f4ed81070b0 2024-04-20T21:43:58.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.493+0000 7f4ecfe00640 1 -- 172.21.15.22:0/1367077145 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec0010910 con 0x7f4ed81070b0 2024-04-20T21:43:58.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.493+0000 7f4ecfe00640 1 -- 172.21.15.22:0/1367077145 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec00188c0 con 0x7f4ed81070b0 2024-04-20T21:43:58.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.493+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1367077145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed81a73d0 con 0x7f4ed81070b0 2024-04-20T21:43:58.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.494+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1367077145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed81a7780 con 0x7f4ed81070b0 2024-04-20T21:43:58.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.495+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1367077145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed81034b0 con 0x7f4ed81070b0 2024-04-20T21:43:58.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.497+0000 7f4ecfe00640 1 -- 172.21.15.22:0/1367077145 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4ec0010ac0 con 0x7f4ed81070b0 2024-04-20T21:43:58.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.498+0000 7f4ecfe00640 1 --2- 172.21.15.22:0/1367077145 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f4ebc0a3d90 0x7f4ebc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:58.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.498+0000 7f4ecfe00640 1 -- 172.21.15.22:0/1367077145 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f4ec00d0bf0 con 0x7f4ed81070b0 2024-04-20T21:43:58.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.499+0000 7f4ed6a00640 1 --2- 172.21.15.22:0/1367077145 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f4ebc0a3d90 0x7f4ebc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:58.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.499+0000 7f4ed6a00640 1 --2- 172.21.15.22:0/1367077145 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f4ebc0a3d90 0x7f4ebc0a6250 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009c90 tx=0x7f4ec8005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:58.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.501+0000 7f4ecfe00640 1 -- 172.21.15.22:0/1367077145 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4ec009cb90 con 0x7f4ed81070b0 2024-04-20T21:43:58.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:43:58 smithi122 ceph-mon[22147]: pgmap v315: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:43:58.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:43:58 smithi033 ceph-mon[21626]: pgmap v315: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:43:58.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:43:58 smithi022 ceph-mon[16843]: pgmap v315: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:43:58.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.733+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1367077145 --> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f4ed8102a00 con 0x7f4ebc0a3d90 2024-04-20T21:43:58.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.746+0000 7f4ecfe00640 1 -- 172.21.15.22:0/1367077145 <== mgr.15024 v2:172.21.15.22:6800/975472489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f4ed8102a00 con 0x7f4ebc0a3d90 2024-04-20T21:43:58.745 INFO:teuthology.orchestra.run.smithi022.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-20T00:00:00", "created": "2024-04-20T21:33:56", "first": "2024-04-20T21:34:00", "last": "2024-04-20T21:43:00", "last_pruned": "2024-04-20T21:43:01", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-20T21:43:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.748+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1367077145 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f4ebc0a3d90 msgr2=0x7f4ebc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.749+0000 7f4ed7e00640 1 --2- 172.21.15.22:0/1367077145 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f4ebc0a3d90 0x7f4ebc0a6250 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009c90 tx=0x7f4ec8005f90 comp rx=0 tx=0).stop 2024-04-20T21:43:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.749+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1367077145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ed81070b0 msgr2=0x7f4ed81a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.749+0000 7f4ed7e00640 1 --2- 172.21.15.22:0/1367077145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ed81070b0 0x7f4ed81a0b70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000e9c0 tx=0x7f4ec000ee80 comp rx=0 tx=0).stop 2024-04-20T21:43:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.749+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1367077145 shutdown_connections 2024-04-20T21:43:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.749+0000 7f4ed7e00640 1 --2- 172.21.15.22:0/1367077145 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f4ebc0a3d90 0x7f4ebc0a6250 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.749+0000 7f4ed7e00640 1 --2- 172.21.15.22:0/1367077145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ed819a6f0 0x7f4ed819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.749+0000 7f4ed7e00640 1 --2- 172.21.15.22:0/1367077145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ed81070b0 0x7f4ed81a0b70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.749+0000 7f4ed7e00640 1 --2- 172.21.15.22:0/1367077145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ed8101fe0 0x7f4ed81a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.749+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1367077145 >> 172.21.15.22:0/1367077145 conn(0x7f4ed80fd8a0 msgr2=0x7f4ed80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.749+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1367077145 shutdown_connections 2024-04-20T21:43:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.749+0000 7f4ed7e00640 1 -- 172.21.15.22:0/1367077145 wait complete. 2024-04-20T21:43:58.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-20T21:43:58.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.973+0000 7f2dd4800640 1 -- 172.21.15.22:0/735674809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dcc108f30 msgr2=0x7f2dcc1153f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:58.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.973+0000 7f2dd4800640 1 --2- 172.21.15.22:0/735674809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dcc108f30 0x7f2dcc1153f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009be0 tx=0x7f2dc802bef0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.973+0000 7f2dd4800640 1 -- 172.21.15.22:0/735674809 shutdown_connections 2024-04-20T21:43:58.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.973+0000 7f2dd4800640 1 --2- 172.21.15.22:0/735674809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dcc108f30 0x7f2dcc1153f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.973+0000 7f2dd4800640 1 --2- 172.21.15.22:0/735674809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dcc108570 0x7f2dcc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.973+0000 7f2dd4800640 1 --2- 172.21.15.22:0/735674809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dcc10e630 0x7f2dcc10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.973+0000 7f2dd4800640 1 -- 172.21.15.22:0/735674809 >> 172.21.15.22:0/735674809 conn(0x7f2dcc0764f0 msgr2=0x7f2dcc078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:58.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.973+0000 7f2dd4800640 1 -- 172.21.15.22:0/735674809 shutdown_connections 2024-04-20T21:43:58.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.973+0000 7f2dd4800640 1 -- 172.21.15.22:0/735674809 wait complete. 2024-04-20T21:43:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.974+0000 7f2dd4800640 1 Processor -- start 2024-04-20T21:43:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.974+0000 7f2dd4800640 1 -- start start 2024-04-20T21:43:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.975+0000 7f2dd4800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dcc108570 0x7f2dcc1a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.975+0000 7f2dd4800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dcc108f30 0x7f2dcc1a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.975+0000 7f2dd4800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dcc10e630 0x7f2dcc19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.975+0000 7f2dd4800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc118940 con 0x7f2dcc10e630 2024-04-20T21:43:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.975+0000 7f2dd4800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc1187c0 con 0x7f2dcc108570 2024-04-20T21:43:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.975+0000 7f2dd4800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc118ac0 con 0x7f2dcc108f30 2024-04-20T21:43:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.975+0000 7f2dd2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dcc10e630 0x7f2dcc19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:58.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.975+0000 7f2dd2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dcc10e630 0x7f2dcc19eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42266/0 (socket says 172.21.15.22:42266) 2024-04-20T21:43:58.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.975+0000 7f2dd2800640 1 -- 172.21.15.22:0/3235699567 learned_addr learned my addr 172.21.15.22:0/3235699567 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:43:58.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.975+0000 7f2dd1400640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dcc108f30 0x7f2dcc1a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:58.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.975+0000 7f2dd1e00640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dcc108570 0x7f2dcc1a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:58.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.976+0000 7f2dd2800640 1 -- 172.21.15.22:0/3235699567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dcc108f30 msgr2=0x7f2dcc1a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:58.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.976+0000 7f2dd2800640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dcc108f30 0x7f2dcc1a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.976+0000 7f2dd2800640 1 -- 172.21.15.22:0/3235699567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dcc108570 msgr2=0x7f2dcc1a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:58.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.976+0000 7f2dd2800640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dcc108570 0x7f2dcc1a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:58.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.976+0000 7f2dd2800640 1 -- 172.21.15.22:0/3235699567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dcc19f260 con 0x7f2dcc10e630 2024-04-20T21:43:58.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.976+0000 7f2dd2800640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dcc10e630 0x7f2dcc19eae0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8033420 tx=0x7f2dc80336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:58.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.976+0000 7f2dc6a00640 1 -- 172.21.15.22:0/3235699567 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8043020 con 0x7f2dcc10e630 2024-04-20T21:43:58.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.976+0000 7f2dc6a00640 1 -- 172.21.15.22:0/3235699567 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc8003e50 con 0x7f2dcc10e630 2024-04-20T21:43:58.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.976+0000 7f2dc6a00640 1 -- 172.21.15.22:0/3235699567 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc803e6b0 con 0x7f2dcc10e630 2024-04-20T21:43:58.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.976+0000 7f2dd4800640 1 -- 172.21.15.22:0/3235699567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dcc19f4f0 con 0x7f2dcc10e630 2024-04-20T21:43:58.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.976+0000 7f2dd4800640 1 -- 172.21.15.22:0/3235699567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dcc1ab7f0 con 0x7f2dcc10e630 2024-04-20T21:43:58.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.977+0000 7f2dd4800640 1 -- 172.21.15.22:0/3235699567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dcc04fcf0 con 0x7f2dcc10e630 2024-04-20T21:43:58.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.979+0000 7f2dc6a00640 1 -- 172.21.15.22:0/3235699567 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2dc8046020 con 0x7f2dcc10e630 2024-04-20T21:43:58.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.981+0000 7f2dc6a00640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2db40a3e40 0x7f2db40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:58.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.981+0000 7f2dc6a00640 1 -- 172.21.15.22:0/3235699567 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8600+0+0 (secure 0 0 0) 0x7f2dc80f2c30 con 0x7f2dcc10e630 2024-04-20T21:43:58.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.981+0000 7f2dd1e00640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2db40a3e40 0x7f2db40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:58.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.982+0000 7f2dd1e00640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2db40a3e40 0x7f2db40a6300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005fd0 tx=0x7f2dc0009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:58.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:58.985+0000 7f2dc6a00640 1 -- 172.21.15.22:0/3235699567 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2dc804d020 con 0x7f2dcc10e630 2024-04-20T21:43:59.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.205+0000 7f2dd4800640 1 -- 172.21.15.22:0/3235699567 --> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2dcc0781e0 con 0x7f2db40a3e40 2024-04-20T21:43:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.219+0000 7f2dc6a00640 1 -- 172.21.15.22:0/3235699567 <== mgr.15024 v2:172.21.15.22:6800/975472489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2dcc0781e0 con 0x7f2db40a3e40 2024-04-20T21:43:59.218 INFO:teuthology.orchestra.run.smithi022.stdout:/ 1m 6m3h 2024-04-20T21:43:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.221+0000 7f2dd4800640 1 -- 172.21.15.22:0/3235699567 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2db40a3e40 msgr2=0x7f2db40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.221+0000 7f2dd4800640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2db40a3e40 0x7f2db40a6300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005fd0 tx=0x7f2dc0009260 comp rx=0 tx=0).stop 2024-04-20T21:43:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.221+0000 7f2dd4800640 1 -- 172.21.15.22:0/3235699567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dcc10e630 msgr2=0x7f2dcc19eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.221+0000 7f2dd4800640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dcc10e630 0x7f2dcc19eae0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8033420 tx=0x7f2dc80336b0 comp rx=0 tx=0).stop 2024-04-20T21:43:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.222+0000 7f2dd4800640 1 -- 172.21.15.22:0/3235699567 shutdown_connections 2024-04-20T21:43:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.222+0000 7f2dd4800640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f2db40a3e40 0x7f2db40a6300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.222+0000 7f2dd4800640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2dcc10e630 0x7f2dcc19eae0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.222+0000 7f2dd4800640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dcc108f30 0x7f2dcc1a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.222+0000 7f2dd4800640 1 --2- 172.21.15.22:0/3235699567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2dcc108570 0x7f2dcc1a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:59.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.222+0000 7f2dd4800640 1 -- 172.21.15.22:0/3235699567 >> 172.21.15.22:0/3235699567 conn(0x7f2dcc0764f0 msgr2=0x7f2dcc077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:59.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.222+0000 7f2dd4800640 1 -- 172.21.15.22:0/3235699567 shutdown_connections 2024-04-20T21:43:59.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:43:59.222+0000 7f2dd4800640 1 -- 172.21.15.22:0/3235699567 wait complete. 2024-04-20T21:43:59.239 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-20T21:43:59.250 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-20T21:44:00.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:00 smithi122 ceph-mon[22147]: from='client.26887 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:44:00.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:00 smithi122 ceph-mon[22147]: pgmap v316: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:44:00.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:00 smithi122 ceph-mon[22147]: from='client.16716 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:44:00.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:00 smithi033 ceph-mon[21626]: from='client.26887 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:44:00.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:00 smithi033 ceph-mon[21626]: pgmap v316: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:44:00.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:00 smithi033 ceph-mon[21626]: from='client.16716 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:44:00.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:00 smithi022 ceph-mon[16843]: from='client.26887 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:44:00.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:00 smithi022 ceph-mon[16843]: pgmap v316: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T21:44:00.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:00 smithi022 ceph-mon[16843]: from='client.16716 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:44:02.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:02 smithi122 ceph-mon[22147]: pgmap v317: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-20T21:44:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:02 smithi033 ceph-mon[21626]: pgmap v317: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-20T21:44:02.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:02 smithi022 ceph-mon[16843]: pgmap v317: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-20T21:44:04.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:04 smithi122 ceph-mon[22147]: pgmap v318: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-20T21:44:04.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:04 smithi033 ceph-mon[21626]: pgmap v318: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-20T21:44:04.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:04 smithi022 ceph-mon[16843]: pgmap v318: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-20T21:44:06.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:06 smithi033 ceph-mon[21626]: pgmap v319: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-20T21:44:06.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:06 smithi022 ceph-mon[16843]: pgmap v319: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-20T21:44:06.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:06 smithi122 ceph-mon[22147]: pgmap v319: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-20T21:44:07.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:07 smithi033 ceph-mon[21626]: osdmap e102: 12 total, 12 up, 12 in 2024-04-20T21:44:07.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:07 smithi022 ceph-mon[16843]: osdmap e102: 12 total, 12 up, 12 in 2024-04-20T21:44:07.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:07 smithi122 ceph-mon[22147]: osdmap e102: 12 total, 12 up, 12 in 2024-04-20T21:44:08.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:08 smithi033 ceph-mon[21626]: pgmap v321: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-04-20T21:44:08.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:08 smithi022 ceph-mon[16843]: pgmap v321: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-04-20T21:44:08.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:08 smithi122 ceph-mon[22147]: pgmap v321: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-04-20T21:44:10.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:10 smithi022 ceph-mon[16843]: pgmap v322: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-04-20T21:44:10.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:10 smithi122 ceph-mon[22147]: pgmap v322: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-04-20T21:44:10.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:10 smithi033 ceph-mon[21626]: pgmap v322: 97 pgs: 97 active+clean; 302 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-04-20T21:44:12.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:12 smithi022 ceph-mon[16843]: pgmap v323: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:44:12.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:12 smithi122 ceph-mon[22147]: pgmap v323: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:44:12.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:12 smithi033 ceph-mon[21626]: pgmap v323: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:44:14.577 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:14 smithi122 ceph-mon[22147]: pgmap v324: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:44:14.577 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:14 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:44:14.577 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:14 smithi122 ceph-mon[22147]: osdmap e103: 12 total, 12 up, 12 in 2024-04-20T21:44:14.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:14 smithi022 ceph-mon[16843]: pgmap v324: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:44:14.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:14 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:44:14.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:14 smithi022 ceph-mon[16843]: osdmap e103: 12 total, 12 up, 12 in 2024-04-20T21:44:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:14 smithi033 ceph-mon[21626]: pgmap v324: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-20T21:44:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:14 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:44:14.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:14 smithi033 ceph-mon[21626]: osdmap e103: 12 total, 12 up, 12 in 2024-04-20T21:44:15.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:15 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:15.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:15 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:15.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:15 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:15.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:15 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:15.743 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:15 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:15.744 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:15 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:15.744 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:15 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:15.744 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:15 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:15.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:15 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:15.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:15 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:15.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:15 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:15.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:15 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:16.684 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:16 smithi122 ceph-mon[22147]: pgmap v326: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 231 B/s wr, 2 op/s 2024-04-20T21:44:16.684 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:16 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:16.685 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:16 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:16.685 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:16 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:16.685 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:16 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:16.685 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:16 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:16.685 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:16 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:16.685 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:16 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:16.685 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:16 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:16 smithi033 ceph-mon[21626]: pgmap v326: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 231 B/s wr, 2 op/s 2024-04-20T21:44:16.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:16 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:16.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:16 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:16.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:16 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:16.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:16 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:16.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:16 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:16.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:16 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:16.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:16 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:16.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:16 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:16.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:16 smithi022 ceph-mon[16843]: pgmap v326: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 231 B/s wr, 2 op/s 2024-04-20T21:44:16.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:16 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:16.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:16 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:16.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:16 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:16.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:16 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:16.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:16 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:16.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:16 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:16.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:16 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:16.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:16 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:17 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-20T21:44:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:17 smithi122 ceph-mon[22147]: Unable to set osd_memory_target on smithi122 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-20T21:44:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:17 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:17 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:17 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:17.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:17 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:17.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:17 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-20T21:44:17.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:17 smithi033 ceph-mon[21626]: Unable to set osd_memory_target on smithi122 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-20T21:44:17.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:17 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:17.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:17 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:17.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:17 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:17.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:17 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:17.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:17 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-20T21:44:17.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:17 smithi022 ceph-mon[16843]: Unable to set osd_memory_target on smithi122 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-20T21:44:17.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:17 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:17.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:17 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:17.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:17 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:17.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:17 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:18 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-20T21:44:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:18 smithi122 ceph-mon[22147]: Unable to set osd_memory_target on smithi033 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-20T21:44:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:18 smithi122 ceph-mon[22147]: pgmap v327: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:44:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:18 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:18.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:18 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:18.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:18 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-20T21:44:18.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:18 smithi033 ceph-mon[21626]: Unable to set osd_memory_target on smithi033 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-20T21:44:18.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:18 smithi033 ceph-mon[21626]: pgmap v327: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:44:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:18 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:18.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:18 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:18.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:18 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-20T21:44:18.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:18 smithi022 ceph-mon[16843]: Unable to set osd_memory_target on smithi033 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-20T21:44:18.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:18 smithi022 ceph-mon[16843]: pgmap v327: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:44:18.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:18 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:18.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:18 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:19.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:19 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:19.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:19 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:19.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:19 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:19.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:19 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:19.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:19 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:44:19.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:19 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:44:19.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:19 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:19.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:19 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:19.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:19 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:19.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:19 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:19.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:19 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:19.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:19 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:44:19.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:19 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:44:19.804 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:19 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:19.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:19 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:19.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:19 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:19.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:19 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:19.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:19 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:44:19.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:19 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:44:19.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:19 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:44:19.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:19 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' 2024-04-20T21:44:19.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:44:19.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:44:20.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.223+0000 7f5771400640 1 -- 172.21.15.22:0/2673826154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c1038e0 msgr2=0x7f576c10bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.223+0000 7f5771400640 1 --2- 172.21.15.22:0/2673826154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c1038e0 0x7f576c10bdc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f575c009900 tx=0x7f575c02bcc0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.223+0000 7f5771400640 1 -- 172.21.15.22:0/2673826154 shutdown_connections 2024-04-20T21:44:20.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.223+0000 7f5771400640 1 --2- 172.21.15.22:0/2673826154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c10c3e0 0x7f576c10e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.223+0000 7f5771400640 1 --2- 172.21.15.22:0/2673826154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c1038e0 0x7f576c10bdc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.223+0000 7f5771400640 1 --2- 172.21.15.22:0/2673826154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f576c102fc0 0x7f576c1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.223+0000 7f5771400640 1 -- 172.21.15.22:0/2673826154 >> 172.21.15.22:0/2673826154 conn(0x7f576c0fc7e0 msgr2=0x7f576c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:20.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.223+0000 7f5771400640 1 -- 172.21.15.22:0/2673826154 shutdown_connections 2024-04-20T21:44:20.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.223+0000 7f5771400640 1 -- 172.21.15.22:0/2673826154 wait complete. 2024-04-20T21:44:20.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.224+0000 7f5771400640 1 Processor -- start 2024-04-20T21:44:20.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.224+0000 7f5771400640 1 -- start start 2024-04-20T21:44:20.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.225+0000 7f5771400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c102fc0 0x7f576c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.225+0000 7f5771400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c1038e0 0x7f576c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.225+0000 7f5771400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f576c10c3e0 0x7f576c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.225+0000 7f5771400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1121c0 con 0x7f576c102fc0 2024-04-20T21:44:20.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.225+0000 7f5771400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f576c112040 con 0x7f576c1038e0 2024-04-20T21:44:20.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.225+0000 7f5771400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f576c112340 con 0x7f576c10c3e0 2024-04-20T21:44:20.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.225+0000 7f576be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c102fc0 0x7f576c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.225+0000 7f576be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c102fc0 0x7f576c1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44094/0 (socket says 172.21.15.22:44094) 2024-04-20T21:44:20.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.225+0000 7f576b400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c1038e0 0x7f576c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.225+0000 7f576be00640 1 -- 172.21.15.22:0/1163579570 learned_addr learned my addr 172.21.15.22:0/1163579570 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:20.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.225+0000 7f5770a00640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f576c10c3e0 0x7f576c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.226+0000 7f576b400640 1 -- 172.21.15.22:0/1163579570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f576c10c3e0 msgr2=0x7f576c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.226+0000 7f576b400640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f576c10c3e0 0x7f576c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.226+0000 7f576b400640 1 -- 172.21.15.22:0/1163579570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c102fc0 msgr2=0x7f576c1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.226+0000 7f576b400640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c102fc0 0x7f576c1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.226+0000 7f576b400640 1 -- 172.21.15.22:0/1163579570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c19afe0 con 0x7f576c1038e0 2024-04-20T21:44:20.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.226+0000 7f576be00640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c102fc0 0x7f576c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:20.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.226+0000 7f576b400640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c1038e0 0x7f576c1a0ba0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f575c000c00 tx=0x7f575c02cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:20.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.226+0000 7f5768c00640 1 -- 172.21.15.22:0/1163579570 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c043020 con 0x7f576c1038e0 2024-04-20T21:44:20.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.226+0000 7f5771400640 1 -- 172.21.15.22:0/1163579570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c19b270 con 0x7f576c1038e0 2024-04-20T21:44:20.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.226+0000 7f5768c00640 1 -- 172.21.15.22:0/1163579570 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f575c0054e0 con 0x7f576c1038e0 2024-04-20T21:44:20.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.227+0000 7f5768c00640 1 -- 172.21.15.22:0/1163579570 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c003a10 con 0x7f576c1038e0 2024-04-20T21:44:20.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.227+0000 7f5771400640 1 -- 172.21.15.22:0/1163579570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c1a75e0 con 0x7f576c1038e0 2024-04-20T21:44:20.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.230+0000 7f577581f740 1 Processor -- start 2024-04-20T21:44:20.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.230+0000 7f5768c00640 1 -- 172.21.15.22:0/1163579570 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f575c046020 con 0x7f576c1038e0 2024-04-20T21:44:20.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.230+0000 7f577581f740 1 -- start start 2024-04-20T21:44:20.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.230+0000 7f577581f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e28380 0x55e7c0e28760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.230+0000 7f577581f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e31340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.230+0000 7f577581f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e31880 0x55e7c0e33c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.230+0000 7f577581f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e7c0d8e150 con 0x55e7c0e31880 2024-04-20T21:44:20.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.230+0000 7f577581f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e7c0d34370 con 0x55e7c0e28ca0 2024-04-20T21:44:20.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.230+0000 7f577581f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e7c0d8a110 con 0x55e7c0e28380 2024-04-20T21:44:20.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.231+0000 7f576b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e31880 0x55e7c0e33c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.231+0000 7f576be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e31340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.231+0000 7f576be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e31340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50834/0 (socket says 172.21.15.22:50834) 2024-04-20T21:44:20.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.231+0000 7f576be00640 1 -- 172.21.15.22:0/495449232 learned_addr learned my addr 172.21.15.22:0/495449232 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:20.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.231+0000 7f5770a00640 1 --2- 172.21.15.22:0/495449232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e28380 0x55e7c0e28760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.231+0000 7f576be00640 1 -- 172.21.15.22:0/495449232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e28380 msgr2=0x55e7c0e28760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:20.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.231+0000 7f576be00640 1 --2- 172.21.15.22:0/495449232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e28380 0x55e7c0e28760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.231+0000 7f576be00640 1 -- 172.21.15.22:0/495449232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e31880 msgr2=0x55e7c0e33c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:20.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.231+0000 7f576be00640 1 --2- 172.21.15.22:0/495449232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e31880 0x55e7c0e33c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.231+0000 7f576be00640 1 -- 172.21.15.22:0/495449232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7c0b6a7d0 con 0x55e7c0e28ca0 2024-04-20T21:44:20.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.232+0000 7f5768c00640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57500a3d20 0x7f57500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.232+0000 7f576be00640 1 --2- 172.21.15.22:0/495449232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e31340 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5754011630 tx=0x7f5754011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=999d35cfd9ffb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:20.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.232+0000 7f5768c00640 1 -- 172.21.15.22:0/1163579570 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f575c0f2590 con 0x7f576c1038e0 2024-04-20T21:44:20.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.232+0000 7f5770a00640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57500a3d20 0x7f57500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.232+0000 7f5761200640 1 -- 172.21.15.22:0/495449232 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575401e070 con 0x55e7c0e28ca0 2024-04-20T21:44:20.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.232+0000 7f5761200640 1 -- 172.21.15.22:0/495449232 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f575400fbd0 con 0x55e7c0e28ca0 2024-04-20T21:44:20.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.232+0000 7f5761200640 1 -- 172.21.15.22:0/495449232 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754019d00 con 0x55e7c0e28ca0 2024-04-20T21:44:20.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.233+0000 7f5770a00640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57500a3d20 0x7f57500a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5764005fd0 tx=0x7f5764009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:20.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.233+0000 7f577581f740 1 -- 172.21.15.22:0/495449232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 msgr2=0x55e7c0e31340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.233+0000 7f577581f740 1 --2- 172.21.15.22:0/495449232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e31340 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5754011630 tx=0x7f5754011af0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.233+0000 7f577581f740 1 -- 172.21.15.22:0/495449232 shutdown_connections 2024-04-20T21:44:20.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.233+0000 7f577581f740 1 --2- 172.21.15.22:0/495449232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e31880 0x55e7c0e33c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.233+0000 7f577581f740 1 --2- 172.21.15.22:0/495449232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e31340 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.233+0000 7f577581f740 1 --2- 172.21.15.22:0/495449232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e28380 0x55e7c0e28760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.233+0000 7f577581f740 1 -- 172.21.15.22:0/495449232 >> 172.21.15.22:0/495449232 conn(0x55e7c0e22ab0 msgr2=0x55e7c0e22e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:20.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.233+0000 7f577581f740 1 -- 172.21.15.22:0/495449232 shutdown_connections 2024-04-20T21:44:20.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.233+0000 7f577581f740 1 -- 172.21.15.22:0/495449232 wait complete. 2024-04-20T21:44:20.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.233+0000 7f577581f740 1 -- start start 2024-04-20T21:44:20.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.234+0000 7f577581f740 1 Processor -- start 2024-04-20T21:44:20.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.234+0000 7f577581f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28380 0x55e7c0e2e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.234+0000 7f577581f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e2edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.234+0000 7f577581f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e31880 0x55e7c0e2f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.234+0000 7f577581f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e7c0e34670 con 0x55e7c0e28380 2024-04-20T21:44:20.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.234+0000 7f577581f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e7c0d47740 con 0x55e7c0e28ca0 2024-04-20T21:44:20.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.234+0000 7f577581f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e7c0e347f0 con 0x55e7c0e31880 2024-04-20T21:44:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.234+0000 7f577581f740 10 client.0 _ll_register_callbacks cb 0x55e7c0d42b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:44:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.234+0000 7f576be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28380 0x55e7c0e2e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.234+0000 7f576be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28380 0x55e7c0e2e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44124/0 (socket says 172.21.15.22:44124) 2024-04-20T21:44:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.234+0000 7f576be00640 1 -- 172.21.15.22:0/3477017294 learned_addr learned my addr 172.21.15.22:0/3477017294 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.235+0000 7f5770a00640 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e31880 0x55e7c0e2f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.235+0000 7f576b400640 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e2edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.235+0000 7f576be00640 1 -- 172.21.15.22:0/3477017294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e31880 msgr2=0x55e7c0e2f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.235+0000 7f576be00640 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e31880 0x55e7c0e2f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.235+0000 7f576be00640 1 -- 172.21.15.22:0/3477017294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 msgr2=0x55e7c0e2edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.235+0000 7f576be00640 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e2edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.235+0000 7f576be00640 1 -- 172.21.15.22:0/3477017294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7c0aa03f0 con 0x55e7c0e28380 2024-04-20T21:44:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.235+0000 7f576be00640 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28380 0x55e7c0e2e8b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f575400ffe0 tx=0x7f575400fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae75e9e244c70178 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.235+0000 7f5759600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:44:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.236+0000 7f5759600640 1 -- 172.21.15.22:0/3477017294 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575401e040 con 0x55e7c0e28380 2024-04-20T21:44:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.236+0000 7f5759600640 1 -- 172.21.15.22:0/3477017294 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f5754009ce0 con 0x55e7c0e28380 2024-04-20T21:44:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.236+0000 7f577581f740 1 -- 172.21.15.22:0/3477017294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f575400fbd0 con 0x55e7c0e28380 2024-04-20T21:44:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.236+0000 7f5759600640 1 -- 172.21.15.22:0/3477017294 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5754030050 con 0x55e7c0e28380 2024-04-20T21:44:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.236+0000 7f577581f740 10 client.16728 fetch_fsmap learned FSMap version 20 2024-04-20T21:44:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.236+0000 7f577581f740 1 -- 172.21.15.22:0/3477017294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e7c0a600f0 con 0x55e7c0e28380 2024-04-20T21:44:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.236+0000 7f5759600640 1 -- 172.21.15.22:0/3477017294 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f575400fbd0 con 0x55e7c0e28380 2024-04-20T21:44:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.236+0000 7f5759600640 1 -- 172.21.15.22:0/3477017294 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f5754021070 con 0x55e7c0e28380 2024-04-20T21:44:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.236+0000 7f577581f740 10 client.16728 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:44:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.236+0000 7f577581f740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:44:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.237+0000 7f577581f740 20 client.16728 populate_metadata read hostname 'smithi022' 2024-04-20T21:44:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.237+0000 7f577581f740 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e7c0e41b40 0x55e7c0e43f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.237+0000 7f577581f740 4 client.16728 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:44:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.237+0000 7f577581f740 1 -- 172.21.15.22:0/3477017294 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7c0d9bb70 con 0x55e7c0e41b40 2024-04-20T21:44:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.237+0000 7f576b400640 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e7c0e41b40 0x55e7c0e43f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.237+0000 7f576b400640 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e7c0e41b40 0x55e7c0e43f30 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=248dd8fe270b5c37 server_cookie=eaf7298a65b7a4b0 in_seq=0 out_seq=0 2024-04-20T21:44:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.237+0000 7f5759600640 10 client.16728 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:44:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.238+0000 7f5759600640 1 -- 172.21.15.22:0/3477017294 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7c0d9bb70 con 0x55e7c0e41b40 2024-04-20T21:44:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.238+0000 7f5759600640 10 client.16728 handle_command_reply: tid=0 2024-04-20T21:44:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f577581f740 1 client.16728 shutdown 2024-04-20T21:44:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f577581f740 1 -- 172.21.15.22:0/3477017294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28380 msgr2=0x55e7c0e2e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f577581f740 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28380 0x55e7c0e2e8b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f575400ffe0 tx=0x7f575400fb00 comp rx=0 tx=0).stop 2024-04-20T21:44:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f577581f740 1 -- 172.21.15.22:0/3477017294 shutdown_connections 2024-04-20T21:44:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f577581f740 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e31880 0x55e7c0e2f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f577581f740 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e7c0e41b40 0x55e7c0e43f30 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f577581f740 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e2edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f577581f740 1 --2- 172.21.15.22:0/3477017294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28380 0x55e7c0e2e8b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f577581f740 1 -- 172.21.15.22:0/3477017294 >> 172.21.15.22:0/3477017294 conn(0x55e7c0e22ab0 msgr2=0x55e7c0e24800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f5759600640 0 client.16728 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:44:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f577581f740 1 -- 172.21.15.22:0/3477017294 shutdown_connections 2024-04-20T21:44:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f577581f740 1 -- 172.21.15.22:0/3477017294 wait complete. 2024-04-20T21:44:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.239+0000 7f577581f740 20 client.16728 trim_cache size 0 max 16384 2024-04-20T21:44:20.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.258+0000 7f577581f740 1 Processor -- start 2024-04-20T21:44:20.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.259+0000 7f577581f740 1 -- start start 2024-04-20T21:44:20.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.259+0000 7f577581f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28380 0x55e7c0e30fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.259+0000 7f577581f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e3d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.259+0000 7f577581f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e31880 0x55e7c0e496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.259+0000 7f577581f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1ade20 con 0x55e7c0e28380 2024-04-20T21:44:20.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.259+0000 7f577581f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1b0680 con 0x55e7c0e28ca0 2024-04-20T21:44:20.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.259+0000 7f577581f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1ae470 con 0x55e7c0e31880 2024-04-20T21:44:20.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.259+0000 7f5770a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e31880 0x55e7c0e496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.259+0000 7f5770a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e31880 0x55e7c0e496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:47098/0 (socket says 172.21.15.22:47098) 2024-04-20T21:44:20.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.259+0000 7f5770a00640 1 -- 172.21.15.22:0/3052294364 learned_addr learned my addr 172.21.15.22:0/3052294364 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:20.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.259+0000 7f576b400640 1 --2- 172.21.15.22:0/3052294364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e3d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.259+0000 7f576be00640 1 --2- 172.21.15.22:0/3052294364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28380 0x55e7c0e30fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.260+0000 7f5770a00640 1 -- 172.21.15.22:0/3052294364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 msgr2=0x55e7c0e3d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:20.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.260+0000 7f5770a00640 1 --2- 172.21.15.22:0/3052294364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e3d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.260+0000 7f5770a00640 1 -- 172.21.15.22:0/3052294364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28380 msgr2=0x55e7c0e30fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:20.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.260+0000 7f5770a00640 1 --2- 172.21.15.22:0/3052294364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28380 0x55e7c0e30fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.260+0000 7f5770a00640 1 -- 172.21.15.22:0/3052294364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7c0e442a0 con 0x55e7c0e31880 2024-04-20T21:44:20.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.260+0000 7f5770a00640 1 --2- 172.21.15.22:0/3052294364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e31880 0x55e7c0e496f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5764003ee0 tx=0x7f576401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=786564d82febb3d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:20.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.261+0000 7f575be00640 1 -- 172.21.15.22:0/3052294364 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764004b20 con 0x55e7c0e31880 2024-04-20T21:44:20.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.261+0000 7f575be00640 1 -- 172.21.15.22:0/3052294364 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5764004cc0 con 0x55e7c0e31880 2024-04-20T21:44:20.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.261+0000 7f575be00640 1 -- 172.21.15.22:0/3052294364 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57640054c0 con 0x55e7c0e31880 2024-04-20T21:44:20.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.261+0000 7f577581f740 1 -- 172.21.15.22:0/3052294364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e31880 msgr2=0x55e7c0e496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.261+0000 7f577581f740 1 --2- 172.21.15.22:0/3052294364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e31880 0x55e7c0e496f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5764003ee0 tx=0x7f576401a000 comp rx=0 tx=0).stop 2024-04-20T21:44:20.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.261+0000 7f577581f740 1 -- 172.21.15.22:0/3052294364 shutdown_connections 2024-04-20T21:44:20.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.261+0000 7f577581f740 1 --2- 172.21.15.22:0/3052294364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e31880 0x55e7c0e496f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.261+0000 7f577581f740 1 --2- 172.21.15.22:0/3052294364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e3d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.261+0000 7f577581f740 1 --2- 172.21.15.22:0/3052294364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28380 0x55e7c0e30fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.261+0000 7f577581f740 1 -- 172.21.15.22:0/3052294364 >> 172.21.15.22:0/3052294364 conn(0x55e7c0e22ab0 msgr2=0x55e7c0e2d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:20.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.262+0000 7f577581f740 1 -- 172.21.15.22:0/3052294364 shutdown_connections 2024-04-20T21:44:20.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.262+0000 7f577581f740 1 -- 172.21.15.22:0/3052294364 wait complete. 2024-04-20T21:44:20.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.262+0000 7f577581f740 1 -- start start 2024-04-20T21:44:20.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.262+0000 7f577581f740 1 Processor -- start 2024-04-20T21:44:20.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f577581f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e28380 0x55e7c0e34c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f577581f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e35170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f577581f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e31880 0x55e7c0ea5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f577581f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1aeb40 con 0x55e7c0e28ca0 2024-04-20T21:44:20.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f577581f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1affb0 con 0x55e7c0e31880 2024-04-20T21:44:20.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f577581f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1af210 con 0x55e7c0e28380 2024-04-20T21:44:20.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f577581f740 10 client.0 _ll_register_callbacks cb 0x7f575401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:44:20.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f576be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e28380 0x55e7c0e34c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f576b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e35170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f576be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e28380 0x55e7c0e34c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:47114/0 (socket says 172.21.15.22:47114) 2024-04-20T21:44:20.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f576be00640 1 -- 172.21.15.22:0/21956301 learned_addr learned my addr 172.21.15.22:0/21956301 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:20.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f5770a00640 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e31880 0x55e7c0ea5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f576be00640 1 -- 172.21.15.22:0/21956301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e31880 msgr2=0x55e7c0ea5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:20.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f576be00640 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e31880 0x55e7c0ea5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f576be00640 1 -- 172.21.15.22:0/21956301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28ca0 msgr2=0x55e7c0e35170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:20.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f576be00640 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e35170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f576be00640 1 -- 172.21.15.22:0/21956301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7c0ea6540 con 0x55e7c0e28380 2024-04-20T21:44:20.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.263+0000 7f5770a00640 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e31880 0x55e7c0ea5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:44:20.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.264+0000 7f576be00640 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e28380 0x55e7c0e34c30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5754010010 tx=0x7f57540095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3874966d7aae415c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:20.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.264+0000 7f5759600640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-20T21:44:20.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.264+0000 7f5759600640 1 -- 172.21.15.22:0/21956301 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754039060 con 0x55e7c0e28380 2024-04-20T21:44:20.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.264+0000 7f577581f740 1 -- 172.21.15.22:0/21956301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5764004cc0 con 0x55e7c0e28380 2024-04-20T21:44:20.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.264+0000 7f5759600640 1 -- 172.21.15.22:0/21956301 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f5754019610 con 0x55e7c0e28380 2024-04-20T21:44:20.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.264+0000 7f5759600640 1 -- 172.21.15.22:0/21956301 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f575401e040 con 0x55e7c0e28380 2024-04-20T21:44:20.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.264+0000 7f577581f740 10 client.26911 fetch_fsmap learned FSMap version 20 2024-04-20T21:44:20.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.264+0000 7f577581f740 1 -- 172.21.15.22:0/21956301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e7c0ea7830 con 0x55e7c0e28380 2024-04-20T21:44:20.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.264+0000 7f5759600640 1 -- 172.21.15.22:0/21956301 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5764004cc0 con 0x55e7c0e28380 2024-04-20T21:44:20.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.265+0000 7f5759600640 1 -- 172.21.15.22:0/21956301 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f5754021040 con 0x55e7c0e28380 2024-04-20T21:44:20.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.265+0000 7f577581f740 10 client.26911 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:44:20.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.265+0000 7f577581f740 10 client.26911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:44:20.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.265+0000 7f577581f740 20 client.26911 populate_metadata read hostname 'smithi022' 2024-04-20T21:44:20.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.265+0000 7f577581f740 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e7c0ea7f20 0x55e7c0eaa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.265+0000 7f577581f740 4 client.26911 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:44:20.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.265+0000 7f577581f740 1 -- 172.21.15.22:0/21956301 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e7c0d959e0 con 0x55e7c0ea7f20 2024-04-20T21:44:20.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.265+0000 7f576b400640 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e7c0ea7f20 0x55e7c0eaa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.265+0000 7f576b400640 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e7c0ea7f20 0x55e7c0eaa340 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98d85c4ce5c87d95 server_cookie=62e984e21681bd08 in_seq=0 out_seq=0 2024-04-20T21:44:20.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.265+0000 7f5759600640 10 client.26911 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:44:20.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f5759600640 1 -- 172.21.15.22:0/21956301 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e7c0d959e0 con 0x55e7c0ea7f20 2024-04-20T21:44:20.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f5759600640 10 client.26911 handle_command_reply: tid=0 2024-04-20T21:44:20.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f577581f740 1 client.26911 shutdown 2024-04-20T21:44:20.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f577581f740 1 -- 172.21.15.22:0/21956301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e28380 msgr2=0x55e7c0e34c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f577581f740 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e28380 0x55e7c0e34c30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5754010010 tx=0x7f57540095b0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f577581f740 1 -- 172.21.15.22:0/21956301 shutdown_connections 2024-04-20T21:44:20.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f577581f740 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e7c0ea7f20 0x55e7c0eaa340 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f577581f740 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7c0e31880 0x55e7c0ea5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f577581f740 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7c0e28ca0 0x55e7c0e35170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f577581f740 1 --2- 172.21.15.22:0/21956301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e7c0e28380 0x55e7c0e34c30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f5759600640 0 client.26911 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:44:20.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f577581f740 1 -- 172.21.15.22:0/21956301 >> 172.21.15.22:0/21956301 conn(0x55e7c0e22ab0 msgr2=0x55e7c0e2d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:20.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f577581f740 1 -- 172.21.15.22:0/21956301 shutdown_connections 2024-04-20T21:44:20.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f577581f740 1 -- 172.21.15.22:0/21956301 wait complete. 2024-04-20T21:44:20.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.266+0000 7f577581f740 20 client.26911 trim_cache size 0 max 16384 2024-04-20T21:44:20.273 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:44:20.273 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-04-20T21:44:20.273 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-04-20T21:44:20.273 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:44:20.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.267+0000 7f5771400640 1 -- 172.21.15.22:0/1163579570 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57500a3d20 msgr2=0x7f57500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.267+0000 7f5771400640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57500a3d20 0x7f57500a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5764005fd0 tx=0x7f5764009500 comp rx=0 tx=0).stop 2024-04-20T21:44:20.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.267+0000 7f5771400640 1 -- 172.21.15.22:0/1163579570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c1038e0 msgr2=0x7f576c1a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.267+0000 7f5771400640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c1038e0 0x7f576c1a0ba0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f575c000c00 tx=0x7f575c02cbc0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.267+0000 7f5771400640 1 -- 172.21.15.22:0/1163579570 shutdown_connections 2024-04-20T21:44:20.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.267+0000 7f5771400640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f57500a3d20 0x7f57500a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.267+0000 7f5771400640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f576c10c3e0 0x7f576c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.267+0000 7f5771400640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c1038e0 0x7f576c1a0ba0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.267+0000 7f5771400640 1 --2- 172.21.15.22:0/1163579570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f576c102fc0 0x7f576c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.267+0000 7f5771400640 1 -- 172.21.15.22:0/1163579570 >> 172.21.15.22:0/1163579570 conn(0x7f576c0fc7e0 msgr2=0x7f576c100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:20.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.267+0000 7f5771400640 1 -- 172.21.15.22:0/1163579570 shutdown_connections 2024-04-20T21:44:20.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.267+0000 7f5771400640 1 -- 172.21.15.22:0/1163579570 wait complete. 2024-04-20T21:44:20.284 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:44:20.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:44:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.497+0000 7f1ce9a00640 1 -- 172.21.15.22:0/874370026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce4109490 msgr2=0x7f1ce4111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.497+0000 7f1ce9a00640 1 --2- 172.21.15.22:0/874370026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce4109490 0x7f1ce4111060 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009900 tx=0x7f1cdc02bc40 comp rx=0 tx=0).stop 2024-04-20T21:44:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.497+0000 7f1ce9a00640 1 -- 172.21.15.22:0/874370026 shutdown_connections 2024-04-20T21:44:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.497+0000 7f1ce9a00640 1 --2- 172.21.15.22:0/874370026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce4109490 0x7f1ce4111060 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.497+0000 7f1ce9a00640 1 --2- 172.21.15.22:0/874370026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce4104d70 0x7f1ce4108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.497+0000 7f1ce9a00640 1 --2- 172.21.15.22:0/874370026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce41043c0 0x7f1ce41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.497+0000 7f1ce9a00640 1 -- 172.21.15.22:0/874370026 >> 172.21.15.22:0/874370026 conn(0x7f1ce40ffc20 msgr2=0x7f1ce4102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:20.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.498+0000 7f1ce9a00640 1 -- 172.21.15.22:0/874370026 shutdown_connections 2024-04-20T21:44:20.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.498+0000 7f1ce9a00640 1 -- 172.21.15.22:0/874370026 wait complete. 2024-04-20T21:44:20.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.499+0000 7f1ce9a00640 1 Processor -- start 2024-04-20T21:44:20.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.499+0000 7f1ce9a00640 1 -- start start 2024-04-20T21:44:20.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.499+0000 7f1ce9a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce41043c0 0x7f1ce41a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.499+0000 7f1ce9a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce4109490 0x7f1ce41a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.499+0000 7f1ce9a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce419a870 0x7f1ce419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.499+0000 7f1ce9a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4114550 con 0x7f1ce41043c0 2024-04-20T21:44:20.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.499+0000 7f1ce9a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41143d0 con 0x7f1ce4109490 2024-04-20T21:44:20.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.499+0000 7f1ce9a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41146d0 con 0x7f1ce419a870 2024-04-20T21:44:20.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce41043c0 0x7f1ce41a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce2000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce4109490 0x7f1ce41a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce419a870 0x7f1ce419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce41043c0 0x7f1ce41a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44170/0 (socket says 172.21.15.22:44170) 2024-04-20T21:44:20.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce2000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce4109490 0x7f1ce41a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50882/0 (socket says 172.21.15.22:50882) 2024-04-20T21:44:20.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce419a870 0x7f1ce419ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:47132/0 (socket says 172.21.15.22:47132) 2024-04-20T21:44:20.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce2a00640 1 -- 172.21.15.22:0/3559145688 learned_addr learned my addr 172.21.15.22:0/3559145688 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:20.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce2a00640 1 -- 172.21.15.22:0/3559145688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce419a870 msgr2=0x7f1ce419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce2a00640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce419a870 0x7f1ce419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce2a00640 1 -- 172.21.15.22:0/3559145688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce4109490 msgr2=0x7f1ce41a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce2a00640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce4109490 0x7f1ce41a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce2a00640 1 -- 172.21.15.22:0/3559145688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce419b390 con 0x7f1ce41043c0 2024-04-20T21:44:20.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce2000640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce4109490 0x7f1ce41a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:20.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.500+0000 7f1ce2a00640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce41043c0 0x7f1ce41a06c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400e9f0 tx=0x7f1cd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:20.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.501+0000 7f1cdb400640 1 -- 172.21.15.22:0/3559145688 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400cd80 con 0x7f1ce41043c0 2024-04-20T21:44:20.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.501+0000 7f1cdb400640 1 -- 172.21.15.22:0/3559145688 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd4010910 con 0x7f1ce41043c0 2024-04-20T21:44:20.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.501+0000 7f1ce9a00640 1 -- 172.21.15.22:0/3559145688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce419b680 con 0x7f1ce41043c0 2024-04-20T21:44:20.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.501+0000 7f1cdb400640 1 -- 172.21.15.22:0/3559145688 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4018990 con 0x7f1ce41043c0 2024-04-20T21:44:20.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.501+0000 7f1ce9a00640 1 -- 172.21.15.22:0/3559145688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce41a78d0 con 0x7f1ce41043c0 2024-04-20T21:44:20.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.502+0000 7f1ce9a00640 1 -- 172.21.15.22:0/3559145688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce4068d80 con 0x7f1ce41043c0 2024-04-20T21:44:20.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.504+0000 7f1cdb400640 1 -- 172.21.15.22:0/3559145688 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1cd4010ac0 con 0x7f1ce41043c0 2024-04-20T21:44:20.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.506+0000 7f1cdb400640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f1cc40a3d20 0x7f1cc40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.507+0000 7f1cdb400640 1 -- 172.21.15.22:0/3559145688 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f1cd40d0b80 con 0x7f1ce41043c0 2024-04-20T21:44:20.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.507+0000 7f1ce2000640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f1cc40a3d20 0x7f1cc40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:20.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.507+0000 7f1ce2000640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f1cc40a3d20 0x7f1cc40a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ce410aab0 tx=0x7f1ccc009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:20.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.512+0000 7f1cdb400640 1 -- 172.21.15.22:0/3559145688 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1cd409cb80 con 0x7f1ce41043c0 2024-04-20T21:44:20.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:20 smithi022 ceph-mon[16843]: Adjusting osd_memory_target on smithi022 to 177.5M 2024-04-20T21:44:20.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:20 smithi022 ceph-mon[16843]: Unable to set osd_memory_target on smithi022 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T21:44:20.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:20 smithi022 ceph-mon[16843]: pgmap v328: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:44:20.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.758+0000 7f1ce9a00640 1 -- 172.21.15.22:0/3559145688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ce4104d70 con 0x7f1ce41043c0 2024-04-20T21:44:20.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.760+0000 7f1cdb400640 1 -- 172.21.15.22:0/3559145688 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f1cd40a0e80 con 0x7f1ce41043c0 2024-04-20T21:44:20.760 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:44:20.760 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:44:20.760 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:44:20.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.763+0000 7f1ce9a00640 1 -- 172.21.15.22:0/3559145688 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f1cc40a3d20 msgr2=0x7f1cc40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.763+0000 7f1ce9a00640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f1cc40a3d20 0x7f1cc40a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ce410aab0 tx=0x7f1ccc009500 comp rx=0 tx=0).stop 2024-04-20T21:44:20.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.763+0000 7f1ce9a00640 1 -- 172.21.15.22:0/3559145688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce41043c0 msgr2=0x7f1ce41a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.763+0000 7f1ce9a00640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce41043c0 0x7f1ce41a06c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400e9f0 tx=0x7f1cd400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.763+0000 7f1ce9a00640 1 -- 172.21.15.22:0/3559145688 shutdown_connections 2024-04-20T21:44:20.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.763+0000 7f1ce9a00640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f1cc40a3d20 0x7f1cc40a61e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.763+0000 7f1ce9a00640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce419a870 0x7f1ce419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.763+0000 7f1ce9a00640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce4109490 0x7f1ce41a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.763+0000 7f1ce9a00640 1 --2- 172.21.15.22:0/3559145688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ce41043c0 0x7f1ce41a06c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.763+0000 7f1ce9a00640 1 -- 172.21.15.22:0/3559145688 >> 172.21.15.22:0/3559145688 conn(0x7f1ce40ffc20 msgr2=0x7f1ce4101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:20.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.763+0000 7f1ce9a00640 1 -- 172.21.15.22:0/3559145688 shutdown_connections 2024-04-20T21:44:20.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.764+0000 7f1ce9a00640 1 -- 172.21.15.22:0/3559145688 wait complete. 2024-04-20T21:44:20.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:20 smithi122 ceph-mon[22147]: Adjusting osd_memory_target on smithi022 to 177.5M 2024-04-20T21:44:20.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:20 smithi122 ceph-mon[22147]: Unable to set osd_memory_target on smithi022 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T21:44:20.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:20 smithi122 ceph-mon[22147]: pgmap v328: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:44:20.783 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:44:20.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:20 smithi033 ceph-mon[21626]: Adjusting osd_memory_target on smithi022 to 177.5M 2024-04-20T21:44:20.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:20 smithi033 ceph-mon[21626]: Unable to set osd_memory_target on smithi022 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T21:44:20.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:20 smithi033 ceph-mon[21626]: pgmap v328: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-20T21:44:20.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.996+0000 7f67b3200640 1 -- 172.21.15.22:0/2438405906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b4109340 msgr2=0x7f67b4110f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:20.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.996+0000 7f67b3200640 1 --2- 172.21.15.22:0/2438405906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b4109340 0x7f67b4110f10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009ca0 tx=0x7f67ac02c100 comp rx=0 tx=0).stop 2024-04-20T21:44:20.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.997+0000 7f67b3200640 1 -- 172.21.15.22:0/2438405906 shutdown_connections 2024-04-20T21:44:20.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.997+0000 7f67b3200640 1 --2- 172.21.15.22:0/2438405906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b4109340 0x7f67b4110f10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.997+0000 7f67b3200640 1 --2- 172.21.15.22:0/2438405906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67b4104c20 0x7f67b4108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.997+0000 7f67b3200640 1 --2- 172.21.15.22:0/2438405906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b4104270 0x7f67b4104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.997+0000 7f67b3200640 1 -- 172.21.15.22:0/2438405906 >> 172.21.15.22:0/2438405906 conn(0x7f67b40ffad0 msgr2=0x7f67b4101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.997+0000 7f67b3200640 1 -- 172.21.15.22:0/2438405906 shutdown_connections 2024-04-20T21:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.997+0000 7f67b3200640 1 -- 172.21.15.22:0/2438405906 wait complete. 2024-04-20T21:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.998+0000 7f67b3200640 1 Processor -- start 2024-04-20T21:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.998+0000 7f67b3200640 1 -- start start 2024-04-20T21:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.998+0000 7f67b3200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b4104270 0x7f67b41a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.998+0000 7f67b3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b4109340 0x7f67b41a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.998+0000 7f67b3200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67b419a680 0x7f67b419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.998+0000 7f67b3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41143e0 con 0x7f67b4109340 2024-04-20T21:44:21.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.998+0000 7f67b3200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f67b4114260 con 0x7f67b4104270 2024-04-20T21:44:21.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.998+0000 7f67b3200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67b4114560 con 0x7f67b419a680 2024-04-20T21:44:21.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.999+0000 7f67abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b4109340 0x7f67b41a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.999+0000 7f67abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b4109340 0x7f67b41a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44194/0 (socket says 172.21.15.22:44194) 2024-04-20T21:44:21.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.999+0000 7f67abe00640 1 -- 172.21.15.22:0/3627848686 learned_addr learned my addr 172.21.15.22:0/3627848686 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:21.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.999+0000 7f67b1e00640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b4104270 0x7f67b41a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.999+0000 7f67b2800640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67b419a680 0x7f67b419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.999+0000 7f67abe00640 1 -- 172.21.15.22:0/3627848686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67b419a680 msgr2=0x7f67b419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:21.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.999+0000 7f67abe00640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67b419a680 0x7f67b419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.999+0000 7f67abe00640 1 -- 172.21.15.22:0/3627848686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b4104270 msgr2=0x7f67b41a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:21.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.999+0000 7f67abe00640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b4104270 0x7f67b41a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:20.999+0000 7f67abe00640 1 -- 172.21.15.22:0/3627848686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b419b280 con 0x7f67b4109340 2024-04-20T21:44:21.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.000+0000 7f67abe00640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b4109340 0x7f67b41a0b00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f679c00e9a0 tx=0x7f679c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:21.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.000+0000 7f67aaa00640 1 -- 172.21.15.22:0/3627848686 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c00cd80 con 0x7f67b4109340 2024-04-20T21:44:21.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.000+0000 7f67aaa00640 1 -- 172.21.15.22:0/3627848686 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f679c0035f0 con 0x7f67b4109340 2024-04-20T21:44:21.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.000+0000 7f67aaa00640 1 -- 172.21.15.22:0/3627848686 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c010670 con 0x7f67b4109340 2024-04-20T21:44:21.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.000+0000 7f67b3200640 1 -- 172.21.15.22:0/3627848686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b41a7360 con 0x7f67b4109340 2024-04-20T21:44:21.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.000+0000 7f67b3200640 1 -- 172.21.15.22:0/3627848686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b41a7760 con 0x7f67b4109340 2024-04-20T21:44:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.003+0000 7f67bb804740 1 Processor -- start 2024-04-20T21:44:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.003+0000 7f67bb804740 1 -- start start 2024-04-20T21:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.003+0000 7f67aaa00640 1 -- 172.21.15.22:0/3627848686 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f679c0037a0 con 0x7f67b4109340 2024-04-20T21:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.004+0000 7f67bb804740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebc3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.004+0000 7f67bb804740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebcc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.004+0000 7f67bb804740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebcea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.004+0000 7f67bb804740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e27eb28050 con 0x55e27ebc3a90 2024-04-20T21:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.004+0000 7f67bb804740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e27eace490 con 0x55e27ebc3170 2024-04-20T21:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.004+0000 7f67bb804740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e27eb24010 con 0x55e27ebcc670 2024-04-20T21:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.004+0000 7f67b1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebcc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.004+0000 7f67b1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebcc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44200/0 (socket says 172.21.15.22:44200) 2024-04-20T21:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.004+0000 7f67b1e00640 1 -- 172.21.15.22:0/3463512082 learned_addr learned my addr 172.21.15.22:0/3463512082 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.004+0000 7f67abe00640 1 --2- 172.21.15.22:0/3463512082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebcea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.004+0000 7f67b2800640 1 --2- 172.21.15.22:0/3463512082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebc3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.005+0000 7f67b1e00640 1 -- 172.21.15.22:0/3463512082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 msgr2=0x55e27ebcea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.005+0000 7f67b1e00640 1 --2- 172.21.15.22:0/3463512082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebcea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.005+0000 7f67b1e00640 1 -- 172.21.15.22:0/3463512082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 msgr2=0x55e27ebc3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.005+0000 7f67b1e00640 1 --2- 172.21.15.22:0/3463512082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebc3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.005+0000 7f67b1e00640 1 -- 172.21.15.22:0/3463512082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27ead6070 con 0x55e27ebc3a90 2024-04-20T21:44:21.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.005+0000 7f67b1e00640 1 --2- 172.21.15.22:0/3463512082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebcc130 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009cf0 tx=0x7f67a4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c32986dcaa22170 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:21.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.005+0000 7f67aaa00640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f67980a3d20 0x7f67980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.005+0000 7f67a3000640 1 -- 172.21.15.22:0/3463512082 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4004230 con 0x55e27ebc3a90 2024-04-20T21:44:21.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.005+0000 7f67a3000640 1 -- 172.21.15.22:0/3463512082 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a40043d0 con 0x55e27ebc3a90 2024-04-20T21:44:21.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.005+0000 7f67a3000640 1 -- 172.21.15.22:0/3463512082 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4019900 con 0x55e27ebc3a90 2024-04-20T21:44:21.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.006+0000 7f67aaa00640 1 -- 172.21.15.22:0/3627848686 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f679c0d0a60 con 0x7f67b4109340 2024-04-20T21:44:21.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.006+0000 7f67b2800640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f67980a3d20 0x7f67980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.006+0000 7f67bb804740 1 -- 172.21.15.22:0/3463512082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 msgr2=0x55e27ebcc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:21.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.006+0000 7f67bb804740 1 --2- 172.21.15.22:0/3463512082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebcc130 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009cf0 tx=0x7f67a4011810 comp rx=0 tx=0).stop 2024-04-20T21:44:21.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.006+0000 7f67bb804740 1 -- 172.21.15.22:0/3463512082 shutdown_connections 2024-04-20T21:44:21.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.006+0000 7f67bb804740 1 --2- 172.21.15.22:0/3463512082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebcea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.006+0000 7f67bb804740 1 --2- 172.21.15.22:0/3463512082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebcc130 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.006+0000 7f67bb804740 1 --2- 172.21.15.22:0/3463512082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebc3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.006+0000 7f67bb804740 1 -- 172.21.15.22:0/3463512082 >> 172.21.15.22:0/3463512082 conn(0x55e27ebbd8a0 msgr2=0x55e27ebbdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:21.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.006+0000 7f67bb804740 1 -- 172.21.15.22:0/3463512082 shutdown_connections 2024-04-20T21:44:21.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.007+0000 7f67b2800640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f67980a3d20 0x7f67980a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f67ac005e30 tx=0x7f67ac005d80 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:21.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.007+0000 7f67bb804740 1 -- 172.21.15.22:0/3463512082 wait complete. 2024-04-20T21:44:21.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.007+0000 7f67bb804740 1 -- start start 2024-04-20T21:44:21.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.007+0000 7f67bb804740 1 Processor -- start 2024-04-20T21:44:21.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67bb804740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebc9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67bb804740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebc9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67bb804740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67bb804740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e27ebcf460 con 0x55e27ebc3a90 2024-04-20T21:44:21.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67bb804740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e27eae1860 con 0x55e27ebc3170 2024-04-20T21:44:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67bb804740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e27ebcf5e0 con 0x55e27ebcc670 2024-04-20T21:44:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67bb804740 10 client.0 _ll_register_callbacks cb 0x55e27eaaffe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:44:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67b2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebc9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67b1e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebc9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67b2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:47172/0 (socket says 172.21.15.22:47172) 2024-04-20T21:44:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67b2800640 1 -- 172.21.15.22:0/766142898 learned_addr learned my addr 172.21.15.22:0/766142898 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67b1e00640 1 -- 172.21.15.22:0/766142898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 msgr2=0x55e27ebca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67b1e00640 1 --2- 172.21.15.22:0/766142898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67b1e00640 1 -- 172.21.15.22:0/766142898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 msgr2=0x55e27ebc9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67b1e00640 1 --2- 172.21.15.22:0/766142898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebc9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67b1e00640 1 -- 172.21.15.22:0/766142898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27eae5950 con 0x55e27ebc3170 2024-04-20T21:44:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.008+0000 7f67abe00640 1 --2- 172.21.15.22:0/766142898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebc9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.009+0000 7f67b1e00640 1 --2- 172.21.15.22:0/766142898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebc9690 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f67a4019460 tx=0x7f67a40038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=592b9ce197e41fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.009+0000 7f6797400640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-20T21:44:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.009+0000 7f6797400640 1 -- 172.21.15.22:0/766142898 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4009ea0 con 0x55e27ebc3170 2024-04-20T21:44:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.009+0000 7f6797400640 1 -- 172.21.15.22:0/766142898 <== mon.2 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f67a4005240 con 0x55e27ebc3170 2024-04-20T21:44:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.009+0000 7f67bb804740 1 -- 172.21.15.22:0/766142898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a40043d0 con 0x55e27ebc3170 2024-04-20T21:44:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.010+0000 7f6797400640 1 -- 172.21.15.22:0/766142898 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f67a4032050 con 0x55e27ebc3170 2024-04-20T21:44:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.010+0000 7f67bb804740 10 client.26900 fetch_fsmap learned FSMap version 20 2024-04-20T21:44:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.010+0000 7f67bb804740 1 -- 172.21.15.22:0/766142898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e27eab3820 con 0x55e27ebc3170 2024-04-20T21:44:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.010+0000 7f6797400640 1 -- 172.21.15.22:0/766142898 <== mon.2 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a40043d0 con 0x55e27ebc3170 2024-04-20T21:44:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.010+0000 7f6797400640 1 -- 172.21.15.22:0/766142898 <== mon.2 v2:172.21.15.33:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f67a4004ba0 con 0x55e27ebc3170 2024-04-20T21:44:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.010+0000 7f67bb804740 10 client.26900 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:44:21.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.010+0000 7f67bb804740 10 client.26900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:44:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.010+0000 7f67bb804740 20 client.26900 populate_metadata read hostname 'smithi022' 2024-04-20T21:44:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.011+0000 7f67bb804740 1 --2- 172.21.15.22:0/766142898 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e27ebdc670 0x55e27ebdea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.011+0000 7f67bb804740 4 client.26900 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:44:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.011+0000 7f67bb804740 1 -- 172.21.15.22:0/766142898 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e27eb35a70 con 0x55e27ebdc670 2024-04-20T21:44:21.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.011+0000 7f67abe00640 1 --2- 172.21.15.22:0/766142898 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e27ebdc670 0x55e27ebdea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.011+0000 7f67abe00640 1 --2- 172.21.15.22:0/766142898 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e27ebdc670 0x55e27ebdea60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=780311c6c2fdbbdf server_cookie=f74fbec24bae7b8a in_seq=0 out_seq=0 2024-04-20T21:44:21.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.011+0000 7f6797400640 10 client.26900 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:44:21.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.012+0000 7f6797400640 1 -- 172.21.15.22:0/766142898 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e27eb35a70 con 0x55e27ebdc670 2024-04-20T21:44:21.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.012+0000 7f6797400640 10 client.26900 handle_command_reply: tid=0 2024-04-20T21:44:21.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.012+0000 7f67bb804740 1 client.26900 shutdown 2024-04-20T21:44:21.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.013+0000 7f67bb804740 1 -- 172.21.15.22:0/766142898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 msgr2=0x55e27ebc9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:21.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.013+0000 7f67bb804740 1 --2- 172.21.15.22:0/766142898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebc9690 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f67a4019460 tx=0x7f67a40038a0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.013+0000 7f67bb804740 1 -- 172.21.15.22:0/766142898 shutdown_connections 2024-04-20T21:44:21.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.013+0000 7f67bb804740 1 --2- 172.21.15.22:0/766142898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.013+0000 7f67bb804740 1 --2- 172.21.15.22:0/766142898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebc9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.013+0000 7f67bb804740 1 --2- 172.21.15.22:0/766142898 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e27ebdc670 0x55e27ebdea60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.013+0000 7f67bb804740 1 --2- 172.21.15.22:0/766142898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebc9690 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.013+0000 7f67bb804740 1 -- 172.21.15.22:0/766142898 >> 172.21.15.22:0/766142898 conn(0x55e27ebbd8a0 msgr2=0x55e27ebbf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:21.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.013+0000 7f6797400640 0 client.26900 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:44:21.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.013+0000 7f67bb804740 1 -- 172.21.15.22:0/766142898 shutdown_connections 2024-04-20T21:44:21.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.013+0000 7f67bb804740 1 -- 172.21.15.22:0/766142898 wait complete. 2024-04-20T21:44:21.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.013+0000 7f67bb804740 20 client.26900 trim_cache size 0 max 16384 2024-04-20T21:44:21.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.028+0000 7f67bb804740 1 Processor -- start 2024-04-20T21:44:21.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.028+0000 7f67bb804740 1 -- start start 2024-04-20T21:44:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.028+0000 7f67bb804740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3170 0x55e27ebcbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.028+0000 7f67bb804740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3a90 0x55e27ebd8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.028+0000 7f67bb804740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebc87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.028+0000 7f67bb804740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41ade70 con 0x55e27ebc3170 2024-04-20T21:44:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.028+0000 7f67bb804740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41b06d0 con 0x55e27ebc3a90 2024-04-20T21:44:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.028+0000 7f67bb804740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41ae4c0 con 0x55e27ebcc670 2024-04-20T21:44:21.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.028+0000 7f67b1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3170 0x55e27ebcbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.028+0000 7f67b1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3170 0x55e27ebcbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44218/0 (socket says 172.21.15.22:44218) 2024-04-20T21:44:21.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.028+0000 7f67b1e00640 1 -- 172.21.15.22:0/1673997511 learned_addr learned my addr 172.21.15.22:0/1673997511 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:21.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.028+0000 7f67abe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3a90 0x55e27ebd8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.029+0000 7f67b2800640 1 --2- 172.21.15.22:0/1673997511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebc87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.029+0000 7f67b1e00640 1 -- 172.21.15.22:0/1673997511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 msgr2=0x55e27ebc87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:21.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.029+0000 7f67b1e00640 1 --2- 172.21.15.22:0/1673997511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebc87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.029+0000 7f67b1e00640 1 -- 172.21.15.22:0/1673997511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3a90 msgr2=0x55e27ebd8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:21.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.029+0000 7f67b1e00640 1 --2- 172.21.15.22:0/1673997511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3a90 0x55e27ebd8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.029+0000 7f67b1e00640 1 -- 172.21.15.22:0/1673997511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27e83a3f0 con 0x55e27ebc3170 2024-04-20T21:44:21.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.029+0000 7f67abe00640 1 --2- 172.21.15.22:0/1673997511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3a90 0x55e27ebd8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:21.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.029+0000 7f67b1e00640 1 --2- 172.21.15.22:0/1673997511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3170 0x55e27ebcbdc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009c40 tx=0x7f67a4005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d4059cd029ffe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.029+0000 7f67a2600640 1 -- 172.21.15.22:0/1673997511 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a403b070 con 0x55e27ebc3170 2024-04-20T21:44:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.029+0000 7f67a2600640 1 -- 172.21.15.22:0/1673997511 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a40054a0 con 0x55e27ebc3170 2024-04-20T21:44:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.029+0000 7f67a2600640 1 -- 172.21.15.22:0/1673997511 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4023460 con 0x55e27ebc3170 2024-04-20T21:44:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.030+0000 7f67bb804740 1 -- 172.21.15.22:0/1673997511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3170 msgr2=0x55e27ebcbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.030+0000 7f67bb804740 1 --2- 172.21.15.22:0/1673997511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3170 0x55e27ebcbdc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009c40 tx=0x7f67a4005d30 comp rx=0 tx=0).stop 2024-04-20T21:44:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.030+0000 7f67bb804740 1 -- 172.21.15.22:0/1673997511 shutdown_connections 2024-04-20T21:44:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.030+0000 7f67bb804740 1 --2- 172.21.15.22:0/1673997511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebc87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.030+0000 7f67bb804740 1 --2- 172.21.15.22:0/1673997511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3a90 0x55e27ebd8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.030+0000 7f67bb804740 1 --2- 172.21.15.22:0/1673997511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3170 0x55e27ebcbdc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.030+0000 7f67bb804740 1 -- 172.21.15.22:0/1673997511 >> 172.21.15.22:0/1673997511 conn(0x55e27ebbd8a0 msgr2=0x55e27ebc3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.030+0000 7f67bb804740 1 -- 172.21.15.22:0/1673997511 shutdown_connections 2024-04-20T21:44:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.030+0000 7f67bb804740 1 -- 172.21.15.22:0/1673997511 wait complete. 2024-04-20T21:44:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.031+0000 7f67bb804740 1 -- start start 2024-04-20T21:44:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.031+0000 7f67bb804740 1 Processor -- start 2024-04-20T21:44:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.031+0000 7f67bb804740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebd4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.031+0000 7f67bb804740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebd4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.031+0000 7f67bb804740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebc5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.031+0000 7f67bb804740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41aeb90 con 0x55e27ebc3a90 2024-04-20T21:44:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.031+0000 7f67bb804740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41b0000 con 0x55e27ebc3170 2024-04-20T21:44:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.031+0000 7f67bb804740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41af260 con 0x55e27ebcc670 2024-04-20T21:44:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.032+0000 7f67bb804740 10 client.0 _ll_register_callbacks cb 0x7f67a4009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:44:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.032+0000 7f67abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebd4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.032+0000 7f67abe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebd4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44224/0 (socket says 172.21.15.22:44224) 2024-04-20T21:44:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.032+0000 7f67abe00640 1 -- 172.21.15.22:0/255994546 learned_addr learned my addr 172.21.15.22:0/255994546 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.032+0000 7f67b2800640 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebc5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.032+0000 7f67b1e00640 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebd4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.032+0000 7f67abe00640 1 -- 172.21.15.22:0/255994546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 msgr2=0x55e27ebc5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.032+0000 7f67abe00640 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebc5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.032+0000 7f67abe00640 1 -- 172.21.15.22:0/255994546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 msgr2=0x55e27ebd4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:44:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.032+0000 7f67abe00640 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebd4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.032+0000 7f67abe00640 1 -- 172.21.15.22:0/255994546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27e9047d0 con 0x55e27ebc3a90 2024-04-20T21:44:21.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.033+0000 7f67abe00640 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebd4da0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f679c00c6a0 tx=0x7f679c0d5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddc384f527b62f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:21.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.033+0000 7f6797400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-04-20T21:44:21.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.033+0000 7f6797400640 1 -- 172.21.15.22:0/255994546 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c09c950 con 0x55e27ebc3a90 2024-04-20T21:44:21.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.033+0000 7f6797400640 1 -- 172.21.15.22:0/255994546 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f679c003b10 con 0x55e27ebc3a90 2024-04-20T21:44:21.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.033+0000 7f67bb804740 1 -- 172.21.15.22:0/255994546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a40054a0 con 0x55e27ebc3a90 2024-04-20T21:44:21.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.033+0000 7f6797400640 1 -- 172.21.15.22:0/255994546 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f679c0b8020 con 0x55e27ebc3a90 2024-04-20T21:44:21.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.033+0000 7f67bb804740 10 client.16758 fetch_fsmap learned FSMap version 20 2024-04-20T21:44:21.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.033+0000 7f67bb804740 1 -- 172.21.15.22:0/255994546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e27ec420a0 con 0x55e27ebc3a90 2024-04-20T21:44:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.033+0000 7f6797400640 1 -- 172.21.15.22:0/255994546 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a40054a0 con 0x55e27ebc3a90 2024-04-20T21:44:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.033+0000 7f6797400640 1 -- 172.21.15.22:0/255994546 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 20) ==== 8323+0+0 (secure 0 0 0) 0x7f679c0870b0 con 0x55e27ebc3a90 2024-04-20T21:44:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.034+0000 7f67bb804740 10 client.16758 fetch_fsmap finished waiting for FSMap version 20 2024-04-20T21:44:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.034+0000 7f67bb804740 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:44:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.034+0000 7f67bb804740 20 client.16758 populate_metadata read hostname 'smithi022' 2024-04-20T21:44:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.034+0000 7f67bb804740 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e27ec427a0 0x55e27ec44bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.034+0000 7f67bb804740 4 client.16758 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:44:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.034+0000 7f67bb804740 1 -- 172.21.15.22:0/255994546 --> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e27eb2f8e0 con 0x55e27ec427a0 2024-04-20T21:44:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.034+0000 7f67b1e00640 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e27ec427a0 0x55e27ec44bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.034+0000 7f67b1e00640 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e27ec427a0 0x55e27ec44bc0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c652360566b6db server_cookie=dca662c5eca7520e in_seq=0 out_seq=0 2024-04-20T21:44:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.034+0000 7f6797400640 10 client.16758 ms_handle_connect on v2:172.21.15.33:6838/300232041 2024-04-20T21:44:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f6797400640 1 -- 172.21.15.22:0/255994546 <== mds.0 v2:172.21.15.33:6838/300232041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e27eb2f8e0 con 0x55e27ec427a0 2024-04-20T21:44:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f6797400640 10 client.16758 handle_command_reply: tid=0 2024-04-20T21:44:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f67bb804740 1 client.16758 shutdown 2024-04-20T21:44:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f67bb804740 1 -- 172.21.15.22:0/255994546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 msgr2=0x55e27ebd4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f67bb804740 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebd4da0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f679c00c6a0 tx=0x7f679c0d5040 comp rx=0 tx=0).stop 2024-04-20T21:44:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f67bb804740 1 -- 172.21.15.22:0/255994546 shutdown_connections 2024-04-20T21:44:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f67bb804740 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e27ebcc670 0x55e27ebc5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f67bb804740 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e27ebc3a90 0x55e27ebd4da0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f67bb804740 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.33:6838/300232041,v1:172.21.15.33:6839/300232041] conn(0x55e27ec427a0 0x55e27ec44bc0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f67bb804740 1 --2- 172.21.15.22:0/255994546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e27ebc3170 0x55e27ebd4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f67bb804740 1 -- 172.21.15.22:0/255994546 >> 172.21.15.22:0/255994546 conn(0x55e27ebbd8a0 msgr2=0x55e27ebc87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f6797400640 0 client.16758 ms_handle_reset on v2:172.21.15.33:6838/300232041 2024-04-20T21:44:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f67bb804740 1 -- 172.21.15.22:0/255994546 shutdown_connections 2024-04-20T21:44:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f67bb804740 1 -- 172.21.15.22:0/255994546 wait complete. 2024-04-20T21:44:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.035+0000 7f67bb804740 20 client.16758 trim_cache size 0 max 16384 2024-04-20T21:44:21.044 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-04-20T21:44:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.036+0000 7f67b3200640 1 -- 172.21.15.22:0/3627848686 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f67980a3d20 msgr2=0x7f67980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.036+0000 7f67b3200640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f67980a3d20 0x7f67980a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f67ac005e30 tx=0x7f67ac005d80 comp rx=0 tx=0).stop 2024-04-20T21:44:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.036+0000 7f67b3200640 1 -- 172.21.15.22:0/3627848686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b4109340 msgr2=0x7f67b41a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.036+0000 7f67b3200640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b4109340 0x7f67b41a0b00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f679c00e9a0 tx=0x7f679c00ee60 comp rx=0 tx=0).stop 2024-04-20T21:44:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.036+0000 7f67b3200640 1 -- 172.21.15.22:0/3627848686 shutdown_connections 2024-04-20T21:44:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.036+0000 7f67b3200640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f67980a3d20 0x7f67980a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.036+0000 7f67b3200640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67b419a680 0x7f67b419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.036+0000 7f67b3200640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67b4109340 0x7f67b41a0b00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.036+0000 7f67b3200640 1 --2- 172.21.15.22:0/3627848686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b4104270 0x7f67b41a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:21.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.036+0000 7f67b3200640 1 -- 172.21.15.22:0/3627848686 >> 172.21.15.22:0/3627848686 conn(0x7f67b40ffad0 msgr2=0x7f67b4101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:21.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.036+0000 7f67b3200640 1 -- 172.21.15.22:0/3627848686 shutdown_connections 2024-04-20T21:44:21.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:21.036+0000 7f67b3200640 1 -- 172.21.15.22:0/3627848686 wait complete. 2024-04-20T21:44:21.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:21 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3559145688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:44:21.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:21 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3559145688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:44:21.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:21 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3559145688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:44:22.055 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-20T21:44:22.056 INFO:tasks.fwd_scrub:done joining 2024-04-20T21:44:22.056 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-20T21:44:22.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:44:22.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.284+0000 7f100fe00640 1 -- 172.21.15.22:0/3467787183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1010102960 msgr2=0x7f10101069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:22.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.284+0000 7f100fe00640 1 --2- 172.21.15.22:0/3467787183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1010102960 0x7f10101069a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009900 tx=0x7f0ff802bc30 comp rx=0 tx=0).stop 2024-04-20T21:44:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.284+0000 7f100fe00640 1 -- 172.21.15.22:0/3467787183 shutdown_connections 2024-04-20T21:44:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.284+0000 7f100fe00640 1 --2- 172.21.15.22:0/3467787183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1010107080 0x7f101010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.284+0000 7f100fe00640 1 --2- 172.21.15.22:0/3467787183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1010102960 0x7f10101069a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.284+0000 7f100fe00640 1 --2- 172.21.15.22:0/3467787183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1010101fb0 0x7f1010102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.284+0000 7f100fe00640 1 -- 172.21.15.22:0/3467787183 >> 172.21.15.22:0/3467787183 conn(0x7f10100fd890 msgr2=0x7f10100ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.284+0000 7f100fe00640 1 -- 172.21.15.22:0/3467787183 shutdown_connections 2024-04-20T21:44:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.284+0000 7f100fe00640 1 -- 172.21.15.22:0/3467787183 wait complete. 2024-04-20T21:44:22.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.285+0000 7f100fe00640 1 Processor -- start 2024-04-20T21:44:22.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.285+0000 7f100fe00640 1 -- start start 2024-04-20T21:44:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.286+0000 7f100fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1010101fb0 0x7f10101a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.286+0000 7f100fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1010107080 0x7f10101a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.286+0000 7f100fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f101019a700 0x7f101019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.286+0000 7f100fe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1010112140 con 0x7f1010101fb0 2024-04-20T21:44:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.286+0000 7f100fe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1010111fc0 con 0x7f101019a700 2024-04-20T21:44:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.286+0000 7f100fe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f10101122c0 con 0x7f1010107080 2024-04-20T21:44:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.286+0000 7f100ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1010101fb0 0x7f10101a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.286+0000 7f100ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1010101fb0 0x7f10101a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44250/0 (socket says 172.21.15.22:44250) 2024-04-20T21:44:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.286+0000 7f100ea00640 1 -- 172.21.15.22:0/436607078 learned_addr learned my addr 172.21.15.22:0/436607078 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.286+0000 7f100e000640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1010107080 0x7f10101a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.286+0000 7f100f400640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f101019a700 0x7f101019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.287+0000 7f100ea00640 1 -- 172.21.15.22:0/436607078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1010107080 msgr2=0x7f10101a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.287+0000 7f100ea00640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1010107080 0x7f10101a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.287+0000 7f100ea00640 1 -- 172.21.15.22:0/436607078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f101019a700 msgr2=0x7f101019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.287+0000 7f100ea00640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f101019a700 0x7f101019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.287+0000 7f100ea00640 1 -- 172.21.15.22:0/436607078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101019b300 con 0x7f1010101fb0 2024-04-20T21:44:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.287+0000 7f100ea00640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1010101fb0 0x7f10101a0640 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1000009d50 tx=0x7f100000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.287+0000 7f1014a00640 1 -- 172.21.15.22:0/436607078 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000003b70 con 0x7f1010101fb0 2024-04-20T21:44:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.287+0000 7f100fe00640 1 -- 172.21.15.22:0/436607078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10101a73e0 con 0x7f1010101fb0 2024-04-20T21:44:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.287+0000 7f1014a00640 1 -- 172.21.15.22:0/436607078 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1000003d10 con 0x7f1010101fb0 2024-04-20T21:44:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.287+0000 7f1014a00640 1 -- 172.21.15.22:0/436607078 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000004d20 con 0x7f1010101fb0 2024-04-20T21:44:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.287+0000 7f100fe00640 1 -- 172.21.15.22:0/436607078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10101a7790 con 0x7f1010101fb0 2024-04-20T21:44:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.289+0000 7f100fe00640 1 -- 172.21.15.22:0/436607078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1010103480 con 0x7f1010101fb0 2024-04-20T21:44:22.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.291+0000 7f1014a00640 1 -- 172.21.15.22:0/436607078 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1000004000 con 0x7f1010101fb0 2024-04-20T21:44:22.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.292+0000 7f1014a00640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0ff40a3d20 0x7f0ff40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:22.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.292+0000 7f1014a00640 1 -- 172.21.15.22:0/436607078 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f10000d0ec0 con 0x7f1010101fb0 2024-04-20T21:44:22.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.292+0000 7f100e000640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0ff40a3d20 0x7f0ff40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:22.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.293+0000 7f100e000640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0ff40a3d20 0x7f0ff40a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ff802c240 tx=0x7f0ff802df90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:22.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.297+0000 7f1014a00640 1 -- 172.21.15.22:0/436607078 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f100009d510 con 0x7f1010101fb0 2024-04-20T21:44:22.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.507+0000 7f100fe00640 1 -- 172.21.15.22:0/436607078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10100690f0 con 0x7f1010101fb0 2024-04-20T21:44:22.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.509+0000 7f1014a00640 1 -- 172.21.15.22:0/436607078 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23239 (secure 0 0 0) 0x7f100009ce10 con 0x7f1010101fb0 2024-04-20T21:44:22.509 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:44:22.509 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":103,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:44:14.296211+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/1815958949":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/3359736029":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/3485677447":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/1911146506":"2024-04-21T21:33:48.815984+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1408863783":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/1911146506":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:44:22.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.512+0000 7f100fe00640 1 -- 172.21.15.22:0/436607078 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0ff40a3d20 msgr2=0x7f0ff40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:22.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.512+0000 7f100fe00640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0ff40a3d20 0x7f0ff40a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ff802c240 tx=0x7f0ff802df90 comp rx=0 tx=0).stop 2024-04-20T21:44:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.512+0000 7f100fe00640 1 -- 172.21.15.22:0/436607078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1010101fb0 msgr2=0x7f10101a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.512+0000 7f100fe00640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1010101fb0 0x7f10101a0640 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1000009d50 tx=0x7f100000c6a0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.512+0000 7f100fe00640 1 -- 172.21.15.22:0/436607078 shutdown_connections 2024-04-20T21:44:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.512+0000 7f100fe00640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f0ff40a3d20 0x7f0ff40a61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.512+0000 7f100fe00640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f101019a700 0x7f101019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.512+0000 7f100fe00640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1010107080 0x7f10101a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.512+0000 7f100fe00640 1 --2- 172.21.15.22:0/436607078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1010101fb0 0x7f10101a0640 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.512+0000 7f100fe00640 1 -- 172.21.15.22:0/436607078 >> 172.21.15.22:0/436607078 conn(0x7f10100fd890 msgr2=0x7f10100ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:22.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.513+0000 7f100fe00640 1 -- 172.21.15.22:0/436607078 shutdown_connections 2024-04-20T21:44:22.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.513+0000 7f100fe00640 1 -- 172.21.15.22:0/436607078 wait complete. 2024-04-20T21:44:22.530 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:26:15.688818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:31:25.506074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:31:26.126261+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:44:22.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:44:22.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.725+0000 7f98e9800640 1 -- 172.21.15.22:0/2141170436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98e4108570 msgr2=0x7f98e41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:22.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.725+0000 7f98e9800640 1 --2- 172.21.15.22:0/2141170436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98e4108570 0x7f98e41089f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009930 tx=0x7f98d002bc20 comp rx=0 tx=0).stop 2024-04-20T21:44:22.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.725+0000 7f98e9800640 1 -- 172.21.15.22:0/2141170436 shutdown_connections 2024-04-20T21:44:22.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.725+0000 7f98e9800640 1 --2- 172.21.15.22:0/2141170436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98e4108f30 0x7f98e41153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.725+0000 7f98e9800640 1 --2- 172.21.15.22:0/2141170436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98e4108570 0x7f98e41089f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.725+0000 7f98e9800640 1 --2- 172.21.15.22:0/2141170436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98e410e630 0x7f98e410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.725+0000 7f98e9800640 1 -- 172.21.15.22:0/2141170436 >> 172.21.15.22:0/2141170436 conn(0x7f98e40764d0 msgr2=0x7f98e40788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:22.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.725+0000 7f98e9800640 1 -- 172.21.15.22:0/2141170436 shutdown_connections 2024-04-20T21:44:22.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.725+0000 7f98e9800640 1 -- 172.21.15.22:0/2141170436 wait complete. 2024-04-20T21:44:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.726+0000 7f98e9800640 1 Processor -- start 2024-04-20T21:44:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.727+0000 7f98e9800640 1 -- start start 2024-04-20T21:44:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.727+0000 7f98e9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98e4108570 0x7f98e419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.727+0000 7f98e9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98e4108f30 0x7f98e419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.727+0000 7f98e9800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98e410e630 0x7f98e41a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.727+0000 7f98e9800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4118940 con 0x7f98e4108f30 2024-04-20T21:44:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.727+0000 7f98e9800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41187c0 con 0x7f98e410e630 2024-04-20T21:44:22.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.727+0000 7f98e9800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4118ac0 con 0x7f98e4108570 2024-04-20T21:44:22.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.727+0000 7f98e2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98e4108f30 0x7f98e419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:22.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.727+0000 7f98e2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98e4108f30 0x7f98e419f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43366/0 (socket says 172.21.15.22:43366) 2024-04-20T21:44:22.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.727+0000 7f98e2000640 1 -- 172.21.15.22:0/3462187217 learned_addr learned my addr 172.21.15.22:0/3462187217 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:22.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.727+0000 7f98e2a00640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98e4108570 0x7f98e419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:22.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.728+0000 7f98e3400640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98e410e630 0x7f98e41a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:22.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.728+0000 7f98e2000640 1 -- 172.21.15.22:0/3462187217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98e4108570 msgr2=0x7f98e419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:22.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.728+0000 7f98e2000640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98e4108570 0x7f98e419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.728+0000 7f98e2000640 1 -- 172.21.15.22:0/3462187217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98e410e630 msgr2=0x7f98e41a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:22.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.728+0000 7f98e2000640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98e410e630 0x7f98e41a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.728+0000 7f98e2000640 1 -- 172.21.15.22:0/3462187217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e41a3dd0 con 0x7f98e4108f30 2024-04-20T21:44:22.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.728+0000 7f98e2a00640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98e4108570 0x7f98e419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:22.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.728+0000 7f98e3400640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98e410e630 0x7f98e41a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:22.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.728+0000 7f98e2000640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98e4108f30 0x7f98e419f320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f98d0005ab0 tx=0x7f98d00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:22.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.729+0000 7f98db400640 1 -- 172.21.15.22:0/3462187217 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0043020 con 0x7f98e4108f30 2024-04-20T21:44:22.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.729+0000 7f98e9800640 1 -- 172.21.15.22:0/3462187217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e41a4060 con 0x7f98e4108f30 2024-04-20T21:44:22.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.729+0000 7f98db400640 1 -- 172.21.15.22:0/3462187217 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98d00044d0 con 0x7f98e4108f30 2024-04-20T21:44:22.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.729+0000 7f98db400640 1 -- 172.21.15.22:0/3462187217 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0003bb0 con 0x7f98e4108f30 2024-04-20T21:44:22.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.729+0000 7f98e9800640 1 -- 172.21.15.22:0/3462187217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e41ab950 con 0x7f98e4108f30 2024-04-20T21:44:22.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.730+0000 7f98e9800640 1 -- 172.21.15.22:0/3462187217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e40691d0 con 0x7f98e4108f30 2024-04-20T21:44:22.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.733+0000 7f98db400640 1 -- 172.21.15.22:0/3462187217 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f98d0046020 con 0x7f98e4108f30 2024-04-20T21:44:22.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.734+0000 7f98db400640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f98c80a3d20 0x7f98c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:22.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.734+0000 7f98db400640 1 -- 172.21.15.22:0/3462187217 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f98d00f2680 con 0x7f98e4108f30 2024-04-20T21:44:22.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.734+0000 7f98e2a00640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f98c80a3d20 0x7f98c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:22.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.735+0000 7f98e2a00640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f98c80a3d20 0x7f98c80a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009fc0 tx=0x7f98d4005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:22.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.739+0000 7f98db400640 1 -- 172.21.15.22:0/3462187217 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f98d004d020 con 0x7f98e4108f30 2024-04-20T21:44:22.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:22 smithi122 ceph-mon[22147]: pgmap v329: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:44:22.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:22 smithi033 ceph-mon[21626]: pgmap v329: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:44:22.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:22 smithi022 ceph-mon[16843]: pgmap v329: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:44:22.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.945+0000 7f98e9800640 1 -- 172.21.15.22:0/3462187217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98e4110980 con 0x7f98e4108f30 2024-04-20T21:44:22.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.946+0000 7f98db400640 1 -- 172.21.15.22:0/3462187217 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7f98d00bede0 con 0x7f98e4108f30 2024-04-20T21:44:22.946 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-20T21:44:22.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.949+0000 7f98e9800640 1 -- 172.21.15.22:0/3462187217 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f98c80a3d20 msgr2=0x7f98c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.949+0000 7f98e9800640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f98c80a3d20 0x7f98c80a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009fc0 tx=0x7f98d4005f90 comp rx=0 tx=0).stop 2024-04-20T21:44:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.949+0000 7f98e9800640 1 -- 172.21.15.22:0/3462187217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98e4108f30 msgr2=0x7f98e419f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.949+0000 7f98e9800640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98e4108f30 0x7f98e419f320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f98d0005ab0 tx=0x7f98d00042e0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.949+0000 7f98e9800640 1 -- 172.21.15.22:0/3462187217 shutdown_connections 2024-04-20T21:44:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.949+0000 7f98e9800640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f98c80a3d20 0x7f98c80a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.949+0000 7f98e9800640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98e410e630 0x7f98e41a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.949+0000 7f98e9800640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98e4108f30 0x7f98e419f320 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.949+0000 7f98e9800640 1 --2- 172.21.15.22:0/3462187217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98e4108570 0x7f98e419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.949+0000 7f98e9800640 1 -- 172.21.15.22:0/3462187217 >> 172.21.15.22:0/3462187217 conn(0x7f98e40764d0 msgr2=0x7f98e4077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:22.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.950+0000 7f98e9800640 1 -- 172.21.15.22:0/3462187217 shutdown_connections 2024-04-20T21:44:22.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:22.950+0000 7f98e9800640 1 -- 172.21.15.22:0/3462187217 wait complete. 2024-04-20T21:44:22.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:44:23.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.186+0000 7fdbf3e00640 1 -- 172.21.15.22:0/3244203399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbec108590 msgr2=0x7fdbec108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:23.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.186+0000 7fdbf3e00640 1 --2- 172.21.15.22:0/3244203399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbec108590 0x7fdbec108a10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009890 tx=0x7fdbd802bb70 comp rx=0 tx=0).stop 2024-04-20T21:44:23.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.186+0000 7fdbf3e00640 1 -- 172.21.15.22:0/3244203399 shutdown_connections 2024-04-20T21:44:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.186+0000 7fdbf3e00640 1 --2- 172.21.15.22:0/3244203399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbec108f50 0x7fdbec1153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.186+0000 7fdbf3e00640 1 --2- 172.21.15.22:0/3244203399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbec108590 0x7fdbec108a10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.186+0000 7fdbf3e00640 1 --2- 172.21.15.22:0/3244203399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbec10e600 0x7fdbec10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.186+0000 7fdbf3e00640 1 -- 172.21.15.22:0/3244203399 >> 172.21.15.22:0/3244203399 conn(0x7fdbec0764a0 msgr2=0x7fdbec0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.186+0000 7fdbf3e00640 1 -- 172.21.15.22:0/3244203399 shutdown_connections 2024-04-20T21:44:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.186+0000 7fdbf3e00640 1 -- 172.21.15.22:0/3244203399 wait complete. 2024-04-20T21:44:23.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.187+0000 7fdbf3e00640 1 Processor -- start 2024-04-20T21:44:23.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.187+0000 7fdbf3e00640 1 -- start start 2024-04-20T21:44:23.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.188+0000 7fdbf3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbec108590 0x7fdbec19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:23.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.188+0000 7fdbf3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbec108f50 0x7fdbec19f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:23.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.188+0000 7fdbf3e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbec10e600 0x7fdbec1a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:23.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.188+0000 7fdbf3e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec1188f0 con 0x7fdbec108f50 2024-04-20T21:44:23.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.188+0000 7fdbf3e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec118770 con 0x7fdbec10e600 2024-04-20T21:44:23.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.188+0000 7fdbf3e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec118a70 con 0x7fdbec108590 2024-04-20T21:44:23.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.188+0000 7fdbf1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbec108590 0x7fdbec19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:23.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.188+0000 7fdbf0a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbec108f50 0x7fdbec19f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:23.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.188+0000 7fdbf1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbec108590 0x7fdbec19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:38210/0 (socket says 172.21.15.22:38210) 2024-04-20T21:44:23.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.188+0000 7fdbf1400640 1 -- 172.21.15.22:0/549666468 learned_addr learned my addr 172.21.15.22:0/549666468 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:23.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.188+0000 7fdbf1e00640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbec10e600 0x7fdbec1a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:23.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.189+0000 7fdbf1400640 1 -- 172.21.15.22:0/549666468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbec10e600 msgr2=0x7fdbec1a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:23.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.189+0000 7fdbf1400640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbec10e600 0x7fdbec1a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.189+0000 7fdbf1400640 1 -- 172.21.15.22:0/549666468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbec108f50 msgr2=0x7fdbec19f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:23.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.189+0000 7fdbf1400640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbec108f50 0x7fdbec19f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.189+0000 7fdbf1400640 1 -- 172.21.15.22:0/549666468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec1a3e00 con 0x7fdbec108590 2024-04-20T21:44:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.189+0000 7fdbf0a00640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbec108f50 0x7fdbec19f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.189+0000 7fdbf1400640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbec108590 0x7fdbec19ee10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000e9f0 tx=0x7fdbe000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.189+0000 7fdbea000640 1 -- 172.21.15.22:0/549666468 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000ccc0 con 0x7fdbec108590 2024-04-20T21:44:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.189+0000 7fdbf3e00640 1 -- 172.21.15.22:0/549666468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbec1a40f0 con 0x7fdbec108590 2024-04-20T21:44:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.189+0000 7fdbea000640 1 -- 172.21.15.22:0/549666468 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe0005500 con 0x7fdbec108590 2024-04-20T21:44:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.189+0000 7fdbea000640 1 -- 172.21.15.22:0/549666468 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0010690 con 0x7fdbec108590 2024-04-20T21:44:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.190+0000 7fdbf3e00640 1 -- 172.21.15.22:0/549666468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbec1ab980 con 0x7fdbec108590 2024-04-20T21:44:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.191+0000 7fdbf3e00640 1 -- 172.21.15.22:0/549666468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbec0691d0 con 0x7fdbec108590 2024-04-20T21:44:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.193+0000 7fdbea000640 1 -- 172.21.15.22:0/549666468 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fdbe00046c0 con 0x7fdbec108590 2024-04-20T21:44:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.194+0000 7fdbea000640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdbd40a3d20 0x7fdbd40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.194+0000 7fdbea000640 1 -- 172.21.15.22:0/549666468 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7fdbe00d0d20 con 0x7fdbec108590 2024-04-20T21:44:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.194+0000 7fdbf0a00640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdbd40a3d20 0x7fdbd40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:23.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.195+0000 7fdbf0a00640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdbd40a3d20 0x7fdbd40a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8037040 tx=0x7fdbd802e040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.197+0000 7fdbea000640 1 -- 172.21.15.22:0/549666468 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdbe0053ff0 con 0x7fdbec108590 2024-04-20T21:44:23.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.407+0000 7fdbf3e00640 1 -- 172.21.15.22:0/549666468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdbec110a70 con 0x7fdbec108590 2024-04-20T21:44:23.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.408+0000 7fdbea000640 1 -- 172.21.15.22:0/549666468 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7fdbe009d310 con 0x7fdbec108590 2024-04-20T21:44:23.407 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-20T21:44:23.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.411+0000 7fdbf3e00640 1 -- 172.21.15.22:0/549666468 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdbd40a3d20 msgr2=0x7fdbd40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:23.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.411+0000 7fdbf3e00640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdbd40a3d20 0x7fdbd40a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8037040 tx=0x7fdbd802e040 comp rx=0 tx=0).stop 2024-04-20T21:44:23.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.411+0000 7fdbf3e00640 1 -- 172.21.15.22:0/549666468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbec108590 msgr2=0x7fdbec19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:23.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.411+0000 7fdbf3e00640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbec108590 0x7fdbec19ee10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000e9f0 tx=0x7fdbe000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.411+0000 7fdbf3e00640 1 -- 172.21.15.22:0/549666468 shutdown_connections 2024-04-20T21:44:23.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.411+0000 7fdbf3e00640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fdbd40a3d20 0x7fdbd40a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.411+0000 7fdbf3e00640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbec10e600 0x7fdbec1a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.411+0000 7fdbf3e00640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdbec108f50 0x7fdbec19f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.411+0000 7fdbf3e00640 1 --2- 172.21.15.22:0/549666468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbec108590 0x7fdbec19ee10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.411+0000 7fdbf3e00640 1 -- 172.21.15.22:0/549666468 >> 172.21.15.22:0/549666468 conn(0x7fdbec0764a0 msgr2=0x7fdbec077d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:23.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.411+0000 7fdbf3e00640 1 -- 172.21.15.22:0/549666468 shutdown_connections 2024-04-20T21:44:23.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.412+0000 7fdbf3e00640 1 -- 172.21.15.22:0/549666468 wait complete. 2024-04-20T21:44:23.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:44:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.642+0000 7f447ce00640 1 -- 172.21.15.22:0/992778070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4478104b50 msgr2=0x7f4478108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.642+0000 7f447ce00640 1 --2- 172.21.15.22:0/992778070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4478104b50 0x7f4478108b90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4460009900 tx=0x7f446002bc40 comp rx=0 tx=0).stop 2024-04-20T21:44:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.642+0000 7f447ce00640 1 -- 172.21.15.22:0/992778070 shutdown_connections 2024-04-20T21:44:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.642+0000 7f447ce00640 1 --2- 172.21.15.22:0/992778070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4478109270 0x7f4478110e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.642+0000 7f447ce00640 1 --2- 172.21.15.22:0/992778070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4478104b50 0x7f4478108b90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.642+0000 7f447ce00640 1 --2- 172.21.15.22:0/992778070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44781041a0 0x7f4478104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.642+0000 7f447ce00640 1 -- 172.21.15.22:0/992778070 >> 172.21.15.22:0/992778070 conn(0x7f44780ffa00 msgr2=0x7f4478101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.642+0000 7f447ce00640 1 -- 172.21.15.22:0/992778070 shutdown_connections 2024-04-20T21:44:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.642+0000 7f447ce00640 1 -- 172.21.15.22:0/992778070 wait complete. 2024-04-20T21:44:23.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.643+0000 7f447ce00640 1 Processor -- start 2024-04-20T21:44:23.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.644+0000 7f447ce00640 1 -- start start 2024-04-20T21:44:23.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.644+0000 7f447ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44781041a0 0x7f44781a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:23.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.644+0000 7f447ce00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4478109270 0x7f44781a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:23.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.644+0000 7f447ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447819a650 0x7f447819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:23.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.644+0000 7f447ce00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4478114350 con 0x7f447819a650 2024-04-20T21:44:23.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.644+0000 7f447ce00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44781141d0 con 0x7f4478109270 2024-04-20T21:44:23.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.644+0000 7f447ce00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f44781144d0 con 0x7f44781041a0 2024-04-20T21:44:23.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.645+0000 7f4477400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44781041a0 0x7f44781a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:23.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.645+0000 7f446fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4478109270 0x7f44781a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:23.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.645+0000 7f4477400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44781041a0 0x7f44781a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:38232/0 (socket says 172.21.15.22:38232) 2024-04-20T21:44:23.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.645+0000 7f4477400640 1 -- 172.21.15.22:0/2445671870 learned_addr learned my addr 172.21.15.22:0/2445671870 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:23.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.645+0000 7f446fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4478109270 0x7f44781a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:42108/0 (socket says 172.21.15.22:42108) 2024-04-20T21:44:23.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.645+0000 7f4477e00640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447819a650 0x7f447819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:23.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.645+0000 7f446fe00640 1 -- 172.21.15.22:0/2445671870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44781041a0 msgr2=0x7f44781a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:23.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.645+0000 7f446fe00640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44781041a0 0x7f44781a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.645+0000 7f446fe00640 1 -- 172.21.15.22:0/2445671870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447819a650 msgr2=0x7f447819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:23.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.645+0000 7f446fe00640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447819a650 0x7f447819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.645+0000 7f446fe00640 1 -- 172.21.15.22:0/2445671870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447819b250 con 0x7f4478109270 2024-04-20T21:44:23.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.645+0000 7f446fe00640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4478109270 0x7f44781a0ad0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4460036040 tx=0x7f44600042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:23.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.646+0000 7f4474c00640 1 -- 172.21.15.22:0/2445671870 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460041070 con 0x7f4478109270 2024-04-20T21:44:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.646+0000 7f447ce00640 1 -- 172.21.15.22:0/2445671870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447819b4e0 con 0x7f4478109270 2024-04-20T21:44:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.646+0000 7f4474c00640 1 -- 172.21.15.22:0/2445671870 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f446002cca0 con 0x7f4478109270 2024-04-20T21:44:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.646+0000 7f4474c00640 1 -- 172.21.15.22:0/2445671870 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460003cf0 con 0x7f4478109270 2024-04-20T21:44:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.646+0000 7f447ce00640 1 -- 172.21.15.22:0/2445671870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44781a7620 con 0x7f4478109270 2024-04-20T21:44:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.647+0000 7f447ce00640 1 -- 172.21.15.22:0/2445671870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4478105670 con 0x7f4478109270 2024-04-20T21:44:23.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.649+0000 7f4474c00640 1 -- 172.21.15.22:0/2445671870 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4460044070 con 0x7f4478109270 2024-04-20T21:44:23.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.651+0000 7f4474c00640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f44580a3d20 0x7f44580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:23.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.651+0000 7f4474c00640 1 -- 172.21.15.22:0/2445671870 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f44600f25e0 con 0x7f4478109270 2024-04-20T21:44:23.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.651+0000 7f4477400640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f44580a3d20 0x7f44580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:23.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.652+0000 7f4477400640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f44580a3d20 0x7f44580a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4468009c90 tx=0x7f4468005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:23.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.653+0000 7f4474c00640 1 -- 172.21.15.22:0/2445671870 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f44600be690 con 0x7f4478109270 2024-04-20T21:44:23.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:23 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/436607078' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:44:23.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:23 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3462187217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:44:23.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:23 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/549666468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:44:23.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:23 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/436607078' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:44:23.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:23 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3462187217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:44:23.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:23 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/549666468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:44:23.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.859+0000 7f447ce00640 1 -- 172.21.15.22:0/2445671870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f447819bb80 con 0x7f4478109270 2024-04-20T21:44:23.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.860+0000 7f4474c00640 1 -- 172.21.15.22:0/2445671870 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7f44600c2990 con 0x7f4478109270 2024-04-20T21:44:23.859 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-20T21:44:23.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.863+0000 7f447ce00640 1 -- 172.21.15.22:0/2445671870 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f44580a3d20 msgr2=0x7f44580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:23.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.863+0000 7f447ce00640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f44580a3d20 0x7f44580a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4468009c90 tx=0x7f4468005f90 comp rx=0 tx=0).stop 2024-04-20T21:44:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.863+0000 7f447ce00640 1 -- 172.21.15.22:0/2445671870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4478109270 msgr2=0x7f44781a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.863+0000 7f447ce00640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4478109270 0x7f44781a0ad0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4460036040 tx=0x7f44600042f0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.863+0000 7f447ce00640 1 -- 172.21.15.22:0/2445671870 shutdown_connections 2024-04-20T21:44:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.863+0000 7f447ce00640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f44580a3d20 0x7f44580a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.863+0000 7f447ce00640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447819a650 0x7f447819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.863+0000 7f447ce00640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4478109270 0x7f44781a0ad0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.863+0000 7f447ce00640 1 --2- 172.21.15.22:0/2445671870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44781041a0 0x7f44781a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.863+0000 7f447ce00640 1 -- 172.21.15.22:0/2445671870 >> 172.21.15.22:0/2445671870 conn(0x7f44780ffa00 msgr2=0x7f4478101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:23.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.863+0000 7f447ce00640 1 -- 172.21.15.22:0/2445671870 shutdown_connections 2024-04-20T21:44:23.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:23.863+0000 7f447ce00640 1 -- 172.21.15.22:0/2445671870 wait complete. 2024-04-20T21:44:23.879 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:23 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/436607078' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:44:23.879 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:23 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3462187217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:44:23.880 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:23 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/549666468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:44:23.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-20T21:44:24.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.090+0000 7f5bbf400640 1 -- 172.21.15.22:0/260823953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bb8102b50 msgr2=0x7f5bb8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:24.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.090+0000 7f5bbf400640 1 --2- 172.21.15.22:0/260823953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bb8102b50 0x7f5bb8106b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009900 tx=0x7f5ba402bc30 comp rx=0 tx=0).stop 2024-04-20T21:44:24.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.090+0000 7f5bbf400640 1 -- 172.21.15.22:0/260823953 shutdown_connections 2024-04-20T21:44:24.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.090+0000 7f5bbf400640 1 --2- 172.21.15.22:0/260823953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8107270 0x7f5bb810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:24.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.090+0000 7f5bbf400640 1 --2- 172.21.15.22:0/260823953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bb8102b50 0x7f5bb8106b90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:24.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.090+0000 7f5bbf400640 1 --2- 172.21.15.22:0/260823953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bb81021a0 0x7f5bb8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:24.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.090+0000 7f5bbf400640 1 -- 172.21.15.22:0/260823953 >> 172.21.15.22:0/260823953 conn(0x7f5bb80793f0 msgr2=0x7f5bb8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:24.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.090+0000 7f5bbf400640 1 -- 172.21.15.22:0/260823953 shutdown_connections 2024-04-20T21:44:24.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.090+0000 7f5bbf400640 1 -- 172.21.15.22:0/260823953 wait complete. 2024-04-20T21:44:24.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.092+0000 7f5bbf400640 1 Processor -- start 2024-04-20T21:44:24.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.092+0000 7f5bbf400640 1 -- start start 2024-04-20T21:44:24.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.092+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb81021a0 0x7f5bb81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:24.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.092+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bb8107270 0x7f5bb81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:24.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.092+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bb819a860 0x7f5bb819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:24.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.092+0000 7f5bbf400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8112350 con 0x7f5bb8107270 2024-04-20T21:44:24.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.092+0000 7f5bbf400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb81121d0 con 0x7f5bb81021a0 2024-04-20T21:44:24.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.092+0000 7f5bbf400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb81124d0 con 0x7f5bb819a860 2024-04-20T21:44:24.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.093+0000 7f5bb7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bb8107270 0x7f5bb81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:24.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.093+0000 7f5bb7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bb8107270 0x7f5bb81a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43416/0 (socket says 172.21.15.22:43416) 2024-04-20T21:44:24.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.093+0000 7f5bb7e00640 1 -- 172.21.15.22:0/1666299652 learned_addr learned my addr 172.21.15.22:0/1666299652 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:44:24.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.093+0000 7f5bbca00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb81021a0 0x7f5bb81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:24.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.093+0000 7f5bbd400640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bb819a860 0x7f5bb819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:24.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.093+0000 7f5bb7e00640 1 -- 172.21.15.22:0/1666299652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bb819a860 msgr2=0x7f5bb819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:24.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.093+0000 7f5bb7e00640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bb819a860 0x7f5bb819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:24.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.093+0000 7f5bb7e00640 1 -- 172.21.15.22:0/1666299652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb81021a0 msgr2=0x7f5bb81a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:24.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.093+0000 7f5bb7e00640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb81021a0 0x7f5bb81a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:24.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.093+0000 7f5bb7e00640 1 -- 172.21.15.22:0/1666299652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb819b380 con 0x7f5bb8107270 2024-04-20T21:44:24.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.093+0000 7f5bbca00640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb81021a0 0x7f5bb81a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:24.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.094+0000 7f5bb7e00640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bb8107270 0x7f5bb81a0bf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5ba402c240 tx=0x7f5ba4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:24.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.094+0000 7f5bb5600640 1 -- 172.21.15.22:0/1666299652 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4043020 con 0x7f5bb8107270 2024-04-20T21:44:24.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.094+0000 7f5bbf400640 1 -- 172.21.15.22:0/1666299652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb819b610 con 0x7f5bb8107270 2024-04-20T21:44:24.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.094+0000 7f5bb5600640 1 -- 172.21.15.22:0/1666299652 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ba40054e0 con 0x7f5bb8107270 2024-04-20T21:44:24.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.094+0000 7f5bbf400640 1 -- 172.21.15.22:0/1666299652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb81a78b0 con 0x7f5bb8107270 2024-04-20T21:44:24.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.094+0000 7f5bb5600640 1 -- 172.21.15.22:0/1666299652 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4003b20 con 0x7f5bb8107270 2024-04-20T21:44:24.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.095+0000 7f5bbf400640 1 -- 172.21.15.22:0/1666299652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb8069240 con 0x7f5bb8107270 2024-04-20T21:44:24.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.097+0000 7f5bb5600640 1 -- 172.21.15.22:0/1666299652 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5ba4046020 con 0x7f5bb8107270 2024-04-20T21:44:24.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.099+0000 7f5bb5600640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5ba00a3d20 0x7f5ba00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:24.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.099+0000 7f5bb5600640 1 -- 172.21.15.22:0/1666299652 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f5ba40f27a0 con 0x7f5bb8107270 2024-04-20T21:44:24.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.099+0000 7f5bbca00640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5ba00a3d20 0x7f5ba00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:24.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.100+0000 7f5bbca00640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5ba00a3d20 0x7f5ba00a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5bb819b9a0 tx=0x7f5bac00b410 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:24.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.103+0000 7f5bb5600640 1 -- 172.21.15.22:0/1666299652 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5ba404d020 con 0x7f5bb8107270 2024-04-20T21:44:24.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.364+0000 7f5bbf400640 1 -- 172.21.15.22:0/1666299652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f5bb8102cf0 con 0x7f5bb8107270 2024-04-20T21:44:24.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.378+0000 7f5bb5600640 1 -- 172.21.15.22:0/1666299652 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+746060 (secure 0 0 0) 0x7f5ba40bef00 con 0x7f5bb8107270 2024-04-20T21:44:24.390 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:44:24.390 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:44:24.390 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 27, 2024-04-20T21:44:24.390 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.390 INFO:teuthology.orchestra.run.smithi022.stdout: "active_gid": 15024, 2024-04-20T21:44:24.390 INFO:teuthology.orchestra.run.smithi022.stdout: "active_name": "x", 2024-04-20T21:44:24.391 INFO:teuthology.orchestra.run.smithi022.stdout: "active_addrs": { 2024-04-20T21:44:24.391 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-04-20T21:44:24.391 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.391 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-04-20T21:44:24.391 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6800", 2024-04-20T21:44:24.391 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 975472489 2024-04-20T21:44:24.391 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.391 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.391 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-04-20T21:44:24.392 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2024-04-20T21:44:24.392 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 975472489 2024-04-20T21:44:24.392 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.392 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-20T21:44:24.392 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.392 INFO:teuthology.orchestra.run.smithi022.stdout: "active_addr": "172.21.15.22:6801/975472489", 2024-04-20T21:44:24.392 INFO:teuthology.orchestra.run.smithi022.stdout: "active_change": "2024-04-20T21:33:48.816323+0000", 2024-04-20T21:44:24.392 INFO:teuthology.orchestra.run.smithi022.stdout: "active_mgr_features": 4540701547738038271, 2024-04-20T21:44:24.393 INFO:teuthology.orchestra.run.smithi022.stdout: "available": true, 2024-04-20T21:44:24.393 INFO:teuthology.orchestra.run.smithi022.stdout: "standbys": [ 2024-04-20T21:44:24.393 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.393 INFO:teuthology.orchestra.run.smithi022.stdout: "gid": 25124, 2024-04-20T21:44:24.393 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "z", 2024-04-20T21:44:24.393 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_features": 4540701547738038271, 2024-04-20T21:44:24.393 INFO:teuthology.orchestra.run.smithi022.stdout: "available_modules": [ 2024-04-20T21:44:24.393 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.393 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "alerts", 2024-04-20T21:44:24.394 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.394 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.394 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.394 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:24.394 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:24.394 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.394 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.394 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.394 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:24.395 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.395 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.395 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.395 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T21:44:24.395 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.395 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.395 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.395 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.396 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.396 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.396 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.396 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.396 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.396 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.396 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.396 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.396 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.397 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.397 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.397 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.397 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.397 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.397 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.397 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.397 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.397 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.398 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.398 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.398 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.398 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.398 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.398 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.398 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.398 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.399 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.399 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.399 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.399 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.399 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.399 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.399 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.399 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.399 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.400 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.400 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.400 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.400 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.400 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.400 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.400 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.400 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.400 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.401 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.401 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.401 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.401 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.401 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.401 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.401 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.401 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.401 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.402 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.402 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.402 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.402 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.402 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.402 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.402 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.402 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.403 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.403 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.403 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.403 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.403 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.403 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.403 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.403 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.404 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.404 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_destination": { 2024-04-20T21:44:24.404 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_destination", 2024-04-20T21:44:24.404 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.404 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.404 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.404 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.404 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.404 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.405 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.405 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email address to send alerts to", 2024-04-20T21:44:24.405 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.405 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.405 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.405 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.405 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_from_name": { 2024-04-20T21:44:24.405 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_from_name", 2024-04-20T21:44:24.405 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.405 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.406 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.406 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "Ceph", 2024-04-20T21:44:24.406 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.406 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.406 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.406 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email From: name", 2024-04-20T21:44:24.406 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.406 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.407 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.407 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.407 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_host": { 2024-04-20T21:44:24.407 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_host", 2024-04-20T21:44:24.407 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.407 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.407 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.407 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.407 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.408 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.408 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.408 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP server", 2024-04-20T21:44:24.408 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.408 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.408 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.408 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.408 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_password": { 2024-04-20T21:44:24.408 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_password", 2024-04-20T21:44:24.409 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.409 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.409 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.409 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.409 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.409 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.409 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.409 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Password to authenticate with", 2024-04-20T21:44:24.409 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.410 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.410 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.410 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.410 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_port": { 2024-04-20T21:44:24.410 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_port", 2024-04-20T21:44:24.410 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.410 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.410 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "465", 2024-04-20T21:44:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP port", 2024-04-20T21:44:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.412 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_sender": { 2024-04-20T21:44:24.412 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_sender", 2024-04-20T21:44:24.412 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.412 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.412 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.412 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.412 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.412 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.412 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.413 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP envelope sender", 2024-04-20T21:44:24.413 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.413 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.413 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.413 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.413 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_ssl": { 2024-04-20T21:44:24.413 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_ssl", 2024-04-20T21:44:24.413 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.414 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.414 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.414 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.414 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.414 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.414 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.414 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T21:44:24.414 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_user": { 2024-04-20T21:44:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_user", 2024-04-20T21:44:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.416 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.416 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.416 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.416 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "User to authenticate as", 2024-04-20T21:44:24.416 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.416 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.416 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.416 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.417 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.417 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.417 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.417 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "balancer", 2024-04-20T21:44:24.417 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.417 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.417 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.417 INFO:teuthology.orchestra.run.smithi022.stdout: "active": { 2024-04-20T21:44:24.417 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "active", 2024-04-20T21:44:24.418 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.418 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.418 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.418 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.418 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.418 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.418 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.418 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T21:44:24.419 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.419 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.419 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.419 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.419 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_time": { 2024-04-20T21:44:24.419 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_time", 2024-04-20T21:44:24.419 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.419 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.420 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.420 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0000", 2024-04-20T21:44:24.420 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.420 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.420 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.420 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T21:44:24.420 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T21:44:24.420 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.420 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.421 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_weekday": { 2024-04-20T21:44:24.421 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_weekday", 2024-04-20T21:44:24.421 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:24.421 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.421 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.421 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:24.421 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-04-20T21:44:24.421 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-04-20T21:44:24.422 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.422 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T21:44:24.422 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T21:44:24.422 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.422 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.422 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.422 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_max_iterations": { 2024-04-20T21:44:24.422 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_max_iterations", 2024-04-20T21:44:24.423 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:24.423 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.423 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.423 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "25", 2024-04-20T21:44:24.423 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-04-20T21:44:24.423 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "250", 2024-04-20T21:44:24.423 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.423 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T21:44:24.423 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.424 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.424 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.424 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_metrics": { 2024-04-20T21:44:24.424 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_metrics", 2024-04-20T21:44:24.424 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.424 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.425 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.425 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T21:44:24.425 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.425 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.425 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.425 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T21:44:24.425 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T21:44:24.425 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.425 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.426 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.426 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_step": { 2024-04-20T21:44:24.426 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_step", 2024-04-20T21:44:24.426 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:24.426 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.426 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.426 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.5", 2024-04-20T21:44:24.427 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0.001", 2024-04-20T21:44:24.427 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "0.999", 2024-04-20T21:44:24.427 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.427 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "aggressiveness of optimization", 2024-04-20T21:44:24.427 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T21:44:24.427 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.427 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.427 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.427 INFO:teuthology.orchestra.run.smithi022.stdout: "end_time": { 2024-04-20T21:44:24.428 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_time", 2024-04-20T21:44:24.428 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.428 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.428 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.428 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2359", 2024-04-20T21:44:24.428 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.428 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.428 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.428 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T21:44:24.429 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T21:44:24.429 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.429 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.429 INFO:teuthology.orchestra.run.smithi022.stdout: "end_weekday": { 2024-04-20T21:44:24.429 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_weekday", 2024-04-20T21:44:24.429 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:24.429 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.429 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.429 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:24.430 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-04-20T21:44:24.430 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-04-20T21:44:24.430 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.430 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T21:44:24.430 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T21:44:24.430 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.430 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.430 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.430 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.431 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.431 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.431 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.431 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.431 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.431 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.431 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.431 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.431 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.435 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.435 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.435 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.435 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.435 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.435 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.435 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.435 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.435 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.436 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.436 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.436 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.436 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.436 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.436 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.436 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.436 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.436 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.437 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.437 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.437 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.437 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.437 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.437 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.437 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.437 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.438 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.438 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.438 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.438 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.438 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.438 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.439 INFO:teuthology.orchestra.run.smithi022.stdout: "min_score": { 2024-04-20T21:44:24.439 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_score", 2024-04-20T21:44:24.439 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:24.439 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.439 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.439 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:24.439 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.440 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.440 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.440 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T21:44:24.440 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.440 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.440 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.440 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.441 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-04-20T21:44:24.441 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-04-20T21:44:24.441 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.441 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.441 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.441 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "upmap", 2024-04-20T21:44:24.442 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.442 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.442 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.442 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-compat", 2024-04-20T21:44:24.442 INFO:teuthology.orchestra.run.smithi022.stdout: "none", 2024-04-20T21:44:24.442 INFO:teuthology.orchestra.run.smithi022.stdout: "read", 2024-04-20T21:44:24.442 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap", 2024-04-20T21:44:24.443 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap-read" 2024-04-20T21:44:24.443 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.443 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Balancer mode", 2024-04-20T21:44:24.443 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.443 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.443 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.443 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.444 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_ids": { 2024-04-20T21:44:24.444 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_ids", 2024-04-20T21:44:24.444 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.444 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.444 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.444 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.444 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.444 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.445 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.445 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T21:44:24.445 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.445 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.445 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.445 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.445 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:24.445 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:24.445 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.445 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.446 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.446 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:24.446 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.446 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.446 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.446 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T21:44:24.446 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.449 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.449 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.450 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.450 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_deviation": { 2024-04-20T21:44:24.450 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_deviation", 2024-04-20T21:44:24.450 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.450 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.450 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.450 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-04-20T21:44:24.450 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-04-20T21:44:24.450 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.451 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.451 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T21:44:24.451 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T21:44:24.451 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.451 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.451 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.451 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_optimizations": { 2024-04-20T21:44:24.451 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_optimizations", 2024-04-20T21:44:24.451 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:24.452 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.452 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.452 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:24.452 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.452 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.452 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.452 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T21:44:24.452 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.452 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.453 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.453 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.453 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.453 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.453 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm", 2024-04-20T21:44:24.453 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.453 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.453 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.454 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_down_multiplier": { 2024-04-20T21:44:24.454 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_down_multiplier", 2024-04-20T21:44:24.454 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:24.454 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.454 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.454 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-04-20T21:44:24.454 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.454 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.455 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.455 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T21:44:24.455 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.455 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.455 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.455 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.455 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_refresh_rate": { 2024-04-20T21:44:24.455 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_refresh_rate", 2024-04-20T21:44:24.455 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.456 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.456 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.456 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "20", 2024-04-20T21:44:24.456 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.456 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.456 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.456 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T21:44:24.456 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.456 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.456 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.457 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.457 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_starting_port": { 2024-04-20T21:44:24.457 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_starting_port", 2024-04-20T21:44:24.457 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.457 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.457 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.457 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4721", 2024-04-20T21:44:24.457 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.457 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.458 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.458 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T21:44:24.458 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.458 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.458 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.458 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_ptrace": { 2024-04-20T21:44:24.459 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_ptrace", 2024-04-20T21:44:24.459 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.459 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.459 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.459 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.459 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.459 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.459 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.459 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T21:44:24.460 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T21:44:24.460 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.460 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.460 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.460 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_interval": { 2024-04-20T21:44:24.460 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_interval", 2024-04-20T21:44:24.460 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.460 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.461 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.461 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:24.461 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.461 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.461 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.461 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T21:44:24.461 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.461 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.461 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.462 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.462 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_memory_target_ratio": { 2024-04-20T21:44:24.462 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T21:44:24.462 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:24.462 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.462 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.462 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.7", 2024-04-20T21:44:24.463 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.463 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.463 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.463 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T21:44:24.463 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.463 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.463 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.463 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.464 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm_log_destination": { 2024-04-20T21:44:24.464 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm_log_destination", 2024-04-20T21:44:24.464 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.464 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.464 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.464 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.464 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.464 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.465 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.465 INFO:teuthology.orchestra.run.smithi022.stdout: "file", 2024-04-20T21:44:24.465 INFO:teuthology.orchestra.run.smithi022.stdout: "file,syslog", 2024-04-20T21:44:24.465 INFO:teuthology.orchestra.run.smithi022.stdout: "syslog" 2024-04-20T21:44:24.465 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.465 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T21:44:24.465 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.465 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.465 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.466 INFO:teuthology.orchestra.run.smithi022.stdout: "cgroups_split": { 2024-04-20T21:44:24.466 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cgroups_split", 2024-04-20T21:44:24.466 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.466 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.466 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.466 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.466 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.467 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.467 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.467 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T21:44:24.467 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.467 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.467 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.467 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.467 INFO:teuthology.orchestra.run.smithi022.stdout: "config_checks_enabled": { 2024-04-20T21:44:24.467 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_checks_enabled", 2024-04-20T21:44:24.468 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.468 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.468 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.468 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.468 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.468 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.468 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.468 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T21:44:24.469 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.469 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.469 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.469 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.469 INFO:teuthology.orchestra.run.smithi022.stdout: "config_dashboard": { 2024-04-20T21:44:24.469 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_dashboard", 2024-04-20T21:44:24.469 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.469 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.469 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.470 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.470 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.470 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.470 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.470 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T21:44:24.470 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.470 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.470 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.471 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.471 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_alertmanager": { 2024-04-20T21:44:24.471 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_alertmanager", 2024-04-20T21:44:24.471 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.471 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.471 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.471 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T21:44:24.471 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.471 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.472 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.472 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-04-20T21:44:24.472 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.472 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.472 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.472 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.472 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_base": { 2024-04-20T21:44:24.472 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_base", 2024-04-20T21:44:24.472 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.473 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.473 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.473 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T21:44:24.473 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.473 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.473 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.473 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Container image name, without the tag", 2024-04-20T21:44:24.473 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.473 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.474 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.474 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.474 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_elasticsearch": { 2024-04-20T21:44:24.474 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_elasticsearch", 2024-04-20T21:44:24.474 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.474 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.474 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.474 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T21:44:24.474 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.475 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.475 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.475 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "elasticsearch container image", 2024-04-20T21:44:24.475 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.475 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.475 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.475 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.475 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_grafana": { 2024-04-20T21:44:24.475 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_grafana", 2024-04-20T21:44:24.476 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.476 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.476 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.476 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T21:44:24.476 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.476 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.476 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.476 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-04-20T21:44:24.476 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.477 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.477 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.477 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.477 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_haproxy": { 2024-04-20T21:44:24.477 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_haproxy", 2024-04-20T21:44:24.477 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.477 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.477 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.477 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T21:44:24.478 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.478 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.478 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.478 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "HAproxy container image", 2024-04-20T21:44:24.478 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.478 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.478 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.478 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.479 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_agent": { 2024-04-20T21:44:24.479 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_agent", 2024-04-20T21:44:24.479 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.479 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.479 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.479 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T21:44:24.479 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.479 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.480 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.480 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger agent container image", 2024-04-20T21:44:24.480 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.480 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.480 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.480 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.480 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_collector": { 2024-04-20T21:44:24.480 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_collector", 2024-04-20T21:44:24.480 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.481 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.481 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.481 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T21:44:24.481 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.481 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.481 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.481 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger collector container image", 2024-04-20T21:44:24.481 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.482 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.482 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.482 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.482 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_query": { 2024-04-20T21:44:24.482 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_query", 2024-04-20T21:44:24.482 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.482 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.482 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.483 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T21:44:24.483 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.483 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.483 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.483 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger query container image", 2024-04-20T21:44:24.483 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.483 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.483 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.483 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.484 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_keepalived": { 2024-04-20T21:44:24.484 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_keepalived", 2024-04-20T21:44:24.484 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.484 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.484 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.484 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T21:44:24.484 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.484 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.485 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.485 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Keepalived container image", 2024-04-20T21:44:24.485 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.485 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.485 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.485 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.485 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_loki": { 2024-04-20T21:44:24.485 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_loki", 2024-04-20T21:44:24.485 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.486 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.486 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.486 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T21:44:24.486 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.486 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.486 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.486 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Loki container image", 2024-04-20T21:44:24.486 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.486 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.487 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.487 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.487 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_node_exporter": { 2024-04-20T21:44:24.487 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_node_exporter", 2024-04-20T21:44:24.487 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.487 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.487 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.487 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T21:44:24.487 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.488 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.488 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.488 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-04-20T21:44:24.488 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.488 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.488 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.488 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.488 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_nvmeof": { 2024-04-20T21:44:24.488 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_nvmeof", 2024-04-20T21:44:24.489 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.489 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.489 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.489 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T21:44:24.489 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.489 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.489 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.489 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Nvme-of container image", 2024-04-20T21:44:24.489 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.489 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.490 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.490 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.490 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_prometheus": { 2024-04-20T21:44:24.490 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_prometheus", 2024-04-20T21:44:24.490 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.490 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.490 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.490 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T21:44:24.491 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.491 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.491 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.491 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-04-20T21:44:24.491 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.491 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.491 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.491 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.492 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_promtail": { 2024-04-20T21:44:24.492 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_promtail", 2024-04-20T21:44:24.492 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.492 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.492 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.492 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T21:44:24.492 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.492 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.493 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.493 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Promtail container image", 2024-04-20T21:44:24.493 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.493 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.493 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.493 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.493 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_samba": { 2024-04-20T21:44:24.494 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_samba", 2024-04-20T21:44:24.494 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.494 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.494 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.494 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T21:44:24.494 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.494 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.494 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.495 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Samba/SMB container image", 2024-04-20T21:44:24.495 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.495 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.495 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.495 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.495 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_snmp_gateway": { 2024-04-20T21:44:24.495 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_snmp_gateway", 2024-04-20T21:44:24.496 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.496 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.496 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.496 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T21:44:24.496 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.496 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.496 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.496 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SNMP Gateway container image", 2024-04-20T21:44:24.496 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.497 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.497 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.497 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.497 INFO:teuthology.orchestra.run.smithi022.stdout: "container_init": { 2024-04-20T21:44:24.497 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_init", 2024-04-20T21:44:24.497 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.497 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.497 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.498 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.498 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.498 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.498 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.498 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T21:44:24.498 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.498 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.499 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.499 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.499 INFO:teuthology.orchestra.run.smithi022.stdout: "daemon_cache_timeout": { 2024-04-20T21:44:24.499 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "daemon_cache_timeout", 2024-04-20T21:44:24.499 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.499 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.499 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.499 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:24.500 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.500 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.500 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.500 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T21:44:24.500 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.500 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.500 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.500 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.501 INFO:teuthology.orchestra.run.smithi022.stdout: "default_cephadm_command_timeout": { 2024-04-20T21:44:24.501 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T21:44:24.501 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.501 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.501 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.501 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "900", 2024-04-20T21:44:24.501 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.501 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.502 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.502 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T21:44:24.502 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.502 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.502 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.502 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.502 INFO:teuthology.orchestra.run.smithi022.stdout: "default_registry": { 2024-04-20T21:44:24.502 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_registry", 2024-04-20T21:44:24.503 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.503 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.503 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.503 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io", 2024-04-20T21:44:24.503 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.503 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.503 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.504 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T21:44:24.504 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.504 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.504 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.504 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.504 INFO:teuthology.orchestra.run.smithi022.stdout: "device_cache_timeout": { 2024-04-20T21:44:24.505 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_cache_timeout", 2024-04-20T21:44:24.505 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.505 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.505 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.505 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1800", 2024-04-20T21:44:24.505 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.505 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.505 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.505 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache device inventory", 2024-04-20T21:44:24.506 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.506 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.506 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.506 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.506 INFO:teuthology.orchestra.run.smithi022.stdout: "device_enhanced_scan": { 2024-04-20T21:44:24.506 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_enhanced_scan", 2024-04-20T21:44:24.506 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.506 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.506 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.507 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.507 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.507 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.507 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.507 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T21:44:24.507 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.507 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.507 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.507 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.508 INFO:teuthology.orchestra.run.smithi022.stdout: "facts_cache_timeout": { 2024-04-20T21:44:24.508 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "facts_cache_timeout", 2024-04-20T21:44:24.508 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.508 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.508 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.508 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:24.508 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.508 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.508 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.509 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache host facts data", 2024-04-20T21:44:24.509 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.509 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.509 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.509 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.509 INFO:teuthology.orchestra.run.smithi022.stdout: "grafana_dashboards_path": { 2024-04-20T21:44:24.509 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "grafana_dashboards_path", 2024-04-20T21:44:24.509 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.509 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.510 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.510 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T21:44:24.510 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.510 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.510 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.510 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T21:44:24.510 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.510 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.510 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.511 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.511 INFO:teuthology.orchestra.run.smithi022.stdout: "host_check_interval": { 2024-04-20T21:44:24.511 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "host_check_interval", 2024-04-20T21:44:24.511 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.511 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.511 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.511 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:24.511 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.511 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.512 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.512 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to perform a host check", 2024-04-20T21:44:24.512 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.512 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.512 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.512 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.512 INFO:teuthology.orchestra.run.smithi022.stdout: "hw_monitoring": { 2024-04-20T21:44:24.512 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hw_monitoring", 2024-04-20T21:44:24.512 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.513 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.513 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.513 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.513 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.513 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.513 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.513 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T21:44:24.513 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.513 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.514 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.514 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.514 INFO:teuthology.orchestra.run.smithi022.stdout: "inventory_list_all": { 2024-04-20T21:44:24.514 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "inventory_list_all", 2024-04-20T21:44:24.514 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.514 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.514 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.514 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.515 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.515 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.515 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.515 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T21:44:24.515 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.515 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.515 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.515 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.515 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.516 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.516 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.516 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.516 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.516 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.516 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.516 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.516 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.516 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.517 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.517 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.517 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.517 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.517 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.517 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.517 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.517 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.518 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.518 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.518 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.518 INFO:teuthology.orchestra.run.smithi022.stdout: "log_refresh_metadata": { 2024-04-20T21:44:24.518 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_refresh_metadata", 2024-04-20T21:44:24.518 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.518 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.518 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.519 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.519 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.519 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.519 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.519 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T21:44:24.519 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.519 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.519 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.520 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.520 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.520 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.520 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.520 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.520 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.520 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.520 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.520 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.521 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.521 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T21:44:24.521 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.521 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.521 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.521 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.521 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.521 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.521 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.522 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.522 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.522 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.522 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.522 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.522 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.522 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.522 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.522 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.523 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.523 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.523 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.523 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.523 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.523 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.523 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.523 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.524 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.524 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.524 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.524 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.524 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.524 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.524 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.524 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.524 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.525 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.525 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.525 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.525 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.525 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.525 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.525 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T21:44:24.525 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T21:44:24.526 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.526 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.526 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.526 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.526 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.526 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.526 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.526 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T21:44:24.527 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.527 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.527 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.527 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.527 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T21:44:24.527 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T21:44:24.527 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.527 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.527 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.528 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "*", 2024-04-20T21:44:24.528 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.528 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.528 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.528 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T21:44:24.528 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.528 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.528 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.529 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.529 INFO:teuthology.orchestra.run.smithi022.stdout: "max_count_per_host": { 2024-04-20T21:44:24.529 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_count_per_host", 2024-04-20T21:44:24.529 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.529 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.529 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.529 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:24.529 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.529 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.530 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.530 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of daemons per service per host", 2024-04-20T21:44:24.530 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.530 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.530 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.530 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.530 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd_draining_count": { 2024-04-20T21:44:24.530 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_osd_draining_count", 2024-04-20T21:44:24.531 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.531 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.531 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.531 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:24.531 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.531 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.531 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.531 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T21:44:24.531 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.532 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.532 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.532 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.532 INFO:teuthology.orchestra.run.smithi022.stdout: "migration_current": { 2024-04-20T21:44:24.532 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "migration_current", 2024-04-20T21:44:24.532 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.532 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.532 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.533 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.533 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.533 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.533 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.533 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "internal - do not modify", 2024-04-20T21:44:24.533 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.533 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.533 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.534 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.534 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-04-20T21:44:24.534 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-04-20T21:44:24.534 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.534 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.534 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.534 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "root", 2024-04-20T21:44:24.535 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.535 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.535 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.535 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm-package", 2024-04-20T21:44:24.535 INFO:teuthology.orchestra.run.smithi022.stdout: "root" 2024-04-20T21:44:24.535 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.535 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T21:44:24.536 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.536 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.536 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.536 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.536 INFO:teuthology.orchestra.run.smithi022.stdout: "oob_default_addr": { 2024-04-20T21:44:24.536 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "oob_default_addr", 2024-04-20T21:44:24.536 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.536 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.536 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.537 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "169.254.1.1", 2024-04-20T21:44:24.537 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.537 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.537 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.537 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T21:44:24.537 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.537 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.538 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.538 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.538 INFO:teuthology.orchestra.run.smithi022.stdout: "prometheus_alerts_path": { 2024-04-20T21:44:24.538 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus_alerts_path", 2024-04-20T21:44:24.538 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.538 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.538 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.538 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T21:44:24.539 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.539 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.539 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.539 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T21:44:24.539 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.539 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.539 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.540 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.540 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_insecure": { 2024-04-20T21:44:24.540 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_insecure", 2024-04-20T21:44:24.540 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.540 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.540 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.540 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.541 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.541 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.541 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.541 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T21:44:24.541 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.541 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.541 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.542 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.542 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_password": { 2024-04-20T21:44:24.542 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_password", 2024-04-20T21:44:24.542 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.542 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.542 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.542 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.543 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.543 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.543 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.543 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T21:44:24.543 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.543 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.543 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.544 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.544 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_url": { 2024-04-20T21:44:24.544 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_url", 2024-04-20T21:44:24.544 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.544 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.544 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.544 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.544 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.544 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.545 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.545 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T21:44:24.545 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.545 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.545 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.545 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.545 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_username": { 2024-04-20T21:44:24.545 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_username", 2024-04-20T21:44:24.545 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.546 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.546 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.546 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.546 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.546 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.546 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.546 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T21:44:24.546 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.547 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.547 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.547 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.547 INFO:teuthology.orchestra.run.smithi022.stdout: "secure_monitoring_stack": { 2024-04-20T21:44:24.547 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secure_monitoring_stack", 2024-04-20T21:44:24.547 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.547 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.547 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.547 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.548 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.548 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.548 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.548 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T21:44:24.548 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.548 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.548 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.548 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.548 INFO:teuthology.orchestra.run.smithi022.stdout: "service_discovery_port": { 2024-04-20T21:44:24.549 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "service_discovery_port", 2024-04-20T21:44:24.549 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.549 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.549 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.549 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8765", 2024-04-20T21:44:24.549 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.549 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.549 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.549 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "cephadm service discovery port", 2024-04-20T21:44:24.550 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.550 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.550 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.550 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.550 INFO:teuthology.orchestra.run.smithi022.stdout: "ssh_config_file": { 2024-04-20T21:44:24.550 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssh_config_file", 2024-04-20T21:44:24.550 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.550 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.551 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.551 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.551 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.551 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.551 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.551 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T21:44:24.551 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.551 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.551 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.552 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.552 INFO:teuthology.orchestra.run.smithi022.stdout: "use_agent": { 2024-04-20T21:44:24.552 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_agent", 2024-04-20T21:44:24.552 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.552 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.552 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.552 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.552 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.552 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.553 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.553 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T21:44:24.553 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.553 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.553 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.553 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.553 INFO:teuthology.orchestra.run.smithi022.stdout: "use_repo_digest": { 2024-04-20T21:44:24.553 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_repo_digest", 2024-04-20T21:44:24.553 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.554 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.554 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.554 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.554 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.554 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.554 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.554 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T21:44:24.554 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.555 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.555 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.555 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.555 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_failed_host_check": { 2024-04-20T21:44:24.555 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_failed_host_check", 2024-04-20T21:44:24.555 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.555 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.555 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.555 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.556 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.556 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.556 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.556 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T21:44:24.556 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.556 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.556 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.556 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.556 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_daemons": { 2024-04-20T21:44:24.557 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_daemons", 2024-04-20T21:44:24.557 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.557 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.557 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.557 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.557 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.557 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.557 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.557 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T21:44:24.558 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.558 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.558 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.558 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.558 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_hosts": { 2024-04-20T21:44:24.558 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_hosts", 2024-04-20T21:44:24.558 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.558 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.558 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.559 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.559 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.559 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.559 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.559 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T21:44:24.559 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.559 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.559 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.560 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.560 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.560 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.560 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.560 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crash", 2024-04-20T21:44:24.560 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.560 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.560 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.561 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.561 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.561 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.561 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.561 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.561 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.561 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.561 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.561 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.562 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.562 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.562 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.562 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.562 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.562 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.562 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.562 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.563 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.563 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.563 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.563 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.563 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.563 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.563 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.563 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.564 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.564 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.564 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.564 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.564 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.564 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.564 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.564 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.564 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.565 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.565 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.565 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.565 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.565 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.565 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.565 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.565 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.566 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.566 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.566 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.566 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.566 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.566 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.566 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.566 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.566 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.567 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.567 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.567 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.567 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.567 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.567 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.567 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.567 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.568 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.568 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.568 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.568 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.568 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.568 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.568 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.568 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.568 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.569 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.569 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.569 INFO:teuthology.orchestra.run.smithi022.stdout: "retain_interval": { 2024-04-20T21:44:24.569 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retain_interval", 2024-04-20T21:44:24.569 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.569 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.569 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.569 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "31536000", 2024-04-20T21:44:24.570 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.570 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.570 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.570 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T21:44:24.570 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.570 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.570 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.570 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.571 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_recent_interval": { 2024-04-20T21:44:24.571 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_recent_interval", 2024-04-20T21:44:24.571 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.571 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.571 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.571 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1209600", 2024-04-20T21:44:24.571 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.571 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.571 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.572 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T21:44:24.572 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.572 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.572 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.572 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.572 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.572 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.572 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dashboard", 2024-04-20T21:44:24.573 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.573 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.573 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.573 INFO:teuthology.orchestra.run.smithi022.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T21:44:24.573 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T21:44:24.573 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.573 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.573 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.574 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:24.574 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.574 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.574 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.574 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.574 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.574 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.574 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.575 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T21:44:24.575 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T21:44:24.575 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.575 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.575 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.575 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.575 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.575 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.575 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.576 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.576 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.576 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.576 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.576 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T21:44:24.576 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T21:44:24.576 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.576 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.577 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.577 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.577 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.577 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.577 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.577 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.577 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.577 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.578 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.578 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_ENABLED": { 2024-04-20T21:44:24.578 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T21:44:24.578 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.578 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.578 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.578 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.579 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.579 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.579 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.579 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.579 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.579 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.579 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.579 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T21:44:24.580 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T21:44:24.580 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.580 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.580 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.580 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.580 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.580 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.580 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.580 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.581 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.581 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.581 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.581 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.581 INFO:teuthology.orchestra.run.smithi022.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T21:44:24.581 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T21:44:24.581 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.581 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.581 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.582 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.582 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.582 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.582 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.582 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.582 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.582 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.582 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.583 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.583 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T21:44:24.583 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T21:44:24.583 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.583 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.583 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.583 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.583 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.584 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.584 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.584 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.584 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.584 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.584 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.584 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T21:44:24.584 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T21:44:24.585 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.585 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.585 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.585 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.585 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.585 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.585 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.585 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.585 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.586 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.586 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.586 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.586 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T21:44:24.586 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T21:44:24.586 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.586 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.586 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.587 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.587 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.587 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.587 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.587 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.587 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.587 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.587 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.588 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T21:44:24.588 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T21:44:24.588 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.588 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.588 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.588 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.588 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.588 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.589 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.589 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.589 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.589 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.589 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.589 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T21:44:24.589 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T21:44:24.590 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.590 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.590 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.590 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.590 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.590 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.590 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.590 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.591 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.591 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.591 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.591 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T21:44:24.591 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T21:44:24.591 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.591 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.591 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.592 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.592 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.592 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.592 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.592 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.592 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.592 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.592 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.593 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T21:44:24.593 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T21:44:24.593 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.593 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.593 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.593 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.593 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.593 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.594 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.594 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.594 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.594 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.594 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.594 INFO:teuthology.orchestra.run.smithi022.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T21:44:24.594 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T21:44:24.594 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.594 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.595 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.595 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.595 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.595 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.595 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.595 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.595 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.595 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.595 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.596 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T21:44:24.596 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T21:44:24.596 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.596 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.596 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.596 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-04-20T21:44:24.596 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.596 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.597 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.597 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.597 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.597 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.597 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.597 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T21:44:24.597 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T21:44:24.598 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.598 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.598 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.598 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.598 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.598 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.598 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.598 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.599 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.599 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.599 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.599 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_URL": { 2024-04-20T21:44:24.599 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_URL", 2024-04-20T21:44:24.599 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.599 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.599 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.600 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.600 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.600 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.600 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.600 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.600 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.600 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.600 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.601 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T21:44:24.601 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T21:44:24.601 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.601 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.601 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.601 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-04-20T21:44:24.601 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.601 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.601 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.602 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.602 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.602 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.602 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.602 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T21:44:24.602 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T21:44:24.602 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.603 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.603 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.603 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.603 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.603 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.603 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.603 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.603 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.604 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.604 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.604 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T21:44:24.604 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T21:44:24.604 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.604 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.604 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.604 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.605 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.605 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.605 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.605 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.605 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.605 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.605 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.605 INFO:teuthology.orchestra.run.smithi022.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T21:44:24.606 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T21:44:24.606 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.606 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.606 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.606 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.606 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.606 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.606 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.606 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.607 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.607 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.607 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.607 INFO:teuthology.orchestra.run.smithi022.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T21:44:24.607 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T21:44:24.607 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.607 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.608 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.608 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.608 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.608 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.608 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.608 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.608 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.608 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.608 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.609 INFO:teuthology.orchestra.run.smithi022.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T21:44:24.609 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T21:44:24.609 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.609 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.609 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.609 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "[]", 2024-04-20T21:44:24.610 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.610 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.610 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.610 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.610 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.610 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.610 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.611 INFO:teuthology.orchestra.run.smithi022.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T21:44:24.611 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T21:44:24.611 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.611 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.611 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.611 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "{}", 2024-04-20T21:44:24.611 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.611 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.612 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.612 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.612 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.612 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.612 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.612 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T21:44:24.612 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T21:44:24.613 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.613 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.613 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.613 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.613 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.613 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.613 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.613 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.613 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.614 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.614 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.614 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T21:44:24.614 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T21:44:24.614 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.614 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.614 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.615 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.615 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.615 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.615 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.615 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.615 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.615 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.615 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.616 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T21:44:24.616 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T21:44:24.616 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.616 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.616 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.616 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.616 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.616 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.617 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.617 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.617 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.617 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.617 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.617 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T21:44:24.617 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T21:44:24.618 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.618 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.618 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.618 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.618 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.618 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.618 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.618 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.618 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.619 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.619 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.619 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T21:44:24.619 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T21:44:24.619 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.619 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.619 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.620 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.620 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.620 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.620 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.620 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.620 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.620 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.620 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.621 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T21:44:24.621 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T21:44:24.621 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.621 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.621 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.621 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.621 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.621 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.621 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.622 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.622 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.622 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.622 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.622 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T21:44:24.622 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T21:44:24.622 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.623 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.623 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.623 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.623 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.623 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.623 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.623 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.623 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.623 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.624 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.624 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T21:44:24.624 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T21:44:24.624 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.624 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.624 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.624 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.624 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.625 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.625 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.625 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.625 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.625 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.625 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.625 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T21:44:24.626 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T21:44:24.626 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.626 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.626 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.626 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.626 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.626 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.626 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.627 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.627 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.627 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.627 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.627 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T21:44:24.627 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T21:44:24.627 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.628 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.628 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.628 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.628 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.628 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.628 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.628 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.628 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.628 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.629 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.629 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T21:44:24.629 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T21:44:24.629 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.629 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.629 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.629 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T21:44:24.630 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.630 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.630 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.630 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.630 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.630 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.630 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.631 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T21:44:24.631 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T21:44:24.631 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.631 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.631 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.631 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:24.631 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.631 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.631 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.632 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.632 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.632 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.632 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.632 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T21:44:24.632 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T21:44:24.632 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.632 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.632 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.633 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8", 2024-04-20T21:44:24.633 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.633 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.633 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.633 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.633 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.633 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.633 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.634 INFO:teuthology.orchestra.run.smithi022.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T21:44:24.634 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T21:44:24.634 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.634 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.634 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.634 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "45", 2024-04-20T21:44:24.634 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.634 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.634 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.635 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.635 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.635 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.635 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.635 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T21:44:24.635 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T21:44:24.635 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.636 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.636 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.636 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.636 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.636 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.636 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.636 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.636 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.636 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.637 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.637 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T21:44:24.637 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T21:44:24.637 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.637 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.637 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.637 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-04-20T21:44:24.638 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.638 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.638 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.638 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.638 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.638 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.638 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.639 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T21:44:24.639 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T21:44:24.639 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.639 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.639 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.639 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.639 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.639 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.640 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.640 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.640 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.640 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.640 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.640 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T21:44:24.640 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T21:44:24.640 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.641 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.641 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.641 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.641 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.641 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.641 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.641 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.641 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.642 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.642 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.642 INFO:teuthology.orchestra.run.smithi022.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T21:44:24.642 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T21:44:24.642 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.642 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.642 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.642 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.643 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.643 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.643 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.643 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.643 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.643 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.643 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T21:44:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T21:44:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.645 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.645 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.645 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.645 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.645 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T21:44:24.645 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T21:44:24.645 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.645 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.646 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.646 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:24.646 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.646 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.646 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.646 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.646 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.646 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.646 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.647 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T21:44:24.647 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T21:44:24.647 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.647 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.647 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.647 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-04-20T21:44:24.647 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.648 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.648 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.648 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.648 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.648 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.648 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.648 INFO:teuthology.orchestra.run.smithi022.stdout: "cross_origin_url": { 2024-04-20T21:44:24.648 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cross_origin_url", 2024-04-20T21:44:24.649 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.649 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.649 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.649 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.649 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.649 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.649 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.649 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.649 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.650 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.650 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.650 INFO:teuthology.orchestra.run.smithi022.stdout: "crt_file": { 2024-04-20T21:44:24.650 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crt_file", 2024-04-20T21:44:24.650 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.650 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.650 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.650 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.650 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.651 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.651 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.651 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.651 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.651 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.651 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.651 INFO:teuthology.orchestra.run.smithi022.stdout: "debug": { 2024-04-20T21:44:24.652 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "debug", 2024-04-20T21:44:24.652 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.652 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.652 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.652 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.652 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.652 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.652 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.652 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable/disable debug options", 2024-04-20T21:44:24.653 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.653 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.653 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.653 INFO:teuthology.orchestra.run.smithi022.stdout: "jwt_token_ttl": { 2024-04-20T21:44:24.653 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "jwt_token_ttl", 2024-04-20T21:44:24.653 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.653 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.653 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.653 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "28800", 2024-04-20T21:44:24.654 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.654 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.654 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.654 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.654 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.654 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.654 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.654 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-04-20T21:44:24.654 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-04-20T21:44:24.655 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.655 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.655 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.655 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.655 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.655 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.655 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.655 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.656 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.656 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.656 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.656 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.656 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.656 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.656 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.656 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.656 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.657 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.657 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.657 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.657 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.657 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.657 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.657 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.657 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.657 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.658 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.658 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.658 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.658 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.658 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.658 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.658 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.658 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.659 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.659 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.659 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.659 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.659 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.659 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.659 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.659 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.659 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.660 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.660 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.660 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.660 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.660 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.660 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.660 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.660 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.661 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.661 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.661 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.661 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.661 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.661 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.661 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.661 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.661 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.662 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.662 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.662 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.662 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.662 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.662 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.662 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.662 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.662 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.663 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.663 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.663 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.663 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.663 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.663 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.663 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.663 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.664 INFO:teuthology.orchestra.run.smithi022.stdout: "motd": { 2024-04-20T21:44:24.664 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "motd", 2024-04-20T21:44:24.664 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.664 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.664 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.664 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.664 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.665 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.665 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.665 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "The message of the day", 2024-04-20T21:44:24.665 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.665 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.665 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.665 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect_resolve_ip_addr": { 2024-04-20T21:44:24.665 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T21:44:24.666 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.666 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.666 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.666 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.666 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.666 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.666 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.666 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.667 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.667 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.667 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.667 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-04-20T21:44:24.667 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-04-20T21:44:24.667 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.667 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.667 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.668 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-04-20T21:44:24.668 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.668 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.668 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.668 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.668 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.668 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.668 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.669 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-04-20T21:44:24.669 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-04-20T21:44:24.669 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.669 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.669 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.669 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8080", 2024-04-20T21:44:24.669 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.670 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.670 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.670 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.670 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.670 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.670 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.670 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-04-20T21:44:24.670 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-04-20T21:44:24.671 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.671 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.671 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.671 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.671 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.671 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.671 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.671 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.671 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.672 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.672 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.672 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl_server_port": { 2024-04-20T21:44:24.672 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl_server_port", 2024-04-20T21:44:24.672 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.672 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.672 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.673 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8443", 2024-04-20T21:44:24.673 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.673 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.673 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.673 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.673 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.673 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.673 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.674 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-04-20T21:44:24.674 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-04-20T21:44:24.674 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.674 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.674 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.674 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "redirect", 2024-04-20T21:44:24.674 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.674 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.675 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.675 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.675 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect" 2024-04-20T21:44:24.675 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.675 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.675 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.675 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.675 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.676 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-04-20T21:44:24.676 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-04-20T21:44:24.676 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.676 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.676 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.676 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-04-20T21:44:24.676 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-04-20T21:44:24.676 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-04-20T21:44:24.677 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.677 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.677 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.677 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.677 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.677 INFO:teuthology.orchestra.run.smithi022.stdout: "url_prefix": { 2024-04-20T21:44:24.677 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url_prefix", 2024-04-20T21:44:24.677 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.678 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.678 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.678 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.678 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.678 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.678 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.678 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.679 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.679 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.679 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.679 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.679 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.679 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2024-04-20T21:44:24.679 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.680 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.680 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.680 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_monitoring": { 2024-04-20T21:44:24.680 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_monitoring", 2024-04-20T21:44:24.680 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.680 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.680 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.680 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.680 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.681 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.681 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.681 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "monitor device health metrics", 2024-04-20T21:44:24.681 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.681 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.681 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.681 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.682 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.682 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.682 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.682 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.682 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.682 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.682 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.682 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.682 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.683 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.683 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.683 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.683 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.683 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.683 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.683 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.683 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.683 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.684 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.684 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.684 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.684 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.684 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.684 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.684 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.685 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.685 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.685 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.685 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.685 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.685 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.685 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.685 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.686 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.686 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.686 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.686 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.686 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.686 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.686 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.686 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.687 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.687 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.687 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.687 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.687 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.687 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.687 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.687 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.687 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.688 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.688 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.688 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.688 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.688 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.688 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.688 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.688 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.689 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.689 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.689 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.689 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.689 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.689 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.689 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mark_out_threshold": { 2024-04-20T21:44:24.690 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mark_out_threshold", 2024-04-20T21:44:24.690 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.690 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.690 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.690 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2419200", 2024-04-20T21:44:24.690 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.690 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.691 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.691 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T21:44:24.691 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.691 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.691 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.691 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": { 2024-04-20T21:44:24.691 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_name", 2024-04-20T21:44:24.692 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.692 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.692 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.692 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "device_health_metrics", 2024-04-20T21:44:24.692 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.692 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.692 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.692 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T21:44:24.693 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.693 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.693 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.693 INFO:teuthology.orchestra.run.smithi022.stdout: "retention_period": { 2024-04-20T21:44:24.693 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retention_period", 2024-04-20T21:44:24.693 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.693 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.693 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.694 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15552000", 2024-04-20T21:44:24.694 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.694 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.694 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.694 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain device health metrics", 2024-04-20T21:44:24.694 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.694 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.694 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.695 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_frequency": { 2024-04-20T21:44:24.695 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_frequency", 2024-04-20T21:44:24.695 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.695 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.695 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.695 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-04-20T21:44:24.695 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.696 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.696 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.696 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T21:44:24.696 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.696 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.696 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.696 INFO:teuthology.orchestra.run.smithi022.stdout: "self_heal": { 2024-04-20T21:44:24.696 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "self_heal", 2024-04-20T21:44:24.697 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.697 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.697 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.697 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.697 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.697 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.697 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.697 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T21:44:24.698 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.698 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.698 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.698 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:24.698 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:24.698 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.698 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.698 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.699 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:24.699 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.699 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.699 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.699 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T21:44:24.699 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.699 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.699 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.700 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_threshold": { 2024-04-20T21:44:24.700 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_threshold", 2024-04-20T21:44:24.700 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.700 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.700 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.700 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7257600", 2024-04-20T21:44:24.700 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.700 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.700 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.701 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T21:44:24.701 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.701 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.701 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.701 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.701 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.701 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "diskprediction_local", 2024-04-20T21:44:24.702 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.702 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.702 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.702 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.702 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.702 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.702 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.702 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.702 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.703 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.703 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.703 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.703 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.703 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.703 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.703 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.703 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.703 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.704 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.704 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.704 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.704 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.704 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.704 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.704 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.705 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.705 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.705 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.705 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.705 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.705 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.705 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.705 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.706 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.706 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.706 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.706 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.706 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.706 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.706 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.707 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.707 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.707 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.707 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.707 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.707 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.707 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.707 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.708 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.708 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.708 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.708 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.708 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.708 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.709 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.709 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.709 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.709 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.709 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.709 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.709 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.709 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.710 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.710 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.710 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.710 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.710 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.710 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.710 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.710 INFO:teuthology.orchestra.run.smithi022.stdout: "predict_interval": { 2024-04-20T21:44:24.711 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predict_interval", 2024-04-20T21:44:24.711 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.711 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.711 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.711 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-04-20T21:44:24.711 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.711 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.711 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.711 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.712 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.712 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.713 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.713 INFO:teuthology.orchestra.run.smithi022.stdout: "predictor_model": { 2024-04-20T21:44:24.713 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predictor_model", 2024-04-20T21:44:24.714 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.714 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.714 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.714 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "prophetstor", 2024-04-20T21:44:24.714 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.714 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.714 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.715 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.715 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.715 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.715 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.715 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:24.715 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:24.715 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.716 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.716 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.716 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:24.716 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.716 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.716 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.716 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.716 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.716 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.717 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.717 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.717 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.717 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "influx", 2024-04-20T21:44:24.717 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": false, 2024-04-20T21:44:24.717 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "influxdb python module not found", 2024-04-20T21:44:24.717 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.718 INFO:teuthology.orchestra.run.smithi022.stdout: "batch_size": { 2024-04-20T21:44:24.718 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "batch_size", 2024-04-20T21:44:24.718 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.718 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.718 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.718 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5000", 2024-04-20T21:44:24.718 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.718 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.718 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.719 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T21:44:24.719 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.719 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.719 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.719 INFO:teuthology.orchestra.run.smithi022.stdout: "database": { 2024-04-20T21:44:24.719 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "database", 2024-04-20T21:44:24.719 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.720 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.720 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.720 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "ceph", 2024-04-20T21:44:24.720 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.720 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.720 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.720 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T21:44:24.720 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.720 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.720 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.721 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": { 2024-04-20T21:44:24.721 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hostname", 2024-04-20T21:44:24.721 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.721 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.721 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.721 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.721 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.721 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.722 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.722 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server hostname", 2024-04-20T21:44:24.722 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.722 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.722 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.722 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.722 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:24.722 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:24.723 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.723 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.723 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.723 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "30", 2024-04-20T21:44:24.723 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "5", 2024-04-20T21:44:24.723 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.723 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.723 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T21:44:24.723 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.724 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.724 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.724 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.724 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.724 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.724 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.724 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.724 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.725 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.725 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.725 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.725 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.725 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.725 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.725 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.725 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.726 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.726 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.726 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.726 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.726 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.726 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.726 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.726 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.727 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.727 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.727 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.727 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.727 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.727 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.727 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.727 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.727 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.728 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.728 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.728 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.728 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.728 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.728 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.728 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.728 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.728 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.729 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.729 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.729 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.729 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.729 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.729 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.729 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.729 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.729 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.730 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.730 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.730 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.730 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.730 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.730 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.730 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.731 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.731 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.731 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.731 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.731 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.731 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.731 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.731 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.731 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.732 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.732 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.732 INFO:teuthology.orchestra.run.smithi022.stdout: "password": { 2024-04-20T21:44:24.732 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "password", 2024-04-20T21:44:24.732 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.732 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.732 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.732 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.733 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.733 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.733 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.733 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "password of InfluxDB server user", 2024-04-20T21:44:24.733 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.733 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.733 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.733 INFO:teuthology.orchestra.run.smithi022.stdout: "port": { 2024-04-20T21:44:24.734 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "port", 2024-04-20T21:44:24.734 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.734 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.734 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.734 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8086", 2024-04-20T21:44:24.734 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.734 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.734 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.734 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server port", 2024-04-20T21:44:24.735 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.735 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.735 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.735 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-04-20T21:44:24.735 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-04-20T21:44:24.735 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.735 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.736 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.736 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "false", 2024-04-20T21:44:24.736 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.736 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.736 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.736 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T21:44:24.736 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.736 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.736 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.737 INFO:teuthology.orchestra.run.smithi022.stdout: "threads": { 2024-04-20T21:44:24.737 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threads", 2024-04-20T21:44:24.737 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.737 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.737 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.737 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-04-20T21:44:24.737 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-04-20T21:44:24.737 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "32", 2024-04-20T21:44:24.738 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.738 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T21:44:24.738 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.738 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.738 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.738 INFO:teuthology.orchestra.run.smithi022.stdout: "username": { 2024-04-20T21:44:24.738 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "username", 2024-04-20T21:44:24.739 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.739 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.739 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.739 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.739 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.739 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.739 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.739 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "username of InfluxDB server user", 2024-04-20T21:44:24.739 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.740 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.740 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.740 INFO:teuthology.orchestra.run.smithi022.stdout: "verify_ssl": { 2024-04-20T21:44:24.740 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "verify_ssl", 2024-04-20T21:44:24.740 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.740 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.740 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.740 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "true", 2024-04-20T21:44:24.741 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.741 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.741 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.741 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T21:44:24.741 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.741 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.741 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.741 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.742 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.742 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.742 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "insights", 2024-04-20T21:44:24.742 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.742 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.742 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.742 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.742 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.743 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.743 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.743 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.743 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.743 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.743 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.743 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.743 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.743 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.744 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.744 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.744 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.744 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.744 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.744 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.744 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.744 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.744 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.745 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.745 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.745 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.745 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.745 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.745 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.745 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.745 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.745 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.746 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.746 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.746 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.746 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.746 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.746 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.746 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.746 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.746 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.747 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.747 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.747 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.747 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.747 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.747 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.747 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.747 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.748 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.748 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.748 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.748 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.748 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.748 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.748 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.748 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.749 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.749 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.749 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.749 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.749 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.749 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.749 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.749 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.749 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.750 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.750 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.750 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.750 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.750 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.750 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.750 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.750 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.751 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.751 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.751 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "iostat", 2024-04-20T21:44:24.751 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.751 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.751 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.751 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.751 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.751 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.752 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.752 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.752 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.752 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.752 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.752 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.752 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.752 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.752 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.753 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.753 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.753 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.753 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.753 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.753 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.753 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.753 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.753 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.754 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.754 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.754 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.754 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.754 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.754 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.754 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.754 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.755 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.755 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.755 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.755 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.755 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.755 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.755 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.755 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.755 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.756 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.756 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.756 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.756 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.756 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.756 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.756 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.756 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.756 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.757 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.757 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.757 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.757 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.757 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.757 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.757 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.757 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.758 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.758 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.758 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.758 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.758 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.758 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.758 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.758 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.759 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.759 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.759 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.759 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.759 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.759 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.759 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.759 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.759 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.760 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.760 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "k8sevents", 2024-04-20T21:44:24.760 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.760 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.760 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.760 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_event_retention_days": { 2024-04-20T21:44:24.760 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ceph_event_retention_days", 2024-04-20T21:44:24.760 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.760 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.761 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.761 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7", 2024-04-20T21:44:24.761 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.761 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.761 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.761 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-20T21:44:24.761 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.761 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.762 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.762 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.762 INFO:teuthology.orchestra.run.smithi022.stdout: "config_check_secs": { 2024-04-20T21:44:24.762 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_check_secs", 2024-04-20T21:44:24.762 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.762 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.762 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.762 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:24.763 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "10", 2024-04-20T21:44:24.763 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.763 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.763 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-20T21:44:24.763 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.763 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.763 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.763 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.763 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.764 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.764 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.764 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.764 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.764 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.764 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.764 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.764 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.764 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.765 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.765 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.765 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.765 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.765 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.765 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.765 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.765 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.765 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.766 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.766 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.766 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.766 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.766 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.766 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.766 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.766 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.766 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.767 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.767 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.767 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.767 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.767 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.767 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.767 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.767 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.768 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.768 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.768 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.768 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.768 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.768 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.768 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.768 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.768 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.769 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.769 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.769 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.769 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.769 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.769 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.769 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.769 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.769 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.770 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.770 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.770 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.770 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.770 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.770 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.770 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.770 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.770 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.771 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.771 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.771 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.771 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.771 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.771 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.771 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.771 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.772 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "localpool", 2024-04-20T21:44:24.772 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.772 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.772 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.772 INFO:teuthology.orchestra.run.smithi022.stdout: "failure_domain": { 2024-04-20T21:44:24.772 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "failure_domain", 2024-04-20T21:44:24.772 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.772 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.772 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.772 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "host", 2024-04-20T21:44:24.773 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.773 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.773 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.773 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "failure domain for any created local pool", 2024-04-20T21:44:24.773 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T21:44:24.773 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.773 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.774 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.774 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.774 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.774 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.774 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.774 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.774 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.774 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.774 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.775 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.775 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.775 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.775 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.775 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.775 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.775 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.775 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.775 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.776 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.776 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.776 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.776 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.776 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.776 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.776 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.776 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.777 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.777 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.777 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.777 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.777 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.777 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.777 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.777 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.777 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.778 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.778 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.778 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.778 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.778 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.778 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.778 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.778 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.779 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.779 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.779 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.779 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.779 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.779 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.779 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.779 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.779 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.780 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.780 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.780 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.780 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.780 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.780 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.780 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.780 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.781 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.781 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.781 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.781 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.781 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.781 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.781 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.781 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": { 2024-04-20T21:44:24.782 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_size", 2024-04-20T21:44:24.782 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.782 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.782 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.782 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.782 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.782 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.782 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.783 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default min_size for any created local pool", 2024-04-20T21:44:24.783 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T21:44:24.783 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.783 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.783 INFO:teuthology.orchestra.run.smithi022.stdout: "num_rep": { 2024-04-20T21:44:24.783 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "num_rep", 2024-04-20T21:44:24.783 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.783 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.784 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.784 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3", 2024-04-20T21:44:24.784 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.784 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.784 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.784 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default replica count for any created local pool", 2024-04-20T21:44:24.784 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.784 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.784 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.785 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.785 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": { 2024-04-20T21:44:24.785 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_num", 2024-04-20T21:44:24.785 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.785 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.785 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.785 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "128", 2024-04-20T21:44:24.785 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.786 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.786 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.786 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T21:44:24.786 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.786 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.786 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.786 INFO:teuthology.orchestra.run.smithi022.stdout: "prefix": { 2024-04-20T21:44:24.786 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prefix", 2024-04-20T21:44:24.787 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.787 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.787 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.787 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.787 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.787 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.787 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.787 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name prefix for any created local pool", 2024-04-20T21:44:24.788 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.788 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.788 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.788 INFO:teuthology.orchestra.run.smithi022.stdout: "subtree": { 2024-04-20T21:44:24.788 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "subtree", 2024-04-20T21:44:24.788 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.788 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.788 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.789 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "rack", 2024-04-20T21:44:24.789 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.789 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.789 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.789 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T21:44:24.789 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T21:44:24.789 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.789 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.789 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.790 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.790 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.790 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mds_autoscaler", 2024-04-20T21:44:24.790 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.790 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.790 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.790 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.790 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.791 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.791 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.791 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.791 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.791 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.791 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.791 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.791 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.791 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.792 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.792 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.792 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.792 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.792 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.792 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.792 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.792 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.792 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.793 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.793 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.793 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.793 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.793 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.793 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.793 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.793 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.794 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.794 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.794 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.794 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.794 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.794 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.794 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.794 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.795 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.795 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.795 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.795 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.795 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.795 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.795 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.795 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.795 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.796 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.796 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.796 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.796 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.796 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.796 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.796 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.796 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.796 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.797 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.797 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.797 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.797 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.797 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.797 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.797 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.797 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.798 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.798 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.798 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.798 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.798 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.798 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.798 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.798 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.799 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.799 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirroring", 2024-04-20T21:44:24.799 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.799 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.799 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.799 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.799 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.799 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.799 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.800 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.800 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.800 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.800 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.800 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.800 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.800 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.800 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.800 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.801 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.801 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.801 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.801 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.801 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.801 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.801 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.802 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.802 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.802 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.802 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.802 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.802 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.802 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.802 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.803 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.803 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.803 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.803 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.803 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.803 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.803 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.803 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.804 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.804 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.804 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.804 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.804 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.804 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.804 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.804 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.804 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.805 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.805 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.805 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.805 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.805 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.805 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.805 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.805 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.805 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.806 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.806 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.806 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.806 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.806 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.806 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.806 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.806 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.807 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.807 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.807 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.807 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.807 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.807 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.807 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.808 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "nfs", 2024-04-20T21:44:24.808 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.808 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.808 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.808 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.808 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.808 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.808 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.808 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.809 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.809 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.809 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.809 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.809 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.809 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.809 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.809 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.810 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.810 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.810 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.810 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.810 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.810 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.810 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.810 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.811 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.811 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.811 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.811 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.811 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.811 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.811 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.811 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.811 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.812 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.812 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.812 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.812 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.812 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.812 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.812 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.812 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.812 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.813 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.813 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.813 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.813 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.813 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.813 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.813 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.813 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.813 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.814 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.814 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.814 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.814 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.814 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.814 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.814 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.815 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.815 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.815 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.815 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.815 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.815 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.815 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.815 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.815 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.816 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.816 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.816 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.816 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.816 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-04-20T21:44:24.816 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.817 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.817 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.817 INFO:teuthology.orchestra.run.smithi022.stdout: "fail_fs": { 2024-04-20T21:44:24.817 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "fail_fs", 2024-04-20T21:44:24.817 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.817 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.817 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.817 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.817 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.818 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.818 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.818 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T21:44:24.818 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.818 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.818 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.818 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.819 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.819 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.819 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.819 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.819 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.819 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.819 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.819 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.819 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.820 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.820 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.820 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.820 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.820 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.820 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.820 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.820 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.821 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.821 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.821 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.821 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.821 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.821 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.821 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.821 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.822 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.822 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.822 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.822 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.822 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.822 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.822 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.823 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.823 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.823 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.823 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.823 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.823 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.823 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.823 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.823 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.824 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.824 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.824 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.824 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.824 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.824 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.824 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.824 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.825 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.825 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.825 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.825 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.825 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.825 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.825 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.826 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.826 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.826 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.826 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.826 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.826 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.826 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.826 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.827 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.827 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator": { 2024-04-20T21:44:24.827 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-04-20T21:44:24.827 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.827 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.827 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.827 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.827 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.828 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.828 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.828 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2024-04-20T21:44:24.828 INFO:teuthology.orchestra.run.smithi022.stdout: "rook", 2024-04-20T21:44:24.828 INFO:teuthology.orchestra.run.smithi022.stdout: "test_orchestrator" 2024-04-20T21:44:24.828 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.828 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Orchestrator backend", 2024-04-20T21:44:24.828 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.828 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.829 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.829 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.829 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.829 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_perf_query", 2024-04-20T21:44:24.829 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.829 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.829 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.830 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.830 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.830 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.830 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.830 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.830 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.830 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.830 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.830 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.831 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.831 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.831 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.831 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.831 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.831 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.831 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.831 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.831 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.832 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.832 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.832 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.832 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.832 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.832 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.832 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.833 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.833 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.833 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.833 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.833 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.833 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.833 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.833 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.834 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.834 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.834 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.834 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.834 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.834 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.834 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.834 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.835 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.835 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.835 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.835 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.835 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.835 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.835 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.835 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.835 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.836 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.836 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.836 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.836 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.836 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.836 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.836 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.836 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.837 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.837 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.837 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.837 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.837 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.837 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.837 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.837 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.837 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.837 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_support", 2024-04-20T21:44:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.839 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.839 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.839 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.839 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.839 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.839 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.839 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.839 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.839 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.840 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.840 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.840 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.840 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.840 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.840 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.840 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.840 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.841 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.841 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.841 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.841 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.841 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.841 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.841 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.841 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.842 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.842 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.842 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.842 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.842 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.842 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.842 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.842 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.843 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.843 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.843 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.843 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.843 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.843 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.843 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.843 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.843 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.844 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.844 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.844 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.844 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.844 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.844 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.844 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.844 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.845 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.845 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.845 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.845 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.845 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.845 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.845 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.845 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.845 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.846 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.846 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.846 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.846 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.846 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.846 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.846 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.847 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_autoscaler", 2024-04-20T21:44:24.847 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.847 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.847 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.847 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.847 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.847 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.847 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.848 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.848 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.848 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.848 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.848 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.848 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.848 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.848 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.848 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.849 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.849 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.849 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.849 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.849 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.849 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.849 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.850 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.850 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.850 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.850 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.850 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.850 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.850 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.850 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.850 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.851 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.851 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.851 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.851 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.851 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.851 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.851 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.851 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.851 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.852 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.852 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.852 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.852 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.852 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.852 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.852 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.852 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.852 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.853 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.853 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.853 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.853 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.853 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.853 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.853 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.853 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.854 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.854 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.854 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.854 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.854 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.854 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.854 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.854 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.854 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.855 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.855 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.855 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.855 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.855 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:24.855 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:24.855 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.855 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.855 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.856 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:24.856 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.856 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.856 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.856 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.856 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.856 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.856 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.857 INFO:teuthology.orchestra.run.smithi022.stdout: "threshold": { 2024-04-20T21:44:24.857 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threshold", 2024-04-20T21:44:24.857 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:24.857 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.857 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.857 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-04-20T21:44:24.857 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1.0", 2024-04-20T21:44:24.857 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.858 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.858 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "scaling threshold", 2024-04-20T21:44:24.858 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T21:44:24.858 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.858 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.858 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.858 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.859 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "progress", 2024-04-20T21:44:24.859 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.859 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.859 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.859 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_pg_recovery_event": { 2024-04-20T21:44:24.859 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_pg_recovery_event", 2024-04-20T21:44:24.859 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.859 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.860 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.860 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.860 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.860 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.860 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.860 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T21:44:24.860 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.860 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.860 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.861 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-04-20T21:44:24.861 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-04-20T21:44:24.861 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.861 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.861 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.861 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.861 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.862 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.862 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.862 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.862 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.862 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.862 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.862 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.862 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.863 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.863 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.863 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.863 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.863 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.863 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.863 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.863 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.863 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.864 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.864 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.864 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.864 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.864 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.864 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.864 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.864 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.865 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.865 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.865 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.865 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.865 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.865 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.865 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.865 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.866 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.866 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.866 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.866 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.866 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.866 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.866 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.866 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.867 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.867 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.867 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.867 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.867 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.867 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.867 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.867 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.867 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.868 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.868 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.868 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.868 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.868 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.868 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.868 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.868 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.869 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.869 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.869 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.869 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.869 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.869 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.869 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.869 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.870 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.870 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.870 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.870 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.870 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.870 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.870 INFO:teuthology.orchestra.run.smithi022.stdout: "max_completed_events": { 2024-04-20T21:44:24.871 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_completed_events", 2024-04-20T21:44:24.871 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.871 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.871 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.871 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "50", 2024-04-20T21:44:24.871 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.871 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.871 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.871 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "number of past completed events to remember", 2024-04-20T21:44:24.872 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.872 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.872 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.872 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:24.872 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:24.872 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:24.872 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.872 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.873 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-04-20T21:44:24.873 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.873 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.873 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.873 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long the module is going to sleep", 2024-04-20T21:44:24.873 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.873 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.873 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.873 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.874 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.874 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus", 2024-04-20T21:44:24.874 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.874 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.874 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.874 INFO:teuthology.orchestra.run.smithi022.stdout: "cache": { 2024-04-20T21:44:24.875 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cache", 2024-04-20T21:44:24.875 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.875 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.875 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.875 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.875 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.875 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.875 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.875 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.876 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.876 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.876 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.876 INFO:teuthology.orchestra.run.smithi022.stdout: "exclude_perf_counters": { 2024-04-20T21:44:24.876 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "exclude_perf_counters", 2024-04-20T21:44:24.876 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.876 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.876 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.877 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.877 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.877 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.877 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.877 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T21:44:24.877 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T21:44:24.877 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.877 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.878 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.878 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.878 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.878 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.878 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.878 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.878 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.878 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.878 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.879 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.879 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.879 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.879 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.879 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.879 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.879 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.879 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.879 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.880 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.880 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.880 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.880 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.880 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.880 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.880 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.880 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.881 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.881 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.881 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.881 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.881 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.881 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.881 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.881 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.881 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.882 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.882 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.882 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.882 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.882 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.882 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.882 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.882 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.883 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.883 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.883 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.883 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.883 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.883 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.883 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.883 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.883 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.884 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.884 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.884 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.884 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.884 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.884 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.884 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.884 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.884 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.884 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.885 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.885 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.885 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.885 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.885 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.885 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools": { 2024-04-20T21:44:24.885 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools", 2024-04-20T21:44:24.885 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.885 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.886 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.886 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.886 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.886 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.886 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.886 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.886 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.886 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.887 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.887 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.887 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T21:44:24.887 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T21:44:24.887 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.887 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.887 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.887 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300", 2024-04-20T21:44:24.887 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.887 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.888 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.888 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.888 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.888 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.888 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.888 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_interval": { 2024-04-20T21:44:24.888 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_interval", 2024-04-20T21:44:24.889 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:24.889 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.889 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.889 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15.0", 2024-04-20T21:44:24.889 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.889 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.889 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.889 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.889 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.890 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.890 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.890 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.890 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-04-20T21:44:24.890 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-04-20T21:44:24.890 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.890 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.890 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.891 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-04-20T21:44:24.891 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.891 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.891 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.891 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T21:44:24.891 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.891 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.891 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.892 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-04-20T21:44:24.892 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-04-20T21:44:24.892 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.892 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.892 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.892 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "9283", 2024-04-20T21:44:24.892 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.892 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.892 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.892 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T21:44:24.893 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.893 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.893 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.893 INFO:teuthology.orchestra.run.smithi022.stdout: "stale_cache_strategy": { 2024-04-20T21:44:24.893 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stale_cache_strategy", 2024-04-20T21:44:24.893 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.893 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.894 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.894 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "log", 2024-04-20T21:44:24.894 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.894 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.894 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.894 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.894 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.894 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.894 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.895 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-04-20T21:44:24.895 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-04-20T21:44:24.895 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.895 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.895 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.895 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "default", 2024-04-20T21:44:24.895 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.895 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.896 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.896 INFO:teuthology.orchestra.run.smithi022.stdout: "default", 2024-04-20T21:44:24.896 INFO:teuthology.orchestra.run.smithi022.stdout: "error" 2024-04-20T21:44:24.896 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.896 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.896 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.896 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.896 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.897 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-04-20T21:44:24.897 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-04-20T21:44:24.897 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.897 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.897 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.897 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-04-20T21:44:24.897 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-04-20T21:44:24.897 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-04-20T21:44:24.897 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.898 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.898 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.898 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.898 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.898 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.898 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.898 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.899 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.899 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2024-04-20T21:44:24.899 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.899 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.899 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.899 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.899 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.899 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.899 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.900 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.900 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.900 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.900 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.900 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.900 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.900 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.900 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.900 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.901 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.901 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.901 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.901 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.901 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.901 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.901 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.901 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.901 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.902 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.902 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.902 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.902 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.902 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.902 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.902 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.902 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.903 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.903 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.903 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.903 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.903 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.903 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.903 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.903 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.904 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.904 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.904 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.904 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.904 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.904 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.904 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.904 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.904 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.905 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.905 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.905 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.905 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.905 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.905 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.905 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.906 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.906 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.906 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.906 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.906 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.906 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.906 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.906 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.906 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.907 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.907 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.907 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.907 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.907 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.907 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_snap_create": { 2024-04-20T21:44:24.907 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_snap_create", 2024-04-20T21:44:24.907 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.907 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.908 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.908 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:24.908 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.908 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.908 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.908 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.908 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.908 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.909 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.909 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.909 INFO:teuthology.orchestra.run.smithi022.stdout: "mirror_snapshot_schedule": { 2024-04-20T21:44:24.909 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T21:44:24.909 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.909 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.909 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.909 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.909 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.910 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.910 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.910 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.910 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.910 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.910 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.910 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.910 INFO:teuthology.orchestra.run.smithi022.stdout: "trash_purge_schedule": { 2024-04-20T21:44:24.911 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "trash_purge_schedule", 2024-04-20T21:44:24.911 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.911 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.911 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.911 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.911 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.911 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.912 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.912 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.912 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.912 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.912 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.912 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.912 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.912 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.912 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.913 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "restful", 2024-04-20T21:44:24.913 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.913 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.913 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.913 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_auth": { 2024-04-20T21:44:24.913 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_auth", 2024-04-20T21:44:24.913 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.913 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.914 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.914 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.914 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.914 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.914 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.914 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.914 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.914 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.914 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.915 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.915 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-04-20T21:44:24.915 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-04-20T21:44:24.915 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.915 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.915 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.915 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.915 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.916 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.916 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.916 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.916 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.916 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.916 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.916 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.916 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.917 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.917 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.917 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.917 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.917 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.917 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.917 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.917 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.917 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.918 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.918 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.918 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.918 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.918 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.918 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.918 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.918 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.918 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.919 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.919 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.919 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.919 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.919 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.919 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.919 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.919 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.919 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.920 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.920 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.920 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.920 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.920 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.920 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.920 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.921 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.921 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.921 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.921 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.921 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.921 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.921 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.921 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.921 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.922 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.922 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.922 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.922 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.922 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.922 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.922 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.922 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.922 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.923 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.923 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.923 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.923 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.923 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.923 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.923 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.923 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.924 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.924 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.924 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.924 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.924 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.924 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.924 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-04-20T21:44:24.924 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-04-20T21:44:24.925 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.925 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.925 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.925 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.925 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.925 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.925 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.925 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.926 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.926 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.926 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.926 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.926 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-04-20T21:44:24.926 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-04-20T21:44:24.926 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.926 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.927 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.927 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.927 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.927 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.927 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.927 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.927 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.927 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.928 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.928 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.928 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.928 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.928 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.928 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rgw", 2024-04-20T21:44:24.928 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.928 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.928 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.929 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.929 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.929 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.929 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.929 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.929 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.929 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.929 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.929 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.930 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.930 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.930 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.930 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.930 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.930 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.930 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.930 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.931 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.931 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.931 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.931 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.931 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.931 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.931 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.932 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.932 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.932 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.932 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.932 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.932 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.932 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.932 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.932 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.933 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.933 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.933 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.933 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.933 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.933 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.933 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.934 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.934 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.934 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.934 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.934 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.934 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.934 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.934 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.935 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.935 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.935 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.935 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.935 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.935 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.935 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.936 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.936 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.936 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.936 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.936 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.936 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.936 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.936 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.936 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.937 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.937 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.937 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.937 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.937 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rook", 2024-04-20T21:44:24.937 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.937 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.938 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.938 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.938 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.938 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.938 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.938 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.938 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.938 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.938 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.939 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.939 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.939 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.939 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.939 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.939 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.939 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.939 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.940 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.940 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.940 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.940 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.940 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.940 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.940 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.940 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.941 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.941 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.941 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.941 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.941 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.941 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.941 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.941 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.941 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.942 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.942 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.942 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.942 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.942 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.942 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.942 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.942 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.943 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.943 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.943 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.943 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.943 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.943 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.943 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.943 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.944 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.944 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.944 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.944 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.944 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.944 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.945 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.945 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.945 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.945 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.945 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.945 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.945 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.945 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.946 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.946 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.946 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.946 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.946 INFO:teuthology.orchestra.run.smithi022.stdout: "storage_class": { 2024-04-20T21:44:24.946 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "storage_class", 2024-04-20T21:44:24.946 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.946 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.947 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.947 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "local", 2024-04-20T21:44:24.947 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.947 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.947 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.947 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T21:44:24.947 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.947 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.948 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.948 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.948 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.948 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.948 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.948 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "selftest", 2024-04-20T21:44:24.948 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.948 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.949 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.949 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.949 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.949 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.949 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.949 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.949 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.949 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.949 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.950 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.950 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.950 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.950 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.950 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.950 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.950 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.950 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.951 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.951 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.951 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.951 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.951 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.951 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.951 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.952 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.952 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.952 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.952 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.952 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.952 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.952 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.952 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.952 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.953 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.953 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.953 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.953 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.953 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.953 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.953 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.953 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.954 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.954 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.954 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.954 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.954 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.954 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.954 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.954 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.954 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.955 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.955 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.955 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.955 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.955 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.955 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.955 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.956 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.956 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.956 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.956 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.956 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.956 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.956 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.957 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.957 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.957 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.957 INFO:teuthology.orchestra.run.smithi022.stdout: "roption1": { 2024-04-20T21:44:24.957 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption1", 2024-04-20T21:44:24.957 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.957 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.958 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.958 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.958 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.958 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.958 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.958 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.958 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.958 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.959 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.959 INFO:teuthology.orchestra.run.smithi022.stdout: "roption2": { 2024-04-20T21:44:24.959 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption2", 2024-04-20T21:44:24.959 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.959 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.959 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.959 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "xyz", 2024-04-20T21:44:24.959 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.960 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.960 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.960 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.960 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.960 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.960 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.960 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption1": { 2024-04-20T21:44:24.960 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption1", 2024-04-20T21:44:24.960 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.961 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.961 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.961 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.961 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.961 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.961 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.961 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.961 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.961 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.962 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.962 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption2": { 2024-04-20T21:44:24.962 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption2", 2024-04-20T21:44:24.962 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.962 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.962 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.962 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.963 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.963 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.963 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.963 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.963 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.963 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.963 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.964 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption3": { 2024-04-20T21:44:24.964 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption3", 2024-04-20T21:44:24.964 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:24.964 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.964 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.964 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.964 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.964 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.964 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.964 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.965 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.965 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.965 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.965 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption4": { 2024-04-20T21:44:24.965 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption4", 2024-04-20T21:44:24.965 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.965 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.966 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.966 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.966 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.966 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.966 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.966 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.966 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.966 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.967 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.967 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption5": { 2024-04-20T21:44:24.967 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption5", 2024-04-20T21:44:24.967 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.967 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.967 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.967 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.967 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.968 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.968 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.968 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.968 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.968 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.968 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.968 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.968 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption6": { 2024-04-20T21:44:24.968 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption6", 2024-04-20T21:44:24.969 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.969 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.969 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.969 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:24.969 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.969 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.969 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.969 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.970 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.970 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.970 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.970 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption7": { 2024-04-20T21:44:24.970 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption7", 2024-04-20T21:44:24.970 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:24.970 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.970 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.971 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.971 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-04-20T21:44:24.971 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "42", 2024-04-20T21:44:24.971 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.971 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.971 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.971 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.971 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.972 INFO:teuthology.orchestra.run.smithi022.stdout: "testkey": { 2024-04-20T21:44:24.972 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testkey", 2024-04-20T21:44:24.972 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.972 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.972 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.972 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.972 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.973 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.973 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.973 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.973 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.973 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.973 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.973 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.973 INFO:teuthology.orchestra.run.smithi022.stdout: "testlkey": { 2024-04-20T21:44:24.973 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testlkey", 2024-04-20T21:44:24.974 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.974 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.974 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.974 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.974 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.974 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.974 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.974 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.974 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.975 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.975 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.975 INFO:teuthology.orchestra.run.smithi022.stdout: "testnewline": { 2024-04-20T21:44:24.975 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testnewline", 2024-04-20T21:44:24.975 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.975 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.975 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:24.976 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.976 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.976 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.976 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.976 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.976 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.976 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.976 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.976 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.977 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.977 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.977 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.977 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-04-20T21:44:24.977 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.977 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.977 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.977 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_m_granularity": { 2024-04-20T21:44:24.977 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_m_granularity", 2024-04-20T21:44:24.978 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.978 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.978 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.978 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.978 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.978 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.978 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.978 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T21:44:24.979 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.979 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.979 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.979 INFO:teuthology.orchestra.run.smithi022.stdout: "dump_on_update": { 2024-04-20T21:44:24.979 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dump_on_update", 2024-04-20T21:44:24.979 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.979 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.979 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.980 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.980 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.980 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.980 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.980 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "dump database to debug log on update", 2024-04-20T21:44:24.980 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.980 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.980 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.980 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.981 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.981 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.981 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.981 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.981 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.981 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.981 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.981 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.981 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.982 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.982 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.982 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.982 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.982 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.982 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.982 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.982 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.982 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.983 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.983 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.983 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.983 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.983 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.983 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.983 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.984 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.984 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.984 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.984 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.984 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.984 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.984 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.984 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.985 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.985 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.985 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.985 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.985 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.985 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.985 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.985 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.985 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.986 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.986 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.986 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.986 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.986 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.986 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.986 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.986 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.987 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.987 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.987 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.987 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.987 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.987 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.987 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.987 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.988 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.988 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.988 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.988 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.988 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.988 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.988 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.988 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.988 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.989 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.989 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.989 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stats", 2024-04-20T21:44:24.989 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.989 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.989 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.989 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.989 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.990 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.990 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.990 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.990 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.990 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.990 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.990 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.990 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.991 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.991 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.991 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.991 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.991 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.991 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.991 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.991 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.991 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.992 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.992 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:24.992 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:24.992 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.992 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.992 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.992 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.992 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.993 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.993 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.993 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.993 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.993 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.993 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.993 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:24.994 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:24.994 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.994 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.994 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.994 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:24.994 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.994 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.994 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.994 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.995 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.995 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.995 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:24.995 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:24.995 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:24.995 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:24.995 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.995 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.995 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.996 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.996 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:24.996 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:24.996 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:24.996 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.996 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.996 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:24.996 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.996 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.997 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:24.997 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:24.997 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:24.997 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:24.997 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:24.997 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.997 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:24.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:24.997 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:24.997 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "status", 2024-04-20T21:44:24.998 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:24.998 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:24.998 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:24.998 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:24.998 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:24.998 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:24.998 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:24.998 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:24.999 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:24.999 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:24.999 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:24.999 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:24.999 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:24.999 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:24.999 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:24.999 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.000 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.000 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.000 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.000 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.000 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.000 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.000 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.000 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.001 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.001 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.001 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.001 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.001 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.001 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.001 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.001 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.001 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.002 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.002 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.002 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.002 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.002 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.002 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.002 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.003 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.003 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.003 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.003 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.003 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.003 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.003 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.003 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.004 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.004 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.004 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.004 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.004 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.004 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.004 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.004 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.005 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.005 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.005 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.005 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.005 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.005 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.005 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.005 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.005 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.006 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.006 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.006 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.006 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.006 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.006 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.006 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.007 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telegraf", 2024-04-20T21:44:25.007 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.007 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.007 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.007 INFO:teuthology.orchestra.run.smithi022.stdout: "address": { 2024-04-20T21:44:25.007 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "address", 2024-04-20T21:44:25.007 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.008 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.008 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.008 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T21:44:25.008 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.008 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.008 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.008 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.009 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.009 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.009 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.009 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:25.009 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:25.009 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.009 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.010 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.010 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15", 2024-04-20T21:44:25.010 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.010 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.010 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.010 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.010 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.011 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.011 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.011 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.011 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.011 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.011 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.012 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.012 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.012 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.012 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.012 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.012 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.012 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.012 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.014 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.014 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.014 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.014 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.014 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.015 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.015 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.015 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.015 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.015 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.015 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.015 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.015 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.016 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.016 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.016 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.016 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.016 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.016 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.017 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.017 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.017 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.017 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.017 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.017 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.017 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.017 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.018 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.018 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.018 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.018 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.018 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.018 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.019 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.019 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.019 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.019 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.019 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.019 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.019 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.020 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.020 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.020 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.020 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.020 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.020 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.020 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.021 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.021 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.021 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telemetry", 2024-04-20T21:44:25.021 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.021 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.021 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.021 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_basic": { 2024-04-20T21:44:25.021 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_basic", 2024-04-20T21:44:25.022 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.022 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.022 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.022 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.022 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.022 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.022 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.022 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T21:44:25.023 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.023 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.023 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.023 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_crash": { 2024-04-20T21:44:25.023 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_crash", 2024-04-20T21:44:25.023 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.024 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.024 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.024 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.024 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.024 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.024 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.024 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T21:44:25.024 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.025 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.025 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.025 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_device": { 2024-04-20T21:44:25.025 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_device", 2024-04-20T21:44:25.025 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.025 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.025 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.026 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.026 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.026 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.026 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.026 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T21:44:25.026 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.026 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.027 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.027 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_ident": { 2024-04-20T21:44:25.027 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_ident", 2024-04-20T21:44:25.027 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.027 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.027 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.027 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.028 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.028 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.028 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.028 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T21:44:25.028 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.028 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.028 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.029 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_perf": { 2024-04-20T21:44:25.029 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_perf", 2024-04-20T21:44:25.029 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.029 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.029 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.029 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.029 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.030 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.030 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.030 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T21:44:25.030 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.030 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.030 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.031 INFO:teuthology.orchestra.run.smithi022.stdout: "contact": { 2024-04-20T21:44:25.031 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "contact", 2024-04-20T21:44:25.031 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.031 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.031 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.031 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.031 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.032 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.032 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.032 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.032 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.032 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.032 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.033 INFO:teuthology.orchestra.run.smithi022.stdout: "description": { 2024-04-20T21:44:25.033 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "description", 2024-04-20T21:44:25.033 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.033 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.033 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.033 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.033 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.033 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.034 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.034 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.034 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.034 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.034 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.034 INFO:teuthology.orchestra.run.smithi022.stdout: "device_url": { 2024-04-20T21:44:25.035 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_url", 2024-04-20T21:44:25.035 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.035 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.035 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.035 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T21:44:25.035 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.035 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.036 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.036 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.036 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.036 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.036 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.036 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-04-20T21:44:25.037 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-04-20T21:44:25.037 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.037 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.037 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.037 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.037 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.038 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.038 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.038 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.038 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.038 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.038 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.038 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.039 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:25.039 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:25.039 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.039 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.039 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.039 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "24", 2024-04-20T21:44:25.039 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "8", 2024-04-20T21:44:25.039 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.040 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.040 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.040 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.040 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.040 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.040 INFO:teuthology.orchestra.run.smithi022.stdout: "last_opt_revision": { 2024-04-20T21:44:25.041 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "last_opt_revision", 2024-04-20T21:44:25.041 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.041 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.041 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.041 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1", 2024-04-20T21:44:25.041 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.041 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.042 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.042 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.042 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.042 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.042 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.042 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard": { 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard", 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard_description": { 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard_description", 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi022.stdout: "organization": { 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "organization", 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi022.stdout: "proxy": { 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "proxy", 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi022.stdout: "url": { 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url", 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "test_orchestrator", 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_clones": { 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_clones", 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4", 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi022.stdout: "periodic_async_work": { 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "periodic_async_work", 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Periodically check for async work", 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_delay": { 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_delay", 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_no_wait": { 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix", 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi022.stdout: "discovery_interval": { 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "discovery_interval", 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "100", 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi022.stdout: "identifier": { 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "identifier", 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_host": { 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_host", 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_port": { 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_port", 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10051", 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_sender": { 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_sender", 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi022.stdout: "gid": 25127, 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "y", 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_features": 4540701547738038271, 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi022.stdout: "available_modules": [ 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "alerts", 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_destination": { 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_destination", 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email address to send alerts to", 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_from_name": { 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_from_name", 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "Ceph", 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email From: name", 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_host": { 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_host", 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.118 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.118 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.118 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP server", 2024-04-20T21:44:25.118 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.118 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.118 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.119 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_password": { 2024-04-20T21:44:25.121 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_password", 2024-04-20T21:44:25.121 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.122 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.122 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.122 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.122 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.122 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.122 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.122 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Password to authenticate with", 2024-04-20T21:44:25.123 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.123 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.123 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.124 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_port": { 2024-04-20T21:44:25.124 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_port", 2024-04-20T21:44:25.124 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.124 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.124 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.124 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "465", 2024-04-20T21:44:25.124 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.124 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.124 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.125 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP port", 2024-04-20T21:44:25.125 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.125 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.125 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.125 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_sender": { 2024-04-20T21:44:25.125 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_sender", 2024-04-20T21:44:25.125 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.126 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.126 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.126 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.126 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.126 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.126 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.126 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP envelope sender", 2024-04-20T21:44:25.126 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.127 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.127 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.127 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_ssl": { 2024-04-20T21:44:25.127 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_ssl", 2024-04-20T21:44:25.127 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.128 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.128 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.128 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.128 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.128 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.128 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.128 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T21:44:25.129 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.129 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.129 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.129 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_user": { 2024-04-20T21:44:25.129 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_user", 2024-04-20T21:44:25.129 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.130 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.130 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.130 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.130 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.130 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.130 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.130 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "User to authenticate as", 2024-04-20T21:44:25.131 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.131 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.131 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.131 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.131 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.132 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "balancer", 2024-04-20T21:44:25.132 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.132 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.132 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.132 INFO:teuthology.orchestra.run.smithi022.stdout: "active": { 2024-04-20T21:44:25.132 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "active", 2024-04-20T21:44:25.132 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.133 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.133 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.133 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.133 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.133 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.133 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.133 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T21:44:25.133 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.134 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.134 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.134 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_time": { 2024-04-20T21:44:25.134 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_time", 2024-04-20T21:44:25.134 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.134 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.135 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.135 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0000", 2024-04-20T21:44:25.135 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.135 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.135 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.135 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T21:44:25.135 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T21:44:25.136 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.136 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.136 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_weekday": { 2024-04-20T21:44:25.136 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_weekday", 2024-04-20T21:44:25.136 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:25.136 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.137 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.137 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:25.137 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-04-20T21:44:25.137 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-04-20T21:44:25.137 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.137 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T21:44:25.137 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T21:44:25.137 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.137 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.138 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.138 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_max_iterations": { 2024-04-20T21:44:25.138 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_max_iterations", 2024-04-20T21:44:25.138 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:25.138 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.138 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.138 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "25", 2024-04-20T21:44:25.138 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-04-20T21:44:25.139 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "250", 2024-04-20T21:44:25.139 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.139 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T21:44:25.139 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.139 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.139 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.139 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_metrics": { 2024-04-20T21:44:25.139 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_metrics", 2024-04-20T21:44:25.140 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.140 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.140 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.140 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T21:44:25.140 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.140 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.140 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.140 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T21:44:25.141 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T21:44:25.141 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.141 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.141 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_step": { 2024-04-20T21:44:25.141 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_step", 2024-04-20T21:44:25.141 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:25.141 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.141 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.142 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.5", 2024-04-20T21:44:25.142 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0.001", 2024-04-20T21:44:25.142 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "0.999", 2024-04-20T21:44:25.142 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.142 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "aggressiveness of optimization", 2024-04-20T21:44:25.142 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T21:44:25.142 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.142 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.143 INFO:teuthology.orchestra.run.smithi022.stdout: "end_time": { 2024-04-20T21:44:25.143 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_time", 2024-04-20T21:44:25.143 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.143 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.143 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.143 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2359", 2024-04-20T21:44:25.143 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.143 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.144 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.144 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T21:44:25.144 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T21:44:25.144 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.144 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.144 INFO:teuthology.orchestra.run.smithi022.stdout: "end_weekday": { 2024-04-20T21:44:25.144 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_weekday", 2024-04-20T21:44:25.145 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:25.145 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.145 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.145 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:25.145 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-04-20T21:44:25.145 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-04-20T21:44:25.145 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.145 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T21:44:25.146 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T21:44:25.146 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.146 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.146 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.146 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.146 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.146 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.147 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.147 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.147 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.147 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.147 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.147 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.147 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.147 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.147 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.148 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.148 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.148 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.148 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.148 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.148 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.148 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.149 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.149 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.149 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.149 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.149 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.149 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.149 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.149 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.149 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.150 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.150 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.150 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.150 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.150 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.150 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.150 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.150 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.151 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.151 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.151 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.151 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.151 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.151 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.151 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.151 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.152 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.152 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.152 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.152 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.152 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.152 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.152 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.152 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.152 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.153 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.153 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.153 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.153 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.153 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.153 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.153 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.153 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.153 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.154 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.154 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.154 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.154 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.154 INFO:teuthology.orchestra.run.smithi022.stdout: "min_score": { 2024-04-20T21:44:25.154 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_score", 2024-04-20T21:44:25.154 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:25.155 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.155 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.155 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:25.155 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.155 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.155 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.155 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T21:44:25.155 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.156 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.156 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.156 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-04-20T21:44:25.156 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-04-20T21:44:25.156 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.156 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.156 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.157 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "upmap", 2024-04-20T21:44:25.157 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.157 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.157 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.157 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-compat", 2024-04-20T21:44:25.157 INFO:teuthology.orchestra.run.smithi022.stdout: "none", 2024-04-20T21:44:25.157 INFO:teuthology.orchestra.run.smithi022.stdout: "read", 2024-04-20T21:44:25.157 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap", 2024-04-20T21:44:25.158 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap-read" 2024-04-20T21:44:25.158 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.158 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Balancer mode", 2024-04-20T21:44:25.158 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.158 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.158 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.158 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_ids": { 2024-04-20T21:44:25.159 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_ids", 2024-04-20T21:44:25.159 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.159 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.159 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.159 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.159 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.159 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.159 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.160 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T21:44:25.160 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.160 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.160 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.160 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:25.160 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:25.160 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.161 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.161 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.161 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:25.161 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.161 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.161 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.161 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T21:44:25.162 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.162 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.162 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.162 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_deviation": { 2024-04-20T21:44:25.162 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_deviation", 2024-04-20T21:44:25.162 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.162 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.162 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.163 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-04-20T21:44:25.163 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-04-20T21:44:25.163 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.163 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.163 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T21:44:25.163 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T21:44:25.163 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.163 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.164 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_optimizations": { 2024-04-20T21:44:25.164 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_optimizations", 2024-04-20T21:44:25.164 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:25.164 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.164 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.164 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:25.164 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.164 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.165 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.165 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T21:44:25.165 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.165 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.165 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.165 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.165 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.166 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.166 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm", 2024-04-20T21:44:25.166 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.166 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.166 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.166 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_down_multiplier": { 2024-04-20T21:44:25.166 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_down_multiplier", 2024-04-20T21:44:25.166 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:25.166 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.167 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.167 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-04-20T21:44:25.167 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.167 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.167 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.167 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T21:44:25.167 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.167 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.168 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.168 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_refresh_rate": { 2024-04-20T21:44:25.168 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_refresh_rate", 2024-04-20T21:44:25.168 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.168 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.168 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.168 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "20", 2024-04-20T21:44:25.169 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.169 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.169 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.169 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T21:44:25.169 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.169 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.169 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.170 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_starting_port": { 2024-04-20T21:44:25.170 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_starting_port", 2024-04-20T21:44:25.170 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.170 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.170 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.170 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4721", 2024-04-20T21:44:25.170 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.170 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.171 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.171 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T21:44:25.171 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.171 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.171 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.171 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_ptrace": { 2024-04-20T21:44:25.171 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_ptrace", 2024-04-20T21:44:25.171 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.172 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.172 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.172 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.172 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.172 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.172 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.172 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T21:44:25.172 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T21:44:25.172 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.173 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.173 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_interval": { 2024-04-20T21:44:25.173 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_interval", 2024-04-20T21:44:25.173 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.173 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.173 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.173 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:25.173 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.174 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.174 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.174 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T21:44:25.174 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.174 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.174 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.174 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_memory_target_ratio": { 2024-04-20T21:44:25.175 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T21:44:25.175 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:25.175 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.175 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.175 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.7", 2024-04-20T21:44:25.175 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.175 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.175 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.175 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T21:44:25.176 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.176 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.176 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.176 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.176 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm_log_destination": { 2024-04-20T21:44:25.176 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm_log_destination", 2024-04-20T21:44:25.176 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.176 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.177 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.177 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.177 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.177 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.177 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.177 INFO:teuthology.orchestra.run.smithi022.stdout: "file", 2024-04-20T21:44:25.177 INFO:teuthology.orchestra.run.smithi022.stdout: "file,syslog", 2024-04-20T21:44:25.177 INFO:teuthology.orchestra.run.smithi022.stdout: "syslog" 2024-04-20T21:44:25.177 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.178 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T21:44:25.178 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.178 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.178 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.178 INFO:teuthology.orchestra.run.smithi022.stdout: "cgroups_split": { 2024-04-20T21:44:25.178 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cgroups_split", 2024-04-20T21:44:25.178 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.179 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.179 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.179 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.179 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.179 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.179 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.179 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T21:44:25.179 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.180 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.180 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.180 INFO:teuthology.orchestra.run.smithi022.stdout: "config_checks_enabled": { 2024-04-20T21:44:25.180 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_checks_enabled", 2024-04-20T21:44:25.180 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.180 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.180 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.180 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.181 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.181 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.181 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.181 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T21:44:25.181 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.181 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.181 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.182 INFO:teuthology.orchestra.run.smithi022.stdout: "config_dashboard": { 2024-04-20T21:44:25.182 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_dashboard", 2024-04-20T21:44:25.182 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.182 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.182 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.182 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.182 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.182 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.182 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.183 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T21:44:25.183 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.183 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.183 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.183 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_alertmanager": { 2024-04-20T21:44:25.183 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_alertmanager", 2024-04-20T21:44:25.184 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.184 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.184 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.184 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T21:44:25.184 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.184 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.184 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.184 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-04-20T21:44:25.184 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.185 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.185 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.185 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.185 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_base": { 2024-04-20T21:44:25.185 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_base", 2024-04-20T21:44:25.185 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.185 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.185 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.186 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T21:44:25.186 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.186 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.186 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.186 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Container image name, without the tag", 2024-04-20T21:44:25.186 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.186 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.186 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.187 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_elasticsearch": { 2024-04-20T21:44:25.187 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_elasticsearch", 2024-04-20T21:44:25.187 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.187 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.187 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.187 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T21:44:25.187 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.187 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.188 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.188 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "elasticsearch container image", 2024-04-20T21:44:25.188 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.188 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.188 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.188 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.188 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_grafana": { 2024-04-20T21:44:25.188 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_grafana", 2024-04-20T21:44:25.188 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T21:44:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-04-20T21:44:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.190 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.190 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_haproxy": { 2024-04-20T21:44:25.190 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_haproxy", 2024-04-20T21:44:25.190 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.190 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.190 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.190 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T21:44:25.191 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.191 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.191 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.191 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "HAproxy container image", 2024-04-20T21:44:25.191 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.191 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.191 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.191 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_agent": { 2024-04-20T21:44:25.192 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_agent", 2024-04-20T21:44:25.192 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.192 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.192 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.192 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T21:44:25.192 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.192 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.192 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.193 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger agent container image", 2024-04-20T21:44:25.193 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.193 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.193 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.193 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_collector": { 2024-04-20T21:44:25.193 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_collector", 2024-04-20T21:44:25.193 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.193 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.194 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.194 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T21:44:25.194 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.194 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.194 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.194 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger collector container image", 2024-04-20T21:44:25.194 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.194 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.194 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.195 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_query": { 2024-04-20T21:44:25.195 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_query", 2024-04-20T21:44:25.195 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.195 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.195 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.195 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T21:44:25.195 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.196 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.196 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.196 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger query container image", 2024-04-20T21:44:25.196 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.196 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.196 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.196 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_keepalived": { 2024-04-20T21:44:25.196 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_keepalived", 2024-04-20T21:44:25.197 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.197 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.197 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.197 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T21:44:25.197 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.197 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.197 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.197 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Keepalived container image", 2024-04-20T21:44:25.197 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.198 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.198 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.198 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_loki": { 2024-04-20T21:44:25.198 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_loki", 2024-04-20T21:44:25.198 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.198 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.198 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.199 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T21:44:25.199 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.199 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.199 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.199 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Loki container image", 2024-04-20T21:44:25.199 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.199 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.199 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.200 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_node_exporter": { 2024-04-20T21:44:25.200 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_node_exporter", 2024-04-20T21:44:25.200 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.200 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.200 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.200 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T21:44:25.200 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.200 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.201 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.201 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-04-20T21:44:25.201 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.201 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.201 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.201 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.201 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_nvmeof": { 2024-04-20T21:44:25.201 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_nvmeof", 2024-04-20T21:44:25.201 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.202 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.202 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.202 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T21:44:25.202 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.202 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.202 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.202 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Nvme-of container image", 2024-04-20T21:44:25.202 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.203 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.203 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.203 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.203 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_prometheus": { 2024-04-20T21:44:25.203 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_prometheus", 2024-04-20T21:44:25.203 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.203 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.203 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.203 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T21:44:25.204 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.204 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.204 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.204 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-04-20T21:44:25.204 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.204 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.204 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.204 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_promtail": { 2024-04-20T21:44:25.205 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_promtail", 2024-04-20T21:44:25.205 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.205 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.205 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.205 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T21:44:25.205 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.205 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.205 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.205 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Promtail container image", 2024-04-20T21:44:25.206 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.206 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.206 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.206 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_samba": { 2024-04-20T21:44:25.206 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_samba", 2024-04-20T21:44:25.206 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.206 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.206 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.207 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T21:44:25.207 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.207 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.207 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.207 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Samba/SMB container image", 2024-04-20T21:44:25.207 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.207 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.207 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.208 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_snmp_gateway": { 2024-04-20T21:44:25.208 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_snmp_gateway", 2024-04-20T21:44:25.208 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.208 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.208 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.208 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T21:44:25.208 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.208 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.209 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.209 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SNMP Gateway container image", 2024-04-20T21:44:25.209 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.209 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.209 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.209 INFO:teuthology.orchestra.run.smithi022.stdout: "container_init": { 2024-04-20T21:44:25.209 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_init", 2024-04-20T21:44:25.209 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.210 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.210 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.210 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.210 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.210 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.210 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.210 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T21:44:25.210 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.211 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.211 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.211 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.211 INFO:teuthology.orchestra.run.smithi022.stdout: "daemon_cache_timeout": { 2024-04-20T21:44:25.211 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "daemon_cache_timeout", 2024-04-20T21:44:25.211 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.211 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.211 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.211 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:25.212 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.212 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.212 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.212 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T21:44:25.212 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.212 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.212 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.212 INFO:teuthology.orchestra.run.smithi022.stdout: "default_cephadm_command_timeout": { 2024-04-20T21:44:25.213 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T21:44:25.213 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.213 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.213 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.213 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "900", 2024-04-20T21:44:25.213 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.213 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.213 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.213 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T21:44:25.214 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.214 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.214 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.214 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.214 INFO:teuthology.orchestra.run.smithi022.stdout: "default_registry": { 2024-04-20T21:44:25.214 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_registry", 2024-04-20T21:44:25.214 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.214 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.214 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.215 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io", 2024-04-20T21:44:25.215 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.215 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.215 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.215 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T21:44:25.215 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.215 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.215 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.216 INFO:teuthology.orchestra.run.smithi022.stdout: "device_cache_timeout": { 2024-04-20T21:44:25.216 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_cache_timeout", 2024-04-20T21:44:25.216 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.216 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.216 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.216 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1800", 2024-04-20T21:44:25.216 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.216 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.217 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.217 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache device inventory", 2024-04-20T21:44:25.217 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.217 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.217 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.217 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.217 INFO:teuthology.orchestra.run.smithi022.stdout: "device_enhanced_scan": { 2024-04-20T21:44:25.217 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_enhanced_scan", 2024-04-20T21:44:25.217 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.218 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.218 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.218 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.218 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.218 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.218 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.218 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T21:44:25.218 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.218 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.219 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.219 INFO:teuthology.orchestra.run.smithi022.stdout: "facts_cache_timeout": { 2024-04-20T21:44:25.219 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "facts_cache_timeout", 2024-04-20T21:44:25.219 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.219 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.219 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.219 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:25.219 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.220 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.220 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.220 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache host facts data", 2024-04-20T21:44:25.220 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.220 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.220 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.220 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.220 INFO:teuthology.orchestra.run.smithi022.stdout: "grafana_dashboards_path": { 2024-04-20T21:44:25.220 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "grafana_dashboards_path", 2024-04-20T21:44:25.221 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.221 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.221 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.221 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T21:44:25.221 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.221 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.221 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.221 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T21:44:25.222 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.222 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.222 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.222 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.222 INFO:teuthology.orchestra.run.smithi022.stdout: "host_check_interval": { 2024-04-20T21:44:25.222 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "host_check_interval", 2024-04-20T21:44:25.222 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.222 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.222 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.223 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:25.223 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.223 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.223 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.223 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to perform a host check", 2024-04-20T21:44:25.223 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.223 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.223 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.224 INFO:teuthology.orchestra.run.smithi022.stdout: "hw_monitoring": { 2024-04-20T21:44:25.224 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hw_monitoring", 2024-04-20T21:44:25.224 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.224 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.224 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.224 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.224 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.224 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.225 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.225 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T21:44:25.225 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.225 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.225 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.225 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.225 INFO:teuthology.orchestra.run.smithi022.stdout: "inventory_list_all": { 2024-04-20T21:44:25.225 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "inventory_list_all", 2024-04-20T21:44:25.226 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.226 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.226 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.226 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.226 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.226 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.226 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.226 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T21:44:25.227 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.227 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.227 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.227 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.227 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.227 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.227 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.228 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.228 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.228 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.228 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.228 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.228 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.228 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.228 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.228 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.229 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.229 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.229 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.229 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.229 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.229 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.229 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.229 INFO:teuthology.orchestra.run.smithi022.stdout: "log_refresh_metadata": { 2024-04-20T21:44:25.230 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_refresh_metadata", 2024-04-20T21:44:25.230 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.230 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.230 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.230 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.230 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.230 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.230 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.231 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T21:44:25.231 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.231 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.231 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.231 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.231 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.231 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.231 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.232 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.232 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.232 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.232 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.232 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.232 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.232 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T21:44:25.232 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.232 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.233 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.233 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.233 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.233 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.233 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.233 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.233 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.233 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.233 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.234 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.234 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.234 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.234 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.234 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.234 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.234 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.234 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.235 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.235 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.235 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.235 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.235 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.235 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.235 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.235 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.236 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.236 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.236 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.236 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.236 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.236 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.236 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.236 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.237 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.237 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.237 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T21:44:25.237 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T21:44:25.237 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.237 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.237 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.237 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.238 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.238 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.238 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.238 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T21:44:25.238 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.238 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.238 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.238 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.239 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T21:44:25.239 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T21:44:25.239 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.239 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.239 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.239 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "*", 2024-04-20T21:44:25.239 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.239 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.239 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.240 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T21:44:25.240 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.240 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.240 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.240 INFO:teuthology.orchestra.run.smithi022.stdout: "max_count_per_host": { 2024-04-20T21:44:25.240 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_count_per_host", 2024-04-20T21:44:25.240 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.241 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.241 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.241 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:25.241 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.241 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.241 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.241 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of daemons per service per host", 2024-04-20T21:44:25.241 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.241 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.242 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.242 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd_draining_count": { 2024-04-20T21:44:25.242 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_osd_draining_count", 2024-04-20T21:44:25.242 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.242 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.242 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.242 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:25.243 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.243 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.243 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.243 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T21:44:25.243 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.243 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.243 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.243 INFO:teuthology.orchestra.run.smithi022.stdout: "migration_current": { 2024-04-20T21:44:25.244 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "migration_current", 2024-04-20T21:44:25.244 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.244 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.244 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.244 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.244 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.244 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.244 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.244 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "internal - do not modify", 2024-04-20T21:44:25.245 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.245 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.245 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.245 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.245 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-04-20T21:44:25.245 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-04-20T21:44:25.245 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.245 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.245 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.246 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "root", 2024-04-20T21:44:25.246 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.246 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.246 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.246 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm-package", 2024-04-20T21:44:25.246 INFO:teuthology.orchestra.run.smithi022.stdout: "root" 2024-04-20T21:44:25.246 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.246 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T21:44:25.246 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.247 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.247 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.247 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.247 INFO:teuthology.orchestra.run.smithi022.stdout: "oob_default_addr": { 2024-04-20T21:44:25.247 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "oob_default_addr", 2024-04-20T21:44:25.247 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.247 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.247 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.248 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "169.254.1.1", 2024-04-20T21:44:25.248 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.248 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.248 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.248 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T21:44:25.248 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.248 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.248 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.248 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.249 INFO:teuthology.orchestra.run.smithi022.stdout: "prometheus_alerts_path": { 2024-04-20T21:44:25.249 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus_alerts_path", 2024-04-20T21:44:25.249 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.249 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.249 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.249 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T21:44:25.249 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.249 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.249 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.250 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T21:44:25.250 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.250 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.250 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.250 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_insecure": { 2024-04-20T21:44:25.250 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_insecure", 2024-04-20T21:44:25.250 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.250 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.251 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.251 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.251 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.251 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.251 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.251 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T21:44:25.251 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.251 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.252 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.252 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_password": { 2024-04-20T21:44:25.252 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_password", 2024-04-20T21:44:25.252 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.252 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.252 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.252 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.252 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.253 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.253 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.253 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T21:44:25.253 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.253 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.253 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.253 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_url": { 2024-04-20T21:44:25.253 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_url", 2024-04-20T21:44:25.254 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.254 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.254 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.254 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.254 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.254 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.254 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.254 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T21:44:25.255 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.255 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.255 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.255 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.255 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_username": { 2024-04-20T21:44:25.255 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_username", 2024-04-20T21:44:25.255 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.255 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.255 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.256 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.256 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.256 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.256 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.256 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T21:44:25.256 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.256 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.256 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.257 INFO:teuthology.orchestra.run.smithi022.stdout: "secure_monitoring_stack": { 2024-04-20T21:44:25.257 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secure_monitoring_stack", 2024-04-20T21:44:25.257 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.257 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.257 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.257 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.257 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.257 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.258 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.258 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T21:44:25.258 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.258 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.258 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.258 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.258 INFO:teuthology.orchestra.run.smithi022.stdout: "service_discovery_port": { 2024-04-20T21:44:25.258 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "service_discovery_port", 2024-04-20T21:44:25.258 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.259 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.259 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.259 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8765", 2024-04-20T21:44:25.259 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.259 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.259 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.259 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "cephadm service discovery port", 2024-04-20T21:44:25.259 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.260 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.260 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.260 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.260 INFO:teuthology.orchestra.run.smithi022.stdout: "ssh_config_file": { 2024-04-20T21:44:25.260 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssh_config_file", 2024-04-20T21:44:25.260 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.260 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.260 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.260 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.261 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.261 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.261 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.261 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T21:44:25.261 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.261 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.261 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.261 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.262 INFO:teuthology.orchestra.run.smithi022.stdout: "use_agent": { 2024-04-20T21:44:25.262 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_agent", 2024-04-20T21:44:25.262 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.262 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.262 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.262 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.262 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.262 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.263 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.263 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T21:44:25.263 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.263 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.263 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.263 INFO:teuthology.orchestra.run.smithi022.stdout: "use_repo_digest": { 2024-04-20T21:44:25.263 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_repo_digest", 2024-04-20T21:44:25.263 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.264 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.264 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.264 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.264 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.264 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.264 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.264 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T21:44:25.264 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.265 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.265 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.265 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.265 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_failed_host_check": { 2024-04-20T21:44:25.265 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_failed_host_check", 2024-04-20T21:44:25.265 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.265 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.265 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.266 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.266 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.266 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.266 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.266 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T21:44:25.266 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.266 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.266 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.266 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.267 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_daemons": { 2024-04-20T21:44:25.267 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_daemons", 2024-04-20T21:44:25.267 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.267 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.267 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.267 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.267 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.267 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.268 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.268 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T21:44:25.268 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.268 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.268 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.268 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.268 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_hosts": { 2024-04-20T21:44:25.268 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_hosts", 2024-04-20T21:44:25.268 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.269 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.269 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.269 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.269 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.269 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.269 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.269 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T21:44:25.269 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.270 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.270 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.270 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.270 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.270 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.270 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.270 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crash", 2024-04-20T21:44:25.270 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.271 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.271 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.271 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.271 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.271 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.271 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.271 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.271 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.272 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.272 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.272 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.272 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.272 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.272 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.272 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.272 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.272 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.273 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.273 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.273 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.273 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.273 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.273 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.273 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.273 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.273 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.274 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.274 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.274 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.274 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.274 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.274 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.274 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.274 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.275 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.275 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.275 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.275 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.275 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.275 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.275 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.275 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.275 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.276 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.276 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.276 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.276 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.276 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.276 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.276 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.276 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.276 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.277 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.277 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.277 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.277 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.277 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.277 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.277 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.277 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.277 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.278 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.278 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.278 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.278 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.278 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.278 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.278 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.278 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.279 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.279 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.279 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.279 INFO:teuthology.orchestra.run.smithi022.stdout: "retain_interval": { 2024-04-20T21:44:25.279 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retain_interval", 2024-04-20T21:44:25.279 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.279 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.279 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.279 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "31536000", 2024-04-20T21:44:25.280 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.280 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.280 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.280 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T21:44:25.280 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.280 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.280 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.280 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.280 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_recent_interval": { 2024-04-20T21:44:25.281 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_recent_interval", 2024-04-20T21:44:25.281 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.281 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.281 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.281 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1209600", 2024-04-20T21:44:25.281 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.281 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.281 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.281 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T21:44:25.282 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.282 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.282 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.282 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.282 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.282 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.282 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.282 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dashboard", 2024-04-20T21:44:25.283 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.283 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.283 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.283 INFO:teuthology.orchestra.run.smithi022.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T21:44:25.283 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T21:44:25.283 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.283 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.283 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.283 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:25.284 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.284 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.284 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.284 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.284 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.284 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.284 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.284 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.284 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T21:44:25.285 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T21:44:25.285 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.285 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.285 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.285 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.285 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.285 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.285 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.285 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.286 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.286 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.286 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.286 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.286 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T21:44:25.286 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T21:44:25.286 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.286 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.287 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.287 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.287 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.287 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.287 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.287 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.287 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.287 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.288 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.288 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.288 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_ENABLED": { 2024-04-20T21:44:25.288 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T21:44:25.288 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.288 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.288 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.288 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.288 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.289 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.289 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.289 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.289 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.289 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.289 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.289 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T21:44:25.290 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T21:44:25.290 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.290 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.290 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.290 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.290 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.290 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.290 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.291 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.291 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.291 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.291 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.291 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.291 INFO:teuthology.orchestra.run.smithi022.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T21:44:25.291 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T21:44:25.291 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.291 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.292 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.292 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.292 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.292 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.292 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.292 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.292 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.292 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.293 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.293 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.293 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T21:44:25.293 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T21:44:25.293 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.293 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.293 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.293 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.293 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.294 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.294 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.294 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.294 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.294 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.294 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.294 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T21:44:25.295 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T21:44:25.295 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.295 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.295 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.295 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.295 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.295 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.295 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.295 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.296 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.296 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.296 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.296 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.296 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T21:44:25.296 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T21:44:25.296 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.296 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.296 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.297 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.297 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.297 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.297 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.297 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.297 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.297 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.297 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.298 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T21:44:25.298 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T21:44:25.298 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.298 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.298 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.298 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.298 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.299 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.299 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.299 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.299 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.299 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.299 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.299 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T21:44:25.299 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T21:44:25.300 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.300 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.300 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.300 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.300 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.300 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.300 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.300 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.300 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T21:44:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T21:44:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.302 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.302 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.302 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.302 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.302 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.302 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.302 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.302 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.303 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.303 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T21:44:25.303 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T21:44:25.303 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.303 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.303 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.303 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.303 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.304 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.304 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.304 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.304 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.304 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.304 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.304 INFO:teuthology.orchestra.run.smithi022.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T21:44:25.304 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T21:44:25.305 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.305 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.305 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.305 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.305 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.305 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.305 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.305 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.305 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.306 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.306 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.306 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.306 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T21:44:25.306 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T21:44:25.306 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.306 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.306 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.306 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-04-20T21:44:25.307 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.307 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.307 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.307 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.307 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.307 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.307 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.308 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T21:44:25.308 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T21:44:25.308 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.308 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.308 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.308 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.308 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.308 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.308 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.309 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.309 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.309 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.309 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.309 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.309 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_URL": { 2024-04-20T21:44:25.309 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_URL", 2024-04-20T21:44:25.309 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.309 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.310 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.310 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.310 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.310 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.310 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.310 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.310 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.310 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.311 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.311 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.311 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T21:44:25.311 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T21:44:25.311 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.311 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.311 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.311 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-04-20T21:44:25.312 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.312 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.312 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.312 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.312 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.312 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.312 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.312 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.312 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T21:44:25.313 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T21:44:25.313 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.313 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.313 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.313 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.313 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.313 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.313 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.313 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.314 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.314 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.314 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.314 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.314 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T21:44:25.314 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T21:44:25.314 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.314 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.314 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.315 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.315 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.315 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.315 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.315 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.315 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.315 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.315 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.315 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.316 INFO:teuthology.orchestra.run.smithi022.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T21:44:25.316 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T21:44:25.316 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.316 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.316 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.316 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.316 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.316 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.316 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.317 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.317 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.317 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.317 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.317 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.317 INFO:teuthology.orchestra.run.smithi022.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T21:44:25.317 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T21:44:25.317 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.317 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.318 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.318 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.318 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.318 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.318 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.318 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.318 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.318 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.319 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.319 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.319 INFO:teuthology.orchestra.run.smithi022.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T21:44:25.319 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T21:44:25.319 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.319 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.319 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.319 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "[]", 2024-04-20T21:44:25.319 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.320 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.320 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.320 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.320 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.320 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.320 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.320 INFO:teuthology.orchestra.run.smithi022.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T21:44:25.320 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T21:44:25.321 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.321 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.321 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.321 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "{}", 2024-04-20T21:44:25.321 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.321 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.321 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.321 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.321 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.322 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.322 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.322 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.322 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T21:44:25.322 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T21:44:25.322 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.322 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.322 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.322 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.323 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.323 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.323 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.323 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.323 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.323 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.323 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.323 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.323 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T21:44:25.324 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T21:44:25.324 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.324 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.324 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.324 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.324 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.324 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.324 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.325 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.325 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.325 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.325 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.325 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.325 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T21:44:25.325 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T21:44:25.325 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.325 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.326 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.326 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.326 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.326 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.326 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.326 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.326 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.326 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.327 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.327 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.327 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T21:44:25.327 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T21:44:25.327 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.327 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.327 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.327 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.327 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.328 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.328 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.328 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.328 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.328 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.328 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.328 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.328 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T21:44:25.328 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T21:44:25.329 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.329 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.329 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.329 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.329 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.329 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.329 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.329 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.329 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.330 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.330 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.330 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.330 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T21:44:25.330 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T21:44:25.330 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.330 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.331 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.331 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.331 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.331 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.331 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.331 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.331 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.331 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.331 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.332 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.332 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T21:44:25.332 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T21:44:25.332 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.332 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.332 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.332 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.333 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.333 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.333 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.333 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.333 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.333 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.333 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.333 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.333 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T21:44:25.334 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T21:44:25.334 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.334 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.334 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.334 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.334 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.334 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.334 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.334 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.335 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.335 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.335 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.335 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.335 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T21:44:25.335 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T21:44:25.335 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.335 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.336 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.336 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.336 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.336 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.336 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.336 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.336 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.336 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.336 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.337 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.337 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T21:44:25.337 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T21:44:25.337 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.337 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.337 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.337 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.337 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.337 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.338 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.338 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.338 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.338 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.338 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.338 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.338 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T21:44:25.338 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T21:44:25.339 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.339 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.339 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.339 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T21:44:25.339 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.339 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.339 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.339 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.339 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.340 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.340 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.340 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.340 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T21:44:25.340 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T21:44:25.340 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.340 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.340 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.340 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:25.341 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.341 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.341 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.341 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.341 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.341 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.341 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.341 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.342 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T21:44:25.342 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T21:44:25.342 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.342 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.342 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.342 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8", 2024-04-20T21:44:25.342 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.342 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.342 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.343 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.343 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.343 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.343 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.343 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.343 INFO:teuthology.orchestra.run.smithi022.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T21:44:25.343 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T21:44:25.343 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.343 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.344 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.344 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "45", 2024-04-20T21:44:25.344 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.344 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.344 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.344 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.344 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.344 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.345 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.345 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.345 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T21:44:25.345 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T21:44:25.345 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.345 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.345 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.345 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.345 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.346 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.346 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.346 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.346 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.346 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.346 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.346 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.346 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T21:44:25.346 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T21:44:25.347 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.347 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.347 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.347 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-04-20T21:44:25.347 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.347 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.347 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.347 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.347 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.348 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.348 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.348 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.348 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T21:44:25.348 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T21:44:25.348 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.348 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.348 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.348 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.349 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.349 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.349 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.349 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.349 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.349 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.349 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.349 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.349 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T21:44:25.350 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T21:44:25.350 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.350 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.350 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.350 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.350 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.350 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.350 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.350 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.351 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.351 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.351 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.351 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.351 INFO:teuthology.orchestra.run.smithi022.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T21:44:25.351 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T21:44:25.351 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.351 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.352 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.352 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.352 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.352 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.352 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.352 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.352 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.352 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.352 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.353 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T21:44:25.353 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T21:44:25.353 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.353 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.353 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.353 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:25.353 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.354 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.354 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.354 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.354 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.354 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.354 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.354 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.354 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T21:44:25.355 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T21:44:25.355 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.355 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.355 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.355 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:25.355 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.355 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.355 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.355 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.356 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.356 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.356 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.356 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T21:44:25.356 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T21:44:25.356 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.356 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.356 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.357 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-04-20T21:44:25.357 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.357 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.357 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.357 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.357 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.357 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.357 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.357 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.358 INFO:teuthology.orchestra.run.smithi022.stdout: "cross_origin_url": { 2024-04-20T21:44:25.358 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cross_origin_url", 2024-04-20T21:44:25.358 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.358 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.358 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.358 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.358 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.358 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.359 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.359 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.359 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.359 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.359 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.359 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.359 INFO:teuthology.orchestra.run.smithi022.stdout: "crt_file": { 2024-04-20T21:44:25.359 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crt_file", 2024-04-20T21:44:25.360 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.360 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.360 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.360 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.360 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.360 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.360 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.360 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.360 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.361 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.361 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.361 INFO:teuthology.orchestra.run.smithi022.stdout: "debug": { 2024-04-20T21:44:25.361 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "debug", 2024-04-20T21:44:25.361 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.361 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.361 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable/disable debug options", 2024-04-20T21:44:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.362 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "jwt_token_ttl": { 2024-04-20T21:44:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "jwt_token_ttl", 2024-04-20T21:44:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "28800", 2024-04-20T21:44:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.363 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-04-20T21:44:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-04-20T21:44:25.364 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.365 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.365 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.365 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.365 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.365 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.365 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.365 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.365 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.366 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.366 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.366 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.366 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.366 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.366 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.366 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.366 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.367 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.367 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.367 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.367 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.367 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.367 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.367 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.367 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.367 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.368 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.368 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.368 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.368 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.368 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.368 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.368 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.368 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.369 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.369 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.369 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.369 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.369 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.369 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.369 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.369 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.369 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.370 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.370 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.370 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.370 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.370 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.370 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.370 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.370 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.371 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.371 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.371 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.371 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.371 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.371 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.371 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.371 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.371 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.373 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.373 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.373 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.373 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.373 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.373 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.373 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.373 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.373 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.374 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.374 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.374 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.374 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.374 INFO:teuthology.orchestra.run.smithi022.stdout: "motd": { 2024-04-20T21:44:25.374 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "motd", 2024-04-20T21:44:25.374 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.374 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.375 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.375 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.375 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.375 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.375 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.375 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "The message of the day", 2024-04-20T21:44:25.375 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.375 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.376 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.376 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.376 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect_resolve_ip_addr": { 2024-04-20T21:44:25.376 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T21:44:25.376 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.376 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.376 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.376 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.376 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.377 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.377 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.377 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.377 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.377 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.377 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.377 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.378 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-04-20T21:44:25.378 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-04-20T21:44:25.378 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.378 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.378 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.378 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-04-20T21:44:25.378 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.378 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.379 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.379 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.379 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.379 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.379 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.379 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.379 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-04-20T21:44:25.379 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-04-20T21:44:25.380 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.380 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.380 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.380 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8080", 2024-04-20T21:44:25.380 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.380 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.380 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.380 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.380 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.381 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.381 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.381 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.381 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-04-20T21:44:25.381 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-04-20T21:44:25.381 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.381 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.381 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.381 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.382 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.382 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.382 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.382 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.382 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.382 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.382 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.382 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.383 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl_server_port": { 2024-04-20T21:44:25.383 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl_server_port", 2024-04-20T21:44:25.383 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.383 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.383 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.383 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8443", 2024-04-20T21:44:25.383 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.383 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.383 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.384 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.384 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.384 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.384 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.384 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.384 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-04-20T21:44:25.384 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-04-20T21:44:25.385 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.385 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.385 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.385 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "redirect", 2024-04-20T21:44:25.385 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.385 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.385 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.385 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.385 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect" 2024-04-20T21:44:25.386 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.386 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.386 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.386 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.386 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.386 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.386 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-04-20T21:44:25.386 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-04-20T21:44:25.386 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.387 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.387 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.387 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-04-20T21:44:25.387 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-04-20T21:44:25.387 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-04-20T21:44:25.387 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.387 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.387 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.387 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.388 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.388 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.388 INFO:teuthology.orchestra.run.smithi022.stdout: "url_prefix": { 2024-04-20T21:44:25.388 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url_prefix", 2024-04-20T21:44:25.388 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.388 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.388 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.388 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.389 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.389 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.389 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.389 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.389 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.389 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.389 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.389 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.390 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.390 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.390 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.390 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2024-04-20T21:44:25.390 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.390 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.390 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.390 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_monitoring": { 2024-04-20T21:44:25.391 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_monitoring", 2024-04-20T21:44:25.391 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.391 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.391 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.391 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.391 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.391 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.391 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.392 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "monitor device health metrics", 2024-04-20T21:44:25.392 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.392 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.392 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.392 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.392 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.392 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.392 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.392 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.392 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.393 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.393 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.393 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.393 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.393 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.393 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.393 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.393 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.394 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.394 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.394 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.394 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.394 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.394 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.394 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.394 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.394 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.395 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.395 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.395 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.395 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.395 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.395 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.395 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.395 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.396 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.396 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.396 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.396 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.396 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.396 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.396 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.396 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.396 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.397 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.397 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.397 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.397 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.397 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.397 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.397 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.397 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.398 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.398 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.398 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.398 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.398 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.398 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.398 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.398 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.399 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.399 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.399 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.399 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.399 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.399 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.399 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.400 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.400 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.400 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.400 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.400 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.400 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.400 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.400 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.400 INFO:teuthology.orchestra.run.smithi022.stdout: "mark_out_threshold": { 2024-04-20T21:44:25.401 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mark_out_threshold", 2024-04-20T21:44:25.401 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.401 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.401 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.401 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2419200", 2024-04-20T21:44:25.401 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.401 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.401 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.402 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T21:44:25.402 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.402 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.402 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.402 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.402 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": { 2024-04-20T21:44:25.402 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_name", 2024-04-20T21:44:25.403 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.403 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.403 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.403 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "device_health_metrics", 2024-04-20T21:44:25.403 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.403 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.403 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.403 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T21:44:25.403 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.404 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.404 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.404 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.404 INFO:teuthology.orchestra.run.smithi022.stdout: "retention_period": { 2024-04-20T21:44:25.404 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retention_period", 2024-04-20T21:44:25.404 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.404 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.404 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.404 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15552000", 2024-04-20T21:44:25.405 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.405 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.405 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.405 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain device health metrics", 2024-04-20T21:44:25.405 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.405 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.405 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.405 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.406 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_frequency": { 2024-04-20T21:44:25.406 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_frequency", 2024-04-20T21:44:25.406 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.406 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.406 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.406 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-04-20T21:44:25.406 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.406 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.406 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.407 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T21:44:25.407 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.407 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.407 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.407 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.407 INFO:teuthology.orchestra.run.smithi022.stdout: "self_heal": { 2024-04-20T21:44:25.407 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "self_heal", 2024-04-20T21:44:25.407 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.408 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.408 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.408 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.408 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.408 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.408 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.408 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T21:44:25.408 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.408 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.409 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.409 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.409 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:25.409 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:25.409 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.409 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.409 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.409 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:25.409 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.410 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.410 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.410 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T21:44:25.410 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.410 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.410 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.410 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.410 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_threshold": { 2024-04-20T21:44:25.411 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_threshold", 2024-04-20T21:44:25.411 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.411 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.411 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.411 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7257600", 2024-04-20T21:44:25.411 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.411 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.411 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.411 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T21:44:25.412 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.412 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.412 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.412 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.412 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.412 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.412 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.412 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "diskprediction_local", 2024-04-20T21:44:25.412 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.413 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.413 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.413 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.413 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.413 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.413 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.413 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.414 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.414 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.414 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.414 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.414 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.414 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.414 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.414 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.414 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.415 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.415 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.415 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.415 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.415 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.415 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.415 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.416 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.416 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.416 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.416 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.416 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.416 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.416 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.416 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.416 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.417 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.417 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.417 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.417 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.417 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.417 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.417 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.417 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.418 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.418 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.418 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.418 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.418 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.418 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.418 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.418 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.419 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.419 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.419 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.419 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.419 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.419 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.419 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.419 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.419 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.420 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.420 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.420 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.420 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.420 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.420 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.420 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.420 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.421 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.421 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.421 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.421 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.421 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.421 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.421 INFO:teuthology.orchestra.run.smithi022.stdout: "predict_interval": { 2024-04-20T21:44:25.422 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predict_interval", 2024-04-20T21:44:25.422 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.422 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.422 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.422 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-04-20T21:44:25.422 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.422 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.422 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.422 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.423 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.423 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.423 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.423 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.423 INFO:teuthology.orchestra.run.smithi022.stdout: "predictor_model": { 2024-04-20T21:44:25.423 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predictor_model", 2024-04-20T21:44:25.423 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.423 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.424 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.424 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "prophetstor", 2024-04-20T21:44:25.424 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.424 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.424 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.424 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.424 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.424 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.424 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.425 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.425 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:25.425 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:25.425 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.425 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.425 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.425 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:25.425 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.425 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.426 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.426 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.426 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.426 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.426 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.426 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.426 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.426 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.427 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.427 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "influx", 2024-04-20T21:44:25.427 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": false, 2024-04-20T21:44:25.427 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "influxdb python module not found", 2024-04-20T21:44:25.427 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.427 INFO:teuthology.orchestra.run.smithi022.stdout: "batch_size": { 2024-04-20T21:44:25.427 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "batch_size", 2024-04-20T21:44:25.427 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.427 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.428 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.428 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5000", 2024-04-20T21:44:25.428 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.428 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.428 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.428 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T21:44:25.428 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.428 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.428 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.429 INFO:teuthology.orchestra.run.smithi022.stdout: "database": { 2024-04-20T21:44:25.429 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "database", 2024-04-20T21:44:25.429 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.429 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.429 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.429 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "ceph", 2024-04-20T21:44:25.429 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.429 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.430 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.430 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T21:44:25.430 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.430 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.430 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.430 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.430 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": { 2024-04-20T21:44:25.430 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hostname", 2024-04-20T21:44:25.431 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.431 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.431 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.431 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.431 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.431 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.431 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.431 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server hostname", 2024-04-20T21:44:25.431 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.432 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.432 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.432 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:25.432 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:25.432 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.432 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.432 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.433 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "30", 2024-04-20T21:44:25.433 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "5", 2024-04-20T21:44:25.433 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.433 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.433 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T21:44:25.433 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.433 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.433 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.433 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.434 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.434 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.434 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.434 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.434 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.434 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.434 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.434 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.435 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.435 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.435 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.435 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.435 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.435 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.435 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.435 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.435 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.436 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.436 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.436 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.436 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.436 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.436 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.436 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.436 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.437 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.437 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.437 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.437 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.437 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.437 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.437 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.437 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.437 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.438 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.438 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.438 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.438 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.438 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.438 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.438 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.438 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.439 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.439 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.439 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.439 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.439 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.439 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.439 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.439 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.439 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.440 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.440 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.440 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.440 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.440 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.440 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.440 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.440 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.441 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.441 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.441 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.441 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.441 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.441 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.441 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.441 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.441 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.442 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.442 INFO:teuthology.orchestra.run.smithi022.stdout: "password": { 2024-04-20T21:44:25.442 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "password", 2024-04-20T21:44:25.442 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.442 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.442 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.442 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.442 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.443 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.443 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.443 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "password of InfluxDB server user", 2024-04-20T21:44:25.443 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.443 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.443 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.443 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.443 INFO:teuthology.orchestra.run.smithi022.stdout: "port": { 2024-04-20T21:44:25.443 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "port", 2024-04-20T21:44:25.444 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.444 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.444 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.444 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8086", 2024-04-20T21:44:25.444 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.444 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.444 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.444 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server port", 2024-04-20T21:44:25.444 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.445 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.445 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.445 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.445 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-04-20T21:44:25.445 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-04-20T21:44:25.445 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.445 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.445 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.445 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "false", 2024-04-20T21:44:25.446 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.446 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.446 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.446 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T21:44:25.446 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.446 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.446 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.446 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.447 INFO:teuthology.orchestra.run.smithi022.stdout: "threads": { 2024-04-20T21:44:25.447 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threads", 2024-04-20T21:44:25.447 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.447 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.447 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.447 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-04-20T21:44:25.447 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-04-20T21:44:25.447 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "32", 2024-04-20T21:44:25.447 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.448 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T21:44:25.448 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.448 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.448 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.448 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.448 INFO:teuthology.orchestra.run.smithi022.stdout: "username": { 2024-04-20T21:44:25.448 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "username", 2024-04-20T21:44:25.448 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.448 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.449 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.449 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.449 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.449 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.449 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.449 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "username of InfluxDB server user", 2024-04-20T21:44:25.449 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.449 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.450 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.450 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.450 INFO:teuthology.orchestra.run.smithi022.stdout: "verify_ssl": { 2024-04-20T21:44:25.450 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "verify_ssl", 2024-04-20T21:44:25.450 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.450 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.450 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.451 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "true", 2024-04-20T21:44:25.451 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.451 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.451 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.451 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T21:44:25.451 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.451 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.452 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.452 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.452 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.452 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.452 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.452 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "insights", 2024-04-20T21:44:25.452 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.452 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.452 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.453 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.453 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.453 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.453 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.453 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.453 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.453 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.453 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.453 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.454 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.454 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.454 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.454 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.454 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.454 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.454 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.455 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.455 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.455 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.455 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.455 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.455 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.455 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.455 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.455 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.456 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.456 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.456 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.456 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.456 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.456 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.456 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.456 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.457 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.457 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.457 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.457 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.457 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.457 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.457 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.457 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.458 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.458 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.458 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.458 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.458 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.458 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.458 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.458 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.458 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.459 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.459 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.459 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.459 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.459 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.459 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.459 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.459 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.459 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.460 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.460 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.460 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.460 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.460 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.460 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.460 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.460 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.461 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.461 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.461 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.461 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.461 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.461 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.461 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "iostat", 2024-04-20T21:44:25.461 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.462 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.462 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.462 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.462 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.462 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.462 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.462 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.462 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.463 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.463 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.463 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.463 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.463 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.463 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.463 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.463 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.463 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.464 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.464 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.464 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.464 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.464 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.464 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.464 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.464 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.464 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.465 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.465 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.465 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.465 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.465 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.465 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.465 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.465 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.466 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.466 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.466 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.466 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.466 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.466 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.466 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.467 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.467 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.467 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.467 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.467 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.467 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.467 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.467 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.467 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.468 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.468 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.468 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.468 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.468 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.468 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.468 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.468 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.469 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.469 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.469 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.469 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.469 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.469 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.469 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.469 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.469 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.470 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.470 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.470 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.470 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.470 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.470 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.470 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.470 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "k8sevents", 2024-04-20T21:44:25.470 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.471 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.471 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.471 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_event_retention_days": { 2024-04-20T21:44:25.471 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ceph_event_retention_days", 2024-04-20T21:44:25.471 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.471 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.471 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.471 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7", 2024-04-20T21:44:25.472 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.472 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.472 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.472 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-20T21:44:25.472 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.472 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.472 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.472 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.472 INFO:teuthology.orchestra.run.smithi022.stdout: "config_check_secs": { 2024-04-20T21:44:25.473 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_check_secs", 2024-04-20T21:44:25.473 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.473 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.473 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.473 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:25.473 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "10", 2024-04-20T21:44:25.473 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.473 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.473 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-20T21:44:25.474 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.474 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.474 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.474 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.474 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.474 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.474 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.474 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.474 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.475 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.475 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.475 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.475 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.475 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.475 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.475 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.475 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.476 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.476 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.476 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.476 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.476 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.476 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.476 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.476 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.476 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.477 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.477 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.477 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.477 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.477 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.477 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.477 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.477 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.478 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.478 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.478 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.478 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.478 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.478 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.478 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.478 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.479 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.479 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.479 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.479 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.479 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.479 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.479 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.479 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.479 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.480 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.480 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.480 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.480 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.480 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.480 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.480 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.480 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.480 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.481 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.481 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.481 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.481 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.481 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.481 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.481 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.481 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.482 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.482 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.482 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.482 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.482 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.482 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.482 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.482 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.482 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.483 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "localpool", 2024-04-20T21:44:25.483 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.483 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.483 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.483 INFO:teuthology.orchestra.run.smithi022.stdout: "failure_domain": { 2024-04-20T21:44:25.483 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "failure_domain", 2024-04-20T21:44:25.483 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.483 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.484 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.484 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "host", 2024-04-20T21:44:25.484 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.484 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.484 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.484 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "failure domain for any created local pool", 2024-04-20T21:44:25.484 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T21:44:25.484 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.485 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.485 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.485 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.485 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.485 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.485 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.485 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.485 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.485 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.486 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.486 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.486 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.486 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.486 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.486 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.486 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.486 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.487 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.487 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.487 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.487 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.487 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.487 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.487 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.488 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.488 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.488 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.488 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.488 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.488 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.488 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.488 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.489 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.489 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.489 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.489 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.489 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.489 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.489 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.489 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.489 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.490 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.490 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.490 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.490 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.490 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.490 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.490 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.490 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.491 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.491 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.491 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.491 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.491 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.491 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.491 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.491 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.491 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.492 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.492 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.492 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.492 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.492 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.492 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.492 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.492 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": { 2024-04-20T21:44:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_size", 2024-04-20T21:44:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default min_size for any created local pool", 2024-04-20T21:44:25.495 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T21:44:25.495 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.495 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.495 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.495 INFO:teuthology.orchestra.run.smithi022.stdout: "num_rep": { 2024-04-20T21:44:25.495 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "num_rep", 2024-04-20T21:44:25.495 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.495 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.495 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.496 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3", 2024-04-20T21:44:25.496 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.496 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.496 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.496 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default replica count for any created local pool", 2024-04-20T21:44:25.496 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.496 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.496 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.497 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.497 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": { 2024-04-20T21:44:25.497 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_num", 2024-04-20T21:44:25.497 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.497 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.497 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.497 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "128", 2024-04-20T21:44:25.497 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.497 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.498 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.498 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T21:44:25.498 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.498 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.498 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.498 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.498 INFO:teuthology.orchestra.run.smithi022.stdout: "prefix": { 2024-04-20T21:44:25.498 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prefix", 2024-04-20T21:44:25.499 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.499 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.499 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.499 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.499 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.499 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.499 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.499 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name prefix for any created local pool", 2024-04-20T21:44:25.500 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.500 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.500 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.500 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.500 INFO:teuthology.orchestra.run.smithi022.stdout: "subtree": { 2024-04-20T21:44:25.500 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "subtree", 2024-04-20T21:44:25.500 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.500 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.500 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.501 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "rack", 2024-04-20T21:44:25.501 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.501 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.501 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.501 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T21:44:25.501 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T21:44:25.501 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.501 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.501 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.502 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.502 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.502 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.502 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mds_autoscaler", 2024-04-20T21:44:25.502 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.502 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.502 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.502 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.503 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.503 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.503 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.503 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.503 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.503 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.503 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.503 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.504 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.504 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.504 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.504 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.504 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.504 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.504 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.504 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.505 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.505 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.505 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.505 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.505 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.505 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.505 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.505 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.505 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.506 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.506 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.506 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.506 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.506 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.506 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.506 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.506 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.507 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.507 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.507 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.507 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.507 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.507 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.507 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.507 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.507 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.508 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.508 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.508 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.508 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.508 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.508 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.508 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.508 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.508 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.509 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.509 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.509 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.509 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.509 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.509 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.509 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.509 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.509 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.510 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.510 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.510 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.510 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.510 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.510 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.510 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.510 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.510 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.511 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.511 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.511 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.511 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirroring", 2024-04-20T21:44:25.511 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.511 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.511 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.511 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.512 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.512 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.512 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.512 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.512 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.512 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.512 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.512 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.512 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.513 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.513 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.513 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.513 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.513 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.513 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.513 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.513 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.514 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.514 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.514 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.514 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.514 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.514 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.514 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.514 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.514 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.515 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.515 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.515 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.515 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.515 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.515 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.515 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.515 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.515 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.516 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.516 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.516 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.516 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.516 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.516 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.516 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.516 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.516 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.517 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.517 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.517 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.517 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.517 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.517 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.517 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.517 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.518 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.518 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.518 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.518 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.518 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.518 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.518 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.518 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.518 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.519 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.519 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.519 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.519 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.519 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.519 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.519 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.519 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.519 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.520 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.520 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.520 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "nfs", 2024-04-20T21:44:25.520 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.520 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.520 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.520 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.520 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.521 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.521 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.521 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.521 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.521 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.521 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.521 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.521 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.521 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.522 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.522 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.522 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.522 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.522 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.522 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.522 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.522 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.523 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.523 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.523 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.523 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.523 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.523 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.523 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.523 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.524 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.524 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.524 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.524 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.524 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.524 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.524 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.524 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.524 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.525 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.525 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.525 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.525 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.525 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.525 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.525 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.525 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.526 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.526 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.526 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.526 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.526 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.526 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.526 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.526 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.527 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.527 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.527 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.527 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.527 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.527 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.527 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.527 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.527 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.529 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.529 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.529 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.529 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.529 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.529 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.530 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.530 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.530 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.530 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.530 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.530 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.530 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-04-20T21:44:25.530 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.531 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.531 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.531 INFO:teuthology.orchestra.run.smithi022.stdout: "fail_fs": { 2024-04-20T21:44:25.531 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "fail_fs", 2024-04-20T21:44:25.531 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.531 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.531 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.531 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.532 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.532 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.532 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.532 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T21:44:25.532 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.532 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.532 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.532 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.532 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.533 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.533 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.533 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.533 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.533 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.533 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.533 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.533 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.533 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.534 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.534 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.534 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.534 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.534 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.534 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.534 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.534 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.535 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.535 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.535 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.535 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.535 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.535 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.535 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.535 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.535 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.536 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.536 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.536 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.536 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.536 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.536 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.536 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.536 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.537 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.537 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.537 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.537 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.537 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.537 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.537 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.537 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.537 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.538 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.538 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.538 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.538 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.538 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.538 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.538 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.538 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.539 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.539 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.539 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.539 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.539 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.539 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.539 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.539 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.539 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.540 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.540 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.540 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.540 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.540 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.540 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.540 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.540 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.541 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.541 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator": { 2024-04-20T21:44:25.541 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-04-20T21:44:25.541 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.541 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.541 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.541 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.541 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.542 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.542 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.542 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2024-04-20T21:44:25.542 INFO:teuthology.orchestra.run.smithi022.stdout: "rook", 2024-04-20T21:44:25.542 INFO:teuthology.orchestra.run.smithi022.stdout: "test_orchestrator" 2024-04-20T21:44:25.542 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.542 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Orchestrator backend", 2024-04-20T21:44:25.542 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.543 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.543 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.543 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.543 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.543 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.543 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.543 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_perf_query", 2024-04-20T21:44:25.543 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.544 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.544 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.544 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.544 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.544 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.544 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.544 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.544 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.544 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.545 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.545 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.545 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.545 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.545 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.545 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.545 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.545 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.546 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.546 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.546 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.546 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.546 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.546 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.546 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.547 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.547 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.547 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.547 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.547 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.547 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.547 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.547 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.547 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.548 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.548 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.548 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.548 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.548 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.548 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.548 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.549 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.549 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.549 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.549 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.549 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.549 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.549 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.550 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.550 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.550 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.550 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.550 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.550 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.551 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.551 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.551 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.551 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.551 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.551 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.551 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.552 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.552 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.552 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.552 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.552 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.552 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.552 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.552 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.553 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.553 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.553 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.553 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.553 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.553 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.553 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.553 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_support", 2024-04-20T21:44:25.553 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.554 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.554 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.554 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.554 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.554 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.554 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.554 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.554 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.555 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.555 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.555 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.555 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.555 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.555 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.555 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.555 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.556 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.556 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.556 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.556 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.556 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.556 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.556 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.556 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.557 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.557 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.557 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.557 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.557 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.557 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.557 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.557 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.557 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.558 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.558 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.558 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.558 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.558 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.558 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.558 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.558 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.559 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.559 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.559 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.559 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.559 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.559 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.559 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.559 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.560 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.560 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.560 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.560 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.560 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.560 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.560 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.560 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.560 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.560 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.561 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.561 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.561 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.561 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.561 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.561 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.561 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.561 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.562 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.562 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.562 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.562 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.562 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.562 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.562 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.562 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.563 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_autoscaler", 2024-04-20T21:44:25.563 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.563 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.563 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.563 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.563 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.563 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.563 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.564 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.564 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.564 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.564 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.564 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.564 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.564 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.564 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.565 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.565 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.565 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.565 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.565 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.565 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.565 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.565 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.565 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.566 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.566 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.566 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.566 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.566 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.566 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.566 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.566 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.567 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.567 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.567 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.567 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.567 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.567 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.567 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.568 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.568 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.568 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.568 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.568 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.568 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.568 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.569 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.569 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.569 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.569 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.569 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.569 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.569 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.570 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.570 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.570 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.570 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.570 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.570 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.571 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.571 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.571 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.571 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.571 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.571 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.571 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.572 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.572 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.572 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.572 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.572 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.572 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.573 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:25.573 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:25.573 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.573 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.573 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.573 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:25.573 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.574 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.574 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.574 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.574 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.574 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.574 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.575 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.575 INFO:teuthology.orchestra.run.smithi022.stdout: "threshold": { 2024-04-20T21:44:25.575 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threshold", 2024-04-20T21:44:25.575 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:25.575 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.575 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.575 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-04-20T21:44:25.576 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1.0", 2024-04-20T21:44:25.576 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.576 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.576 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "scaling threshold", 2024-04-20T21:44:25.576 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T21:44:25.576 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.577 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.577 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.577 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.577 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.577 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "progress", 2024-04-20T21:44:25.577 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.578 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.578 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.578 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_pg_recovery_event": { 2024-04-20T21:44:25.578 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_pg_recovery_event", 2024-04-20T21:44:25.578 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.578 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.578 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.579 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.579 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.579 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.579 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.579 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T21:44:25.579 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.580 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.580 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.580 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-04-20T21:44:25.580 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-04-20T21:44:25.580 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.580 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.581 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.581 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.581 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.581 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.581 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.581 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.581 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.582 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.582 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.582 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.582 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.582 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.582 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.582 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.583 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.583 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.583 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.583 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.583 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.583 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.583 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.584 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.584 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.584 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.584 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.584 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.584 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.584 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.585 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.585 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.585 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.585 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.586 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.586 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.586 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.586 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.586 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.586 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.586 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.587 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.587 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.587 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.587 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.587 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.587 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.587 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.588 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.588 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.588 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.588 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.588 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.588 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.588 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.589 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.589 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.589 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.589 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.589 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.589 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.589 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.590 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.590 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.590 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.590 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.590 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.590 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.591 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.591 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.591 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.591 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.591 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.591 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.591 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.592 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.592 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.592 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.592 INFO:teuthology.orchestra.run.smithi022.stdout: "max_completed_events": { 2024-04-20T21:44:25.592 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_completed_events", 2024-04-20T21:44:25.592 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.592 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.593 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.593 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "50", 2024-04-20T21:44:25.593 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.593 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.593 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.593 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "number of past completed events to remember", 2024-04-20T21:44:25.593 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.593 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.594 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.594 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:25.594 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:25.594 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.594 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.594 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.594 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-04-20T21:44:25.595 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.595 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.595 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.595 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long the module is going to sleep", 2024-04-20T21:44:25.595 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.595 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.595 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.595 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.595 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.596 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus", 2024-04-20T21:44:25.596 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.596 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.596 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.596 INFO:teuthology.orchestra.run.smithi022.stdout: "cache": { 2024-04-20T21:44:25.597 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cache", 2024-04-20T21:44:25.597 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.597 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.597 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.597 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.597 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.597 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.597 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.597 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.598 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.598 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.598 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.598 INFO:teuthology.orchestra.run.smithi022.stdout: "exclude_perf_counters": { 2024-04-20T21:44:25.598 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "exclude_perf_counters", 2024-04-20T21:44:25.598 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.598 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.599 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.599 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.599 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.599 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.599 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.599 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T21:44:25.599 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T21:44:25.599 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.600 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.600 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.600 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.600 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.600 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.600 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.600 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.600 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.601 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.601 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.601 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.601 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.601 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.601 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.601 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.601 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.602 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.602 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.602 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.602 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.602 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.602 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.602 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.602 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.603 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.603 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.603 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.603 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.603 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.603 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.603 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.604 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.604 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.604 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.604 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.604 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.604 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.605 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.605 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.605 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.605 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.605 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.605 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.605 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.606 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.606 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.606 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.606 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.606 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.606 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.607 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.607 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.607 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.607 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.607 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.607 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.608 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.608 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.608 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.608 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.608 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.608 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.608 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.609 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.609 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.609 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.609 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.609 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools": { 2024-04-20T21:44:25.609 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools", 2024-04-20T21:44:25.610 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.610 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.610 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.610 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.610 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.610 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.611 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.611 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.611 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.611 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.611 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.611 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T21:44:25.612 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T21:44:25.612 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.612 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.612 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.612 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300", 2024-04-20T21:44:25.612 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.612 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.613 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.613 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.613 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.613 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.613 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.613 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_interval": { 2024-04-20T21:44:25.613 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_interval", 2024-04-20T21:44:25.614 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:25.614 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.614 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.614 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15.0", 2024-04-20T21:44:25.614 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.614 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.614 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.614 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.615 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.615 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.615 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.615 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-04-20T21:44:25.615 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-04-20T21:44:25.615 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.615 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.615 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.616 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-04-20T21:44:25.616 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.616 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.616 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.616 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T21:44:25.616 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.616 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.617 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.617 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-04-20T21:44:25.617 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-04-20T21:44:25.617 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.617 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.617 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "9283", 2024-04-20T21:44:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T21:44:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.619 INFO:teuthology.orchestra.run.smithi022.stdout: "stale_cache_strategy": { 2024-04-20T21:44:25.619 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stale_cache_strategy", 2024-04-20T21:44:25.619 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.619 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.619 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.620 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "log", 2024-04-20T21:44:25.620 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.620 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.620 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.620 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.620 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.621 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.621 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.621 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-04-20T21:44:25.621 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-04-20T21:44:25.621 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.621 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.621 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.622 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "default", 2024-04-20T21:44:25.622 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.622 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.622 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.622 INFO:teuthology.orchestra.run.smithi022.stdout: "default", 2024-04-20T21:44:25.622 INFO:teuthology.orchestra.run.smithi022.stdout: "error" 2024-04-20T21:44:25.623 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.623 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.623 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.623 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.623 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.624 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-04-20T21:44:25.624 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-04-20T21:44:25.624 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.624 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.624 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.624 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-04-20T21:44:25.624 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-04-20T21:44:25.625 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-04-20T21:44:25.625 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.625 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.625 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.625 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.625 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.626 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.626 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.626 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2024-04-20T21:44:25.626 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.626 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.626 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.627 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.627 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.627 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.627 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.627 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.627 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.627 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.628 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.628 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.628 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.628 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.628 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.628 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.628 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.628 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.628 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.629 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.629 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.629 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.629 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.629 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.629 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.629 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.629 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.630 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.630 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.630 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.630 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.630 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.630 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.630 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.630 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.631 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.631 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.631 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.631 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.631 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.631 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.631 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.632 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.632 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.632 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.632 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.632 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.632 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.632 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.632 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.632 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.633 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.633 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.633 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.633 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.633 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.633 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.633 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.634 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.634 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.634 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.634 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.634 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.634 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.634 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.634 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.634 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.635 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.635 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.635 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_snap_create": { 2024-04-20T21:44:25.635 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_snap_create", 2024-04-20T21:44:25.635 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.635 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.636 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.636 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:25.636 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.636 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.636 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.636 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.636 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.636 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.636 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.637 INFO:teuthology.orchestra.run.smithi022.stdout: "mirror_snapshot_schedule": { 2024-04-20T21:44:25.637 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T21:44:25.637 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.637 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.637 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.637 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.637 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.637 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.638 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.638 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.638 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.638 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.638 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.638 INFO:teuthology.orchestra.run.smithi022.stdout: "trash_purge_schedule": { 2024-04-20T21:44:25.639 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "trash_purge_schedule", 2024-04-20T21:44:25.639 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.639 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.639 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.639 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.639 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.639 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.639 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.639 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.640 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.640 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.640 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.640 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.640 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.640 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "restful", 2024-04-20T21:44:25.641 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.641 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.641 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.641 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_auth": { 2024-04-20T21:44:25.641 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_auth", 2024-04-20T21:44:25.641 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.641 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.641 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.642 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.642 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.642 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.642 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.642 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.642 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.642 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.642 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.643 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-04-20T21:44:25.643 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-04-20T21:44:25.643 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.643 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.643 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.643 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.643 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.643 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.644 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.644 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.644 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.644 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.644 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.644 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.644 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.645 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.645 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.645 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.645 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.645 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.645 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.645 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.645 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.645 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.646 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.646 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.646 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.646 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.646 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.646 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.646 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.646 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.647 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.647 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.647 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.647 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.647 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.647 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.647 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.647 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.648 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.648 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.648 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.648 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.648 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.648 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.648 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.649 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.649 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.649 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.649 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.649 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.649 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.649 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.649 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.649 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.650 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.650 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.650 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.650 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.650 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.650 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.650 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.650 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.651 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.651 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.651 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.651 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.651 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.651 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.651 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.651 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.652 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.652 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.652 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.652 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.652 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.652 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.652 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.652 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-04-20T21:44:25.653 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-04-20T21:44:25.653 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.653 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.653 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.653 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.653 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.653 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.653 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.654 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.654 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.654 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.654 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.654 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-04-20T21:44:25.654 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-04-20T21:44:25.654 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.655 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.655 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.655 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.655 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.655 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.655 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.655 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.655 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.656 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.656 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.656 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.656 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.656 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rgw", 2024-04-20T21:44:25.657 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.657 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.657 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.657 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.657 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.657 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.657 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.657 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.658 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.658 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.658 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.658 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.658 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.658 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.658 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.658 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.659 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.659 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.659 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.659 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.659 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.659 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.659 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.660 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.660 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.660 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.660 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.660 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.660 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.661 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.661 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.661 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.661 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.661 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.661 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.661 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.662 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.662 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.662 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.662 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.662 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.662 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.663 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.663 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.663 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.663 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.663 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.663 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.664 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.664 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.664 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.664 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.664 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.664 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.664 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.664 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.665 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.665 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.665 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.665 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.665 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.665 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.665 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.666 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.666 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.666 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.666 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.666 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.666 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.666 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.666 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.667 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rook", 2024-04-20T21:44:25.667 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.667 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.667 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.667 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.667 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.667 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.668 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.668 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.668 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.668 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.668 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.668 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.668 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.668 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.669 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.669 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.669 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.669 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.669 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.669 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.669 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.669 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.670 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.670 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.670 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.670 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.670 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.670 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.670 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.670 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.671 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.671 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.671 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.671 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.671 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.671 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.671 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.672 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.672 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.672 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.672 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.672 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.672 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.672 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.672 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.673 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.673 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.673 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.673 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.673 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.673 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.673 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.673 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.674 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.674 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.674 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.674 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.674 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.674 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.674 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.674 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.675 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.675 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.675 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.675 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.675 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.675 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.675 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: "storage_class": { 2024-04-20T21:44:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "storage_class", 2024-04-20T21:44:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "local", 2024-04-20T21:44:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T21:44:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "selftest", 2024-04-20T21:44:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "roption1": { 2024-04-20T21:44:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption1", 2024-04-20T21:44:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "roption2": { 2024-04-20T21:44:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption2", 2024-04-20T21:44:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "xyz", 2024-04-20T21:44:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption1": { 2024-04-20T21:44:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption1", 2024-04-20T21:44:25.691 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.691 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.691 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.691 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.691 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.691 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption2": { 2024-04-20T21:44:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption2", 2024-04-20T21:44:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption3": { 2024-04-20T21:44:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption3", 2024-04-20T21:44:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption4": { 2024-04-20T21:44:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption4", 2024-04-20T21:44:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption5": { 2024-04-20T21:44:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption5", 2024-04-20T21:44:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption6": { 2024-04-20T21:44:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption6", 2024-04-20T21:44:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption7": { 2024-04-20T21:44:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption7", 2024-04-20T21:44:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-04-20T21:44:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "42", 2024-04-20T21:44:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "testkey": { 2024-04-20T21:44:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testkey", 2024-04-20T21:44:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "testlkey": { 2024-04-20T21:44:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testlkey", 2024-04-20T21:44:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "testnewline": { 2024-04-20T21:44:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testnewline", 2024-04-20T21:44:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-04-20T21:44:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_m_granularity": { 2024-04-20T21:44:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_m_granularity", 2024-04-20T21:44:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T21:44:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "dump_on_update": { 2024-04-20T21:44:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dump_on_update", 2024-04-20T21:44:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "dump database to debug log on update", 2024-04-20T21:44:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.719 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.719 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.719 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.719 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.719 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.719 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.720 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.720 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.720 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.720 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.720 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.720 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.721 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.721 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.721 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.721 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.721 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.721 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.721 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.722 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.722 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.722 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.722 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.722 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.722 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.722 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.723 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stats", 2024-04-20T21:44:25.723 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.723 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.723 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.723 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.724 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.724 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.724 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.724 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.724 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.724 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.725 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.725 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.725 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.725 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.725 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.725 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.725 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.726 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.726 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.726 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.726 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.726 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.726 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.727 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.727 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.727 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.727 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.727 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.728 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.728 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.728 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.728 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.728 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.728 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.728 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.728 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.729 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.729 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.729 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.729 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.729 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.729 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.729 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.730 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.730 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.730 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.730 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.730 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.730 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.730 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.730 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.731 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.731 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.731 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.731 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.731 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.731 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.731 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.732 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.732 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.732 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.732 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.732 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.732 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.733 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.733 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.733 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.733 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.733 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.733 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.733 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.734 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.734 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.734 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "status", 2024-04-20T21:44:25.734 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.734 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.734 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.734 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.734 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.735 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.735 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.735 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.735 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.735 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.735 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.735 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.735 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.736 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.736 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.736 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.736 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.736 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.736 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.736 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.736 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.736 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.737 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.737 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.737 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.737 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.737 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.737 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.737 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.737 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.738 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.738 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.738 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.738 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.738 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.738 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.738 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.739 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.740 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.742 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telegraf", 2024-04-20T21:44:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: "address": { 2024-04-20T21:44:25.743 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "address", 2024-04-20T21:44:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T21:44:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.744 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.745 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15", 2024-04-20T21:44:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.746 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.747 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.748 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.749 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.750 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.751 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.753 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.754 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.755 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.755 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.755 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.755 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.755 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.755 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.755 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.755 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.756 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.756 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telemetry", 2024-04-20T21:44:25.756 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.756 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.756 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.756 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_basic": { 2024-04-20T21:44:25.756 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_basic", 2024-04-20T21:44:25.756 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.757 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.757 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.757 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.757 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.757 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.757 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.757 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T21:44:25.757 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.757 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.758 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.758 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.758 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_crash": { 2024-04-20T21:44:25.758 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_crash", 2024-04-20T21:44:25.758 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.758 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.758 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.758 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.759 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.759 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.759 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.759 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T21:44:25.759 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.759 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.759 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.760 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.760 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_device": { 2024-04-20T21:44:25.760 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_device", 2024-04-20T21:44:25.760 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.760 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.760 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.760 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.760 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.760 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.761 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.761 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T21:44:25.761 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.761 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.761 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.761 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.761 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_ident": { 2024-04-20T21:44:25.761 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_ident", 2024-04-20T21:44:25.762 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.762 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.762 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.762 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.762 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.762 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.762 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.762 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T21:44:25.763 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.763 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.763 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.763 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.763 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_perf": { 2024-04-20T21:44:25.763 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_perf", 2024-04-20T21:44:25.763 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.763 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.764 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.764 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.764 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.764 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.764 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.764 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T21:44:25.764 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.764 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.764 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.765 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.765 INFO:teuthology.orchestra.run.smithi022.stdout: "contact": { 2024-04-20T21:44:25.765 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "contact", 2024-04-20T21:44:25.765 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.765 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.765 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.765 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.765 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.766 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.766 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.766 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.766 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.766 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.766 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.766 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.766 INFO:teuthology.orchestra.run.smithi022.stdout: "description": { 2024-04-20T21:44:25.767 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "description", 2024-04-20T21:44:25.767 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.767 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.767 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.767 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.767 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.767 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.767 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.768 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.768 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.768 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.768 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.768 INFO:teuthology.orchestra.run.smithi022.stdout: "device_url": { 2024-04-20T21:44:25.768 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_url", 2024-04-20T21:44:25.768 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.768 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.769 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.769 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T21:44:25.769 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.769 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.769 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.769 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.769 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.769 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.770 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.770 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.770 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-04-20T21:44:25.770 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-04-20T21:44:25.770 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.770 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.770 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.770 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.771 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.771 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.771 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.771 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.771 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.771 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.771 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.771 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:25.772 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:25.772 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.772 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.772 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.772 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "24", 2024-04-20T21:44:25.772 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "8", 2024-04-20T21:44:25.772 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.772 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.772 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.773 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.773 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.773 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.773 INFO:teuthology.orchestra.run.smithi022.stdout: "last_opt_revision": { 2024-04-20T21:44:25.773 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "last_opt_revision", 2024-04-20T21:44:25.773 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.773 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.773 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.774 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1", 2024-04-20T21:44:25.774 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.774 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.774 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.774 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.774 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.774 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.774 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.775 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard": { 2024-04-20T21:44:25.775 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard", 2024-04-20T21:44:25.775 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.775 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.775 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.775 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.775 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.776 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.776 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.776 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.776 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.776 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.776 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.776 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard_description": { 2024-04-20T21:44:25.776 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard_description", 2024-04-20T21:44:25.777 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.777 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.777 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.777 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.777 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.777 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.777 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.777 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.777 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.778 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.778 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.778 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.778 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.778 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.778 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.778 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.778 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.779 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.779 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.779 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.779 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.779 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.779 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.779 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.779 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.779 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.780 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.780 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.780 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.780 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.780 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.780 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.780 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.781 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.781 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.781 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.781 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.781 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.781 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.781 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.781 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.781 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.782 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.782 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.782 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.782 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.782 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.782 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.782 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.782 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.783 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.783 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.783 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.783 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.783 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.783 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.783 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.783 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.783 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.784 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.784 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.784 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.784 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.784 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.784 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.784 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.785 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.785 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.785 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.785 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.785 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.785 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.785 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.785 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.786 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.786 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.786 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.786 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.786 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.786 INFO:teuthology.orchestra.run.smithi022.stdout: "organization": { 2024-04-20T21:44:25.786 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "organization", 2024-04-20T21:44:25.787 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.787 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.787 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.787 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.787 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.787 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.787 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.787 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.787 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.788 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.788 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.788 INFO:teuthology.orchestra.run.smithi022.stdout: "proxy": { 2024-04-20T21:44:25.788 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "proxy", 2024-04-20T21:44:25.788 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.788 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.788 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.789 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.789 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.789 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.789 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.789 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.789 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.789 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.789 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.790 INFO:teuthology.orchestra.run.smithi022.stdout: "url": { 2024-04-20T21:44:25.790 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url", 2024-04-20T21:44:25.790 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.790 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.790 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.790 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T21:44:25.790 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.791 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.791 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.791 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.791 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.791 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.791 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.791 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.791 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.791 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.792 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.792 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "test_orchestrator", 2024-04-20T21:44:25.792 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.792 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.792 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.792 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.792 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.792 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.792 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.793 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.793 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.793 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.793 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.793 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.793 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.793 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.793 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.793 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.794 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.794 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.794 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.794 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.794 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.794 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.794 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.795 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.795 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.795 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.795 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.795 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.795 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.795 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.795 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.795 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.796 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.796 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.796 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.796 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.796 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.796 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.796 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.796 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.797 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.797 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.797 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.797 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.797 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.797 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.797 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.797 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.797 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.798 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.798 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.798 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.798 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.798 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.798 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.798 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.798 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.799 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.799 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.799 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.799 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.799 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.799 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.799 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.800 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.800 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.800 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.800 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.800 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.800 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.800 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.800 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.801 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2024-04-20T21:44:25.801 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.801 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.801 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.801 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.801 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.802 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.802 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.802 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.802 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.802 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.802 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.802 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.802 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.803 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.803 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.803 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.803 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.803 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.803 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.803 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.803 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.804 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.804 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.804 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.804 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.804 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.804 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.804 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.805 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.805 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.805 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.805 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.805 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.805 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.805 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.805 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.806 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.806 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.806 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.806 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.806 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.806 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.806 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.806 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.807 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.807 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.807 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.807 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.807 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.807 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.807 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.807 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.808 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.808 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.808 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.808 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.808 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.808 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.808 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.809 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.809 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.809 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.809 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.809 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.809 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.809 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.809 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.809 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.810 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.810 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_clones": { 2024-04-20T21:44:25.810 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_clones", 2024-04-20T21:44:25.810 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.810 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.810 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.810 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4", 2024-04-20T21:44:25.811 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.811 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.811 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.811 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T21:44:25.811 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.811 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.811 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.811 INFO:teuthology.orchestra.run.smithi022.stdout: "periodic_async_work": { 2024-04-20T21:44:25.812 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "periodic_async_work", 2024-04-20T21:44:25.812 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.812 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.812 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.812 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.812 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.812 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.812 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.813 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Periodically check for async work", 2024-04-20T21:44:25.813 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.813 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.813 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.813 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_delay": { 2024-04-20T21:44:25.813 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_delay", 2024-04-20T21:44:25.813 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.813 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.813 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.814 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:25.814 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.814 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.814 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.814 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T21:44:25.814 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.814 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.814 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.815 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_no_wait": { 2024-04-20T21:44:25.815 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T21:44:25.815 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.815 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.815 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.815 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.815 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.815 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.815 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.816 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T21:44:25.816 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.816 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.816 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.816 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.816 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.816 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix", 2024-04-20T21:44:25.817 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.817 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.817 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.817 INFO:teuthology.orchestra.run.smithi022.stdout: "discovery_interval": { 2024-04-20T21:44:25.817 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "discovery_interval", 2024-04-20T21:44:25.817 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:25.817 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.818 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.818 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "100", 2024-04-20T21:44:25.818 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.818 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.818 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.818 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.818 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.818 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.819 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.819 INFO:teuthology.orchestra.run.smithi022.stdout: "identifier": { 2024-04-20T21:44:25.819 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "identifier", 2024-04-20T21:44:25.819 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.819 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.819 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.819 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.820 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.820 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.820 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.820 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.820 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.820 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.820 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.820 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:25.821 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:25.821 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.821 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.821 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.821 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:25.821 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.821 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.821 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.822 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.822 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.822 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.822 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.822 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.822 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.822 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.822 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.823 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.823 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.823 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.823 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.823 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.823 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.823 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.823 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.824 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.824 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.824 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.824 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.824 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.824 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.824 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.824 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.825 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.825 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.825 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.825 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.825 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.825 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.825 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.825 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.825 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.826 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.826 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.826 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.826 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.826 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.826 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.826 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.827 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.827 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.827 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.827 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.827 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.827 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.827 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.827 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.828 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.828 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.828 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.828 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.828 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.828 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.828 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.828 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.829 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.829 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.829 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.829 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.829 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.829 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.829 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.829 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.830 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.830 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.830 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.830 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.830 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.830 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.830 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_host": { 2024-04-20T21:44:25.831 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_host", 2024-04-20T21:44:25.831 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.831 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.831 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.831 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.831 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.831 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.831 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.832 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.832 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.832 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.832 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.832 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_port": { 2024-04-20T21:44:25.832 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_port", 2024-04-20T21:44:25.832 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.832 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.833 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.833 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10051", 2024-04-20T21:44:25.833 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.833 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.833 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.833 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.833 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.833 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.834 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.834 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_sender": { 2024-04-20T21:44:25.834 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_sender", 2024-04-20T21:44:25.834 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.834 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.834 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.834 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T21:44:25.834 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.835 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.835 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.835 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.835 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.835 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.835 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.835 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.835 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.836 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.836 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-20T21:44:25.836 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.836 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.836 INFO:teuthology.orchestra.run.smithi022.stdout: "modules": [ 2024-04-20T21:44:25.836 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2024-04-20T21:44:25.836 INFO:teuthology.orchestra.run.smithi022.stdout: "dashboard", 2024-04-20T21:44:25.836 INFO:teuthology.orchestra.run.smithi022.stdout: "iostat", 2024-04-20T21:44:25.836 INFO:teuthology.orchestra.run.smithi022.stdout: "nfs", 2024-04-20T21:44:25.837 INFO:teuthology.orchestra.run.smithi022.stdout: "restful", 2024-04-20T21:44:25.837 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_schedule" 2024-04-20T21:44:25.837 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.837 INFO:teuthology.orchestra.run.smithi022.stdout: "available_modules": [ 2024-04-20T21:44:25.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.837 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "alerts", 2024-04-20T21:44:25.837 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.837 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.837 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.838 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:25.838 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:25.838 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.838 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.838 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.838 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:25.838 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.838 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.838 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.839 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T21:44:25.839 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.839 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.839 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.839 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.839 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.839 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.840 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.840 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.840 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.840 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.840 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.840 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.840 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.840 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.840 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.841 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.841 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.841 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.841 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.841 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.841 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.841 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.841 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.842 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.842 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.842 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.842 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.842 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.842 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.842 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.843 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.843 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.843 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.843 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.843 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.843 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.843 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.843 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.844 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.844 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.844 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.844 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.844 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.844 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.844 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.844 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.844 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.845 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.845 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.845 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.845 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.845 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.845 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.845 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.845 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.845 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.846 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.846 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.846 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.846 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.846 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.846 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.846 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.847 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.847 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.847 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.847 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.847 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.847 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.847 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_destination": { 2024-04-20T21:44:25.847 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_destination", 2024-04-20T21:44:25.848 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.848 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.848 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.848 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.848 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.848 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.848 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.848 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email address to send alerts to", 2024-04-20T21:44:25.848 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.849 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.849 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.849 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_from_name": { 2024-04-20T21:44:25.849 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_from_name", 2024-04-20T21:44:25.849 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.849 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.849 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.849 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "Ceph", 2024-04-20T21:44:25.850 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.850 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.850 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.850 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email From: name", 2024-04-20T21:44:25.850 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.850 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.850 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.850 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_host": { 2024-04-20T21:44:25.851 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_host", 2024-04-20T21:44:25.851 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.851 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.851 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.851 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.851 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.851 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.851 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.852 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP server", 2024-04-20T21:44:25.852 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.852 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.852 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.852 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.852 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_password": { 2024-04-20T21:44:25.852 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_password", 2024-04-20T21:44:25.853 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.853 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.853 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.853 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.853 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.853 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.853 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.854 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Password to authenticate with", 2024-04-20T21:44:25.854 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.854 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.854 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.854 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.854 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_port": { 2024-04-20T21:44:25.854 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_port", 2024-04-20T21:44:25.854 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.855 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.855 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.855 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "465", 2024-04-20T21:44:25.855 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.855 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.855 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.855 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP port", 2024-04-20T21:44:25.856 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.856 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.856 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.856 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.856 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_sender": { 2024-04-20T21:44:25.856 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_sender", 2024-04-20T21:44:25.856 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.857 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.857 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.857 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.857 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.857 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.857 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.857 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP envelope sender", 2024-04-20T21:44:25.857 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.858 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.858 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.858 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_ssl": { 2024-04-20T21:44:25.858 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_ssl", 2024-04-20T21:44:25.858 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.858 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.859 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.859 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.859 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.859 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.859 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.859 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T21:44:25.859 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.860 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.860 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.860 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_user": { 2024-04-20T21:44:25.860 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_user", 2024-04-20T21:44:25.860 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.860 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.861 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.861 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.861 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.861 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.861 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.861 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "User to authenticate as", 2024-04-20T21:44:25.861 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.862 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.862 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.862 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.862 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.862 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "balancer", 2024-04-20T21:44:25.862 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.863 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.863 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.863 INFO:teuthology.orchestra.run.smithi022.stdout: "active": { 2024-04-20T21:44:25.863 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "active", 2024-04-20T21:44:25.863 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.863 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.863 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.864 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.864 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.864 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.864 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.864 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T21:44:25.864 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.864 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.864 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.865 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_time": { 2024-04-20T21:44:25.865 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_time", 2024-04-20T21:44:25.865 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.865 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.865 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.865 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0000", 2024-04-20T21:44:25.865 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.865 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.866 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.866 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T21:44:25.866 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T21:44:25.866 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.866 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.866 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_weekday": { 2024-04-20T21:44:25.867 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_weekday", 2024-04-20T21:44:25.867 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:25.867 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.867 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.867 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:25.867 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-04-20T21:44:25.867 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-04-20T21:44:25.868 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.868 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T21:44:25.868 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T21:44:25.868 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.868 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.869 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_max_iterations": { 2024-04-20T21:44:25.869 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_max_iterations", 2024-04-20T21:44:25.869 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:25.869 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.869 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.869 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "25", 2024-04-20T21:44:25.869 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-04-20T21:44:25.870 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "250", 2024-04-20T21:44:25.870 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.870 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T21:44:25.870 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.870 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.870 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.871 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_metrics": { 2024-04-20T21:44:25.871 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_metrics", 2024-04-20T21:44:25.871 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.871 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.871 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.871 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T21:44:25.871 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.871 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.872 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.872 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T21:44:25.872 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T21:44:25.872 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.872 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.872 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_step": { 2024-04-20T21:44:25.872 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_step", 2024-04-20T21:44:25.873 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:25.873 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.873 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.873 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.5", 2024-04-20T21:44:25.873 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0.001", 2024-04-20T21:44:25.873 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "0.999", 2024-04-20T21:44:25.873 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.873 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "aggressiveness of optimization", 2024-04-20T21:44:25.874 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T21:44:25.874 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.874 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.874 INFO:teuthology.orchestra.run.smithi022.stdout: "end_time": { 2024-04-20T21:44:25.874 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_time", 2024-04-20T21:44:25.874 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.874 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.875 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.875 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2359", 2024-04-20T21:44:25.875 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.875 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.875 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.875 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T21:44:25.875 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T21:44:25.875 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.876 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.876 INFO:teuthology.orchestra.run.smithi022.stdout: "end_weekday": { 2024-04-20T21:44:25.876 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_weekday", 2024-04-20T21:44:25.876 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:25.876 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.876 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.876 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:25.876 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-04-20T21:44:25.877 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-04-20T21:44:25.877 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.877 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T21:44:25.877 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T21:44:25.877 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.877 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.877 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.877 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.878 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.878 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.878 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.878 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.878 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.878 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.878 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.878 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.878 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.879 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.879 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.879 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.879 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.879 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.879 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.879 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.879 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.879 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.880 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.880 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.880 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.880 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.880 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.880 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.880 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.880 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.881 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.881 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.881 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.881 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.881 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.881 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.881 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.882 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.882 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.882 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.882 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.882 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.882 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.882 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.882 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.882 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.883 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.883 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.883 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.883 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.883 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.883 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.883 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.883 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.883 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.884 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.884 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.884 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.884 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.884 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.884 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.884 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.884 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.884 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.885 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.885 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.885 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.885 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.885 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.885 INFO:teuthology.orchestra.run.smithi022.stdout: "min_score": { 2024-04-20T21:44:25.885 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_score", 2024-04-20T21:44:25.886 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:25.886 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.886 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.886 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:25.886 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.886 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.886 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.887 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T21:44:25.887 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.887 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.887 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.887 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.887 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-04-20T21:44:25.887 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-04-20T21:44:25.888 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.888 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.888 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.888 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "upmap", 2024-04-20T21:44:25.888 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.888 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.888 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.888 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-compat", 2024-04-20T21:44:25.888 INFO:teuthology.orchestra.run.smithi022.stdout: "none", 2024-04-20T21:44:25.889 INFO:teuthology.orchestra.run.smithi022.stdout: "read", 2024-04-20T21:44:25.889 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap", 2024-04-20T21:44:25.889 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap-read" 2024-04-20T21:44:25.889 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.889 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Balancer mode", 2024-04-20T21:44:25.889 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.889 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.889 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.890 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_ids": { 2024-04-20T21:44:25.890 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_ids", 2024-04-20T21:44:25.890 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.890 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.890 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.890 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.890 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.890 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.891 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.891 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T21:44:25.891 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.891 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.891 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.891 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:25.891 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:25.892 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.892 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.892 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.892 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:25.892 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.892 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.892 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.892 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T21:44:25.893 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.893 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.893 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.893 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_deviation": { 2024-04-20T21:44:25.893 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_deviation", 2024-04-20T21:44:25.893 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.894 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.894 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.894 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-04-20T21:44:25.894 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-04-20T21:44:25.894 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.894 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.894 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T21:44:25.895 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T21:44:25.895 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.895 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.895 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_optimizations": { 2024-04-20T21:44:25.895 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_optimizations", 2024-04-20T21:44:25.895 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:25.896 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.896 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.896 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:25.896 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.896 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.896 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.896 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T21:44:25.897 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.897 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.897 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.897 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.897 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:25.897 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.897 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:25.897 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm", 2024-04-20T21:44:25.898 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:25.898 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:25.898 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:25.898 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_down_multiplier": { 2024-04-20T21:44:25.898 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_down_multiplier", 2024-04-20T21:44:25.898 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:25.899 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.899 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.899 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-04-20T21:44:25.899 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.899 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.899 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.899 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T21:44:25.899 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.900 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.900 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.900 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_refresh_rate": { 2024-04-20T21:44:25.900 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_refresh_rate", 2024-04-20T21:44:25.900 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.900 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.901 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.901 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "20", 2024-04-20T21:44:25.901 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.901 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.901 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.901 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T21:44:25.901 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.902 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.902 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.902 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.902 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_starting_port": { 2024-04-20T21:44:25.902 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_starting_port", 2024-04-20T21:44:25.902 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.903 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.903 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.903 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4721", 2024-04-20T21:44:25.903 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.903 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.903 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.903 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T21:44:25.903 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.903 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.904 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.904 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.904 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_ptrace": { 2024-04-20T21:44:25.904 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_ptrace", 2024-04-20T21:44:25.904 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.904 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.904 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.904 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.905 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.905 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.905 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.905 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T21:44:25.905 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T21:44:25.905 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.905 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.905 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_interval": { 2024-04-20T21:44:25.906 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_interval", 2024-04-20T21:44:25.906 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.906 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.906 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.906 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:25.906 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.906 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.907 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.907 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T21:44:25.907 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.907 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.907 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.907 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.907 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_memory_target_ratio": { 2024-04-20T21:44:25.908 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T21:44:25.908 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:25.908 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.908 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.908 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.7", 2024-04-20T21:44:25.908 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.909 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.909 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.909 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T21:44:25.909 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.909 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.909 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.909 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.910 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm_log_destination": { 2024-04-20T21:44:25.910 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm_log_destination", 2024-04-20T21:44:25.910 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.910 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.910 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.910 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.910 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.910 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.911 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.911 INFO:teuthology.orchestra.run.smithi022.stdout: "file", 2024-04-20T21:44:25.911 INFO:teuthology.orchestra.run.smithi022.stdout: "file,syslog", 2024-04-20T21:44:25.911 INFO:teuthology.orchestra.run.smithi022.stdout: "syslog" 2024-04-20T21:44:25.911 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.911 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T21:44:25.911 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.911 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.911 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.912 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.912 INFO:teuthology.orchestra.run.smithi022.stdout: "cgroups_split": { 2024-04-20T21:44:25.912 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cgroups_split", 2024-04-20T21:44:25.912 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.912 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.912 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.913 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.913 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.913 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.913 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.913 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T21:44:25.913 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.913 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.914 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.914 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.914 INFO:teuthology.orchestra.run.smithi022.stdout: "config_checks_enabled": { 2024-04-20T21:44:25.914 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_checks_enabled", 2024-04-20T21:44:25.914 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.914 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.914 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.914 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.914 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.915 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.915 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.915 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T21:44:25.915 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.915 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.915 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.915 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.915 INFO:teuthology.orchestra.run.smithi022.stdout: "config_dashboard": { 2024-04-20T21:44:25.915 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_dashboard", 2024-04-20T21:44:25.916 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.916 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.916 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.916 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.916 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.916 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.916 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.916 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T21:44:25.916 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.917 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.917 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.917 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.917 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_alertmanager": { 2024-04-20T21:44:25.917 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_alertmanager", 2024-04-20T21:44:25.917 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.917 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.917 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.917 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T21:44:25.918 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.918 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.918 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.918 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-04-20T21:44:25.918 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.918 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.918 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.918 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.918 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_base": { 2024-04-20T21:44:25.919 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_base", 2024-04-20T21:44:25.919 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.919 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.919 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.919 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T21:44:25.919 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.919 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.919 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.919 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Container image name, without the tag", 2024-04-20T21:44:25.920 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.920 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.920 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.920 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_elasticsearch": { 2024-04-20T21:44:25.920 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_elasticsearch", 2024-04-20T21:44:25.920 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.920 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.920 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.921 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T21:44:25.921 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.921 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.921 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.921 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "elasticsearch container image", 2024-04-20T21:44:25.921 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.921 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.921 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.921 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.922 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_grafana": { 2024-04-20T21:44:25.922 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_grafana", 2024-04-20T21:44:25.922 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.922 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.922 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.922 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T21:44:25.922 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.922 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.923 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.923 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-04-20T21:44:25.923 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.923 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.923 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.923 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.923 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_haproxy": { 2024-04-20T21:44:25.923 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_haproxy", 2024-04-20T21:44:25.923 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.924 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.924 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.924 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T21:44:25.924 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.924 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.924 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.924 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "HAproxy container image", 2024-04-20T21:44:25.924 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.924 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.925 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.925 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_agent": { 2024-04-20T21:44:25.925 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_agent", 2024-04-20T21:44:25.925 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.925 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.925 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.926 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T21:44:25.926 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.926 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.926 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.926 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger agent container image", 2024-04-20T21:44:25.926 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.926 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.927 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.927 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.927 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_collector": { 2024-04-20T21:44:25.927 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_collector", 2024-04-20T21:44:25.927 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.927 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.927 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.928 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T21:44:25.928 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.928 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.928 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.928 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger collector container image", 2024-04-20T21:44:25.928 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.928 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.929 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.929 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.929 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_query": { 2024-04-20T21:44:25.929 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_query", 2024-04-20T21:44:25.929 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.929 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.929 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.929 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T21:44:25.930 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.930 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.930 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.930 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger query container image", 2024-04-20T21:44:25.930 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.930 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.930 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.931 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_keepalived": { 2024-04-20T21:44:25.931 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_keepalived", 2024-04-20T21:44:25.931 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.931 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.931 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.931 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T21:44:25.932 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.932 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.932 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.932 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Keepalived container image", 2024-04-20T21:44:25.932 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.932 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.932 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.933 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_loki": { 2024-04-20T21:44:25.933 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_loki", 2024-04-20T21:44:25.933 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.933 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.933 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.934 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T21:44:25.934 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.934 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.934 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.935 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Loki container image", 2024-04-20T21:44:25.935 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.935 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.935 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.936 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_node_exporter": { 2024-04-20T21:44:25.936 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_node_exporter", 2024-04-20T21:44:25.936 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.936 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.937 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.937 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T21:44:25.937 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.937 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.937 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.937 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-04-20T21:44:25.938 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.938 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.938 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.938 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_nvmeof": { 2024-04-20T21:44:25.938 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_nvmeof", 2024-04-20T21:44:25.938 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.938 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.939 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.939 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T21:44:25.939 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.939 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.939 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.939 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Nvme-of container image", 2024-04-20T21:44:25.939 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.939 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.939 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.940 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_prometheus": { 2024-04-20T21:44:25.940 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_prometheus", 2024-04-20T21:44:25.940 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.940 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.940 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.940 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T21:44:25.940 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.941 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.941 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.941 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-04-20T21:44:25.941 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.941 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.941 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.941 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_promtail": { 2024-04-20T21:44:25.942 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_promtail", 2024-04-20T21:44:25.942 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.942 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.942 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.942 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T21:44:25.942 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.942 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.943 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.943 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Promtail container image", 2024-04-20T21:44:25.943 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.943 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.943 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.944 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_samba": { 2024-04-20T21:44:25.944 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_samba", 2024-04-20T21:44:25.944 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.944 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.944 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.944 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T21:44:25.945 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.945 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.945 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.945 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Samba/SMB container image", 2024-04-20T21:44:25.945 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.945 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.945 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.946 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_snmp_gateway": { 2024-04-20T21:44:25.946 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_snmp_gateway", 2024-04-20T21:44:25.946 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.946 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.946 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.946 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T21:44:25.946 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.946 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.947 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.947 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SNMP Gateway container image", 2024-04-20T21:44:25.947 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.947 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.947 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.947 INFO:teuthology.orchestra.run.smithi022.stdout: "container_init": { 2024-04-20T21:44:25.947 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_init", 2024-04-20T21:44:25.947 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.948 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.948 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.948 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.948 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.948 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.948 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.948 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T21:44:25.948 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.949 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.949 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.949 INFO:teuthology.orchestra.run.smithi022.stdout: "daemon_cache_timeout": { 2024-04-20T21:44:25.949 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "daemon_cache_timeout", 2024-04-20T21:44:25.949 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.949 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.949 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.949 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:25.950 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.950 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.950 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.950 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T21:44:25.950 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.950 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.950 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.950 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.950 INFO:teuthology.orchestra.run.smithi022.stdout: "default_cephadm_command_timeout": { 2024-04-20T21:44:25.951 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T21:44:25.951 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.951 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.951 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.951 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "900", 2024-04-20T21:44:25.951 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.951 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.951 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.951 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T21:44:25.952 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.952 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.952 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.952 INFO:teuthology.orchestra.run.smithi022.stdout: "default_registry": { 2024-04-20T21:44:25.952 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_registry", 2024-04-20T21:44:25.952 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.952 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.952 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.953 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io", 2024-04-20T21:44:25.953 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.953 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.953 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.953 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T21:44:25.953 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.953 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.953 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.954 INFO:teuthology.orchestra.run.smithi022.stdout: "device_cache_timeout": { 2024-04-20T21:44:25.954 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_cache_timeout", 2024-04-20T21:44:25.954 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.954 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.954 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.954 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1800", 2024-04-20T21:44:25.954 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.954 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.954 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.955 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache device inventory", 2024-04-20T21:44:25.955 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.955 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.955 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.955 INFO:teuthology.orchestra.run.smithi022.stdout: "device_enhanced_scan": { 2024-04-20T21:44:25.955 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_enhanced_scan", 2024-04-20T21:44:25.955 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.955 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.956 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.956 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.956 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.956 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.956 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.956 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T21:44:25.956 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.956 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.956 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.957 INFO:teuthology.orchestra.run.smithi022.stdout: "facts_cache_timeout": { 2024-04-20T21:44:25.957 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "facts_cache_timeout", 2024-04-20T21:44:25.957 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.957 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.957 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.957 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:25.957 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.957 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.958 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.958 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache host facts data", 2024-04-20T21:44:25.958 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.958 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.958 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.958 INFO:teuthology.orchestra.run.smithi022.stdout: "grafana_dashboards_path": { 2024-04-20T21:44:25.958 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "grafana_dashboards_path", 2024-04-20T21:44:25.959 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.959 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.959 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.959 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T21:44:25.959 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.959 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.959 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.959 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T21:44:25.959 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.960 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.960 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.960 INFO:teuthology.orchestra.run.smithi022.stdout: "host_check_interval": { 2024-04-20T21:44:25.960 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "host_check_interval", 2024-04-20T21:44:25.960 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:25.960 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.960 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.960 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:25.961 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.961 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.961 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.961 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to perform a host check", 2024-04-20T21:44:25.961 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.961 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.961 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.961 INFO:teuthology.orchestra.run.smithi022.stdout: "hw_monitoring": { 2024-04-20T21:44:25.962 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hw_monitoring", 2024-04-20T21:44:25.962 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.962 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.962 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.962 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.962 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.962 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.962 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.963 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T21:44:25.963 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.963 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.963 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.963 INFO:teuthology.orchestra.run.smithi022.stdout: "inventory_list_all": { 2024-04-20T21:44:25.963 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "inventory_list_all", 2024-04-20T21:44:25.963 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.963 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.963 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.964 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.964 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.964 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.964 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.964 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T21:44:25.964 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.964 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.964 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.965 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:25.965 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:25.965 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.965 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.965 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.965 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.965 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.965 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.965 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.966 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.966 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.966 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.966 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.966 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.966 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.966 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.966 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.967 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.967 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.967 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.967 INFO:teuthology.orchestra.run.smithi022.stdout: "log_refresh_metadata": { 2024-04-20T21:44:25.967 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_refresh_metadata", 2024-04-20T21:44:25.967 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.967 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.968 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.968 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.968 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.968 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.968 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.968 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T21:44:25.968 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.968 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.968 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.969 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.969 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:25.969 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:25.969 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.969 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.969 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.969 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:25.969 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.970 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.970 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.970 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T21:44:25.970 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.970 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.970 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.970 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:25.970 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:25.971 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.971 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.971 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.971 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:25.971 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.971 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.971 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.971 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:25.971 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:25.972 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:25.972 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:25.972 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:25.972 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:25.972 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.972 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.972 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.972 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.972 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.973 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.973 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:25.973 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:25.973 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.973 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.973 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:25.973 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.973 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.973 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.974 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.974 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:25.974 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.974 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.974 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.974 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.974 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T21:44:25.975 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T21:44:25.975 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.975 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.975 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.975 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.975 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.975 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.975 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.975 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T21:44:25.976 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.976 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.976 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.976 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.976 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T21:44:25.976 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T21:44:25.976 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.976 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.976 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.977 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "*", 2024-04-20T21:44:25.977 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.977 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.977 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.977 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T21:44:25.977 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.977 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.978 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.978 INFO:teuthology.orchestra.run.smithi022.stdout: "max_count_per_host": { 2024-04-20T21:44:25.978 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_count_per_host", 2024-04-20T21:44:25.978 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.978 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.979 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.979 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:25.979 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.979 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.979 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.979 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of daemons per service per host", 2024-04-20T21:44:25.979 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.980 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.980 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.980 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.980 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd_draining_count": { 2024-04-20T21:44:25.980 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_osd_draining_count", 2024-04-20T21:44:25.980 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.980 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.981 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.981 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:25.981 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.981 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.981 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.981 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T21:44:25.981 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.981 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.982 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.982 INFO:teuthology.orchestra.run.smithi022.stdout: "migration_current": { 2024-04-20T21:44:25.982 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "migration_current", 2024-04-20T21:44:25.982 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.982 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.982 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.983 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.983 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.983 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.983 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.983 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "internal - do not modify", 2024-04-20T21:44:25.983 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.983 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.983 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.984 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-04-20T21:44:25.984 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-04-20T21:44:25.984 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.984 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.984 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.984 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "root", 2024-04-20T21:44:25.985 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.985 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.985 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:25.985 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm-package", 2024-04-20T21:44:25.985 INFO:teuthology.orchestra.run.smithi022.stdout: "root" 2024-04-20T21:44:25.985 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:25.985 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T21:44:25.985 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.986 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.986 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.986 INFO:teuthology.orchestra.run.smithi022.stdout: "oob_default_addr": { 2024-04-20T21:44:25.986 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "oob_default_addr", 2024-04-20T21:44:25.986 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.986 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.986 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.987 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "169.254.1.1", 2024-04-20T21:44:25.987 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.987 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.987 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.987 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T21:44:25.987 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.987 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.987 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.988 INFO:teuthology.orchestra.run.smithi022.stdout: "prometheus_alerts_path": { 2024-04-20T21:44:25.988 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus_alerts_path", 2024-04-20T21:44:25.988 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.988 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.988 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.988 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T21:44:25.988 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.988 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.989 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.989 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T21:44:25.989 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.989 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.989 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.989 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_insecure": { 2024-04-20T21:44:25.989 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_insecure", 2024-04-20T21:44:25.990 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.990 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.990 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.990 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.990 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.990 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.990 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.990 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T21:44:25.991 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.991 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.991 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.991 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_password": { 2024-04-20T21:44:25.991 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_password", 2024-04-20T21:44:25.991 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.991 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.991 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.992 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.992 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.992 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.992 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.992 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T21:44:25.992 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.992 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.992 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.993 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_url": { 2024-04-20T21:44:25.993 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_url", 2024-04-20T21:44:25.993 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.993 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.993 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.993 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.993 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.993 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.993 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.994 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T21:44:25.994 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.994 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.994 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.994 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_username": { 2024-04-20T21:44:25.994 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_username", 2024-04-20T21:44:25.994 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:25.994 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T21:44:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "secure_monitoring_stack": { 2024-04-20T21:44:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secure_monitoring_stack", 2024-04-20T21:44:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T21:44:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "service_discovery_port": { 2024-04-20T21:44:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "service_discovery_port", 2024-04-20T21:44:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8765", 2024-04-20T21:44:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "cephadm service discovery port", 2024-04-20T21:44:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.000 INFO:teuthology.orchestra.run.smithi022.stdout: "ssh_config_file": { 2024-04-20T21:44:26.000 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssh_config_file", 2024-04-20T21:44:26.000 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.000 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.000 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.000 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.000 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.001 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.001 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.001 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T21:44:26.001 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.001 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.001 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "use_agent": { 2024-04-20T21:44:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_agent", 2024-04-20T21:44:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T21:44:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "use_repo_digest": { 2024-04-20T21:44:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_repo_digest", 2024-04-20T21:44:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T21:44:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_failed_host_check": { 2024-04-20T21:44:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_failed_host_check", 2024-04-20T21:44:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T21:44:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_daemons": { 2024-04-20T21:44:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_daemons", 2024-04-20T21:44:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.009 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.009 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.009 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.009 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.009 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.009 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.009 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T21:44:26.010 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.010 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.010 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.010 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_hosts": { 2024-04-20T21:44:26.010 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_hosts", 2024-04-20T21:44:26.011 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.011 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.011 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.011 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.011 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.011 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.011 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.012 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T21:44:26.012 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.012 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.012 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.012 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.012 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.012 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.012 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.013 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crash", 2024-04-20T21:44:26.013 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.013 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.013 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.013 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.013 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.013 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.013 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.014 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.014 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.014 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.014 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.014 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.014 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.014 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.014 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.014 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.015 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.015 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.015 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.015 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.015 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.015 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.015 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.016 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.016 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.016 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.016 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.016 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.016 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.016 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.016 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.017 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.017 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.017 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.017 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.017 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.017 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.017 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.017 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.018 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.018 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.018 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.018 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.018 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.018 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.018 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.018 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.019 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.019 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.019 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.019 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.019 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.019 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.019 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.019 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.019 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.020 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.020 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.020 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.020 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.020 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.020 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.020 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.021 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.021 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.021 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.021 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.021 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.021 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.021 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.021 INFO:teuthology.orchestra.run.smithi022.stdout: "retain_interval": { 2024-04-20T21:44:26.022 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retain_interval", 2024-04-20T21:44:26.022 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:26.022 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.022 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.022 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "31536000", 2024-04-20T21:44:26.022 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.022 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.022 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.022 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T21:44:26.023 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.023 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.023 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.023 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_recent_interval": { 2024-04-20T21:44:26.023 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_recent_interval", 2024-04-20T21:44:26.023 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:26.023 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.024 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.024 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1209600", 2024-04-20T21:44:26.024 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.024 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.024 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.024 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T21:44:26.024 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.024 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.025 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.025 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.025 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.025 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dashboard", 2024-04-20T21:44:26.025 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.025 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.026 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.026 INFO:teuthology.orchestra.run.smithi022.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T21:44:26.026 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T21:44:26.026 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.026 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.026 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.026 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:26.026 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.027 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.027 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.027 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.027 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.027 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.027 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.027 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T21:44:26.028 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T21:44:26.028 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.028 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.028 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.028 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.028 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.028 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.028 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.029 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.029 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.029 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.029 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.029 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T21:44:26.029 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T21:44:26.029 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.029 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.030 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.030 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.030 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.030 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.030 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.030 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.030 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.030 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.031 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.031 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_ENABLED": { 2024-04-20T21:44:26.031 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T21:44:26.031 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.031 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.031 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.031 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.032 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.032 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.032 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.032 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.032 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.032 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.032 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.032 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T21:44:26.033 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T21:44:26.033 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.033 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.033 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.033 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.033 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.033 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.033 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.034 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.034 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.034 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.034 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.034 INFO:teuthology.orchestra.run.smithi022.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T21:44:26.034 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T21:44:26.034 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.035 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.035 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.035 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.035 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.035 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.035 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.035 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.035 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.036 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.036 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.036 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T21:44:26.036 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T21:44:26.036 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.036 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.036 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.036 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.037 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.037 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.037 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.037 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.037 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.037 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.037 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.038 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T21:44:26.038 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T21:44:26.038 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.038 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.038 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.038 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.038 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.038 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.039 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.039 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.039 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.039 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.039 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.039 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T21:44:26.039 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T21:44:26.039 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.040 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.040 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.040 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.040 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.040 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.040 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.040 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.040 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.041 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.041 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.041 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T21:44:26.041 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T21:44:26.041 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.041 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.041 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.042 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.042 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.042 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.042 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.042 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.042 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.042 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.042 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.043 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T21:44:26.043 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T21:44:26.043 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.043 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.043 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.043 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.043 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.043 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.044 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.044 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.044 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.044 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.044 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.044 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T21:44:26.044 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T21:44:26.044 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.045 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.045 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.045 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.045 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.045 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.045 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.045 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.045 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.045 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.046 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.046 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T21:44:26.046 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T21:44:26.046 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.046 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.046 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.046 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.047 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.047 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.047 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.047 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.047 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.047 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.047 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.048 INFO:teuthology.orchestra.run.smithi022.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T21:44:26.048 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T21:44:26.048 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.048 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.048 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.048 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.048 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.048 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.048 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.049 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.049 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.049 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.049 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.049 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T21:44:26.049 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T21:44:26.049 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.050 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.050 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.050 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-04-20T21:44:26.050 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.050 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.050 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.050 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.050 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.050 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T21:44:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T21:44:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.052 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.052 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.052 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.052 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.052 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.052 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.052 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_URL": { 2024-04-20T21:44:26.052 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_URL", 2024-04-20T21:44:26.053 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.053 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.053 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.053 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.053 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.053 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.053 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.053 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.053 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.054 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.054 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.054 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T21:44:26.054 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T21:44:26.054 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.054 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.055 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.055 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-04-20T21:44:26.055 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.055 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.055 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.055 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.055 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.055 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.055 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.056 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.056 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T21:44:26.056 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T21:44:26.056 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.056 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.056 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.056 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.056 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.057 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.057 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.057 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.057 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.057 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.057 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.057 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T21:44:26.057 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T21:44:26.058 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.058 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.058 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.058 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.058 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.058 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.058 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.058 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.059 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.059 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.059 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.059 INFO:teuthology.orchestra.run.smithi022.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T21:44:26.059 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T21:44:26.059 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.059 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.059 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.060 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.060 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.060 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.060 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.060 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.060 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.060 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.060 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.061 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.061 INFO:teuthology.orchestra.run.smithi022.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T21:44:26.061 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T21:44:26.061 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.061 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.061 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.061 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.061 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.062 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.062 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.062 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.062 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.062 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.062 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.062 INFO:teuthology.orchestra.run.smithi022.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T21:44:26.063 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T21:44:26.063 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.063 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.063 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.063 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "[]", 2024-04-20T21:44:26.063 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.063 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.063 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.063 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.064 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.064 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.064 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.064 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.064 INFO:teuthology.orchestra.run.smithi022.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T21:44:26.064 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T21:44:26.064 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.064 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.064 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.065 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "{}", 2024-04-20T21:44:26.065 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.065 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.065 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.065 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.065 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.065 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.065 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.065 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.066 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T21:44:26.066 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T21:44:26.066 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.066 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.066 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.066 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.066 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.066 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.066 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.067 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.067 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.067 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.067 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.067 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.067 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T21:44:26.067 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T21:44:26.067 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.068 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.068 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.068 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.068 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.068 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.068 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.068 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.068 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.068 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.069 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.069 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T21:44:26.069 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T21:44:26.069 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.069 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.069 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.069 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.070 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.070 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.070 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.070 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.070 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.070 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.070 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.070 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.070 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T21:44:26.071 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T21:44:26.071 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.071 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.071 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.071 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.071 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.071 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.071 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.072 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.072 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.072 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.072 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.072 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.072 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T21:44:26.072 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T21:44:26.072 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.073 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.073 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.073 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.073 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.073 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.073 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.073 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.073 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.073 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.074 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.074 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.074 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T21:44:26.074 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T21:44:26.074 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.074 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.074 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.074 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.075 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.075 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.075 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.075 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.075 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.075 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.075 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.075 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.076 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T21:44:26.076 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T21:44:26.076 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.076 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.076 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.076 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.076 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.076 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.076 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.077 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.077 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.077 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.077 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.077 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.077 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T21:44:26.077 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T21:44:26.077 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.078 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.078 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.078 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.078 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.078 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.078 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.078 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.078 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.079 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.079 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.079 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T21:44:26.079 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T21:44:26.079 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.079 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.079 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.080 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.080 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.080 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.080 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.080 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.080 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.080 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.080 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.081 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T21:44:26.081 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T21:44:26.081 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.081 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.081 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.081 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.081 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.081 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.081 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.082 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.082 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.082 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.082 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.082 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T21:44:26.082 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T21:44:26.082 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.083 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.083 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.083 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T21:44:26.083 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.083 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.083 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.083 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.083 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.084 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.084 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.084 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T21:44:26.084 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T21:44:26.084 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.084 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.084 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.084 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:26.085 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.085 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.085 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.085 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.085 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.085 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.085 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.085 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T21:44:26.086 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T21:44:26.086 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.086 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.086 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.086 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8", 2024-04-20T21:44:26.086 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.086 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.086 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.086 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.087 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.087 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.087 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.087 INFO:teuthology.orchestra.run.smithi022.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T21:44:26.087 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T21:44:26.087 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.087 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.088 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.088 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "45", 2024-04-20T21:44:26.088 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.088 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.088 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.088 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.088 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.088 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.088 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.089 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T21:44:26.089 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T21:44:26.089 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.089 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.089 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.089 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.089 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.089 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.090 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.090 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.090 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.090 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.090 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.090 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T21:44:26.090 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T21:44:26.091 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.091 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.091 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.091 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-04-20T21:44:26.091 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.091 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.091 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.091 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.091 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.092 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.092 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.092 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T21:44:26.092 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T21:44:26.092 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.092 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.092 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.093 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.093 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.093 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.093 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.093 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.093 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.093 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.093 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.094 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T21:44:26.094 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T21:44:26.094 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.094 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.094 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.094 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.094 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.094 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.095 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.095 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.095 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.095 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.095 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.095 INFO:teuthology.orchestra.run.smithi022.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T21:44:26.095 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T21:44:26.095 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.096 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.096 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.096 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.096 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.096 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.096 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.096 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.096 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.096 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.097 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.097 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T21:44:26.097 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T21:44:26.097 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.097 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.097 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.097 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:26.097 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.098 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.098 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.098 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.098 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.098 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.098 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.098 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T21:44:26.099 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T21:44:26.099 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.099 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.099 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.099 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:26.099 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.099 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.099 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.099 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.100 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.100 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.100 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.100 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.100 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T21:44:26.100 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T21:44:26.100 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.100 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.100 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.101 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-04-20T21:44:26.101 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.101 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.101 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.101 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.101 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.101 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.101 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.101 INFO:teuthology.orchestra.run.smithi022.stdout: "cross_origin_url": { 2024-04-20T21:44:26.102 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cross_origin_url", 2024-04-20T21:44:26.102 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.102 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.102 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.102 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.102 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.102 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.102 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.103 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.103 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.103 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.103 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.103 INFO:teuthology.orchestra.run.smithi022.stdout: "crt_file": { 2024-04-20T21:44:26.103 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crt_file", 2024-04-20T21:44:26.103 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.103 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.104 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.104 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.104 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.104 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.104 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.104 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.104 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.104 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.104 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.105 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.105 INFO:teuthology.orchestra.run.smithi022.stdout: "debug": { 2024-04-20T21:44:26.105 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "debug", 2024-04-20T21:44:26.105 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.105 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.105 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.105 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.105 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.106 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.106 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.106 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable/disable debug options", 2024-04-20T21:44:26.106 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.106 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.106 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.106 INFO:teuthology.orchestra.run.smithi022.stdout: "jwt_token_ttl": { 2024-04-20T21:44:26.107 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "jwt_token_ttl", 2024-04-20T21:44:26.107 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.107 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.107 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.107 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "28800", 2024-04-20T21:44:26.107 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.107 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.107 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.107 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.108 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.108 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.108 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.108 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-04-20T21:44:26.108 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-04-20T21:44:26.108 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.108 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.109 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.109 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.109 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.109 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.109 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.109 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.109 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.109 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.109 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.110 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.110 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.110 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.110 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.110 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.110 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.110 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.111 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.111 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.111 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.111 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.111 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.111 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.111 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.111 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.111 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.112 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.112 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.112 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.112 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.112 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.112 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.112 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.113 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.113 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.113 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.113 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.113 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.113 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.113 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.113 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.113 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.114 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.114 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.114 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.114 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.114 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.114 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.114 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.114 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.115 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.115 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.115 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.115 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.115 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.115 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.115 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.115 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.116 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.116 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.116 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.116 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.116 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.116 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.116 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.116 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.117 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.117 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.117 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.117 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.117 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.117 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.117 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.117 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.117 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.118 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.118 INFO:teuthology.orchestra.run.smithi022.stdout: "motd": { 2024-04-20T21:44:26.118 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "motd", 2024-04-20T21:44:26.118 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.118 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.118 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.118 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.119 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.119 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.119 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.119 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "The message of the day", 2024-04-20T21:44:26.119 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.119 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.119 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.119 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect_resolve_ip_addr": { 2024-04-20T21:44:26.120 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T21:44:26.120 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.120 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.120 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.120 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.120 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.120 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.120 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.121 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.121 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.121 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.121 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.121 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-04-20T21:44:26.121 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-04-20T21:44:26.121 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.122 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.122 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.122 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-04-20T21:44:26.122 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.122 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.122 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.122 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.122 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.122 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.123 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.123 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-04-20T21:44:26.123 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-04-20T21:44:26.123 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.123 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.123 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.123 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8080", 2024-04-20T21:44:26.123 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.124 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.124 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.124 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.124 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.124 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.124 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.124 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-04-20T21:44:26.125 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-04-20T21:44:26.125 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.125 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.125 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.125 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.125 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.125 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.125 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.125 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.126 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.126 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.126 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.126 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl_server_port": { 2024-04-20T21:44:26.126 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl_server_port", 2024-04-20T21:44:26.126 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.126 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.127 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.127 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8443", 2024-04-20T21:44:26.127 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.127 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.127 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.127 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.127 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.127 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.127 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.128 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-04-20T21:44:26.128 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-04-20T21:44:26.128 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.128 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.128 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.128 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "redirect", 2024-04-20T21:44:26.128 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.128 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.129 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.129 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.129 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect" 2024-04-20T21:44:26.129 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.129 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.129 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.129 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.129 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.130 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-04-20T21:44:26.130 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-04-20T21:44:26.130 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.130 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.130 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.130 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-04-20T21:44:26.130 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-04-20T21:44:26.130 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-04-20T21:44:26.131 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.131 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.131 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.131 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.131 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.131 INFO:teuthology.orchestra.run.smithi022.stdout: "url_prefix": { 2024-04-20T21:44:26.131 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url_prefix", 2024-04-20T21:44:26.132 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.132 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.132 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.132 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.132 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.132 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.132 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.132 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.132 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.132 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2024-04-20T21:44:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.134 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_monitoring": { 2024-04-20T21:44:26.134 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_monitoring", 2024-04-20T21:44:26.134 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.134 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.134 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.134 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.134 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.134 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.135 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.135 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "monitor device health metrics", 2024-04-20T21:44:26.135 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.135 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.135 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.135 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.135 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.135 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.136 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.136 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.136 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.136 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.136 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.136 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.136 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.136 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.137 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.137 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.137 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.137 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.137 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.137 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.137 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.137 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.137 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.138 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.138 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.138 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.138 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.138 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.138 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.138 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.138 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.138 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.139 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.139 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.139 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.141 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.141 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.142 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.142 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.142 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.142 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.142 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.142 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.142 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.143 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.143 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.143 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.143 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.143 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.143 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.143 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.143 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.143 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.144 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.144 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.144 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.144 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.144 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.144 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.144 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.144 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.145 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.145 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.145 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.145 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.145 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.145 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.145 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.145 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.145 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.146 INFO:teuthology.orchestra.run.smithi022.stdout: "mark_out_threshold": { 2024-04-20T21:44:26.146 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mark_out_threshold", 2024-04-20T21:44:26.146 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:26.146 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.146 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.146 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2419200", 2024-04-20T21:44:26.146 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.147 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.147 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.147 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T21:44:26.147 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.147 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.147 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.147 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": { 2024-04-20T21:44:26.147 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_name", 2024-04-20T21:44:26.148 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.148 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.148 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.148 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "device_health_metrics", 2024-04-20T21:44:26.148 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.148 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.148 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.148 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T21:44:26.149 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.149 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.149 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.149 INFO:teuthology.orchestra.run.smithi022.stdout: "retention_period": { 2024-04-20T21:44:26.149 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retention_period", 2024-04-20T21:44:26.149 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:26.149 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.149 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.150 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15552000", 2024-04-20T21:44:26.150 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.150 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.150 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.150 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain device health metrics", 2024-04-20T21:44:26.150 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_frequency": { 2024-04-20T21:44:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_frequency", 2024-04-20T21:44:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.151 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-04-20T21:44:26.152 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.152 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.152 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.152 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T21:44:26.152 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.152 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.152 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.152 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.153 INFO:teuthology.orchestra.run.smithi022.stdout: "self_heal": { 2024-04-20T21:44:26.153 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "self_heal", 2024-04-20T21:44:26.153 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.153 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.153 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.153 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.153 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.153 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.153 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.154 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T21:44:26.154 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.154 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.154 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.154 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:26.154 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:26.154 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:26.154 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.155 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.155 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:26.155 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.155 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.155 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.155 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T21:44:26.155 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.155 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.155 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.156 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_threshold": { 2024-04-20T21:44:26.156 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_threshold", 2024-04-20T21:44:26.156 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:26.156 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.156 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.156 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7257600", 2024-04-20T21:44:26.156 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.157 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.157 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.157 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T21:44:26.157 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.157 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.157 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.157 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.157 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.158 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.158 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "diskprediction_local", 2024-04-20T21:44:26.158 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.158 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.158 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.158 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.158 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.158 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.158 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.159 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.159 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.159 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.159 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.159 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.159 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.159 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.159 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.160 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.160 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.160 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.160 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.160 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.160 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.160 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.160 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.161 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.161 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.161 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.161 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.161 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.161 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.161 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.161 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.161 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.162 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.162 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.162 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.162 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.162 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.162 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.162 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.163 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.163 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.163 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.163 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.163 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.163 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.163 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.163 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.164 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.164 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.164 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.164 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.164 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.164 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.164 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.164 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.164 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.165 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.165 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.165 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.165 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.165 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.165 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.165 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.165 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.166 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.166 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.166 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.166 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.166 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.166 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.166 INFO:teuthology.orchestra.run.smithi022.stdout: "predict_interval": { 2024-04-20T21:44:26.166 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predict_interval", 2024-04-20T21:44:26.167 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.167 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.167 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.167 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-04-20T21:44:26.167 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.167 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.167 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.167 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.167 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.168 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.168 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.168 INFO:teuthology.orchestra.run.smithi022.stdout: "predictor_model": { 2024-04-20T21:44:26.168 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predictor_model", 2024-04-20T21:44:26.168 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.168 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.168 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.169 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "prophetstor", 2024-04-20T21:44:26.169 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.169 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.169 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.169 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.169 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.169 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.169 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.170 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:26.170 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:26.170 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.170 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.170 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.170 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-04-20T21:44:26.170 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.171 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.171 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.171 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.171 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.171 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.171 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.171 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.171 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.172 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "influx", 2024-04-20T21:44:26.172 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": false, 2024-04-20T21:44:26.172 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "influxdb python module not found", 2024-04-20T21:44:26.172 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.172 INFO:teuthology.orchestra.run.smithi022.stdout: "batch_size": { 2024-04-20T21:44:26.172 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "batch_size", 2024-04-20T21:44:26.172 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.173 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.173 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.173 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5000", 2024-04-20T21:44:26.173 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.173 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.173 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.173 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T21:44:26.173 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.173 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.174 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.174 INFO:teuthology.orchestra.run.smithi022.stdout: "database": { 2024-04-20T21:44:26.174 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "database", 2024-04-20T21:44:26.174 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.174 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.174 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.174 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "ceph", 2024-04-20T21:44:26.175 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.175 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.175 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.175 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T21:44:26.175 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.175 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.175 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.176 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": { 2024-04-20T21:44:26.176 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hostname", 2024-04-20T21:44:26.176 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.176 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.176 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.176 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.176 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.176 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.176 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.177 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server hostname", 2024-04-20T21:44:26.177 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.177 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.177 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.177 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:26.177 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:26.177 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:26.178 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.178 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.178 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "30", 2024-04-20T21:44:26.178 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "5", 2024-04-20T21:44:26.178 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.178 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.178 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T21:44:26.178 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.179 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.179 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.179 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.179 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.179 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.179 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.179 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.179 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.180 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.180 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.180 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.180 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.180 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.180 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.180 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.180 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.180 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.181 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.181 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.181 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.181 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.181 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.181 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.181 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.181 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.185 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.185 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.185 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.185 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.185 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.185 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.185 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.185 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.185 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.185 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.186 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.186 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.186 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.186 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.186 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.186 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.186 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.186 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.187 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.187 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.187 INFO:teuthology.orchestra.run.smithi022.stdout: "password": { 2024-04-20T21:44:26.187 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "password", 2024-04-20T21:44:26.187 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.187 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.187 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.187 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.187 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.188 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.188 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.188 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "password of InfluxDB server user", 2024-04-20T21:44:26.188 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.188 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.188 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.188 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.188 INFO:teuthology.orchestra.run.smithi022.stdout: "port": { 2024-04-20T21:44:26.188 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "port", 2024-04-20T21:44:26.189 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.189 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.189 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.189 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8086", 2024-04-20T21:44:26.189 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.189 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.189 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.189 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server port", 2024-04-20T21:44:26.189 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.190 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.190 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.190 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-04-20T21:44:26.190 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-04-20T21:44:26.190 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.190 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.190 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.191 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "false", 2024-04-20T21:44:26.191 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.191 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.191 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.191 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T21:44:26.191 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.191 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.191 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.192 INFO:teuthology.orchestra.run.smithi022.stdout: "threads": { 2024-04-20T21:44:26.192 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threads", 2024-04-20T21:44:26.192 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.192 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.192 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.192 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-04-20T21:44:26.192 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-04-20T21:44:26.192 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "32", 2024-04-20T21:44:26.193 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.193 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T21:44:26.193 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.193 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.193 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.193 INFO:teuthology.orchestra.run.smithi022.stdout: "username": { 2024-04-20T21:44:26.193 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "username", 2024-04-20T21:44:26.194 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.194 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.194 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.194 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.194 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.194 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.194 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.195 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "username of InfluxDB server user", 2024-04-20T21:44:26.195 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.195 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.195 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.195 INFO:teuthology.orchestra.run.smithi022.stdout: "verify_ssl": { 2024-04-20T21:44:26.195 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "verify_ssl", 2024-04-20T21:44:26.195 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.196 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.196 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.196 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "true", 2024-04-20T21:44:26.196 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.196 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.196 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.196 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T21:44:26.196 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.196 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.197 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.197 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.197 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.197 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.197 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.197 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "insights", 2024-04-20T21:44:26.197 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.197 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.198 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.198 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.198 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.198 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.198 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.198 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.198 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.198 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.199 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.199 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.199 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.199 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.199 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.199 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.199 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.199 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.199 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.200 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.200 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.200 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.200 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.200 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.200 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.200 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.201 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.201 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.201 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.201 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.201 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.201 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.201 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.201 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.201 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.202 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.202 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.202 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.202 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.202 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.202 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.202 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.202 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.202 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.203 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.203 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.203 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.203 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.203 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.203 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.203 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.203 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.203 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.204 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.204 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.204 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.204 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.204 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.204 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.204 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.205 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.205 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.205 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.205 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.205 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.205 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.205 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.205 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.205 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.205 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.206 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.206 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.206 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.206 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "iostat", 2024-04-20T21:44:26.206 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.206 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.206 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.207 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.207 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.207 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.207 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.207 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.207 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.207 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.207 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.207 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.208 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.208 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.208 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.208 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.208 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.208 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.208 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.208 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.208 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.209 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.209 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.209 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.209 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.209 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.209 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.209 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.209 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.210 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.210 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.210 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.210 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.210 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.210 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.210 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.210 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.210 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.211 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.211 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.211 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.211 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.211 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.211 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.211 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.211 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.212 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.212 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.212 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.212 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.212 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.212 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.212 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.212 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.212 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.213 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.213 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.213 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.213 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.213 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.213 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.213 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.213 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.213 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.214 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.214 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.214 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.214 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.214 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.214 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.214 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.214 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.215 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.215 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.215 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.215 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "k8sevents", 2024-04-20T21:44:26.215 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.215 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.215 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.215 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_event_retention_days": { 2024-04-20T21:44:26.215 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ceph_event_retention_days", 2024-04-20T21:44:26.216 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.216 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.216 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.216 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7", 2024-04-20T21:44:26.216 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.216 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.216 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.216 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-20T21:44:26.216 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.217 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.217 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.217 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.217 INFO:teuthology.orchestra.run.smithi022.stdout: "config_check_secs": { 2024-04-20T21:44:26.217 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_check_secs", 2024-04-20T21:44:26.217 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.217 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.217 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.218 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:26.218 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "10", 2024-04-20T21:44:26.218 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.218 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.218 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-20T21:44:26.218 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.218 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.218 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.219 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.219 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.219 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.219 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.219 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.219 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.219 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.219 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.220 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.220 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.220 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.220 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.220 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.220 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.220 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.220 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.220 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.221 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.221 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.221 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.221 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.221 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.221 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.221 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.221 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.222 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.222 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.222 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.222 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.222 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.222 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.222 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.223 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.223 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.223 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.223 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.223 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.223 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.223 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.223 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.224 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.224 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.224 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.224 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.224 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.224 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.224 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.224 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.224 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.225 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.225 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.225 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.225 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.225 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.225 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.225 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.225 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.226 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.226 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.226 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.226 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.226 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.226 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.226 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.226 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.227 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.227 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.227 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.227 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.227 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.227 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "localpool", 2024-04-20T21:44:26.227 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.228 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.228 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.228 INFO:teuthology.orchestra.run.smithi022.stdout: "failure_domain": { 2024-04-20T21:44:26.228 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "failure_domain", 2024-04-20T21:44:26.228 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.228 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.228 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.228 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "host", 2024-04-20T21:44:26.228 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.229 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.229 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.229 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "failure domain for any created local pool", 2024-04-20T21:44:26.229 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T21:44:26.229 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.229 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.229 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.229 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.230 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.230 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.230 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.230 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.230 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.230 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.230 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.231 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.231 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.231 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.231 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.231 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.231 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.231 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.231 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.231 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.232 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.232 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.232 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.232 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.232 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.232 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.232 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.233 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.233 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.233 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.233 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.233 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.233 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.233 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.233 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.233 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.234 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.234 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.234 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.234 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.234 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.234 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.234 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.234 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.235 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.235 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.235 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.235 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.235 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.235 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.235 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.235 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.235 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.236 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.236 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.236 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.236 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.236 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.236 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.236 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.236 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.237 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.237 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.237 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.237 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.237 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.237 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.237 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.237 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.237 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.238 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.238 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": { 2024-04-20T21:44:26.238 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_size", 2024-04-20T21:44:26.238 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.238 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.238 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.238 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.238 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.238 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.239 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.239 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default min_size for any created local pool", 2024-04-20T21:44:26.239 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T21:44:26.239 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.239 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.239 INFO:teuthology.orchestra.run.smithi022.stdout: "num_rep": { 2024-04-20T21:44:26.239 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "num_rep", 2024-04-20T21:44:26.240 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.240 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.240 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.240 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3", 2024-04-20T21:44:26.240 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.240 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.240 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.240 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default replica count for any created local pool", 2024-04-20T21:44:26.240 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.241 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.241 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.241 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.241 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": { 2024-04-20T21:44:26.241 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_num", 2024-04-20T21:44:26.241 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.241 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.241 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.241 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "128", 2024-04-20T21:44:26.242 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.242 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.242 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.242 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T21:44:26.242 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.242 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.242 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.243 INFO:teuthology.orchestra.run.smithi022.stdout: "prefix": { 2024-04-20T21:44:26.243 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prefix", 2024-04-20T21:44:26.243 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.243 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.243 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.243 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.243 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.243 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.244 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.244 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name prefix for any created local pool", 2024-04-20T21:44:26.244 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.244 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.244 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.244 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.244 INFO:teuthology.orchestra.run.smithi022.stdout: "subtree": { 2024-04-20T21:44:26.244 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "subtree", 2024-04-20T21:44:26.244 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.245 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.245 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.245 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "rack", 2024-04-20T21:44:26.245 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.245 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.245 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.245 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T21:44:26.245 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T21:44:26.245 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.246 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.246 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.246 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.246 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.246 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mds_autoscaler", 2024-04-20T21:44:26.246 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.246 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.246 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.247 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.247 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.247 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.247 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.247 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.247 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.247 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.247 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.247 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.248 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.248 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.248 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.248 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.248 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.248 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.248 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.248 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.249 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.249 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.249 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.249 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.249 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.249 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.249 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.250 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.250 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.250 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.250 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.250 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.250 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.250 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.250 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.250 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.251 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.251 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.251 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.251 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.251 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.251 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.251 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.251 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.251 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.252 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.252 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.252 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.252 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.252 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.252 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.252 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.252 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.253 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.253 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.253 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.253 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.253 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.253 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.253 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.253 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.254 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.254 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.254 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.254 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.254 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.254 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.254 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.254 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.254 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.255 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.255 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.255 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.255 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.255 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.255 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirroring", 2024-04-20T21:44:26.255 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.255 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.255 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.256 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.256 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.256 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.256 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.256 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.256 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.256 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.256 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.256 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.257 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.257 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.257 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.257 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.257 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.257 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.257 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.257 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.258 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.258 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.258 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.258 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.258 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.258 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.258 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.258 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.258 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.259 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.259 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.259 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.259 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.259 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.259 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.259 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.259 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.260 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.260 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.260 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.260 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.260 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.260 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.260 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.260 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.260 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.261 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.261 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.261 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.261 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.261 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.261 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.261 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.261 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.261 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.262 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.262 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.262 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.262 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.262 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.262 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.262 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.262 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.263 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.263 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.263 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.263 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.263 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.263 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.263 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.263 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.263 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.264 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.264 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.264 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.264 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "nfs", 2024-04-20T21:44:26.264 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.264 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.264 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.264 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.265 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.265 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.265 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.265 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.265 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.265 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.265 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.265 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.266 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.266 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.266 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.266 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.266 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.266 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.266 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.266 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.266 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.267 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.267 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.267 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.267 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.267 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.267 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.267 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.267 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.268 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.268 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.268 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.268 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.268 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.268 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.268 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.268 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.268 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.269 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.269 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.269 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.269 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.269 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.269 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.269 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.269 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.270 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.270 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.270 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.270 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.270 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.270 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.270 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.270 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.271 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.271 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.271 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.271 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.271 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.271 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.271 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.271 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.272 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.272 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.272 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.272 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.272 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.272 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.272 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.272 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.273 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.273 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.273 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.273 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.273 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.273 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-04-20T21:44:26.273 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.273 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.274 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.274 INFO:teuthology.orchestra.run.smithi022.stdout: "fail_fs": { 2024-04-20T21:44:26.274 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "fail_fs", 2024-04-20T21:44:26.274 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.274 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.274 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.274 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.274 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.275 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.275 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.275 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T21:44:26.275 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.275 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.275 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.275 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.275 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.275 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.276 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.276 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.276 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.276 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.276 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.276 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.276 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.276 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.277 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.277 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.277 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.277 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.277 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.277 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.277 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.277 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.277 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.278 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.278 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.278 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.278 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.278 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.278 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.278 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.278 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.279 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.279 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.279 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.279 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.279 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.279 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.279 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.279 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.280 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.280 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.280 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.280 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.280 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.280 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.280 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.280 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.280 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.281 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.281 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.281 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.281 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.281 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.281 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.281 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.281 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.281 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.282 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.282 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.282 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.282 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.282 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.282 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.282 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.282 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.283 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.283 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.283 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.283 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.283 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.283 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.283 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.283 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.283 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.284 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator": { 2024-04-20T21:44:26.284 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-04-20T21:44:26.284 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.284 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.284 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.284 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.284 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.284 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.285 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.285 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2024-04-20T21:44:26.285 INFO:teuthology.orchestra.run.smithi022.stdout: "rook", 2024-04-20T21:44:26.285 INFO:teuthology.orchestra.run.smithi022.stdout: "test_orchestrator" 2024-04-20T21:44:26.285 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.285 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Orchestrator backend", 2024-04-20T21:44:26.285 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.285 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.285 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.286 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.286 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.286 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.286 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.286 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_perf_query", 2024-04-20T21:44:26.286 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.286 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.286 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.286 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.287 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.287 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.287 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.287 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.287 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.287 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.287 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.287 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.288 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.288 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.288 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.288 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.288 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.288 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.288 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.288 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.289 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.289 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.289 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.289 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.289 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.289 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.289 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.290 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.290 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.290 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.290 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.290 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.290 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.290 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.290 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.290 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.291 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.291 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.291 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.291 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.291 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.291 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.291 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.291 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.292 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.292 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.292 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.292 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.292 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.292 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.292 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.292 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.293 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.293 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.293 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.293 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.293 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.293 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.293 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.293 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.293 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.294 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.294 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.294 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.294 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.294 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.294 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.294 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.294 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.295 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.295 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.295 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.295 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.295 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.295 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.295 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_support", 2024-04-20T21:44:26.295 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.295 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.296 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.296 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.296 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.296 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.296 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.296 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.296 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.296 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.296 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.297 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.297 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.297 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.297 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.297 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.297 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.297 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.297 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.297 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.298 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.298 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.298 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.298 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.298 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.298 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.298 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.299 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.299 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.299 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.299 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.299 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.299 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.299 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.299 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.300 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.300 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.300 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.300 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.300 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.300 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.300 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.300 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.300 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.301 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.301 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.301 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.301 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.301 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.301 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.301 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.301 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.302 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.302 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.302 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.302 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.302 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.302 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.302 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.302 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.303 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.303 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.303 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.303 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.303 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.303 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.303 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.303 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.304 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.304 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.304 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.304 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.304 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.304 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_autoscaler", 2024-04-20T21:44:26.304 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.304 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.305 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.305 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.305 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.305 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.305 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.305 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.305 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.305 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.306 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.306 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.306 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.306 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.306 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.306 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.306 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.307 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.307 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.307 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.307 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.307 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.307 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.307 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.307 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.308 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.308 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.308 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.308 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.308 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.308 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.308 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.308 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.309 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.309 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.309 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.309 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.309 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.309 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.309 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.309 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.309 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.310 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.310 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.310 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.310 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.310 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.310 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.310 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.310 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.310 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.311 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.311 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.311 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.311 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.311 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.311 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.311 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.311 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.312 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.312 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.312 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.312 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.312 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.312 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.312 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.312 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.312 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.313 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.313 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.313 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.313 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:26.313 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:26.313 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:26.313 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.313 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.314 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:26.314 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.314 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.314 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.314 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.314 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.314 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.314 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.315 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.315 INFO:teuthology.orchestra.run.smithi022.stdout: "threshold": { 2024-04-20T21:44:26.315 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threshold", 2024-04-20T21:44:26.315 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:26.315 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.315 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.315 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-04-20T21:44:26.315 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1.0", 2024-04-20T21:44:26.315 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.316 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.316 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "scaling threshold", 2024-04-20T21:44:26.316 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T21:44:26.316 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.316 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.316 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.316 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.316 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.317 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "progress", 2024-04-20T21:44:26.317 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.317 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.317 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.317 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_pg_recovery_event": { 2024-04-20T21:44:26.317 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_pg_recovery_event", 2024-04-20T21:44:26.317 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.317 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.317 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.318 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.318 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.318 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.318 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.318 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T21:44:26.318 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.318 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.318 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.319 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-04-20T21:44:26.319 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-04-20T21:44:26.319 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.319 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.319 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.319 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.319 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.319 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.320 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.320 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.320 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.320 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.320 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.320 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.320 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.320 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.321 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.321 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.321 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.321 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.321 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.321 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.321 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.321 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.322 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.322 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.322 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.322 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.322 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.322 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.322 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.322 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.323 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.323 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.323 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.323 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.323 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.323 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.323 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.323 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.323 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.324 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.324 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.324 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.324 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.324 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.324 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.324 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.324 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.325 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.325 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.325 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.325 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.325 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.325 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.325 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.325 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.326 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.326 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.326 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.326 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.326 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.326 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.326 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.326 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.326 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.327 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.327 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.327 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.327 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.327 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.327 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.327 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.327 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.328 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.328 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.328 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.328 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.328 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.328 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.328 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.328 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.328 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: "max_completed_events": { 2024-04-20T21:44:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_completed_events", 2024-04-20T21:44:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "50", 2024-04-20T21:44:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.330 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "number of past completed events to remember", 2024-04-20T21:44:26.330 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.330 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.330 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.330 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.330 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-04-20T21:44:26.330 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-04-20T21:44:26.330 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:26.331 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.331 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.331 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-04-20T21:44:26.331 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.331 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.331 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.331 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long the module is going to sleep", 2024-04-20T21:44:26.331 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.332 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.332 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.332 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.332 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.332 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.332 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.332 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus", 2024-04-20T21:44:26.332 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.332 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.333 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.333 INFO:teuthology.orchestra.run.smithi022.stdout: "cache": { 2024-04-20T21:44:26.333 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cache", 2024-04-20T21:44:26.333 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.333 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.333 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.333 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.333 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.334 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.334 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.334 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.334 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.334 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.334 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.335 INFO:teuthology.orchestra.run.smithi022.stdout: "exclude_perf_counters": { 2024-04-20T21:44:26.335 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "exclude_perf_counters", 2024-04-20T21:44:26.335 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.335 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.335 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.335 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.335 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.336 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.336 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.336 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T21:44:26.336 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T21:44:26.336 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.336 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.336 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.337 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.337 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.337 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.337 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.337 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.337 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.337 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.338 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.338 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.338 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.338 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.338 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.338 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.338 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.339 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.339 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.339 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.339 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.339 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.339 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.340 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.340 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.340 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.340 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.340 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.340 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.340 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.340 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.341 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.341 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.341 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.341 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.341 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.341 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.341 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.341 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.341 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.342 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.342 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.342 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.342 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.342 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.342 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.342 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.342 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.343 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.343 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.343 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.343 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.343 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.343 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.343 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.343 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.343 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.345 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.345 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.345 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.345 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.346 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.346 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.346 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.346 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.346 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.346 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.346 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.346 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.347 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.347 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools": { 2024-04-20T21:44:26.347 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools", 2024-04-20T21:44:26.347 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.347 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.347 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.347 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.348 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.348 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.348 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.348 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.348 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.348 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.348 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.348 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.348 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T21:44:26.349 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T21:44:26.349 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.349 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.349 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.349 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300", 2024-04-20T21:44:26.349 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.349 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.349 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.350 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.350 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.350 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.350 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.350 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.350 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_interval": { 2024-04-20T21:44:26.350 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_interval", 2024-04-20T21:44:26.350 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:26.351 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.351 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.351 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15.0", 2024-04-20T21:44:26.351 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.351 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.351 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.351 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.351 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.351 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.352 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.352 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.352 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-04-20T21:44:26.352 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-04-20T21:44:26.352 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.352 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.352 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.352 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-04-20T21:44:26.353 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.353 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.353 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.353 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T21:44:26.353 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.353 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.353 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.353 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-04-20T21:44:26.354 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-04-20T21:44:26.354 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.354 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.354 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.354 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "9283", 2024-04-20T21:44:26.354 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.354 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.354 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.355 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T21:44:26.355 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.355 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.355 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.355 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.355 INFO:teuthology.orchestra.run.smithi022.stdout: "stale_cache_strategy": { 2024-04-20T21:44:26.355 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stale_cache_strategy", 2024-04-20T21:44:26.355 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.355 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.356 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.356 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "log", 2024-04-20T21:44:26.356 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.356 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.356 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.356 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.356 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.356 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.356 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.357 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.357 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-04-20T21:44:26.357 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-04-20T21:44:26.357 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.357 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.357 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.357 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "default", 2024-04-20T21:44:26.357 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.357 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.358 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.358 INFO:teuthology.orchestra.run.smithi022.stdout: "default", 2024-04-20T21:44:26.358 INFO:teuthology.orchestra.run.smithi022.stdout: "error" 2024-04-20T21:44:26.358 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.358 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.358 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.358 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.358 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.359 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.359 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-04-20T21:44:26.359 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-04-20T21:44:26.359 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.359 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.359 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.359 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-04-20T21:44:26.359 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-04-20T21:44:26.360 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-04-20T21:44:26.360 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.360 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.360 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.360 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.360 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.360 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.360 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.360 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.361 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.361 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2024-04-20T21:44:26.361 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.361 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.361 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.361 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.361 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.361 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.361 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.362 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.362 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.362 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.362 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.362 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.362 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.362 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.362 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.363 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.363 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.363 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.363 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.363 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.363 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.363 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.363 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.363 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.364 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.364 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.364 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.364 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.364 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.364 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.364 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.364 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.364 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.365 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.365 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.365 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.365 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.365 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.365 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.365 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.365 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.365 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.366 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.366 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.366 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.366 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.366 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.366 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.366 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.366 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.367 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.367 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.367 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.367 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.367 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.367 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.367 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.367 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.368 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.368 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.368 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.368 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.368 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.368 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.368 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.368 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.368 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.369 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.369 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.369 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.369 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.369 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.369 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.369 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_snap_create": { 2024-04-20T21:44:26.369 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_snap_create", 2024-04-20T21:44:26.369 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.370 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.370 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.370 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-04-20T21:44:26.370 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.370 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.370 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.370 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.370 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.371 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.371 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.371 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.371 INFO:teuthology.orchestra.run.smithi022.stdout: "mirror_snapshot_schedule": { 2024-04-20T21:44:26.371 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T21:44:26.371 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.371 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.371 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.372 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.372 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.372 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.372 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.372 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.372 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.372 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.372 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.373 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.373 INFO:teuthology.orchestra.run.smithi022.stdout: "trash_purge_schedule": { 2024-04-20T21:44:26.373 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "trash_purge_schedule", 2024-04-20T21:44:26.373 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.373 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.373 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.373 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.374 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.374 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.374 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.374 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.374 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.374 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.374 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.375 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.375 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.375 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.375 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.375 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "restful", 2024-04-20T21:44:26.375 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.375 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.376 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.376 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_auth": { 2024-04-20T21:44:26.376 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_auth", 2024-04-20T21:44:26.376 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.376 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.376 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.376 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.377 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.377 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.377 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.377 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.377 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.377 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.377 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.378 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.378 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-04-20T21:44:26.378 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-04-20T21:44:26.378 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.378 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.378 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.378 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.379 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.379 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.379 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.379 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.379 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.379 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.379 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.380 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.380 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.380 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.380 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.380 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.380 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.380 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.381 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.381 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.381 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.381 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.381 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.381 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.382 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.382 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.382 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.382 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.382 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.382 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.383 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.383 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.383 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.383 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.383 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.383 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.383 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.384 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.384 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.384 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.384 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.384 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.384 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.384 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.385 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.385 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.385 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.385 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.385 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.385 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.385 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.386 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.386 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.386 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.386 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.386 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.386 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.386 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.386 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.387 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.387 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.387 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.387 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.387 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.387 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.387 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.387 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.388 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.388 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.388 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.388 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.388 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.388 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.388 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.388 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.388 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.389 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.389 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.389 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.389 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.389 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.389 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.389 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-04-20T21:44:26.389 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-04-20T21:44:26.389 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.390 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.390 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.390 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.390 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.390 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.390 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.390 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.390 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.391 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.391 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.391 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.391 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-04-20T21:44:26.391 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-04-20T21:44:26.391 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.391 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.391 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.391 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.392 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.392 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.392 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.392 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.392 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.392 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.392 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.392 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.393 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.393 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.393 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.393 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rgw", 2024-04-20T21:44:26.393 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.393 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.393 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.393 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.394 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.394 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.394 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.394 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.394 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.394 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.394 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.394 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.394 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.395 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.395 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.395 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.395 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.395 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.395 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.395 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.395 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.396 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.396 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.396 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.396 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.396 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.396 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.396 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.396 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.396 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.397 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.397 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.397 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.397 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.397 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.397 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.397 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.397 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.398 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.398 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.398 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.398 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.398 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.398 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.398 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.398 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.399 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.399 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.399 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.399 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.399 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.399 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.399 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.399 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.399 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.400 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.400 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.400 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.400 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.400 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.400 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.400 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.400 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.401 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.401 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.401 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.401 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.401 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.401 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.401 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.401 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.402 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.402 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.402 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.402 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.402 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.402 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rook", 2024-04-20T21:44:26.402 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.402 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.403 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.403 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.403 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.403 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.403 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.403 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.403 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.403 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.403 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.404 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.404 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.404 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.404 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.404 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.404 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.404 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.404 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.405 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.405 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.405 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.405 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.405 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.405 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.405 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.405 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.406 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.406 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.406 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.406 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.406 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.406 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.406 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.406 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.407 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.407 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.407 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.407 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.407 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.407 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.407 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.407 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.408 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.408 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.408 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.408 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.408 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.408 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.408 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.408 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.409 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.409 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.409 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.409 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.409 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.409 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.409 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.409 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.409 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.410 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.410 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.410 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.410 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.410 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.410 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.410 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.410 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.411 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.411 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.411 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.411 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.411 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.411 INFO:teuthology.orchestra.run.smithi022.stdout: "storage_class": { 2024-04-20T21:44:26.411 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "storage_class", 2024-04-20T21:44:26.412 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.412 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.412 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.412 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "local", 2024-04-20T21:44:26.412 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.412 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.412 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.412 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T21:44:26.412 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.413 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.413 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.413 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.413 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.413 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.413 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.413 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "selftest", 2024-04-20T21:44:26.413 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.413 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.414 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.414 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.414 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.414 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.414 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.414 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.414 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.414 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.415 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.415 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.415 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.415 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.415 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.415 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.415 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.415 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.415 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.416 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.416 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.416 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.416 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.416 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.416 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.416 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.416 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.416 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.417 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.417 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.417 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.417 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.417 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.417 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.417 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.417 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.417 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.418 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.418 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.418 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.418 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.418 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.418 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.418 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.419 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.419 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.419 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.419 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.419 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.419 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.419 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.419 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.420 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.420 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.420 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.420 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.420 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.420 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.420 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.421 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.421 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.421 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.421 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.421 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.421 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.421 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.422 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.422 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.422 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.422 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.422 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.422 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.422 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.422 INFO:teuthology.orchestra.run.smithi022.stdout: "roption1": { 2024-04-20T21:44:26.423 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption1", 2024-04-20T21:44:26.423 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.423 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.423 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.423 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.423 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.423 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.423 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.424 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.424 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.424 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.424 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.424 INFO:teuthology.orchestra.run.smithi022.stdout: "roption2": { 2024-04-20T21:44:26.424 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption2", 2024-04-20T21:44:26.424 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.424 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.425 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.425 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "xyz", 2024-04-20T21:44:26.425 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.425 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.425 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.425 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.425 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.425 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.426 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.426 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.426 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption1": { 2024-04-20T21:44:26.426 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption1", 2024-04-20T21:44:26.426 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.426 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.426 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.427 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.427 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.427 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.427 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.427 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.427 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.427 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.428 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.428 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.428 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption2": { 2024-04-20T21:44:26.428 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption2", 2024-04-20T21:44:26.428 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.428 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.428 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.428 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.428 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.429 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.429 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.429 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.429 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.429 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.429 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.429 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption3": { 2024-04-20T21:44:26.430 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption3", 2024-04-20T21:44:26.430 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-04-20T21:44:26.430 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.430 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.430 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.430 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.430 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.430 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.431 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.431 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.431 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.431 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.431 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.431 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption4": { 2024-04-20T21:44:26.431 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption4", 2024-04-20T21:44:26.431 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.431 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.432 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.432 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.432 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.432 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.432 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.432 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.432 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.432 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.432 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.433 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption5": { 2024-04-20T21:44:26.433 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption5", 2024-04-20T21:44:26.433 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.433 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.433 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.433 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.433 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.433 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.434 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.434 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.434 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.434 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.434 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.434 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.434 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption6": { 2024-04-20T21:44:26.434 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption6", 2024-04-20T21:44:26.435 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.435 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.435 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.435 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.435 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.435 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.435 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.435 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.435 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.436 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.436 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.436 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.436 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption7": { 2024-04-20T21:44:26.436 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption7", 2024-04-20T21:44:26.436 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.436 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.436 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.436 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.437 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-04-20T21:44:26.437 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "42", 2024-04-20T21:44:26.437 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.437 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.437 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.437 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.437 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.437 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.438 INFO:teuthology.orchestra.run.smithi022.stdout: "testkey": { 2024-04-20T21:44:26.438 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testkey", 2024-04-20T21:44:26.438 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.438 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.438 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.438 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.438 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.438 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.439 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.439 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.439 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.439 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.439 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.439 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.439 INFO:teuthology.orchestra.run.smithi022.stdout: "testlkey": { 2024-04-20T21:44:26.439 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testlkey", 2024-04-20T21:44:26.440 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.440 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.440 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.440 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.440 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.440 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.440 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.440 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.440 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.441 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.441 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.441 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.441 INFO:teuthology.orchestra.run.smithi022.stdout: "testnewline": { 2024-04-20T21:44:26.441 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testnewline", 2024-04-20T21:44:26.441 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.441 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.441 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.441 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.442 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.442 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.442 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.442 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.442 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.442 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.442 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.442 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.443 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.443 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.443 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.443 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-04-20T21:44:26.443 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.443 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.443 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.443 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_m_granularity": { 2024-04-20T21:44:26.444 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_m_granularity", 2024-04-20T21:44:26.444 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.444 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.444 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.444 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.444 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.444 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.444 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.444 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T21:44:26.445 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.445 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.445 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.445 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.445 INFO:teuthology.orchestra.run.smithi022.stdout: "dump_on_update": { 2024-04-20T21:44:26.445 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dump_on_update", 2024-04-20T21:44:26.445 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.445 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.445 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.446 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.446 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.446 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.446 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.446 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "dump database to debug log on update", 2024-04-20T21:44:26.446 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.446 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.446 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.447 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.447 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.447 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.447 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.447 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.447 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.447 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.447 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.448 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.448 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.448 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.448 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.448 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.448 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.448 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.448 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.448 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.449 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.449 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.449 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.449 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.449 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.449 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.449 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.449 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.450 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.450 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.450 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.450 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.450 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.450 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.450 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.450 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.451 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.451 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.451 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.451 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.451 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.451 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.451 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.451 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.451 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.452 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.452 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.452 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.452 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.452 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.452 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.452 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.452 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.453 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.453 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.453 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.453 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.453 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.453 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.453 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.453 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.454 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.454 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.454 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.454 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.454 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.454 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.454 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.454 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.454 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.455 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.455 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.455 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.455 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.455 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.455 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.455 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stats", 2024-04-20T21:44:26.455 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.456 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.456 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.456 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.456 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.456 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.456 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.456 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.456 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.456 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.457 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.457 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.457 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.457 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.457 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.457 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.457 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.457 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.457 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.458 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.458 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.458 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.458 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.458 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.458 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.458 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.459 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.459 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.459 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.459 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.459 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.459 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.459 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.459 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.459 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.460 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.460 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.460 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.460 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.460 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.460 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.460 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.460 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.460 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.461 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.461 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.461 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.461 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.461 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.461 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.461 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.461 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.462 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.462 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.462 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.462 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.462 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.465 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.465 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.465 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.466 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.466 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.466 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.466 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.466 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.466 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.466 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.466 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.467 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.467 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.467 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.467 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.467 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.467 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.467 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.467 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "status", 2024-04-20T21:44:26.467 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.468 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.468 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.468 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.468 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.468 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.468 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.468 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.468 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.468 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.469 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.469 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.469 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.469 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.469 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.469 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.469 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.469 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.469 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.470 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.470 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.470 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.470 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.470 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.470 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.470 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.470 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.471 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.471 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.471 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.471 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.471 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.471 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.471 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.471 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.471 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.472 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.472 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.472 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.472 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.472 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.472 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.472 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.472 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.472 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.473 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.473 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.473 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.473 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.473 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.473 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.473 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.473 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.473 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.474 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.474 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.474 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.474 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.474 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.474 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.474 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.474 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.475 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.475 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.475 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.475 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.475 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.475 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.475 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.475 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.475 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.476 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.476 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.476 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.476 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.476 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.476 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telegraf", 2024-04-20T21:44:26.476 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.476 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.476 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.477 INFO:teuthology.orchestra.run.smithi022.stdout: "address": { 2024-04-20T21:44:26.477 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "address", 2024-04-20T21:44:26.477 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.477 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.477 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.477 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T21:44:26.477 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.478 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.478 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.478 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.478 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.478 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.478 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.478 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.478 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:26.479 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:26.479 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:26.479 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.479 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.479 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15", 2024-04-20T21:44:26.479 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.479 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.479 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.480 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.480 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.480 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.480 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.480 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.480 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.480 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.480 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.480 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.481 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.481 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.481 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.481 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.481 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.481 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.481 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.481 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.482 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.482 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.482 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.482 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.482 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.482 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.482 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.482 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.483 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.483 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.483 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.483 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.483 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.483 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.483 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.483 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.483 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.484 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.484 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.484 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.484 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.484 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.484 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.484 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.484 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.485 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.485 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.485 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.485 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.485 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.485 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.485 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.485 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.486 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.486 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.486 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.486 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.486 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.486 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.486 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.486 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.486 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.487 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.487 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.487 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.487 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.487 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.487 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.487 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.487 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telemetry", 2024-04-20T21:44:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_basic": { 2024-04-20T21:44:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_basic", 2024-04-20T21:44:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.491 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T21:44:26.491 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.491 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.491 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.491 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.491 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_crash": { 2024-04-20T21:44:26.491 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_crash", 2024-04-20T21:44:26.491 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.491 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.492 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.492 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.492 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.492 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.492 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.492 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T21:44:26.492 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.492 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.493 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.493 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.493 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_device": { 2024-04-20T21:44:26.493 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_device", 2024-04-20T21:44:26.493 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.493 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.493 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.493 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.494 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.494 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.494 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.494 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T21:44:26.494 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.494 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.494 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.494 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.494 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_ident": { 2024-04-20T21:44:26.495 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_ident", 2024-04-20T21:44:26.495 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.495 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.495 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.495 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.495 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.495 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.495 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.496 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T21:44:26.496 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.496 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.496 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.496 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.496 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_perf": { 2024-04-20T21:44:26.496 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_perf", 2024-04-20T21:44:26.496 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.497 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.497 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.497 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.497 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.497 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.497 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.497 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T21:44:26.497 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.498 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.498 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.498 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.498 INFO:teuthology.orchestra.run.smithi022.stdout: "contact": { 2024-04-20T21:44:26.498 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "contact", 2024-04-20T21:44:26.498 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.498 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.498 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.499 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.499 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.499 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.499 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.499 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.499 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.499 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.499 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.500 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.500 INFO:teuthology.orchestra.run.smithi022.stdout: "description": { 2024-04-20T21:44:26.500 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "description", 2024-04-20T21:44:26.500 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.500 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.500 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.500 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.501 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.501 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.501 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.501 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.501 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.501 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.501 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.501 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.501 INFO:teuthology.orchestra.run.smithi022.stdout: "device_url": { 2024-04-20T21:44:26.502 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_url", 2024-04-20T21:44:26.502 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.502 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.502 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.502 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T21:44:26.502 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.502 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.502 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.503 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.503 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.503 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.503 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.503 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.503 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-04-20T21:44:26.503 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-04-20T21:44:26.503 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.504 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.504 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.504 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.504 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.504 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.504 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.504 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.504 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.505 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.505 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.505 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.505 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:26.505 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:26.505 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.505 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.506 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.506 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "24", 2024-04-20T21:44:26.506 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "8", 2024-04-20T21:44:26.506 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.506 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.506 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.506 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: "last_opt_revision": { 2024-04-20T21:44:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "last_opt_revision", 2024-04-20T21:44:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1", 2024-04-20T21:44:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard": { 2024-04-20T21:44:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard", 2024-04-20T21:44:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard_description": { 2024-04-20T21:44:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard_description", 2024-04-20T21:44:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.519 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.519 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.519 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.519 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.519 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.519 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.519 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.519 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.520 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.520 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.520 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.520 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.520 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.520 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.520 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.520 INFO:teuthology.orchestra.run.smithi022.stdout: "organization": { 2024-04-20T21:44:26.521 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "organization", 2024-04-20T21:44:26.521 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.521 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.521 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.521 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.521 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.521 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.521 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.522 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.522 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.522 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.522 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.522 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.522 INFO:teuthology.orchestra.run.smithi022.stdout: "proxy": { 2024-04-20T21:44:26.522 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "proxy", 2024-04-20T21:44:26.523 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.523 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.523 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.523 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.523 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.523 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.524 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.524 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.524 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.524 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.524 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.524 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.524 INFO:teuthology.orchestra.run.smithi022.stdout: "url": { 2024-04-20T21:44:26.525 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url", 2024-04-20T21:44:26.525 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.525 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.525 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.525 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T21:44:26.525 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.525 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.526 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.526 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.526 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.526 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.526 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.526 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.526 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.527 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.527 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.527 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "test_orchestrator", 2024-04-20T21:44:26.527 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.527 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.527 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.527 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.528 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.528 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.528 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.528 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.528 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.528 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.529 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.529 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.529 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.529 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.529 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.529 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.529 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.530 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.530 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.530 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.530 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.530 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.530 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.530 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.531 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.531 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.531 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.531 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.531 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.531 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.531 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.532 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.532 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.532 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.532 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.532 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.532 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.533 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.533 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.533 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.533 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.533 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.533 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.533 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.533 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.534 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.534 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.534 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.534 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.534 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.534 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.534 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.534 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.535 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.535 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.535 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.535 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.535 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.535 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.535 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.536 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.536 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.536 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.536 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.536 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.536 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.536 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.537 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.537 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.537 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.537 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.537 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.538 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.538 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.538 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.538 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.538 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2024-04-20T21:44:26.538 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.538 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.539 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.539 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.539 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.539 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.539 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.539 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.539 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.539 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.540 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.540 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.540 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.540 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.540 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.540 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.540 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.540 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.541 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.541 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.541 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.541 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.541 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.541 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.541 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.541 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.542 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.542 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.542 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.542 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.542 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.542 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.543 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.543 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.543 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.543 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.543 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.543 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.543 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.543 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.544 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.544 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.544 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.544 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.544 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.544 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.544 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.544 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.545 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.545 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.545 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.545 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.545 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.545 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.545 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.545 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.546 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.546 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.546 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.546 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.546 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.546 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.546 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.546 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.547 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.547 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.547 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.547 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.547 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.547 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.547 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.547 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.548 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.548 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_clones": { 2024-04-20T21:44:26.548 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_clones", 2024-04-20T21:44:26.548 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.548 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.548 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.548 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4", 2024-04-20T21:44:26.548 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.549 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.549 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.549 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T21:44:26.549 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.549 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.549 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.549 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.549 INFO:teuthology.orchestra.run.smithi022.stdout: "periodic_async_work": { 2024-04-20T21:44:26.549 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "periodic_async_work", 2024-04-20T21:44:26.550 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.550 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.550 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.550 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.550 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.550 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.550 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.550 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Periodically check for async work", 2024-04-20T21:44:26.551 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.551 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.551 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.551 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.551 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_delay": { 2024-04-20T21:44:26.551 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_delay", 2024-04-20T21:44:26.551 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.551 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.552 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.552 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-04-20T21:44:26.552 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.552 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.552 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.552 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T21:44:26.552 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.552 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.553 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.553 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.553 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_no_wait": { 2024-04-20T21:44:26.553 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T21:44:26.553 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.553 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.553 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.553 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-04-20T21:44:26.553 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.554 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.554 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.554 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T21:44:26.554 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.554 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.554 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.554 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.555 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.555 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.555 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.555 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix", 2024-04-20T21:44:26.555 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-04-20T21:44:26.555 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-04-20T21:44:26.555 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-04-20T21:44:26.555 INFO:teuthology.orchestra.run.smithi022.stdout: "discovery_interval": { 2024-04-20T21:44:26.556 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "discovery_interval", 2024-04-20T21:44:26.556 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-04-20T21:44:26.556 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.556 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.556 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "100", 2024-04-20T21:44:26.556 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.556 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.556 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.557 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.557 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.557 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.557 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.557 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.557 INFO:teuthology.orchestra.run.smithi022.stdout: "identifier": { 2024-04-20T21:44:26.558 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "identifier", 2024-04-20T21:44:26.558 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.558 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.558 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.558 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.558 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.558 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.559 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.559 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.559 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.559 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.559 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.559 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.560 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-04-20T21:44:26.560 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-04-20T21:44:26.560 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-04-20T21:44:26.560 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.560 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.560 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-04-20T21:44:26.561 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.561 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.561 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.561 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.561 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.561 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.561 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.562 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.562 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-04-20T21:44:26.562 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-04-20T21:44:26.562 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.562 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.562 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.562 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.563 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.563 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.563 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.563 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.563 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.563 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.563 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.564 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.564 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.564 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.564 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.564 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.564 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.564 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.565 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.565 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-04-20T21:44:26.565 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-04-20T21:44:26.565 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.565 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.565 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.565 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.566 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.566 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.566 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.566 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.566 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.566 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.567 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.567 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.567 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-04-20T21:44:26.567 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-04-20T21:44:26.567 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.567 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.567 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.568 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-04-20T21:44:26.568 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.568 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.568 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-04-20T21:44:26.568 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-04-20T21:44:26.568 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-04-20T21:44:26.568 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-04-20T21:44:26.568 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-04-20T21:44:26.568 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-04-20T21:44:26.569 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-04-20T21:44:26.569 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.569 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.569 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.569 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.569 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.569 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.569 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-04-20T21:44:26.570 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-04-20T21:44:26.570 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-04-20T21:44:26.570 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.570 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-04-20T21:44:26.570 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-04-20T21:44:26.570 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.570 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.570 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.571 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.571 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.571 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.571 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.571 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_host": { 2024-04-20T21:44:26.571 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_host", 2024-04-20T21:44:26.571 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.571 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.572 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.572 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-04-20T21:44:26.572 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.572 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.572 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.572 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.572 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.572 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.573 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.573 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.573 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_port": { 2024-04-20T21:44:26.573 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_port", 2024-04-20T21:44:26.573 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-04-20T21:44:26.573 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.573 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.573 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10051", 2024-04-20T21:44:26.574 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.574 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.574 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.574 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.574 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.574 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.574 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.575 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.575 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_sender": { 2024-04-20T21:44:26.575 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_sender", 2024-04-20T21:44:26.575 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-04-20T21:44:26.575 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-04-20T21:44:26.575 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-04-20T21:44:26.575 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T21:44:26.576 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-04-20T21:44:26.576 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-04-20T21:44:26.576 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-04-20T21:44:26.576 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-04-20T21:44:26.576 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-04-20T21:44:26.576 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-04-20T21:44:26.576 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-04-20T21:44:26.577 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.577 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.577 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.577 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.577 INFO:teuthology.orchestra.run.smithi022.stdout: "services": { 2024-04-20T21:44:26.577 INFO:teuthology.orchestra.run.smithi022.stdout: "dashboard": "https://172.21.15.22:8443/" 2024-04-20T21:44:26.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.578 INFO:teuthology.orchestra.run.smithi022.stdout: "always_on_modules": { 2024-04-20T21:44:26.578 INFO:teuthology.orchestra.run.smithi022.stdout: "octopus": [ 2024-04-20T21:44:26.578 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-04-20T21:44:26.578 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-04-20T21:44:26.578 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-04-20T21:44:26.578 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-04-20T21:44:26.578 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-04-20T21:44:26.578 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-04-20T21:44:26.578 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-04-20T21:44:26.579 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-04-20T21:44:26.579 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-04-20T21:44:26.579 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-04-20T21:44:26.579 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.579 INFO:teuthology.orchestra.run.smithi022.stdout: "pacific": [ 2024-04-20T21:44:26.579 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-04-20T21:44:26.579 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-04-20T21:44:26.580 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-04-20T21:44:26.580 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-04-20T21:44:26.580 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-04-20T21:44:26.580 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-04-20T21:44:26.580 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-04-20T21:44:26.580 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-04-20T21:44:26.580 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-04-20T21:44:26.580 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-04-20T21:44:26.581 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.581 INFO:teuthology.orchestra.run.smithi022.stdout: "quincy": [ 2024-04-20T21:44:26.581 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-04-20T21:44:26.581 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-04-20T21:44:26.581 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-04-20T21:44:26.581 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-04-20T21:44:26.581 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-04-20T21:44:26.581 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-04-20T21:44:26.582 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-04-20T21:44:26.582 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-04-20T21:44:26.582 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-04-20T21:44:26.582 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-04-20T21:44:26.582 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.582 INFO:teuthology.orchestra.run.smithi022.stdout: "reef": [ 2024-04-20T21:44:26.582 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-04-20T21:44:26.582 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-04-20T21:44:26.582 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-04-20T21:44:26.583 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-04-20T21:44:26.583 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-04-20T21:44:26.583 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-04-20T21:44:26.583 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-04-20T21:44:26.583 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-04-20T21:44:26.583 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-04-20T21:44:26.583 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-04-20T21:44:26.584 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-04-20T21:44:26.584 INFO:teuthology.orchestra.run.smithi022.stdout: "squid": [ 2024-04-20T21:44:26.584 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-04-20T21:44:26.584 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-04-20T21:44:26.584 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-04-20T21:44:26.584 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-04-20T21:44:26.584 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-04-20T21:44:26.584 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-04-20T21:44:26.585 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-04-20T21:44:26.585 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-04-20T21:44:26.585 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-04-20T21:44:26.585 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-04-20T21:44:26.585 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-20T21:44:26.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.585 INFO:teuthology.orchestra.run.smithi022.stdout: "last_failure_osd_epoch": 93, 2024-04-20T21:44:26.585 INFO:teuthology.orchestra.run.smithi022.stdout: "active_clients": [ 2024-04-20T21:44:26.585 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.586 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2024-04-20T21:44:26.586 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-04-20T21:44:26.586 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.586 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-04-20T21:44:26.586 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-04-20T21:44:26.586 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3472676332 2024-04-20T21:44:26.586 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.587 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-20T21:44:26.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.587 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.587 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "libcephsqlite", 2024-04-20T21:44:26.587 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-04-20T21:44:26.587 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.587 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-04-20T21:44:26.587 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-04-20T21:44:26.587 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1087151549 2024-04-20T21:44:26.588 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.588 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-20T21:44:26.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.588 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2024-04-20T21:44:26.588 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-04-20T21:44:26.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.588 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-04-20T21:44:26.589 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-04-20T21:44:26.589 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3465459579 2024-04-20T21:44:26.589 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.589 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-20T21:44:26.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.589 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.589 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-04-20T21:44:26.589 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-04-20T21:44:26.590 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.590 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-04-20T21:44:26.590 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-04-20T21:44:26.590 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2579084752 2024-04-20T21:44:26.590 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.590 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-20T21:44:26.590 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.590 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.591 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-04-20T21:44:26.591 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-04-20T21:44:26.591 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.591 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-04-20T21:44:26.591 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-04-20T21:44:26.591 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3904188311 2024-04-20T21:44:26.591 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.591 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-20T21:44:26.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:26.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.592 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2024-04-20T21:44:26.592 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-04-20T21:44:26.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-04-20T21:44:26.592 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-04-20T21:44:26.592 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-04-20T21:44:26.592 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4021571798 2024-04-20T21:44:26.592 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.593 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-20T21:44:26.593 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:26.593 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-04-20T21:44:26.593 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:44:26.594 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:24 smithi033 ceph-mon[21626]: pgmap v330: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:44:26.594 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:24 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2445671870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:44:26.594 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:24 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1666299652' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-20T21:44:26.595 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:24 smithi122 ceph-mon[22147]: pgmap v330: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:44:26.595 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:24 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2445671870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:44:26.595 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:24 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1666299652' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-20T21:44:26.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.392+0000 7f5bbf400640 1 -- 172.21.15.22:0/1666299652 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5ba00a3d20 msgr2=0x7f5ba00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:26.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.392+0000 7f5bbf400640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5ba00a3d20 0x7f5ba00a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5bb819b9a0 tx=0x7f5bac00b410 comp rx=0 tx=0).stop 2024-04-20T21:44:26.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.392+0000 7f5bbf400640 1 -- 172.21.15.22:0/1666299652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bb8107270 msgr2=0x7f5bb81a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:26.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.392+0000 7f5bbf400640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bb8107270 0x7f5bb81a0bf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5ba402c240 tx=0x7f5ba4004360 comp rx=0 tx=0).stop 2024-04-20T21:44:26.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.392+0000 7f5bbf400640 1 -- 172.21.15.22:0/1666299652 shutdown_connections 2024-04-20T21:44:26.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.392+0000 7f5bbf400640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f5ba00a3d20 0x7f5ba00a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:26.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.392+0000 7f5bbf400640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bb819a860 0x7f5bb819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:26.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.392+0000 7f5bbf400640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bb8107270 0x7f5bb81a0bf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:26.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.392+0000 7f5bbf400640 1 --2- 172.21.15.22:0/1666299652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb81021a0 0x7f5bb81a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:26.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.392+0000 7f5bbf400640 1 -- 172.21.15.22:0/1666299652 >> 172.21.15.22:0/1666299652 conn(0x7f5bb80793f0 msgr2=0x7f5bb81000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:26.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.392+0000 7f5bbf400640 1 -- 172.21.15.22:0/1666299652 shutdown_connections 2024-04-20T21:44:26.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:44:24.392+0000 7f5bbf400640 1 -- 172.21.15.22:0/1666299652 wait complete. 2024-04-20T21:44:26.606 DEBUG:tasks.check_counter:Getting stats from a 2024-04-20T21:44:26.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-20T21:44:26.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:24 smithi022 ceph-mon[16843]: pgmap v330: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-20T21:44:26.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:24 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2445671870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:44:26.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:24 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1666299652' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-20T21:44:26.754 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:26 smithi122 ceph-mon[22147]: pgmap v331: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 94 B/s wr, 2 op/s 2024-04-20T21:44:26.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:26 smithi033 ceph-mon[21626]: pgmap v331: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 94 B/s wr, 2 op/s 2024-04-20T21:44:26.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:26 smithi022 ceph-mon[16843]: pgmap v331: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 94 B/s wr, 2 op/s 2024-04-20T21:44:27.066 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:44:27.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:27 smithi122 ceph-mon[22147]: pgmap v332: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:27.803 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:27 smithi033 ceph-mon[21626]: pgmap v332: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:27.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:27 smithi022 ceph-mon[16843]: pgmap v332: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:29.448 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:44:29.449 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:29.449 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 228, 2024-04-20T21:44:29.449 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 214, 2024-04-20T21:44:29.449 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1731760, 2024-04-20T21:44:29.449 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 130912, 2024-04-20T21:44:29.449 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:29.449 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:29.449 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.104234689, 2024-04-20T21:44:29.450 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.034236238, 2024-04-20T21:44:29.450 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.078285467, 2024-04-20T21:44:29.450 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.010304409, 2024-04-20T21:44:29.450 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:29.450 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 214, 2024-04-20T21:44:29.450 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.021592499, 2024-04-20T21:44:29.450 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000100899 2024-04-20T21:44:29.450 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:29.450 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:29.451 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:29.451 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:29.451 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:29.451 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:29.451 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1731760, 2024-04-20T21:44:29.451 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 130912 2024-04-20T21:44:29.451 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:29.451 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:29.452 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 2, 2024-04-20T21:44:29.452 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 161, 2024-04-20T21:44:29.452 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 320, 2024-04-20T21:44:29.452 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 184448, 2024-04-20T21:44:29.452 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 5, 2024-04-20T21:44:29.452 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:29.452 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.034575256, 2024-04-20T21:44:29.452 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.024521889, 2024-04-20T21:44:29.452 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.004568747, 2024-04-20T21:44:29.453 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:29.453 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:29.453 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 158, 2024-04-20T21:44:29.453 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.027847582, 2024-04-20T21:44:29.453 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000176250 2024-04-20T21:44:29.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:29.453 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:29.453 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:29.454 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:29.454 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:29.454 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:29.454 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T21:44:29.454 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 184448 2024-04-20T21:44:29.454 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:29.454 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:29.454 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 0, 2024-04-20T21:44:29.454 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2, 2024-04-20T21:44:29.455 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:44:29.455 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 0, 2024-04-20T21:44:29.455 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:29.455 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:29.455 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.000967776, 2024-04-20T21:44:29.455 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:44:29.455 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000775276, 2024-04-20T21:44:29.455 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:29.455 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:29.456 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:29.456 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:29.456 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:29.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:29.456 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:29.456 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:29.456 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:29.456 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:29.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:29.457 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:29.457 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:29.457 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:29.457 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-04-20T21:44:29.457 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-04-20T21:44:29.457 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-04-20T21:44:29.457 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:29.457 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-04-20T21:44:29.457 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:29.458 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:29.458 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:29.458 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:29.458 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:29.458 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:29.458 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:29.458 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:29.459 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:29.459 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:29.459 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:29.459 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:29.459 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:29.459 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:29.459 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:29.459 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:29.459 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:29.460 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:29.460 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:29.460 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:29.460 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:29.460 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:29.460 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:29.460 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:29.460 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:29.461 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:29.461 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:29.461 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:29.461 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:29.461 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:29.461 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:29.461 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-04-20T21:44:29.461 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:29.461 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:29.462 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:29.462 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:29.462 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 150008, 2024-04-20T21:44:29.462 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 31, 2024-04-20T21:44:29.462 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:29.462 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:29.462 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-04-20T21:44:29.462 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-04-20T21:44:29.462 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:29.463 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:29.463 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:29.463 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:29.463 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:29.463 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-04-20T21:44:29.463 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:29.463 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:29.463 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:29.463 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-04-20T21:44:29.464 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:29.464 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-04-20T21:44:29.464 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:29.464 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-04-20T21:44:29.464 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:29.464 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-04-20T21:44:29.464 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:29.464 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:29.464 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-20T21:44:29.465 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-04-20T21:44:29.465 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-20T21:44:29.465 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 226, 2024-04-20T21:44:29.465 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1697194, 2024-04-20T21:44:29.465 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:29.465 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 226, 2024-04-20T21:44:29.465 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-04-20T21:44:29.465 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-04-20T21:44:29.465 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 226, 2024-04-20T21:44:29.466 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1697194, 2024-04-20T21:44:29.466 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-20T21:44:29.466 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:29.466 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:29.466 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:29.466 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:29.466 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:29.466 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:44:30.340 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-20T21:44:30.341 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-20T21:44:30.341 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-20T21:44:30.341 DEBUG:tasks.check_counter:Getting stats from d 2024-04-20T21:44:30.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-20T21:44:30.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:30 smithi122 ceph-mon[22147]: pgmap v333: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:30.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:30 smithi033 ceph-mon[21626]: pgmap v333: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:30.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:30 smithi022 ceph-mon[16843]: pgmap v333: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:30.772 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:44:32.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:32 smithi022 ceph-mon[16843]: pgmap v334: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:32.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:32 smithi122 ceph-mon[22147]: pgmap v334: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:32.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:32 smithi033 ceph-mon[21626]: pgmap v334: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:32.990 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:44:32.990 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:32.991 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 4, 2024-04-20T21:44:32.991 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 4, 2024-04-20T21:44:32.991 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 2992, 2024-04-20T21:44:32.991 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 448, 2024-04-20T21:44:32.991 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:32.991 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:32.991 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.004844775, 2024-04-20T21:44:32.991 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000103939, 2024-04-20T21:44:32.992 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.001546780, 2024-04-20T21:44:32.992 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:32.992 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:32.992 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 3, 2024-04-20T21:44:32.992 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.001748098, 2024-04-20T21:44:32.992 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000582699 2024-04-20T21:44:32.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:32.992 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:32.992 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:32.993 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:32.993 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:32.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:32.993 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-20T21:44:32.993 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 448 2024-04-20T21:44:32.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:32.993 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:32.993 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 231, 2024-04-20T21:44:32.993 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 219, 2024-04-20T21:44:32.994 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1730672, 2024-04-20T21:44:32.994 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 134944, 2024-04-20T21:44:32.994 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:32.994 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:32.994 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.104258837, 2024-04-20T21:44:32.994 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.036731628, 2024-04-20T21:44:32.994 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.076532553, 2024-04-20T21:44:32.994 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.010805941, 2024-04-20T21:44:32.995 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:32.995 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 218, 2024-04-20T21:44:32.995 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.021109422, 2024-04-20T21:44:32.995 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000096832 2024-04-20T21:44:32.995 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:32.995 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:32.995 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:32.995 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:32.995 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:32.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:32.996 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1730672, 2024-04-20T21:44:32.996 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 134944 2024-04-20T21:44:32.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:32.996 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:32.996 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 2, 2024-04-20T21:44:32.996 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 167, 2024-04-20T21:44:32.996 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 320, 2024-04-20T21:44:32.996 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 185504, 2024-04-20T21:44:32.997 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 5, 2024-04-20T21:44:32.997 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:32.997 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.035862654, 2024-04-20T21:44:32.997 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.027122611, 2024-04-20T21:44:32.997 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.003287667, 2024-04-20T21:44:32.997 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:32.997 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:32.997 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 164, 2024-04-20T21:44:32.997 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.028024511, 2024-04-20T21:44:32.998 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000170881 2024-04-20T21:44:32.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:32.998 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:32.998 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:32.998 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:32.998 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:32.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:32.998 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T21:44:32.998 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 185504 2024-04-20T21:44:32.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:32.999 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-04-20T21:44:32.999 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-04-20T21:44:32.999 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-04-20T21:44:32.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:32.999 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-04-20T21:44:32.999 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:32.999 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:32.999 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:33.000 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:33.000 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:33.000 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:33.000 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:33.000 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:33.000 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:33.000 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:33.000 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:33.000 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:33.001 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:33.001 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:33.001 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:33.001 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:33.001 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:33.001 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:33.001 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:33.001 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:33.001 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:33.001 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:33.002 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:33.002 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:33.002 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:33.002 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:33.002 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:33.002 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:33.002 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:33.002 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:33.003 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:33.003 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-04-20T21:44:33.003 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:33.003 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:33.003 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:33.003 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:33.003 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 150008, 2024-04-20T21:44:33.003 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 31, 2024-04-20T21:44:33.003 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:33.004 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:33.004 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-04-20T21:44:33.004 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-04-20T21:44:33.004 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:33.004 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:33.004 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:33.004 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:33.004 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:33.004 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-04-20T21:44:33.005 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:33.005 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:33.005 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:33.005 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-04-20T21:44:33.005 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:33.005 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-04-20T21:44:33.005 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:33.005 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-04-20T21:44:33.005 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:33.006 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-04-20T21:44:33.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:33.006 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:33.006 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-20T21:44:33.006 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-04-20T21:44:33.006 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-20T21:44:33.006 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 233, 2024-04-20T21:44:33.006 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1698104, 2024-04-20T21:44:33.006 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:33.007 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 233, 2024-04-20T21:44:33.007 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-04-20T21:44:33.007 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-04-20T21:44:33.007 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 233, 2024-04-20T21:44:33.007 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1698104, 2024-04-20T21:44:33.007 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-20T21:44:33.007 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:33.007 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:33.008 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:33.008 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:33.008 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:33.008 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:44:33.789 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-20T21:44:33.790 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-20T21:44:33.790 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-20T21:44:33.790 DEBUG:tasks.check_counter:Getting stats from g 2024-04-20T21:44:33.790 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-20T21:44:34.254 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:44:34.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:34 smithi022 ceph-mon[16843]: pgmap v335: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:34.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:34 smithi122 ceph-mon[22147]: pgmap v335: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:34.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:34 smithi033 ceph-mon[21626]: pgmap v335: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:36.385 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:44:36.385 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:36.385 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 0, 2024-04-20T21:44:36.385 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2, 2024-04-20T21:44:36.385 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:44:36.385 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 0, 2024-04-20T21:44:36.386 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:36.386 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:36.386 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.004225192, 2024-04-20T21:44:36.386 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:44:36.386 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000866867, 2024-04-20T21:44:36.386 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:36.386 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:36.386 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:36.387 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:36.387 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:36.387 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:36.387 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:36.387 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:36.387 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:36.387 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:36.387 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:36.387 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:36.388 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:36.388 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:36.388 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:36.388 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 227, 2024-04-20T21:44:36.388 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 215, 2024-04-20T21:44:36.388 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1729584, 2024-04-20T21:44:36.388 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 132384, 2024-04-20T21:44:36.388 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:36.388 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:36.389 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.098837146, 2024-04-20T21:44:36.389 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.034068159, 2024-04-20T21:44:36.389 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.073547128, 2024-04-20T21:44:36.389 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.010696770, 2024-04-20T21:44:36.389 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:36.389 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 214, 2024-04-20T21:44:36.389 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.020652857, 2024-04-20T21:44:36.389 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000096508 2024-04-20T21:44:36.390 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:36.390 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:36.390 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:36.390 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:36.390 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:36.390 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:36.390 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1729584, 2024-04-20T21:44:36.390 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 132384 2024-04-20T21:44:36.390 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:36.391 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:36.391 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 6, 2024-04-20T21:44:36.391 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 166, 2024-04-20T21:44:36.391 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 3312, 2024-04-20T21:44:36.391 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 185424, 2024-04-20T21:44:36.391 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 5, 2024-04-20T21:44:36.391 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:36.391 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.035087203, 2024-04-20T21:44:36.391 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.025831527, 2024-04-20T21:44:36.392 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.003980915, 2024-04-20T21:44:36.392 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:36.392 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:36.392 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 164, 2024-04-20T21:44:36.392 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.038222865, 2024-04-20T21:44:36.392 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000233066 2024-04-20T21:44:36.392 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:36.392 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:36.392 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:36.393 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:36.393 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:36.393 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:36.393 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-20T21:44:36.393 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 185424 2024-04-20T21:44:36.393 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:36.393 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-04-20T21:44:36.393 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-04-20T21:44:36.393 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-04-20T21:44:36.394 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:36.394 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-04-20T21:44:36.394 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:36.394 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:36.394 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:36.394 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:36.394 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:36.394 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:36.395 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:36.395 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:36.395 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:36.395 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:36.395 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:36.395 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:36.395 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:36.395 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:36.395 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:36.396 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:36.396 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:36.396 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:36.396 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:36.396 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:36.396 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:36.396 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:36.396 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:36.396 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:36.397 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:36.397 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:36.397 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:36.397 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:36.397 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:36.397 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:36.397 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:36.397 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-04-20T21:44:36.397 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:36.398 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:36.398 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:36.398 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:36.398 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 150008, 2024-04-20T21:44:36.398 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 31, 2024-04-20T21:44:36.398 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:36.398 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:36.398 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-04-20T21:44:36.399 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-04-20T21:44:36.399 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:36.399 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:36.399 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:36.399 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:36.399 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:36.399 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-04-20T21:44:36.399 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:36.399 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:36.400 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:36.400 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-04-20T21:44:36.400 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:36.400 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-04-20T21:44:36.400 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:36.400 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-04-20T21:44:36.400 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:36.400 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-04-20T21:44:36.400 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:36.401 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:36.401 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-20T21:44:36.401 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-04-20T21:44:36.401 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-20T21:44:36.401 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 229, 2024-04-20T21:44:36.401 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1697584, 2024-04-20T21:44:36.401 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:36.401 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 229, 2024-04-20T21:44:36.401 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-04-20T21:44:36.402 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-04-20T21:44:36.402 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 229, 2024-04-20T21:44:36.402 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1697584, 2024-04-20T21:44:36.402 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-20T21:44:36.402 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:36.402 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:36.402 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:36.402 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:36.402 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:36.403 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:44:36.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:36 smithi122 ceph-mon[22147]: pgmap v336: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:36.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:36 smithi033 ceph-mon[21626]: pgmap v336: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:36.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:36 smithi022 ceph-mon[16843]: pgmap v336: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:37.162 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-20T21:44:37.163 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-20T21:44:37.163 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-20T21:44:37.163 DEBUG:tasks.check_counter:Getting stats from j 2024-04-20T21:44:37.163 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-20T21:44:37.650 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:44:38.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:38 smithi022 ceph-mon[16843]: pgmap v337: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:38.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:38 smithi122 ceph-mon[22147]: pgmap v337: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:38.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:38 smithi033 ceph-mon[21626]: pgmap v337: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:39.770 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-04-20T21:44:39.771 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:39.771 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 238, 2024-04-20T21:44:39.771 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 224, 2024-04-20T21:44:39.771 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1734480, 2024-04-20T21:44:39.771 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 137312, 2024-04-20T21:44:39.771 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:39.771 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:39.771 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.123215152, 2024-04-20T21:44:39.772 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.036114190, 2024-04-20T21:44:39.772 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.102274019, 2024-04-20T21:44:39.772 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.010810681, 2024-04-20T21:44:39.772 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:39.772 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 224, 2024-04-20T21:44:39.772 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.025419851, 2024-04-20T21:44:39.772 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000113481 2024-04-20T21:44:39.772 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:39.772 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:39.773 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:39.773 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:39.773 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:39.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:39.773 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1734480, 2024-04-20T21:44:39.773 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 137312 2024-04-20T21:44:39.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:39.773 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:39.774 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 2, 2024-04-20T21:44:39.774 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 169, 2024-04-20T21:44:39.774 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 320, 2024-04-20T21:44:39.774 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 185856, 2024-04-20T21:44:39.774 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 5, 2024-04-20T21:44:39.774 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:39.774 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.035923083, 2024-04-20T21:44:39.774 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.025707066, 2024-04-20T21:44:39.775 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.004457842, 2024-04-20T21:44:39.775 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:39.775 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:39.775 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 166, 2024-04-20T21:44:39.775 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.032071629, 2024-04-20T21:44:39.775 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000193202 2024-04-20T21:44:39.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:39.775 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:39.775 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:39.775 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:39.776 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:39.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:39.776 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T21:44:39.776 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 185856 2024-04-20T21:44:39.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:39.776 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:39.776 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 0, 2024-04-20T21:44:39.776 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2, 2024-04-20T21:44:39.776 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:44:39.777 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 0, 2024-04-20T21:44:39.777 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:39.777 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:39.777 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.001006041, 2024-04-20T21:44:39.777 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:44:39.777 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000761931, 2024-04-20T21:44:39.777 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:39.777 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:39.777 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:39.778 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:39.778 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:39.778 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:39.778 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:39.778 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:39.778 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:39.778 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:39.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:39.779 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:39.779 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:39.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:39.779 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-04-20T21:44:39.779 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-04-20T21:44:39.779 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-04-20T21:44:39.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:39.780 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-04-20T21:44:39.780 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:39.780 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:39.780 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:39.780 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:39.780 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:39.780 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:39.780 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:39.780 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:39.781 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:39.781 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:39.781 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:39.781 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:39.781 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:39.781 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:39.781 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:39.781 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:39.781 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:39.782 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:39.782 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:39.782 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:39.782 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:39.782 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:39.782 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:39.782 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:39.783 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:39.783 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:39.783 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:39.783 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:39.783 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:39.783 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:39.783 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:39.783 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-04-20T21:44:39.783 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:39.784 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:39.784 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:39.784 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:39.784 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 151039, 2024-04-20T21:44:39.784 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 32, 2024-04-20T21:44:39.784 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:39.784 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:39.784 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-04-20T21:44:39.784 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-04-20T21:44:39.784 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:39.785 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:39.785 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:39.785 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:39.785 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:39.785 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-04-20T21:44:39.785 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:39.785 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:39.785 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:39.785 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-04-20T21:44:39.786 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:39.786 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-04-20T21:44:39.786 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:39.786 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-04-20T21:44:39.786 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:39.786 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-04-20T21:44:39.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-04-20T21:44:39.786 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:39.786 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-04-20T21:44:39.787 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-04-20T21:44:39.787 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-04-20T21:44:39.787 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 236, 2024-04-20T21:44:39.787 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1698494, 2024-04-20T21:44:39.787 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:39.787 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 236, 2024-04-20T21:44:39.787 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-04-20T21:44:39.787 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-04-20T21:44:39.787 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 236, 2024-04-20T21:44:39.787 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1698494, 2024-04-20T21:44:39.788 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-04-20T21:44:39.788 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-04-20T21:44:39.788 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-04-20T21:44:39.788 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-04-20T21:44:39.788 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:39.788 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-04-20T21:44:39.788 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-04-20T21:44:40.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:40 smithi022 ceph-mon[16843]: pgmap v338: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:40.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:40 smithi122 ceph-mon[22147]: pgmap v338: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:40.538 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-20T21:44:40.538 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-20T21:44:40.538 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-20T21:44:40.538 DEBUG:tasks.check_counter:Getting stats from b 2024-04-20T21:44:40.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-20T21:44:40.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:40 smithi033 ceph-mon[21626]: pgmap v338: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:40.801 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:44:42.524 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:42 smithi033 ceph-mon[21626]: pgmap v339: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:42.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:42 smithi122 ceph-mon[22147]: pgmap v339: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:42.525 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-20T21:44:42.525 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:42.525 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 15857, 2024-04-20T21:44:42.525 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 15235, 2024-04-20T21:44:42.525 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 5794208, 2024-04-20T21:44:42.525 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 83202589, 2024-04-20T21:44:42.526 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 13, 2024-04-20T21:44:42.526 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 5, 2024-04-20T21:44:42.526 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 1.370105700, 2024-04-20T21:44:42.526 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.480542149, 2024-04-20T21:44:42.526 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.350397377, 2024-04-20T21:44:42.526 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.442369743, 2024-04-20T21:44:42.526 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:42.526 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 15234, 2024-04-20T21:44:42.526 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 1.022590683, 2024-04-20T21:44:42.527 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000067125 2024-04-20T21:44:42.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.527 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:42.527 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 923, 2024-04-20T21:44:42.527 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000172835, 2024-04-20T21:44:42.527 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000187 2024-04-20T21:44:42.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.527 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 2108384, 2024-04-20T21:44:42.527 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 373760 2024-04-20T21:44:42.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.528 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:42.528 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 20523, 2024-04-20T21:44:42.528 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 20385, 2024-04-20T21:44:42.528 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 6872569, 2024-04-20T21:44:42.528 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 135229973, 2024-04-20T21:44:42.528 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 171, 2024-04-20T21:44:42.528 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 8, 2024-04-20T21:44:42.529 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 1.755852704, 2024-04-20T21:44:42.529 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.669765436, 2024-04-20T21:44:42.529 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.403661948, 2024-04-20T21:44:42.529 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.549666101, 2024-04-20T21:44:42.529 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:42.529 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 20381, 2024-04-20T21:44:42.529 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 1.212646813, 2024-04-20T21:44:42.529 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000059498 2024-04-20T21:44:42.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.530 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:42.530 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 371, 2024-04-20T21:44:42.530 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000051808, 2024-04-20T21:44:42.530 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000139 2024-04-20T21:44:42.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.530 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 86608, 2024-04-20T21:44:42.530 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 220464 2024-04-20T21:44:42.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.531 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:42.531 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 120479, 2024-04-20T21:44:42.531 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 211139, 2024-04-20T21:44:42.531 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 38826197, 2024-04-20T21:44:42.531 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 265679195, 2024-04-20T21:44:42.531 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 13, 2024-04-20T21:44:42.531 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 6, 2024-04-20T21:44:42.532 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 8.075517915, 2024-04-20T21:44:42.532 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 3.476612376, 2024-04-20T21:44:42.532 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 5.437026278, 2024-04-20T21:44:42.532 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.391600830, 2024-04-20T21:44:42.532 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:42.532 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 211138, 2024-04-20T21:44:42.532 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 4.522947474, 2024-04-20T21:44:42.532 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000021421 2024-04-20T21:44:42.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.533 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:42.533 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 227781, 2024-04-20T21:44:42.533 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.042641322, 2024-04-20T21:44:42.533 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000187 2024-04-20T21:44:42.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.533 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 60272, 2024-04-20T21:44:42.533 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 356464 2024-04-20T21:44:42.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.533 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-20T21:44:42.534 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 1, 2024-04-20T21:44:42.534 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-20T21:44:42.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.534 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-MDSRank": { 2024-04-20T21:44:42.534 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-04-20T21:44:42.534 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-04-20T21:44:42.534 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 53742, 2024-04-20T21:44:42.534 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 47.262263109, 2024-04-20T21:44:42.535 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000879428 2024-04-20T21:44:42.535 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:42.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.535 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-PurgeQueue": { 2024-04-20T21:44:42.535 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-04-20T21:44:42.535 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-04-20T21:44:42.535 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 22286, 2024-04-20T21:44:42.535 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 2.535192948, 2024-04-20T21:44:42.535 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000113757 2024-04-20T21:44:42.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:42.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-20T21:44:42.536 INFO:teuthology.orchestra.run.smithi033.stdout: "request": 76466, 2024-04-20T21:44:42.536 INFO:teuthology.orchestra.run.smithi033.stdout: "reply": 76466, 2024-04-20T21:44:42.536 INFO:teuthology.orchestra.run.smithi033.stdout: "reply_latency": { 2024-04-20T21:44:42.536 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 76466, 2024-04-20T21:44:42.537 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 139.929573486, 2024-04-20T21:44:42.537 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.001829958 2024-04-20T21:44:42.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.537 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_reply": 0, 2024-04-20T21:44:42.537 INFO:teuthology.orchestra.run.smithi033.stdout: "forward": 0, 2024-04-20T21:44:42.537 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_complete": 0, 2024-04-20T21:44:42.537 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_keys": 0, 2024-04-20T21:44:42.537 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_commit": 116, 2024-04-20T21:44:42.538 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_split": 11, 2024-04-20T21:44:42.538 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_merge": 15, 2024-04-20T21:44:42.538 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes": 1737, 2024-04-20T21:44:42.538 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_top": 721, 2024-04-20T21:44:42.538 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_bottom": 1016, 2024-04-20T21:44:42.538 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pin_tail": 0, 2024-04-20T21:44:42.538 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pinned": 707, 2024-04-20T21:44:42.538 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_expired": 0, 2024-04-20T21:44:42.538 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_with_caps": 77, 2024-04-20T21:44:42.539 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": 78, 2024-04-20T21:44:42.539 INFO:teuthology.orchestra.run.smithi033.stdout: "subtrees": 2, 2024-04-20T21:44:42.539 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse": 90433, 2024-04-20T21:44:42.539 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_hit": 70886, 2024-04-20T21:44:42.539 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_forward": 0, 2024-04-20T21:44:42.539 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_discover": 0, 2024-04-20T21:44:42.539 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_dir_fetch": 0, 2024-04-20T21:44:42.539 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_remote_ino": 0, 2024-04-20T21:44:42.539 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_lock": 135, 2024-04-20T21:44:42.540 INFO:teuthology.orchestra.run.smithi033.stdout: "load_cent": 0, 2024-04-20T21:44:42.540 INFO:teuthology.orchestra.run.smithi033.stdout: "q": 0, 2024-04-20T21:44:42.540 INFO:teuthology.orchestra.run.smithi033.stdout: "exported": 0, 2024-04-20T21:44:42.540 INFO:teuthology.orchestra.run.smithi033.stdout: "exported_inodes": 0, 2024-04-20T21:44:42.540 INFO:teuthology.orchestra.run.smithi033.stdout: "imported": 0, 2024-04-20T21:44:42.540 INFO:teuthology.orchestra.run.smithi033.stdout: "imported_inodes": 0, 2024-04-20T21:44:42.540 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_dir_fetch": 0, 2024-04-20T21:44:42.540 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_backtrace_fetch": 1681, 2024-04-20T21:44:42.541 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_peer_discover": 0, 2024-04-20T21:44:42.541 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rfiles": 68, 2024-04-20T21:44:42.541 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rbytes": 21196213, 2024-04-20T21:44:42.541 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rsnaps": 6, 2024-04-20T21:44:42.541 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_fetch": 1681, 2024-04-20T21:44:42.541 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_set_tag": 0, 2024-04-20T21:44:42.541 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T21:44:42.541 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_inotable_repaired": 0, 2024-04-20T21:44:42.541 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_inodes": 401, 2024-04-20T21:44:42.542 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_base_inodes": 21, 2024-04-20T21:44:42.542 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dirfrag_rstats": 380, 2024-04-20T21:44:42.542 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_file_inodes": 1280, 2024-04-20T21:44:42.542 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_inode_file_caps": 0, 2024-04-20T21:44:42.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_revoke": 1213, 2024-04-20T21:44:42.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_grant": 69226, 2024-04-20T21:44:42.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T21:44:42.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T21:44:42.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T21:44:42.543 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps": 21972, 2024-04-20T21:44:42.543 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps_dirty": 13620, 2024-04-20T21:44:42.543 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_cap_release": 6589, 2024-04-20T21:44:42.543 INFO:teuthology.orchestra.run.smithi033.stdout: "process_request_cap_release": 49343 2024-04-20T21:44:42.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.543 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache": { 2024-04-20T21:44:42.543 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update": 0, 2024-04-20T21:44:42.543 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update_receipt": 0, 2024-04-20T21:44:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_try_discover": 0, 2024-04-20T21:44:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_send_discover": 0, 2024-04-20T21:44:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_handle_discover": 0, 2024-04-20T21:44:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays": 0, 2024-04-20T21:44:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_delayed": 0, 2024-04-20T21:44:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_enqueuing": 0, 2024-04-20T21:44:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_created": 17203, 2024-04-20T21:44:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_enqueued": 17203, 2024-04-20T21:44:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_reintegrated": 0, 2024-04-20T21:44:42.545 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_migrated": 0, 2024-04-20T21:44:42.545 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_processing": 0, 2024-04-20T21:44:42.545 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_enqueued": 0, 2024-04-20T21:44:42.545 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_prioritized": 0, 2024-04-20T21:44:42.545 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_started": 0, 2024-04-20T21:44:42.545 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_completed": 0, 2024-04-20T21:44:42.545 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_path": 0, 2024-04-20T21:44:42.545 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_inode": 0, 2024-04-20T21:44:42.545 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_enqueue_scrub": 21, 2024-04-20T21:44:42.546 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_exportdir": 0, 2024-04-20T21:44:42.546 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_flush": 0, 2024-04-20T21:44:42.546 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragmentdir": 26, 2024-04-20T21:44:42.546 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragstats": 0, 2024-04-20T21:44:42.546 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_inodestats": 0 2024-04-20T21:44:42.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.546 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log": { 2024-04-20T21:44:42.546 INFO:teuthology.orchestra.run.smithi033.stdout: "evlrg": 0, 2024-04-20T21:44:42.547 INFO:teuthology.orchestra.run.smithi033.stdout: "evadd": 76928, 2024-04-20T21:44:42.547 INFO:teuthology.orchestra.run.smithi033.stdout: "evex": 0, 2024-04-20T21:44:42.547 INFO:teuthology.orchestra.run.smithi033.stdout: "evtrm": 0, 2024-04-20T21:44:42.547 INFO:teuthology.orchestra.run.smithi033.stdout: "ev": 76928, 2024-04-20T21:44:42.547 INFO:teuthology.orchestra.run.smithi033.stdout: "evexg": 0, 2024-04-20T21:44:42.547 INFO:teuthology.orchestra.run.smithi033.stdout: "evexd": 0, 2024-04-20T21:44:42.547 INFO:teuthology.orchestra.run.smithi033.stdout: "segadd": 118, 2024-04-20T21:44:42.547 INFO:teuthology.orchestra.run.smithi033.stdout: "segex": 0, 2024-04-20T21:44:42.547 INFO:teuthology.orchestra.run.smithi033.stdout: "segtrm": 0, 2024-04-20T21:44:42.548 INFO:teuthology.orchestra.run.smithi033.stdout: "seg": 118, 2024-04-20T21:44:42.548 INFO:teuthology.orchestra.run.smithi033.stdout: "segmjr": 8, 2024-04-20T21:44:42.548 INFO:teuthology.orchestra.run.smithi033.stdout: "segexg": 0, 2024-04-20T21:44:42.548 INFO:teuthology.orchestra.run.smithi033.stdout: "segexd": 0, 2024-04-20T21:44:42.548 INFO:teuthology.orchestra.run.smithi033.stdout: "expos": 4194304, 2024-04-20T21:44:42.548 INFO:teuthology.orchestra.run.smithi033.stdout: "wrpos": 270928470, 2024-04-20T21:44:42.548 INFO:teuthology.orchestra.run.smithi033.stdout: "rdpos": 0, 2024-04-20T21:44:42.548 INFO:teuthology.orchestra.run.smithi033.stdout: "jlat": { 2024-04-20T21:44:42.548 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 26638, 2024-04-20T21:44:42.549 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 78.092978399, 2024-04-20T21:44:42.549 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.002931638 2024-04-20T21:44:42.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.549 INFO:teuthology.orchestra.run.smithi033.stdout: "replayed": 0 2024-04-20T21:44:42.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.549 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_mem": { 2024-04-20T21:44:42.549 INFO:teuthology.orchestra.run.smithi033.stdout: "ino": 394, 2024-04-20T21:44:42.549 INFO:teuthology.orchestra.run.smithi033.stdout: "ino+": 17618, 2024-04-20T21:44:42.550 INFO:teuthology.orchestra.run.smithi033.stdout: "ino-": 17224, 2024-04-20T21:44:42.550 INFO:teuthology.orchestra.run.smithi033.stdout: "dir": 56, 2024-04-20T21:44:42.550 INFO:teuthology.orchestra.run.smithi033.stdout: "dir+": 135, 2024-04-20T21:44:42.550 INFO:teuthology.orchestra.run.smithi033.stdout: "dir-": 79, 2024-04-20T21:44:42.550 INFO:teuthology.orchestra.run.smithi033.stdout: "dn": 1737, 2024-04-20T21:44:42.550 INFO:teuthology.orchestra.run.smithi033.stdout: "dn+": 26562, 2024-04-20T21:44:42.550 INFO:teuthology.orchestra.run.smithi033.stdout: "dn-": 24825, 2024-04-20T21:44:42.550 INFO:teuthology.orchestra.run.smithi033.stdout: "cap": 78, 2024-04-20T21:44:42.550 INFO:teuthology.orchestra.run.smithi033.stdout: "cap+": 17370, 2024-04-20T21:44:42.551 INFO:teuthology.orchestra.run.smithi033.stdout: "cap-": 17292, 2024-04-20T21:44:42.551 INFO:teuthology.orchestra.run.smithi033.stdout: "rss": 77292, 2024-04-20T21:44:42.551 INFO:teuthology.orchestra.run.smithi033.stdout: "heap": 231800 2024-04-20T21:44:42.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.551 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_server": { 2024-04-20T21:44:42.551 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_client_request": 97452, 2024-04-20T21:44:42.551 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_server_request": 0, 2024-04-20T21:44:42.551 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_request": 76466, 2024-04-20T21:44:42.551 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_session": 2386, 2024-04-20T21:44:42.552 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_peer_request": 0, 2024-04-20T21:44:42.552 INFO:teuthology.orchestra.run.smithi033.stdout: "req_create_latency": { 2024-04-20T21:44:42.552 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 17271, 2024-04-20T21:44:42.552 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 16.378659861, 2024-04-20T21:44:42.552 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000948333 2024-04-20T21:44:42.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.552 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getattr_latency": { 2024-04-20T21:44:42.552 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 120, 2024-04-20T21:44:42.553 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.038427079, 2024-04-20T21:44:42.553 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000320225 2024-04-20T21:44:42.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.553 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getfilelock_latency": { 2024-04-20T21:44:42.553 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.553 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.553 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.553 INFO:teuthology.orchestra.run.smithi033.stdout: "req_link_latency": { 2024-04-20T21:44:42.554 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.554 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.554 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.554 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookup_latency": { 2024-04-20T21:44:42.554 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 10158, 2024-04-20T21:44:42.554 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 3.625463249, 2024-04-20T21:44:42.554 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000356907 2024-04-20T21:44:42.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.555 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookuphash_latency": { 2024-04-20T21:44:42.555 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.555 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.555 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.555 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupino_latency": { 2024-04-20T21:44:42.555 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2, 2024-04-20T21:44:42.555 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000365788, 2024-04-20T21:44:42.556 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000182894 2024-04-20T21:44:42.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.556 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupname_latency": { 2024-04-20T21:44:42.556 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.556 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.556 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.556 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupparent_latency": { 2024-04-20T21:44:42.556 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.557 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.557 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.557 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupsnap_latency": { 2024-04-20T21:44:42.557 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.557 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.557 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.558 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lssnap_latency": { 2024-04-20T21:44:42.558 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 11, 2024-04-20T21:44:42.558 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.002548537, 2024-04-20T21:44:42.558 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000231685 2024-04-20T21:44:42.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.559 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mkdir_latency": { 2024-04-20T21:44:42.559 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 20, 2024-04-20T21:44:42.559 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.021211644, 2024-04-20T21:44:42.559 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.001060582 2024-04-20T21:44:42.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.559 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mknod_latency": { 2024-04-20T21:44:42.559 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.560 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.560 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.560 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mksnap_latency": { 2024-04-20T21:44:42.560 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 11, 2024-04-20T21:44:42.560 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.089566207, 2024-04-20T21:44:42.560 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.008142382 2024-04-20T21:44:42.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.560 INFO:teuthology.orchestra.run.smithi033.stdout: "req_open_latency": { 2024-04-20T21:44:42.561 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 1, 2024-04-20T21:44:42.561 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000411031, 2024-04-20T21:44:42.561 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000411031 2024-04-20T21:44:42.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.561 INFO:teuthology.orchestra.run.smithi033.stdout: "req_readdir_latency": { 2024-04-20T21:44:42.561 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 27502, 2024-04-20T21:44:42.561 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 90.729856569, 2024-04-20T21:44:42.561 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.003299027 2024-04-20T21:44:42.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.562 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rename_latency": { 2024-04-20T21:44:42.562 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3602, 2024-04-20T21:44:42.562 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 4.577309048, 2024-04-20T21:44:42.562 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.001270768 2024-04-20T21:44:42.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.562 INFO:teuthology.orchestra.run.smithi033.stdout: "req_renamesnap_latency": { 2024-04-20T21:44:42.562 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.562 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.563 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.563 INFO:teuthology.orchestra.run.smithi033.stdout: "req_snapdiff_latency": { 2024-04-20T21:44:42.563 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.563 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.563 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.563 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmdir_latency": { 2024-04-20T21:44:42.563 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.564 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.564 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.564 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmsnap_latency": { 2024-04-20T21:44:42.564 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 5, 2024-04-20T21:44:42.564 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.053392298, 2024-04-20T21:44:42.564 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.010678459 2024-04-20T21:44:42.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.565 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmxattr_latency": { 2024-04-20T21:44:42.565 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.565 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.565 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.565 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setattr_latency": { 2024-04-20T21:44:42.565 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 1, 2024-04-20T21:44:42.565 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000903016, 2024-04-20T21:44:42.565 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000903016 2024-04-20T21:44:42.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.566 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setdirlayout_latency": { 2024-04-20T21:44:42.566 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.566 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.566 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.566 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setfilelock_latency": { 2024-04-20T21:44:42.566 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 552, 2024-04-20T21:44:42.566 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.293951381, 2024-04-20T21:44:42.567 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000532520 2024-04-20T21:44:42.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.567 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setlayout_latency": { 2024-04-20T21:44:42.567 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.567 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.567 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.567 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setxattr_latency": { 2024-04-20T21:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 7, 2024-04-20T21:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.025280007, 2024-04-20T21:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.003611429 2024-04-20T21:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "req_symlink_latency": { 2024-04-20T21:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "req_unlink_latency": { 2024-04-20T21:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 17203, 2024-04-20T21:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 24.092227771, 2024-04-20T21:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.001400466 2024-04-20T21:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_revoke_eviction": 0, 2024-04-20T21:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_acquisition_throttle": 0, 2024-04-20T21:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getvxattr_latency": { 2024-04-20T21:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_sessions": { 2024-04-20T21:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "session_count": 2, 2024-04-20T21:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "session_add": 5, 2024-04-20T21:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "session_remove": 3, 2024-04-20T21:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_open": 2, 2024-04-20T21:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_stale": 0, 2024-04-20T21:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "total_load": 1552, 2024-04-20T21:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "average_load": 776, 2024-04-20T21:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "avg_session_uptime": 647, 2024-04-20T21:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "mdthresh_evicted": 0 2024-04-20T21:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-20T21:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-20T21:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 326407, 2024-04-20T21:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 320, 2024-04-20T21:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 616, 2024-04-20T21:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 7, 2024-04-20T21:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-20T21:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-20T21:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 8320, 2024-04-20T21:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 173, 2024-04-20T21:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-20T21:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 2438968, 2024-04-20T21:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 25562, 2024-04-20T21:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-20T21:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-20T21:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter": { 2024-04-20T21:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "op_active": 0, 2024-04-20T21:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "op_laggy": 0, 2024-04-20T21:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send": 47383, 2024-04-20T21:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send_bytes": 271200643, 2024-04-20T21:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "op_resend": 32, 2024-04-20T21:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "op_reply": 47383, 2024-04-20T21:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "op_latency": { 2024-04-20T21:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 47383, 2024-04-20T21:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 244.349091504, 2024-04-20T21:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.005156893 2024-04-20T21:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "op_inflight": 0, 2024-04-20T21:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "oplen_avg": { 2024-04-20T21:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 47383, 2024-04-20T21:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 47951 2024-04-20T21:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "op": 47383, 2024-04-20T21:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "op_r": 2011, 2024-04-20T21:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "op_w": 45372, 2024-04-20T21:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "op_rmw": 0, 2024-04-20T21:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "op_pg": 0, 2024-04-20T21:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_stat": 0, 2024-04-20T21:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_create": 11, 2024-04-20T21:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_read": 309, 2024-04-20T21:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_write": 26926, 2024-04-20T21:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writefull": 164, 2024-04-20T21:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writesame": 0, 2024-04-20T21:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_append": 0, 2024-04-20T21:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_zero": 0, 2024-04-20T21:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_truncate": 11, 2024-04-20T21:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_delete": 17846, 2024-04-20T21:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_mapext": 0, 2024-04-20T21:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_sparse_read": 0, 2024-04-20T21:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_clonerange": 0, 2024-04-20T21:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_getxattr": 1702, 2024-04-20T21:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_setxattr": 22, 2024-04-20T21:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_cmpxattr": 0, 2024-04-20T21:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_rmxattr": 0, 2024-04-20T21:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_resetxattrs": 0, 2024-04-20T21:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_call": 0, 2024-04-20T21:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_watch": 0, 2024-04-20T21:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_notify": 0, 2024-04-20T21:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T21:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls": 0, 2024-04-20T21:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls_filter": 0, 2024-04-20T21:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_other": 0, 2024-04-20T21:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_active": 0, 2024-04-20T21:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_send": 0, 2024-04-20T21:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_resend": 0, 2024-04-20T21:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_ping": 0, 2024-04-20T21:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_active": 0, 2024-04-20T21:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_send": 0, 2024-04-20T21:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_resend": 0, 2024-04-20T21:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_active": 0, 2024-04-20T21:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_send": 0, 2024-04-20T21:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_resend": 0, 2024-04-20T21:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_active": 0, 2024-04-20T21:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_send": 0, 2024-04-20T21:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_resend": 0, 2024-04-20T21:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "command_active": 0, 2024-04-20T21:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "command_send": 0, 2024-04-20T21:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "command_resend": 0, 2024-04-20T21:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "map_epoch": 103, 2024-04-20T21:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "map_full": 0, 2024-04-20T21:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "map_inc": 16, 2024-04-20T21:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_sessions": 12, 2024-04-20T21:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_open": 12, 2024-04-20T21:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_close": 0, 2024-04-20T21:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_laggy": 0, 2024-04-20T21:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_wr": 797, 2024-04-20T21:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_rd": 0, 2024-04-20T21:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_del": 163 2024-04-20T21:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "oft": { 2024-04-20T21:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_objs": 1, 2024-04-20T21:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_kv_pairs": 0, 2024-04-20T21:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_updates": 0, 2024-04-20T21:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_removes": 0 2024-04-20T21:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "osd-slow-ops": { 2024-04-20T21:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_ops_count": 0 2024-04-20T21:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "purge_queue": { 2024-04-20T21:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops": 0, 2024-04-20T21:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops_high_water": 32, 2024-04-20T21:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing": 0, 2024-04-20T21:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_high_water": 32, 2024-04-20T21:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed_ops": 17203, 2024-04-20T21:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed": 17203, 2024-04-20T21:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_item_in_journal": 0 2024-04-20T21:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-20T21:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-20T21:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-20T21:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 156855, 2024-04-20T21:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 37306306, 2024-04-20T21:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 156855, 2024-04-20T21:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-20T21:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-20T21:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 156855, 2024-04-20T21:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 37306306, 2024-04-20T21:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-20T21:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_bytes": { 2024-04-20T21:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-20T21:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-20T21:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-20T21:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-04-20T21:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-04-20T21:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 47383, 2024-04-20T21:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 272940370, 2024-04-20T21:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 47383, 2024-04-20T21:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 272940370, 2024-04-20T21:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-20T21:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_ops": { 2024-04-20T21:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-20T21:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 1024, 2024-04-20T21:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-20T21:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-04-20T21:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-04-20T21:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 47383, 2024-04-20T21:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 47383, 2024-04-20T21:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 47383, 2024-04-20T21:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 47383, 2024-04-20T21:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-20T21:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle": { 2024-04-20T21:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-20T21:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-04-20T21:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-20T21:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 17203, 2024-04-20T21:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1739727, 2024-04-20T21:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 17203, 2024-04-20T21:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-20T21:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-20T21:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 288, 2024-04-20T21:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1739727, 2024-04-20T21:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-20T21:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle-0x55b6cef72500": { 2024-04-20T21:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-20T21:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-04-20T21:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-20T21:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 76928, 2024-04-20T21:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 266734166, 2024-04-20T21:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 76928, 2024-04-20T21:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-20T21:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-20T21:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 26638, 2024-04-20T21:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 266734166, 2024-04-20T21:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-20T21:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-20T21:44:42.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:42 smithi022 ceph-mon[16843]: pgmap v339: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:43.401 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-20T21:44:43.402 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-20T21:44:43.402 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-20T21:44:43.402 DEBUG:tasks.check_counter:Getting stats from e 2024-04-20T21:44:43.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-20T21:44:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:44:44.306 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:44 smithi033 ceph-mon[21626]: pgmap v340: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:44.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:44 smithi122 ceph-mon[22147]: pgmap v340: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:44.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:44 smithi022 ceph-mon[16843]: pgmap v340: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:45.354 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-20T21:44:45.354 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:45.354 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-04-20T21:44:45.354 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-04-20T21:44:45.355 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:44:45.355 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-04-20T21:44:45.355 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:45.355 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:45.355 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001170229, 2024-04-20T21:44:45.355 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:44:45.355 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000157713, 2024-04-20T21:44:45.355 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:45.356 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:45.356 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:45.356 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:45.356 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:45.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:45.356 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:45.356 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:45.356 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:45.357 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:45.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:45.357 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:45.357 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:45.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:45.357 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:45.357 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 7, 2024-04-20T21:44:45.357 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 173, 2024-04-20T21:44:45.357 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3472, 2024-04-20T21:44:45.358 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 264976, 2024-04-20T21:44:45.358 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 6, 2024-04-20T21:44:45.358 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:45.358 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.012210445, 2024-04-20T21:44:45.358 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.007927701, 2024-04-20T21:44:45.358 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.002041957, 2024-04-20T21:44:45.358 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:45.358 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:45.359 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 171, 2024-04-20T21:44:45.359 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.026233633, 2024-04-20T21:44:45.359 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000153413 2024-04-20T21:44:45.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:45.359 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:45.359 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:45.359 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:45.359 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:45.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:45.360 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-04-20T21:44:45.360 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 264976 2024-04-20T21:44:45.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:45.360 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:45.360 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 233, 2024-04-20T21:44:45.360 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 221, 2024-04-20T21:44:45.360 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1731216, 2024-04-20T21:44:45.360 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 136224, 2024-04-20T21:44:45.360 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:45.361 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:45.361 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.045214989, 2024-04-20T21:44:45.361 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.012870241, 2024-04-20T21:44:45.361 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.037213195, 2024-04-20T21:44:45.361 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.005310030, 2024-04-20T21:44:45.361 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:45.361 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 220, 2024-04-20T21:44:45.361 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010819450, 2024-04-20T21:44:45.361 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000049179 2024-04-20T21:44:45.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:45.362 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:45.362 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:45.362 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:45.362 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:45.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:45.362 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1731216, 2024-04-20T21:44:45.362 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 136224 2024-04-20T21:44:45.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:45.363 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-20T21:44:45.363 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-04-20T21:44:45.363 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-20T21:44:45.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:45.363 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-20T21:44:45.363 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:45.363 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:45.364 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:45.364 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:45.364 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:45.364 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:45.364 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:45.364 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:45.364 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:45.364 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:45.364 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:45.365 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:45.365 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:45.365 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:45.365 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:45.365 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:45.365 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:45.365 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:45.365 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:45.365 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:45.366 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:45.366 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:45.366 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:45.366 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:45.366 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:45.366 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:45.366 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:45.366 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:45.367 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:45.367 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:45.367 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:45.367 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-20T21:44:45.367 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:45.367 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:45.367 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:45.367 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:45.367 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148955, 2024-04-20T21:44:45.368 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 30, 2024-04-20T21:44:45.368 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:45.368 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:45.368 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-20T21:44:45.368 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-20T21:44:45.368 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:45.368 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:45.368 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:45.369 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:45.369 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:45.369 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-04-20T21:44:45.369 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:45.369 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:45.369 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:45.369 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-20T21:44:45.369 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:45.369 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-04-20T21:44:45.370 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:45.370 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-20T21:44:45.370 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:45.370 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-20T21:44:45.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:45.370 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:45.371 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-20T21:44:45.371 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-20T21:44:45.371 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-20T21:44:45.371 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 236, 2024-04-20T21:44:45.371 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1698391, 2024-04-20T21:44:45.371 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:45.371 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 236, 2024-04-20T21:44:45.371 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-20T21:44:45.371 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-20T21:44:45.372 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 236, 2024-04-20T21:44:45.372 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1698391, 2024-04-20T21:44:45.372 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-20T21:44:45.372 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:45.372 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:45.372 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:45.372 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:45.372 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:45.373 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-20T21:44:46.074 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-20T21:44:46.074 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-20T21:44:46.074 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-20T21:44:46.074 DEBUG:tasks.check_counter:Getting stats from h 2024-04-20T21:44:46.074 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-20T21:44:46.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:46 smithi033 ceph-mon[21626]: pgmap v341: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:44:46.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:46 smithi122 ceph-mon[22147]: pgmap v341: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:46.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:46 smithi022 ceph-mon[16843]: pgmap v341: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:48.009 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-20T21:44:48.010 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:48.010 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 232, 2024-04-20T21:44:48.010 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 220, 2024-04-20T21:44:48.010 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1730944, 2024-04-20T21:44:48.010 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 135584, 2024-04-20T21:44:48.010 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:48.010 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:48.010 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.048004637, 2024-04-20T21:44:48.011 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.012734325, 2024-04-20T21:44:48.011 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.039664749, 2024-04-20T21:44:48.011 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.005692203, 2024-04-20T21:44:48.011 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:48.011 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 219, 2024-04-20T21:44:48.011 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010040588, 2024-04-20T21:44:48.011 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000045847 2024-04-20T21:44:48.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:48.012 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:48.012 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:48.012 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:48.012 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:48.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:48.012 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1730944, 2024-04-20T21:44:48.012 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 135584 2024-04-20T21:44:48.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:48.012 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:48.013 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 6, 2024-04-20T21:44:48.013 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 170, 2024-04-20T21:44:48.013 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3312, 2024-04-20T21:44:48.013 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 186128, 2024-04-20T21:44:48.013 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-04-20T21:44:48.013 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:48.013 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.010568277, 2024-04-20T21:44:48.013 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.006779449, 2024-04-20T21:44:48.013 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001791295, 2024-04-20T21:44:48.014 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:48.014 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:48.014 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 168, 2024-04-20T21:44:48.014 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.025606665, 2024-04-20T21:44:48.014 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000152420 2024-04-20T21:44:48.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:48.014 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:48.014 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:48.014 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:48.015 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:48.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:48.015 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-20T21:44:48.015 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 186128 2024-04-20T21:44:48.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:48.015 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:48.015 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-04-20T21:44:48.015 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-04-20T21:44:48.016 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:44:48.016 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-04-20T21:44:48.016 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:48.016 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:48.016 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.000178466, 2024-04-20T21:44:48.016 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:44:48.016 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000123295, 2024-04-20T21:44:48.016 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:48.016 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:48.017 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:48.017 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:48.017 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:48.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:48.017 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:48.017 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:48.017 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:48.017 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:48.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:48.018 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:48.018 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:48.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:48.018 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-20T21:44:48.018 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-04-20T21:44:48.018 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-20T21:44:48.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:48.018 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-20T21:44:48.019 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:48.019 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:48.019 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:48.019 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:48.019 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:48.019 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:48.019 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:48.019 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:48.019 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:48.020 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:48.020 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:48.020 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:48.020 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:48.020 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:48.020 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:48.020 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:48.020 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:48.020 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:48.021 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:48.021 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:48.021 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:48.021 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:48.021 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:48.021 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:48.021 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:48.021 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:48.021 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:48.022 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:48.022 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:48.022 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:48.022 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:48.022 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-20T21:44:48.022 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:48.022 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:48.022 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:48.022 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:48.023 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148955, 2024-04-20T21:44:48.023 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 30, 2024-04-20T21:44:48.023 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:48.023 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:48.023 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-20T21:44:48.023 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-20T21:44:48.023 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:48.023 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:48.023 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:48.024 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:48.024 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:48.024 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-04-20T21:44:48.024 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:48.024 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:48.024 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:48.024 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-20T21:44:48.024 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:48.025 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-04-20T21:44:48.025 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:48.025 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-20T21:44:48.025 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:48.025 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-20T21:44:48.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:48.025 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:48.025 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-20T21:44:48.025 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-20T21:44:48.025 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-20T21:44:48.026 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 234, 2024-04-20T21:44:48.026 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1698234, 2024-04-20T21:44:48.026 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:48.026 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 234, 2024-04-20T21:44:48.026 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-20T21:44:48.026 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-20T21:44:48.026 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 234, 2024-04-20T21:44:48.026 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1698234, 2024-04-20T21:44:48.027 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-20T21:44:48.027 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:48.027 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:48.027 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:48.027 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:48.027 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:48.027 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-20T21:44:48.303 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:48 smithi033 ceph-mon[21626]: pgmap v342: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:48 smithi122 ceph-mon[22147]: pgmap v342: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:48.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:48 smithi022 ceph-mon[16843]: pgmap v342: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:48.662 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-04-20T21:44:48.662 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-04-20T21:44:48.662 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-04-20T21:44:48.663 DEBUG:tasks.check_counter:Getting stats from k 2024-04-20T21:44:48.663 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-20T21:44:48.913 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:44:50.354 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:50 smithi033 ceph-mon[21626]: pgmap v343: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:50.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:50 smithi122 ceph-mon[22147]: pgmap v343: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:50.605 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-20T21:44:50.606 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:50.606 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-04-20T21:44:50.606 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-04-20T21:44:50.606 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:44:50.606 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-04-20T21:44:50.606 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:50.606 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:50.607 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001152869, 2024-04-20T21:44:50.607 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:44:50.607 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000225085, 2024-04-20T21:44:50.607 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:50.607 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:50.607 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:50.607 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:50.607 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:50.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:50.608 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:50.608 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:50.608 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:50.608 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:50.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:50.608 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:50.608 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:50.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:50.608 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:50.608 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 6, 2024-04-20T21:44:50.609 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 175, 2024-04-20T21:44:50.609 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3312, 2024-04-20T21:44:50.609 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 187008, 2024-04-20T21:44:50.609 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-04-20T21:44:50.609 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:50.609 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.010886399, 2024-04-20T21:44:50.609 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.007244487, 2024-04-20T21:44:50.609 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001479473, 2024-04-20T21:44:50.609 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:50.610 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:50.610 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 173, 2024-04-20T21:44:50.610 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.029203848, 2024-04-20T21:44:50.610 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000168808 2024-04-20T21:44:50.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:50.610 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:50.610 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:50.610 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:50.611 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:50.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:50.611 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-20T21:44:50.611 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 187008 2024-04-20T21:44:50.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:50.611 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:50.611 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 237, 2024-04-20T21:44:50.612 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 225, 2024-04-20T21:44:50.612 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1732304, 2024-04-20T21:44:50.612 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 138784, 2024-04-20T21:44:50.612 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:50.612 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:50.612 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.048286439, 2024-04-20T21:44:50.612 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.012698566, 2024-04-20T21:44:50.612 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.041160440, 2024-04-20T21:44:50.612 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.005375773, 2024-04-20T21:44:50.613 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:50.613 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 224, 2024-04-20T21:44:50.613 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010685200, 2024-04-20T21:44:50.613 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000047701 2024-04-20T21:44:50.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:50.613 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:50.613 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:50.613 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:50.613 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1732304, 2024-04-20T21:44:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 138784 2024-04-20T21:44:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-20T21:44:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-04-20T21:44:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-20T21:44:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:50.615 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-20T21:44:50.615 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:50.615 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:50.615 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:50.615 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:50.615 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:50.615 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:50.615 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:50.615 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:50.615 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:50.616 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:50.616 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:50.616 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:50.616 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:50.616 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:50.616 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:50.616 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:50.616 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:50.616 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:50.617 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:50.617 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:50.617 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:50.617 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:50.617 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:50.617 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:50.617 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:50.617 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:50.617 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:50.617 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:50.618 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:50.618 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:50.618 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:50.618 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-20T21:44:50.618 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:50.618 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:50.618 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:50.618 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:50.619 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148692, 2024-04-20T21:44:50.619 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 30, 2024-04-20T21:44:50.619 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:50.619 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:50.619 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-20T21:44:50.619 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-20T21:44:50.619 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:50.619 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:50.619 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:50.619 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:50.620 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:50.620 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-04-20T21:44:50.620 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:50.620 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:50.620 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:50.620 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-20T21:44:50.620 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:50.620 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-04-20T21:44:50.620 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:50.621 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-20T21:44:50.621 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:50.621 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-20T21:44:50.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-20T21:44:50.621 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:50.621 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-20T21:44:50.621 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-20T21:44:50.621 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-20T21:44:50.621 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 239, 2024-04-20T21:44:50.621 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1698884, 2024-04-20T21:44:50.622 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:50.622 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 239, 2024-04-20T21:44:50.622 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-20T21:44:50.622 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-20T21:44:50.622 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 239, 2024-04-20T21:44:50.622 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1698884, 2024-04-20T21:44:50.622 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-20T21:44:50.623 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-20T21:44:50.623 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-20T21:44:50.623 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-20T21:44:50.623 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:50.623 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-20T21:44:50.623 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-20T21:44:50.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:50 smithi022 ceph-mon[16843]: pgmap v343: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:51.209 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-20T21:44:51.209 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-20T21:44:51.209 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-20T21:44:51.209 DEBUG:tasks.check_counter:Getting stats from c 2024-04-20T21:44:51.210 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-20T21:44:51.458 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:44:52.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:52 smithi122 ceph-mon[22147]: pgmap v344: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:52.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:52 smithi033 ceph-mon[21626]: pgmap v344: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:52.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:52 smithi022 ceph-mon[16843]: pgmap v344: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:53.265 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-20T21:44:53.265 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:53.265 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 0, 2024-04-20T21:44:53.266 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2, 2024-04-20T21:44:53.266 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:44:53.266 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 0, 2024-04-20T21:44:53.266 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:53.266 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:53.266 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.001048845, 2024-04-20T21:44:53.267 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:44:53.267 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.000184079, 2024-04-20T21:44:53.267 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:53.267 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:53.267 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:53.267 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:53.267 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:53.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:53.267 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:53.268 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:53.268 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:53.268 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:53.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:53.268 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:53.268 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:53.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:53.268 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:53.269 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 238, 2024-04-20T21:44:53.269 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 224, 2024-04-20T21:44:53.269 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 1734480, 2024-04-20T21:44:53.269 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 137312, 2024-04-20T21:44:53.269 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:53.269 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:53.269 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.044015457, 2024-04-20T21:44:53.269 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.013346237, 2024-04-20T21:44:53.269 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.035057220, 2024-04-20T21:44:53.270 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.005146808, 2024-04-20T21:44:53.270 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:53.270 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 224, 2024-04-20T21:44:53.270 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.011211003, 2024-04-20T21:44:53.270 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000050049 2024-04-20T21:44:53.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:53.270 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:53.270 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:53.271 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:53.271 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:53.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:53.271 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1734480, 2024-04-20T21:44:53.271 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 137312 2024-04-20T21:44:53.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:53.271 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:53.271 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 2, 2024-04-20T21:44:53.272 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 170, 2024-04-20T21:44:53.272 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 320, 2024-04-20T21:44:53.272 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 264352, 2024-04-20T21:44:53.272 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 6, 2024-04-20T21:44:53.272 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:53.272 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.011401028, 2024-04-20T21:44:53.272 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.007628909, 2024-04-20T21:44:53.272 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.001519427, 2024-04-20T21:44:53.272 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:53.273 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:53.273 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 167, 2024-04-20T21:44:53.273 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.026947887, 2024-04-20T21:44:53.273 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000161364 2024-04-20T21:44:53.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:53.273 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:53.274 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:53.274 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:53.274 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:53.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:53.274 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T21:44:53.274 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 264352 2024-04-20T21:44:53.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:53.274 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-04-20T21:44:53.275 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-04-20T21:44:53.275 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-04-20T21:44:53.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:53.275 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-04-20T21:44:53.275 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:53.275 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:53.275 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:53.275 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:53.276 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:53.276 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:53.276 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:53.276 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:53.276 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:53.276 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:53.276 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:53.276 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:53.276 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:53.277 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:53.277 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:53.277 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:53.277 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:53.277 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:53.277 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:53.277 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:53.277 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:53.278 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:53.278 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:53.278 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:53.278 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:53.278 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-04-20T21:44:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:53.279 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 148955, 2024-04-20T21:44:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 30, 2024-04-20T21:44:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-04-20T21:44:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-04-20T21:44:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:53.281 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:53.281 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:53.281 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:53.281 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:53.281 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-04-20T21:44:53.281 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:53.281 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:53.281 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:53.281 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-04-20T21:44:53.282 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:53.282 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-04-20T21:44:53.282 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:53.282 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-04-20T21:44:53.282 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:53.282 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-04-20T21:44:53.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:53.282 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:53.283 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-20T21:44:53.283 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-20T21:44:53.283 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-20T21:44:53.283 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 236, 2024-04-20T21:44:53.283 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 1698494, 2024-04-20T21:44:53.283 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:53.283 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 236, 2024-04-20T21:44:53.283 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-20T21:44:53.284 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-20T21:44:53.284 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 236, 2024-04-20T21:44:53.284 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1698494, 2024-04-20T21:44:53.284 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-20T21:44:53.284 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:53.284 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:53.284 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:53.284 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:53.284 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:53.285 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-20T21:44:53.981 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-04-20T21:44:53.982 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-04-20T21:44:53.982 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-04-20T21:44:53.982 DEBUG:tasks.check_counter:Getting stats from f 2024-04-20T21:44:53.982 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-20T21:44:54.230 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:44:54.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:54 smithi122 ceph-mon[22147]: pgmap v345: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:54.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:54 smithi033 ceph-mon[21626]: pgmap v345: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:54.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:54 smithi022 ceph-mon[16843]: pgmap v345: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:55.931 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-20T21:44:55.932 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:55.932 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 0, 2024-04-20T21:44:55.932 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2, 2024-04-20T21:44:55.932 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:44:55.932 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 0, 2024-04-20T21:44:55.932 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:55.932 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:55.932 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.000991421, 2024-04-20T21:44:55.932 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:44:55.933 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.000182440, 2024-04-20T21:44:55.933 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:55.933 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:55.933 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:55.933 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:55.933 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:55.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:55.933 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:55.933 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:55.934 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:55.934 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:55.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:55.934 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:55.934 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:55.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:55.934 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:55.934 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 239, 2024-04-20T21:44:55.935 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 225, 2024-04-20T21:44:55.935 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 1734752, 2024-04-20T21:44:55.935 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 137952, 2024-04-20T21:44:55.935 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:55.935 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:55.935 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.046698896, 2024-04-20T21:44:55.935 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.013727405, 2024-04-20T21:44:55.935 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.037825231, 2024-04-20T21:44:55.935 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.005252940, 2024-04-20T21:44:55.936 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:55.936 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 225, 2024-04-20T21:44:55.936 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.011605185, 2024-04-20T21:44:55.936 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000051578 2024-04-20T21:44:55.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:55.936 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:55.936 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:55.936 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:55.937 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:55.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:55.937 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1734752, 2024-04-20T21:44:55.937 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 137952 2024-04-20T21:44:55.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:55.937 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:55.937 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 2, 2024-04-20T21:44:55.937 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 171, 2024-04-20T21:44:55.937 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 320, 2024-04-20T21:44:55.938 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 186208, 2024-04-20T21:44:55.938 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 5, 2024-04-20T21:44:55.938 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:55.938 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.010280723, 2024-04-20T21:44:55.938 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.006965124, 2024-04-20T21:44:55.938 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.001078493, 2024-04-20T21:44:55.938 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:55.938 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:55.938 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 168, 2024-04-20T21:44:55.939 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.026336887, 2024-04-20T21:44:55.939 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000156767 2024-04-20T21:44:55.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:55.939 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:55.939 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:55.939 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:55.939 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:55.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:55.939 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T21:44:55.940 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 186208 2024-04-20T21:44:55.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:55.940 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-04-20T21:44:55.940 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-04-20T21:44:55.940 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-04-20T21:44:55.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:55.940 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-04-20T21:44:55.940 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:55.941 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:55.941 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:55.941 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:55.941 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:55.941 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:55.941 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:55.941 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:55.941 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:55.941 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:55.942 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:55.942 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:55.942 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:55.942 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:55.942 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:55.942 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:55.942 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:55.942 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:55.943 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:55.943 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:55.943 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:55.943 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:55.943 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:55.943 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:55.943 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:55.943 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:55.943 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:55.944 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:55.944 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:55.944 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:55.944 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:55.944 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-04-20T21:44:55.944 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:55.944 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:55.944 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:55.944 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:55.945 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 148955, 2024-04-20T21:44:55.945 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 30, 2024-04-20T21:44:55.945 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:55.945 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:55.945 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-04-20T21:44:55.945 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-04-20T21:44:55.945 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:55.945 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:55.945 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:55.946 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:55.946 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:55.946 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-04-20T21:44:55.946 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:55.946 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:55.946 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:55.946 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-04-20T21:44:55.946 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:55.947 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-04-20T21:44:55.947 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:55.947 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-04-20T21:44:55.947 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:55.947 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-04-20T21:44:55.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:55.947 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:55.947 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-20T21:44:55.948 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-20T21:44:55.948 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-20T21:44:55.948 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 237, 2024-04-20T21:44:55.948 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 1698624, 2024-04-20T21:44:55.948 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:55.948 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 237, 2024-04-20T21:44:55.948 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-20T21:44:55.948 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-20T21:44:55.949 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 237, 2024-04-20T21:44:55.949 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1698624, 2024-04-20T21:44:55.949 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-20T21:44:55.949 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:55.949 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:55.949 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:55.949 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:55.949 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:55.950 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-20T21:44:56.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:56 smithi122 ceph-mon[22147]: pgmap v346: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:56.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:56 smithi033 ceph-mon[21626]: pgmap v346: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:56.629 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-20T21:44:56.629 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-20T21:44:56.629 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-20T21:44:56.630 DEBUG:tasks.check_counter:Getting stats from i 2024-04-20T21:44:56.630 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-20T21:44:56.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:56 smithi022 ceph-mon[16843]: pgmap v346: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:44:56.866 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:44:58.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:44:58 smithi122 ceph-mon[22147]: pgmap v347: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:58.539 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-20T21:44:58.539 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:58.539 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 1480, 2024-04-20T21:44:58.539 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 1482, 2024-04-20T21:44:58.539 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 101108169, 2024-04-20T21:44:58.539 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 470650, 2024-04-20T21:44:58.540 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 8, 2024-04-20T21:44:58.540 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 4, 2024-04-20T21:44:58.540 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.189561807, 2024-04-20T21:44:58.540 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.041074083, 2024-04-20T21:44:58.540 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.089718261, 2024-04-20T21:44:58.540 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.045321858, 2024-04-20T21:44:58.540 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:58.540 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1480, 2024-04-20T21:44:58.541 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.058277629, 2024-04-20T21:44:58.541 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000039376 2024-04-20T21:44:58.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.541 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:58.541 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.541 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.541 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.541 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:58.542 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:58.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.542 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:58.542 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 894, 2024-04-20T21:44:58.542 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 881, 2024-04-20T21:44:58.542 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 85801348, 2024-04-20T21:44:58.542 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 343446, 2024-04-20T21:44:58.542 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 6, 2024-04-20T21:44:58.543 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 5, 2024-04-20T21:44:58.543 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.131466726, 2024-04-20T21:44:58.543 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.030966107, 2024-04-20T21:44:58.543 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.084072800, 2024-04-20T21:44:58.543 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.021990495, 2024-04-20T21:44:58.543 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:58.543 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 880, 2024-04-20T21:44:58.543 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.034115974, 2024-04-20T21:44:58.543 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000038768 2024-04-20T21:44:58.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.544 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:58.544 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.544 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.544 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.544 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1753536, 2024-04-20T21:44:58.544 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 140496 2024-04-20T21:44:58.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.544 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:58.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 880, 2024-04-20T21:44:58.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 1046, 2024-04-20T21:44:58.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 82417247, 2024-04-20T21:44:58.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 819535, 2024-04-20T21:44:58.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 9, 2024-04-20T21:44:58.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 6, 2024-04-20T21:44:58.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.123396370, 2024-04-20T21:44:58.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.032522144, 2024-04-20T21:44:58.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.060162102, 2024-04-20T21:44:58.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.021889573, 2024-04-20T21:44:58.546 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:58.546 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1043, 2024-04-20T21:44:58.546 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.063017394, 2024-04-20T21:44:58.546 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000060419 2024-04-20T21:44:58.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.546 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:58.546 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.546 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.547 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-04-20T21:44:58.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 541920 2024-04-20T21:44:58.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.547 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-04-20T21:44:58.547 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 1, 2024-04-20T21:44:58.547 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-04-20T21:44:58.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-MDSRank": { 2024-04-20T21:44:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-20T21:44:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-20T21:44:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 4844, 2024-04-20T21:44:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.124154071, 2024-04-20T21:44:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000025630 2024-04-20T21:44:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-PurgeQueue": { 2024-04-20T21:44:58.549 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-20T21:44:58.549 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-20T21:44:58.549 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.549 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.549 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.549 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:58.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.549 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-04-20T21:44:58.549 INFO:teuthology.orchestra.run.smithi122.stdout: "request": 0, 2024-04-20T21:44:58.550 INFO:teuthology.orchestra.run.smithi122.stdout: "reply": 0, 2024-04-20T21:44:58.550 INFO:teuthology.orchestra.run.smithi122.stdout: "reply_latency": { 2024-04-20T21:44:58.550 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.550 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.550 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.550 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_reply": 0, 2024-04-20T21:44:58.550 INFO:teuthology.orchestra.run.smithi122.stdout: "forward": 0, 2024-04-20T21:44:58.551 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_complete": 0, 2024-04-20T21:44:58.551 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_keys": 0, 2024-04-20T21:44:58.551 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_commit": 0, 2024-04-20T21:44:58.551 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_split": 0, 2024-04-20T21:44:58.551 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_merge": 0, 2024-04-20T21:44:58.551 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes": 17669, 2024-04-20T21:44:58.551 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_top": 0, 2024-04-20T21:44:58.551 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_bottom": 17669, 2024-04-20T21:44:58.551 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pin_tail": 0, 2024-04-20T21:44:58.552 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pinned": 17669, 2024-04-20T21:44:58.552 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_expired": 0, 2024-04-20T21:44:58.552 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_with_caps": 0, 2024-04-20T21:44:58.552 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": 0, 2024-04-20T21:44:58.552 INFO:teuthology.orchestra.run.smithi122.stdout: "subtrees": 2, 2024-04-20T21:44:58.552 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse": 0, 2024-04-20T21:44:58.552 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_hit": 0, 2024-04-20T21:44:58.552 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_forward": 0, 2024-04-20T21:44:58.552 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_discover": 0, 2024-04-20T21:44:58.553 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_dir_fetch": 0, 2024-04-20T21:44:58.553 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_remote_ino": 0, 2024-04-20T21:44:58.553 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_lock": 0, 2024-04-20T21:44:58.553 INFO:teuthology.orchestra.run.smithi122.stdout: "load_cent": 0, 2024-04-20T21:44:58.553 INFO:teuthology.orchestra.run.smithi122.stdout: "q": 0, 2024-04-20T21:44:58.553 INFO:teuthology.orchestra.run.smithi122.stdout: "exported": 0, 2024-04-20T21:44:58.553 INFO:teuthology.orchestra.run.smithi122.stdout: "exported_inodes": 0, 2024-04-20T21:44:58.553 INFO:teuthology.orchestra.run.smithi122.stdout: "imported": 0, 2024-04-20T21:44:58.554 INFO:teuthology.orchestra.run.smithi122.stdout: "imported_inodes": 0, 2024-04-20T21:44:58.554 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_dir_fetch": 0, 2024-04-20T21:44:58.554 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_backtrace_fetch": 0, 2024-04-20T21:44:58.554 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_peer_discover": 0, 2024-04-20T21:44:58.554 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rfiles": 68, 2024-04-20T21:44:58.554 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rbytes": 21196213, 2024-04-20T21:44:58.554 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rsnaps": 6, 2024-04-20T21:44:58.554 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_fetch": 0, 2024-04-20T21:44:58.555 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_set_tag": 0, 2024-04-20T21:44:58.555 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T21:44:58.555 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_inotable_repaired": 0, 2024-04-20T21:44:58.555 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_inodes": 0, 2024-04-20T21:44:58.555 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T21:44:58.555 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dirfrag_rstats": 0, 2024-04-20T21:44:58.555 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_file_inodes": 0, 2024-04-20T21:44:58.555 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_inode_file_caps": 0, 2024-04-20T21:44:58.555 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_revoke": 0, 2024-04-20T21:44:58.556 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_grant": 0, 2024-04-20T21:44:58.556 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T21:44:58.556 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T21:44:58.556 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T21:44:58.556 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps": 0, 2024-04-20T21:44:58.556 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps_dirty": 0, 2024-04-20T21:44:58.556 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_cap_release": 0, 2024-04-20T21:44:58.556 INFO:teuthology.orchestra.run.smithi122.stdout: "process_request_cap_release": 0 2024-04-20T21:44:58.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.557 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_cache": { 2024-04-20T21:44:58.557 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update": 0, 2024-04-20T21:44:58.557 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update_receipt": 0, 2024-04-20T21:44:58.557 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_try_discover": 0, 2024-04-20T21:44:58.557 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_send_discover": 0, 2024-04-20T21:44:58.557 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_handle_discover": 0, 2024-04-20T21:44:58.557 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays": 0, 2024-04-20T21:44:58.557 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_delayed": 0, 2024-04-20T21:44:58.557 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_enqueuing": 0, 2024-04-20T21:44:58.558 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_created": 0, 2024-04-20T21:44:58.558 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_enqueued": 0, 2024-04-20T21:44:58.558 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_reintegrated": 0, 2024-04-20T21:44:58.558 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_migrated": 0, 2024-04-20T21:44:58.558 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_processing": 0, 2024-04-20T21:44:58.558 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_enqueued": 0, 2024-04-20T21:44:58.558 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_prioritized": 0, 2024-04-20T21:44:58.558 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_started": 0, 2024-04-20T21:44:58.558 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_completed": 0, 2024-04-20T21:44:58.559 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_path": 0, 2024-04-20T21:44:58.559 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_inode": 0, 2024-04-20T21:44:58.559 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T21:44:58.559 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_exportdir": 0, 2024-04-20T21:44:58.559 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_flush": 0, 2024-04-20T21:44:58.559 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragmentdir": 0, 2024-04-20T21:44:58.559 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragstats": 0, 2024-04-20T21:44:58.559 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_inodestats": 0 2024-04-20T21:44:58.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.560 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_log": { 2024-04-20T21:44:58.560 INFO:teuthology.orchestra.run.smithi122.stdout: "evlrg": 0, 2024-04-20T21:44:58.560 INFO:teuthology.orchestra.run.smithi122.stdout: "evadd": 0, 2024-04-20T21:44:58.560 INFO:teuthology.orchestra.run.smithi122.stdout: "evex": 0, 2024-04-20T21:44:58.560 INFO:teuthology.orchestra.run.smithi122.stdout: "evtrm": 0, 2024-04-20T21:44:58.560 INFO:teuthology.orchestra.run.smithi122.stdout: "ev": 76928, 2024-04-20T21:44:58.560 INFO:teuthology.orchestra.run.smithi122.stdout: "evexg": 0, 2024-04-20T21:44:58.560 INFO:teuthology.orchestra.run.smithi122.stdout: "evexd": 0, 2024-04-20T21:44:58.561 INFO:teuthology.orchestra.run.smithi122.stdout: "segadd": 0, 2024-04-20T21:44:58.561 INFO:teuthology.orchestra.run.smithi122.stdout: "segex": 0, 2024-04-20T21:44:58.561 INFO:teuthology.orchestra.run.smithi122.stdout: "segtrm": 0, 2024-04-20T21:44:58.561 INFO:teuthology.orchestra.run.smithi122.stdout: "seg": 118, 2024-04-20T21:44:58.561 INFO:teuthology.orchestra.run.smithi122.stdout: "segmjr": 8, 2024-04-20T21:44:58.561 INFO:teuthology.orchestra.run.smithi122.stdout: "segexg": 0, 2024-04-20T21:44:58.561 INFO:teuthology.orchestra.run.smithi122.stdout: "segexd": 0, 2024-04-20T21:44:58.561 INFO:teuthology.orchestra.run.smithi122.stdout: "expos": 4194304, 2024-04-20T21:44:58.561 INFO:teuthology.orchestra.run.smithi122.stdout: "wrpos": 270928470, 2024-04-20T21:44:58.562 INFO:teuthology.orchestra.run.smithi122.stdout: "rdpos": 270928358, 2024-04-20T21:44:58.562 INFO:teuthology.orchestra.run.smithi122.stdout: "jlat": { 2024-04-20T21:44:58.562 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.562 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.562 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.562 INFO:teuthology.orchestra.run.smithi122.stdout: "replayed": 76928 2024-04-20T21:44:58.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.563 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_mem": { 2024-04-20T21:44:58.563 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 394, 2024-04-20T21:44:58.563 INFO:teuthology.orchestra.run.smithi122.stdout: "ino+": 17597, 2024-04-20T21:44:58.563 INFO:teuthology.orchestra.run.smithi122.stdout: "ino-": 17203, 2024-04-20T21:44:58.563 INFO:teuthology.orchestra.run.smithi122.stdout: "dir": 56, 2024-04-20T21:44:58.563 INFO:teuthology.orchestra.run.smithi122.stdout: "dir+": 135, 2024-04-20T21:44:58.563 INFO:teuthology.orchestra.run.smithi122.stdout: "dir-": 79, 2024-04-20T21:44:58.563 INFO:teuthology.orchestra.run.smithi122.stdout: "dn": 17669, 2024-04-20T21:44:58.563 INFO:teuthology.orchestra.run.smithi122.stdout: "dn+": 17669, 2024-04-20T21:44:58.564 INFO:teuthology.orchestra.run.smithi122.stdout: "dn-": 0, 2024-04-20T21:44:58.564 INFO:teuthology.orchestra.run.smithi122.stdout: "cap": 0, 2024-04-20T21:44:58.564 INFO:teuthology.orchestra.run.smithi122.stdout: "cap+": 0, 2024-04-20T21:44:58.564 INFO:teuthology.orchestra.run.smithi122.stdout: "cap-": 0, 2024-04-20T21:44:58.564 INFO:teuthology.orchestra.run.smithi122.stdout: "rss": 68304, 2024-04-20T21:44:58.564 INFO:teuthology.orchestra.run.smithi122.stdout: "heap": 215416 2024-04-20T21:44:58.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.564 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_server": { 2024-04-20T21:44:58.564 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_client_request": 0, 2024-04-20T21:44:58.564 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_server_request": 0, 2024-04-20T21:44:58.565 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_request": 0, 2024-04-20T21:44:58.565 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_session": 0, 2024-04-20T21:44:58.565 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_peer_request": 0, 2024-04-20T21:44:58.565 INFO:teuthology.orchestra.run.smithi122.stdout: "req_create_latency": { 2024-04-20T21:44:58.565 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.565 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.565 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.565 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getattr_latency": { 2024-04-20T21:44:58.565 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.566 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.566 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.566 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getfilelock_latency": { 2024-04-20T21:44:58.566 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.566 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.566 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.567 INFO:teuthology.orchestra.run.smithi122.stdout: "req_link_latency": { 2024-04-20T21:44:58.567 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.567 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.567 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.567 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookup_latency": { 2024-04-20T21:44:58.567 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.567 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.567 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.568 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookuphash_latency": { 2024-04-20T21:44:58.568 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.568 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.568 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.568 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupino_latency": { 2024-04-20T21:44:58.568 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.568 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.569 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupname_latency": { 2024-04-20T21:44:58.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.569 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.569 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupparent_latency": { 2024-04-20T21:44:58.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.569 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.570 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.570 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupsnap_latency": { 2024-04-20T21:44:58.570 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.570 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.570 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.570 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lssnap_latency": { 2024-04-20T21:44:58.571 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.571 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.571 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.572 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:44:58 smithi033 ceph-mon[21626]: pgmap v347: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:58.572 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mkdir_latency": { 2024-04-20T21:44:58.572 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.572 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.572 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mknod_latency": { 2024-04-20T21:44:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mksnap_latency": { 2024-04-20T21:44:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.573 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: "req_open_latency": { 2024-04-20T21:44:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: "req_readdir_latency": { 2024-04-20T21:44:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.574 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rename_latency": { 2024-04-20T21:44:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: "req_renamesnap_latency": { 2024-04-20T21:44:58.575 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: "req_snapdiff_latency": { 2024-04-20T21:44:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.576 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmdir_latency": { 2024-04-20T21:44:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmsnap_latency": { 2024-04-20T21:44:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmxattr_latency": { 2024-04-20T21:44:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setattr_latency": { 2024-04-20T21:44:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.578 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setdirlayout_latency": { 2024-04-20T21:44:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setfilelock_latency": { 2024-04-20T21:44:58.579 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setlayout_latency": { 2024-04-20T21:44:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.580 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setxattr_latency": { 2024-04-20T21:44:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: "req_symlink_latency": { 2024-04-20T21:44:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: "req_unlink_latency": { 2024-04-20T21:44:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_revoke_eviction": 0, 2024-04-20T21:44:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_acquisition_throttle": 0, 2024-04-20T21:44:58.582 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getvxattr_latency": { 2024-04-20T21:44:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_sessions": { 2024-04-20T21:44:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "session_count": 2, 2024-04-20T21:44:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "session_add": 5, 2024-04-20T21:44:58.583 INFO:teuthology.orchestra.run.smithi122.stdout: "session_remove": 3, 2024-04-20T21:44:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_open": 4, 2024-04-20T21:44:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_stale": 0, 2024-04-20T21:44:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: "total_load": 0, 2024-04-20T21:44:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: "average_load": 0, 2024-04-20T21:44:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: "avg_session_uptime": 4804, 2024-04-20T21:44:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: "mdthresh_evicted": 0 2024-04-20T21:44:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-04-20T21:44:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:58.585 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:58.586 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:58.587 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-04-20T21:44:58.588 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 256516, 2024-04-20T21:44:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 541, 2024-04-20T21:44:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-04-20T21:44:58.589 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-04-20T21:44:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 8320, 2024-04-20T21:44:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 173, 2024-04-20T21:44:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:58.590 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-04-20T21:44:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 13573823, 2024-04-20T21:44:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 81399, 2024-04-20T21:44:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-04-20T21:44:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-04-20T21:44:58.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "objecter": { 2024-04-20T21:44:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "op_active": 0, 2024-04-20T21:44:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "op_laggy": 0, 2024-04-20T21:44:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send": 2991, 2024-04-20T21:44:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send_bytes": 26, 2024-04-20T21:44:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "op_resend": 4, 2024-04-20T21:44:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "op_reply": 2991, 2024-04-20T21:44:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "op_latency": { 2024-04-20T21:44:58.592 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2991, 2024-04-20T21:44:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 2.446944520, 2024-04-20T21:44:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000818102 2024-04-20T21:44:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: "op_inflight": 0, 2024-04-20T21:44:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: "oplen_avg": { 2024-04-20T21:44:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2991, 2024-04-20T21:44:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 2992 2024-04-20T21:44:58.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: "op": 2991, 2024-04-20T21:44:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: "op_r": 2991, 2024-04-20T21:44:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: "op_w": 0, 2024-04-20T21:44:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: "op_rmw": 0, 2024-04-20T21:44:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: "op_pg": 0, 2024-04-20T21:44:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_stat": 1815, 2024-04-20T21:44:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_create": 0, 2024-04-20T21:44:58.594 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_read": 1173, 2024-04-20T21:44:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_write": 0, 2024-04-20T21:44:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writefull": 0, 2024-04-20T21:44:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writesame": 0, 2024-04-20T21:44:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_append": 0, 2024-04-20T21:44:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_zero": 0, 2024-04-20T21:44:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_truncate": 0, 2024-04-20T21:44:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_delete": 0, 2024-04-20T21:44:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_mapext": 0, 2024-04-20T21:44:58.595 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_sparse_read": 0, 2024-04-20T21:44:58.596 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_clonerange": 0, 2024-04-20T21:44:58.596 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_getxattr": 2, 2024-04-20T21:44:58.596 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_setxattr": 0, 2024-04-20T21:44:58.596 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_cmpxattr": 0, 2024-04-20T21:44:58.596 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_rmxattr": 0, 2024-04-20T21:44:58.596 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_resetxattrs": 0, 2024-04-20T21:44:58.596 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_call": 0, 2024-04-20T21:44:58.596 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_watch": 0, 2024-04-20T21:44:58.596 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_notify": 0, 2024-04-20T21:44:58.597 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T21:44:58.597 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls": 0, 2024-04-20T21:44:58.597 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls_filter": 0, 2024-04-20T21:44:58.597 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_other": 0, 2024-04-20T21:44:58.597 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_active": 0, 2024-04-20T21:44:58.597 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_send": 0, 2024-04-20T21:44:58.597 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_resend": 0, 2024-04-20T21:44:58.597 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_ping": 0, 2024-04-20T21:44:58.598 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_active": 0, 2024-04-20T21:44:58.598 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_send": 0, 2024-04-20T21:44:58.598 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_resend": 0, 2024-04-20T21:44:58.598 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_active": 0, 2024-04-20T21:44:58.598 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_send": 0, 2024-04-20T21:44:58.598 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_resend": 0, 2024-04-20T21:44:58.598 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_active": 0, 2024-04-20T21:44:58.598 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_send": 0, 2024-04-20T21:44:58.599 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_resend": 0, 2024-04-20T21:44:58.599 INFO:teuthology.orchestra.run.smithi122.stdout: "command_active": 0, 2024-04-20T21:44:58.599 INFO:teuthology.orchestra.run.smithi122.stdout: "command_send": 0, 2024-04-20T21:44:58.599 INFO:teuthology.orchestra.run.smithi122.stdout: "command_resend": 0, 2024-04-20T21:44:58.599 INFO:teuthology.orchestra.run.smithi122.stdout: "map_epoch": 103, 2024-04-20T21:44:58.599 INFO:teuthology.orchestra.run.smithi122.stdout: "map_full": 0, 2024-04-20T21:44:58.599 INFO:teuthology.orchestra.run.smithi122.stdout: "map_inc": 15, 2024-04-20T21:44:58.599 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_sessions": 11, 2024-04-20T21:44:58.599 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_open": 11, 2024-04-20T21:44:58.600 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_close": 0, 2024-04-20T21:44:58.600 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_laggy": 0, 2024-04-20T21:44:58.600 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_wr": 0, 2024-04-20T21:44:58.600 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_rd": 2, 2024-04-20T21:44:58.600 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_del": 0 2024-04-20T21:44:58.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.600 INFO:teuthology.orchestra.run.smithi122.stdout: "oft": { 2024-04-20T21:44:58.600 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_objs": 0, 2024-04-20T21:44:58.600 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_kv_pairs": 0, 2024-04-20T21:44:58.601 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_updates": 0, 2024-04-20T21:44:58.601 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_removes": 0 2024-04-20T21:44:58.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.601 INFO:teuthology.orchestra.run.smithi122.stdout: "osd-slow-ops": { 2024-04-20T21:44:58.601 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_ops_count": 0 2024-04-20T21:44:58.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.601 INFO:teuthology.orchestra.run.smithi122.stdout: "purge_queue": { 2024-04-20T21:44:58.601 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops": 0, 2024-04-20T21:44:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops_high_water": 0, 2024-04-20T21:44:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing": 0, 2024-04-20T21:44:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_high_water": 0, 2024-04-20T21:44:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed_ops": 0, 2024-04-20T21:44:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed": 0, 2024-04-20T21:44:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_item_in_journal": 0 2024-04-20T21:44:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.602 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-20T21:44:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-20T21:44:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-20T21:44:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 3250, 2024-04-20T21:44:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 269018154, 2024-04-20T21:44:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 3250, 2024-04-20T21:44:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-20T21:44:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-20T21:44:58.603 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 3250, 2024-04-20T21:44:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 269018154, 2024-04-20T21:44:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-20T21:44:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.604 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_bytes": { 2024-04-20T21:44:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-20T21:44:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-20T21:44:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-20T21:44:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-20T21:44:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-20T21:44:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 2991, 2024-04-20T21:44:58.605 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 266740380, 2024-04-20T21:44:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 2991, 2024-04-20T21:44:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 266740380, 2024-04-20T21:44:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-20T21:44:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:58.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_ops": { 2024-04-20T21:44:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-20T21:44:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1024, 2024-04-20T21:44:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-20T21:44:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-20T21:44:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-20T21:44:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 2991, 2024-04-20T21:44:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 2991, 2024-04-20T21:44:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 2991, 2024-04-20T21:44:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 2991, 2024-04-20T21:44:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-20T21:44:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:58.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle": { 2024-04-20T21:44:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-20T21:44:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-20T21:44:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-20T21:44:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-20T21:44:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-20T21:44:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:58.609 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-20T21:44:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-20T21:44:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-04-20T21:44:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-04-20T21:44:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-20T21:44:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:58.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:44:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle-0x55bc67fcc6e0": { 2024-04-20T21:44:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-20T21:44:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-20T21:44:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-20T21:44:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-20T21:44:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-20T21:44:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:58.611 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:58.612 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-20T21:44:58.612 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-20T21:44:58.612 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-04-20T21:44:58.612 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-04-20T21:44:58.612 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-20T21:44:58.612 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:44:58.612 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:44:58.612 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:44:58.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:58.613 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:44:58.613 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-20T21:44:58.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:44:58 smithi022 ceph-mon[16843]: pgmap v347: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:44:59.391 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-20T21:44:59.392 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-20T21:44:59.392 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T21:44:59.392 DEBUG:tasks.check_counter:Getting stats from l 2024-04-20T21:44:59.392 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-20T21:44:59.629 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:45:00.478 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:00 smithi122 ceph-mon[22147]: pgmap v348: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:45:00.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:00 smithi033 ceph-mon[21626]: pgmap v348: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:45:00.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:00 smithi022 ceph-mon[16843]: pgmap v348: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:45:01.317 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-20T21:45:01.317 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:45:01.317 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 245, 2024-04-20T21:45:01.317 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 231, 2024-04-20T21:45:01.317 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 1736384, 2024-04-20T21:45:01.317 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 141792, 2024-04-20T21:45:01.318 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-04-20T21:45:01.318 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-04-20T21:45:01.318 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.046857708, 2024-04-20T21:45:01.318 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.013834728, 2024-04-20T21:45:01.318 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.036149981, 2024-04-20T21:45:01.318 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.005485725, 2024-04-20T21:45:01.318 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:45:01.318 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 231, 2024-04-20T21:45:01.319 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.011651138, 2024-04-20T21:45:01.319 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000050437 2024-04-20T21:45:01.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:45:01.319 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:45:01.319 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:45:01.319 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:45:01.319 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:45:01.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:45:01.319 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1736384, 2024-04-20T21:45:01.319 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 141792 2024-04-20T21:45:01.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:45:01.320 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:45:01.320 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 3, 2024-04-20T21:45:01.320 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 177, 2024-04-20T21:45:01.320 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 480, 2024-04-20T21:45:01.320 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 265408, 2024-04-20T21:45:01.320 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 6, 2024-04-20T21:45:01.320 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 3, 2024-04-20T21:45:01.320 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.011586991, 2024-04-20T21:45:01.321 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.007654402, 2024-04-20T21:45:01.321 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.001586498, 2024-04-20T21:45:01.321 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:45:01.321 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:45:01.321 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 173, 2024-04-20T21:45:01.321 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.029588755, 2024-04-20T21:45:01.321 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000171033 2024-04-20T21:45:01.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:45:01.321 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:45:01.321 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:45:01.322 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:45:01.322 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:45:01.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:45:01.322 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-20T21:45:01.322 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 265408 2024-04-20T21:45:01.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:45:01.322 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:45:01.322 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 0, 2024-04-20T21:45:01.322 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2, 2024-04-20T21:45:01.322 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:45:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 0, 2024-04-20T21:45:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-04-20T21:45:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-04-20T21:45:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.000219028, 2024-04-20T21:45:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:45:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.000166917, 2024-04-20T21:45:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:45:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:45:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:45:01.324 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:45:01.324 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:45:01.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:45:01.324 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:45:01.324 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:45:01.324 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:45:01.324 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:45:01.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:45:01.324 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:45:01.324 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:45:01.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:45:01.325 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-04-20T21:45:01.325 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-04-20T21:45:01.325 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-04-20T21:45:01.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:45:01.325 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-04-20T21:45:01.325 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:45:01.325 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-04-20T21:45:01.325 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:45:01.326 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:45:01.326 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:45:01.326 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:45:01.326 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:45:01.326 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:45:01.326 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:45:01.326 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:45:01.326 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:45:01.327 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:45:01.327 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:45:01.327 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:45:01.327 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:45:01.327 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-04-20T21:45:01.327 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:45:01.327 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-04-20T21:45:01.327 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:45:01.327 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:45:01.327 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:45:01.328 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:45:01.328 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:45:01.328 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:45:01.328 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:45:01.328 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-04-20T21:45:01.328 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:45:01.328 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:45:01.328 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:45:01.328 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-04-20T21:45:01.329 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-04-20T21:45:01.329 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-04-20T21:45:01.329 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:45:01.329 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:45:01.329 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:45:01.329 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:45:01.329 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 148732, 2024-04-20T21:45:01.329 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 31, 2024-04-20T21:45:01.329 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:45:01.330 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-04-20T21:45:01.330 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-04-20T21:45:01.330 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-04-20T21:45:01.330 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:45:01.330 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-04-20T21:45:01.330 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:45:01.330 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-04-20T21:45:01.330 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-04-20T21:45:01.330 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-04-20T21:45:01.331 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:45:01.331 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:45:01.331 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-04-20T21:45:01.331 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-04-20T21:45:01.331 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-04-20T21:45:01.331 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-04-20T21:45:01.331 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-04-20T21:45:01.331 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-04-20T21:45:01.331 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-04-20T21:45:01.332 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-04-20T21:45:01.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-20T21:45:01.332 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:45:01.332 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-20T21:45:01.332 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-20T21:45:01.332 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-20T21:45:01.332 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 244, 2024-04-20T21:45:01.332 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 1699431, 2024-04-20T21:45:01.332 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-20T21:45:01.332 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 244, 2024-04-20T21:45:01.333 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-20T21:45:01.333 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-20T21:45:01.333 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 244, 2024-04-20T21:45:01.333 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1699431, 2024-04-20T21:45:01.333 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-20T21:45:01.333 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-20T21:45:01.333 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-20T21:45:01.333 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-20T21:45:01.333 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:45:01.333 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-20T21:45:01.334 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-20T21:45:01.999 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-20T21:45:01.999 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-20T21:45:01.999 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-20T21:45:02.000 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-20T21:45:02.012 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-20T21:45:02.012 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:45:02.012 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:45:02.074 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-04-20T21:45:02.074 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:45:02.129 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:45:02.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.354+0000 7fbc1b200640 1 -- 172.21.15.22:0/1555386202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c108400 msgr2=0x7fbc1c108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.354+0000 7fbc1b200640 1 --2- 172.21.15.22:0/1555386202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c108400 0x7fbc1c108860 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009900 tx=0x7fbc0402bcc0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.354+0000 7fbc1b200640 1 -- 172.21.15.22:0/1555386202 shutdown_connections 2024-04-20T21:45:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.354+0000 7fbc1b200640 1 --2- 172.21.15.22:0/1555386202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc1c108da0 0x7fbc1c115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.354+0000 7fbc1b200640 1 --2- 172.21.15.22:0/1555386202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c108400 0x7fbc1c108860 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.354+0000 7fbc1b200640 1 --2- 172.21.15.22:0/1555386202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c10e470 0x7fbc1c10e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.354+0000 7fbc1b200640 1 -- 172.21.15.22:0/1555386202 >> 172.21.15.22:0/1555386202 conn(0x7fbc1c077ae0 msgr2=0x7fbc1c079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.354+0000 7fbc1b200640 1 -- 172.21.15.22:0/1555386202 shutdown_connections 2024-04-20T21:45:02.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.354+0000 7fbc1b200640 1 -- 172.21.15.22:0/1555386202 wait complete. 2024-04-20T21:45:02.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.356+0000 7fbc1b200640 1 Processor -- start 2024-04-20T21:45:02.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.356+0000 7fbc1b200640 1 -- start start 2024-04-20T21:45:02.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.356+0000 7fbc1b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c108400 0x7fbc1c1a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:02.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.356+0000 7fbc1b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c108da0 0x7fbc1c1a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:02.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.356+0000 7fbc1b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc1c10e470 0x7fbc1c19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:02.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.356+0000 7fbc1b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1187a0 con 0x7fbc1c108400 2024-04-20T21:45:02.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.356+0000 7fbc1b200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c118620 con 0x7fbc1c108da0 2024-04-20T21:45:02.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.356+0000 7fbc1b200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c118920 con 0x7fbc1c10e470 2024-04-20T21:45:02.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.357+0000 7fbc19400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c108da0 0x7fbc1c1a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:02.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.357+0000 7fbc19400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c108da0 0x7fbc1c1a4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50642/0 (socket says 172.21.15.22:50642) 2024-04-20T21:45:02.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.357+0000 7fbc19400640 1 -- 172.21.15.22:0/71088298 learned_addr learned my addr 172.21.15.22:0/71088298 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:45:02.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.357+0000 7fbc1a800640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc1c10e470 0x7fbc1c19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:02.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.357+0000 7fbc19e00640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c108400 0x7fbc1c1a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:02.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.357+0000 7fbc19400640 1 -- 172.21.15.22:0/71088298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc1c10e470 msgr2=0x7fbc1c19ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:02.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.357+0000 7fbc19400640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc1c10e470 0x7fbc1c19ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.357+0000 7fbc19400640 1 -- 172.21.15.22:0/71088298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c108400 msgr2=0x7fbc1c1a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:02.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.357+0000 7fbc19400640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c108400 0x7fbc1c1a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.357+0000 7fbc19400640 1 -- 172.21.15.22:0/71088298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c19f180 con 0x7fbc1c108da0 2024-04-20T21:45:02.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.358+0000 7fbc19400640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c108da0 0x7fbc1c1a4d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbc04000c00 tx=0x7fbc04004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:02.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.358+0000 7fbc12a00640 1 -- 172.21.15.22:0/71088298 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04042020 con 0x7fbc1c108da0 2024-04-20T21:45:02.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.358+0000 7fbc12a00640 1 -- 172.21.15.22:0/71088298 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc04005000 con 0x7fbc1c108da0 2024-04-20T21:45:02.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.358+0000 7fbc12a00640 1 -- 172.21.15.22:0/71088298 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04003b20 con 0x7fbc1c108da0 2024-04-20T21:45:02.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.358+0000 7fbc1b200640 1 -- 172.21.15.22:0/71088298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1c19f410 con 0x7fbc1c108da0 2024-04-20T21:45:02.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.358+0000 7fbc1b200640 1 -- 172.21.15.22:0/71088298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1c1ab710 con 0x7fbc1c108da0 2024-04-20T21:45:02.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.359+0000 7fbc1b200640 1 -- 172.21.15.22:0/71088298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1c0690f0 con 0x7fbc1c108da0 2024-04-20T21:45:02.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.361+0000 7fbc12a00640 1 -- 172.21.15.22:0/71088298 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbc04045020 con 0x7fbc1c108da0 2024-04-20T21:45:02.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.362+0000 7fbc12a00640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbc000a3d20 0x7fbc000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:02.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.363+0000 7fbc12a00640 1 -- 172.21.15.22:0/71088298 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7fbc040f2350 con 0x7fbc1c108da0 2024-04-20T21:45:02.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.363+0000 7fbc19e00640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbc000a3d20 0x7fbc000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:02.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.363+0000 7fbc19e00640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbc000a3d20 0x7fbc000a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c19fb40 tx=0x7fbc0c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:02.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.367+0000 7fbc12a00640 1 -- 172.21.15.22:0/71088298 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbc0404c020 con 0x7fbc1c108da0 2024-04-20T21:45:02.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:02 smithi122 ceph-mon[22147]: pgmap v349: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:45:02.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:02 smithi033 ceph-mon[21626]: pgmap v349: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:45:02.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.579+0000 7fbc1b200640 1 -- 172.21.15.22:0/71088298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc1c110890 con 0x7fbc1c108da0 2024-04-20T21:45:02.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.582+0000 7fbc12a00640 1 -- 172.21.15.22:0/71088298 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23239 (secure 0 0 0) 0x7fbc040bea50 con 0x7fbc1c108da0 2024-04-20T21:45:02.581 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:45:02.581 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":103,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:44:14.296211+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/1815958949":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/3359736029":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/3485677447":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/1911146506":"2024-04-21T21:33:48.815984+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1408863783":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/1911146506":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:45:02.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.585+0000 7fbc1b200640 1 -- 172.21.15.22:0/71088298 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbc000a3d20 msgr2=0x7fbc000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:02.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.585+0000 7fbc1b200640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbc000a3d20 0x7fbc000a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c19fb40 tx=0x7fbc0c005f90 comp rx=0 tx=0).stop 2024-04-20T21:45:02.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.585+0000 7fbc1b200640 1 -- 172.21.15.22:0/71088298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c108da0 msgr2=0x7fbc1c1a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:02.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.585+0000 7fbc1b200640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c108da0 0x7fbc1c1a4d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbc04000c00 tx=0x7fbc04004360 comp rx=0 tx=0).stop 2024-04-20T21:45:02.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.585+0000 7fbc1b200640 1 -- 172.21.15.22:0/71088298 shutdown_connections 2024-04-20T21:45:02.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.585+0000 7fbc1b200640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fbc000a3d20 0x7fbc000a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.585+0000 7fbc1b200640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc1c10e470 0x7fbc1c19ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.585+0000 7fbc1b200640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c108da0 0x7fbc1c1a4d90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.585+0000 7fbc1b200640 1 --2- 172.21.15.22:0/71088298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc1c108400 0x7fbc1c1a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.585+0000 7fbc1b200640 1 -- 172.21.15.22:0/71088298 >> 172.21.15.22:0/71088298 conn(0x7fbc1c077ae0 msgr2=0x7fbc1c0790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:02.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.585+0000 7fbc1b200640 1 -- 172.21.15.22:0/71088298 shutdown_connections 2024-04-20T21:45:02.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.585+0000 7fbc1b200640 1 -- 172.21.15.22:0/71088298 wait complete. 2024-04-20T21:45:02.601 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:02 smithi022 ceph-mon[16843]: pgmap v349: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T21:45:02.605 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:26:15.688818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:31:25.506074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:31:26.126261+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:45:02.605 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:45:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.820+0000 7fd6d6200640 1 -- 172.21.15.22:0/2301699629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d0109280 msgr2=0x7fd6d0110e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:02.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.820+0000 7fd6d6200640 1 --2- 172.21.15.22:0/2301699629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d0109280 0x7fd6d0110e50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009ca0 tx=0x7fd6c802c100 comp rx=0 tx=0).stop 2024-04-20T21:45:02.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.821+0000 7fd6d6200640 1 -- 172.21.15.22:0/2301699629 shutdown_connections 2024-04-20T21:45:02.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.821+0000 7fd6d6200640 1 --2- 172.21.15.22:0/2301699629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d0109280 0x7fd6d0110e50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.821+0000 7fd6d6200640 1 --2- 172.21.15.22:0/2301699629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d0104b60 0x7fd6d0108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.821+0000 7fd6d6200640 1 --2- 172.21.15.22:0/2301699629 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d01041b0 0x7fd6d0104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.821+0000 7fd6d6200640 1 -- 172.21.15.22:0/2301699629 >> 172.21.15.22:0/2301699629 conn(0x7fd6d00ff9d0 msgr2=0x7fd6d0101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:02.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.821+0000 7fd6d6200640 1 -- 172.21.15.22:0/2301699629 shutdown_connections 2024-04-20T21:45:02.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.821+0000 7fd6d6200640 1 -- 172.21.15.22:0/2301699629 wait complete. 2024-04-20T21:45:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.822+0000 7fd6d6200640 1 Processor -- start 2024-04-20T21:45:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.822+0000 7fd6d6200640 1 -- start start 2024-04-20T21:45:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.822+0000 7fd6d6200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d01041b0 0x7fd6d01a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.822+0000 7fd6d6200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d0109280 0x7fd6d01a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.822+0000 7fd6d6200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d019a620 0x7fd6d019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.822+0000 7fd6d6200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0114340 con 0x7fd6d01041b0 2024-04-20T21:45:02.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.822+0000 7fd6d6200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01141c0 con 0x7fd6d0109280 2024-04-20T21:45:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.822+0000 7fd6d6200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01144c0 con 0x7fd6d019a620 2024-04-20T21:45:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.823+0000 7fd6d4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d01041b0 0x7fd6d01a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.823+0000 7fd6d4e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d01041b0 0x7fd6d01a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40704/0 (socket says 172.21.15.22:40704) 2024-04-20T21:45:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.823+0000 7fd6d4e00640 1 -- 172.21.15.22:0/2551766639 learned_addr learned my addr 172.21.15.22:0/2551766639 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:45:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.823+0000 7fd6cfe00640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d0109280 0x7fd6d01a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.823+0000 7fd6d5800640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d019a620 0x7fd6d019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.823+0000 7fd6d4e00640 1 -- 172.21.15.22:0/2551766639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d019a620 msgr2=0x7fd6d019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:02.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.823+0000 7fd6d4e00640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d019a620 0x7fd6d019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.823+0000 7fd6d4e00640 1 -- 172.21.15.22:0/2551766639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d0109280 msgr2=0x7fd6d01a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:02.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.823+0000 7fd6d4e00640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d0109280 0x7fd6d01a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:02.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.823+0000 7fd6d4e00640 1 -- 172.21.15.22:0/2551766639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d019b140 con 0x7fd6d01041b0 2024-04-20T21:45:02.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.824+0000 7fd6d4e00640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d01041b0 0x7fd6d01a0470 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000e9f0 tx=0x7fd6c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:02.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.824+0000 7fd6cd600640 1 -- 172.21.15.22:0/2551766639 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c000cd80 con 0x7fd6d01041b0 2024-04-20T21:45:02.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.824+0000 7fd6cd600640 1 -- 172.21.15.22:0/2551766639 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6c0005500 con 0x7fd6d01041b0 2024-04-20T21:45:02.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.824+0000 7fd6d6200640 1 -- 172.21.15.22:0/2551766639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d019b430 con 0x7fd6d01041b0 2024-04-20T21:45:02.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.824+0000 7fd6cd600640 1 -- 172.21.15.22:0/2551766639 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0010690 con 0x7fd6d01041b0 2024-04-20T21:45:02.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.824+0000 7fd6d6200640 1 -- 172.21.15.22:0/2551766639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d01a7630 con 0x7fd6d01041b0 2024-04-20T21:45:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.825+0000 7fd6d6200640 1 -- 172.21.15.22:0/2551766639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d0068ca0 con 0x7fd6d01041b0 2024-04-20T21:45:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.827+0000 7fd6cd600640 1 -- 172.21.15.22:0/2551766639 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd6c00046c0 con 0x7fd6d01041b0 2024-04-20T21:45:02.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.829+0000 7fd6cd600640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd6b40a3d20 0x7fd6b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:02.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.830+0000 7fd6cd600640 1 -- 172.21.15.22:0/2551766639 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7fd6c00d0c60 con 0x7fd6d01041b0 2024-04-20T21:45:02.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.830+0000 7fd6cfe00640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd6b40a3d20 0x7fd6b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:02.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.831+0000 7fd6cfe00640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd6b40a3d20 0x7fd6b40a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd6d010a890 tx=0x7fd6bc005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:02.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:02.834+0000 7fd6cd600640 1 -- 172.21.15.22:0/2551766639 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd6c009d2b0 con 0x7fd6d01041b0 2024-04-20T21:45:03.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.040+0000 7fd6d6200640 1 -- 172.21.15.22:0/2551766639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6d0104b60 con 0x7fd6d01041b0 2024-04-20T21:45:03.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.042+0000 7fd6cd600640 1 -- 172.21.15.22:0/2551766639 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7fd6c009cbb0 con 0x7fd6d01041b0 2024-04-20T21:45:03.041 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-04-20T21:45:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.045+0000 7fd6d6200640 1 -- 172.21.15.22:0/2551766639 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd6b40a3d20 msgr2=0x7fd6b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.045+0000 7fd6d6200640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd6b40a3d20 0x7fd6b40a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd6d010a890 tx=0x7fd6bc005f90 comp rx=0 tx=0).stop 2024-04-20T21:45:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.045+0000 7fd6d6200640 1 -- 172.21.15.22:0/2551766639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d01041b0 msgr2=0x7fd6d01a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.045+0000 7fd6d6200640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d01041b0 0x7fd6d01a0470 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000e9f0 tx=0x7fd6c000edf0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.045+0000 7fd6d6200640 1 -- 172.21.15.22:0/2551766639 shutdown_connections 2024-04-20T21:45:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.045+0000 7fd6d6200640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fd6b40a3d20 0x7fd6b40a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.045+0000 7fd6d6200640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d019a620 0x7fd6d019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.045+0000 7fd6d6200640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d0109280 0x7fd6d01a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.045+0000 7fd6d6200640 1 --2- 172.21.15.22:0/2551766639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6d01041b0 0x7fd6d01a0470 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.045+0000 7fd6d6200640 1 -- 172.21.15.22:0/2551766639 >> 172.21.15.22:0/2551766639 conn(0x7fd6d00ff9d0 msgr2=0x7fd6d0101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.045+0000 7fd6d6200640 1 -- 172.21.15.22:0/2551766639 shutdown_connections 2024-04-20T21:45:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.045+0000 7fd6d6200640 1 -- 172.21.15.22:0/2551766639 wait complete. 2024-04-20T21:45:03.065 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:45:03.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.276+0000 7f756b000640 1 -- 172.21.15.22:0/3705850047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75641021f0 msgr2=0x7f75641025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.276+0000 7f756b000640 1 --2- 172.21.15.22:0/3705850047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75641021f0 0x7f75641025d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7554009900 tx=0x7f755402bc40 comp rx=0 tx=0).stop 2024-04-20T21:45:03.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.277+0000 7f756b000640 1 -- 172.21.15.22:0/3705850047 shutdown_connections 2024-04-20T21:45:03.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.277+0000 7f756b000640 1 --2- 172.21.15.22:0/3705850047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75641072c0 0x7f756410ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.277+0000 7f756b000640 1 --2- 172.21.15.22:0/3705850047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7564102ba0 0x7f7564106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.277+0000 7f756b000640 1 --2- 172.21.15.22:0/3705850047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75641021f0 0x7f75641025d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.277+0000 7f756b000640 1 -- 172.21.15.22:0/3705850047 >> 172.21.15.22:0/3705850047 conn(0x7f75640fda70 msgr2=0x7f75640ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:03.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.277+0000 7f756b000640 1 -- 172.21.15.22:0/3705850047 shutdown_connections 2024-04-20T21:45:03.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.277+0000 7f756b000640 1 -- 172.21.15.22:0/3705850047 wait complete. 2024-04-20T21:45:03.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.278+0000 7f756b000640 1 Processor -- start 2024-04-20T21:45:03.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.278+0000 7f756b000640 1 -- start start 2024-04-20T21:45:03.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.279+0000 7f756b000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75641021f0 0x7f75641a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.279+0000 7f756b000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75641072c0 0x7f75641a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.279+0000 7f756b000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f756419a960 0x7f756419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.279+0000 7f756b000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f75641123a0 con 0x7f75641021f0 2024-04-20T21:45:03.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.279+0000 7f756b000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7564112220 con 0x7f75641072c0 2024-04-20T21:45:03.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.279+0000 7f756b000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7564112520 con 0x7f756419a960 2024-04-20T21:45:03.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.279+0000 7f7563e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75641021f0 0x7f75641a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.279+0000 7f755be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75641072c0 0x7f75641a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.279+0000 7f755be00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75641072c0 0x7f75641a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50668/0 (socket says 172.21.15.22:50668) 2024-04-20T21:45:03.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.279+0000 7f755be00640 1 -- 172.21.15.22:0/1032245519 learned_addr learned my addr 172.21.15.22:0/1032245519 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:45:03.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.279+0000 7f7569000640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f756419a960 0x7f756419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.280+0000 7f755be00640 1 -- 172.21.15.22:0/1032245519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f756419a960 msgr2=0x7f756419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.280+0000 7f755be00640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f756419a960 0x7f756419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.280+0000 7f755be00640 1 -- 172.21.15.22:0/1032245519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75641021f0 msgr2=0x7f75641a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.280+0000 7f755be00640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75641021f0 0x7f75641a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.280+0000 7f755be00640 1 -- 172.21.15.22:0/1032245519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756419b480 con 0x7f75641072c0 2024-04-20T21:45:03.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.280+0000 7f755be00640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75641072c0 0x7f75641a0cf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f754c00d770 tx=0x7f754c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:03.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.280+0000 7f7561600640 1 -- 172.21.15.22:0/1032245519 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c017070 con 0x7f75641072c0 2024-04-20T21:45:03.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.280+0000 7f7561600640 1 -- 172.21.15.22:0/1032245519 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f754c009c30 con 0x7f75641072c0 2024-04-20T21:45:03.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.280+0000 7f756b000640 1 -- 172.21.15.22:0/1032245519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756419b770 con 0x7f75641072c0 2024-04-20T21:45:03.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.280+0000 7f7561600640 1 -- 172.21.15.22:0/1032245519 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c00f620 con 0x7f75641072c0 2024-04-20T21:45:03.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.281+0000 7f756b000640 1 -- 172.21.15.22:0/1032245519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75641a7a10 con 0x7f75641072c0 2024-04-20T21:45:03.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.282+0000 7f756b000640 1 -- 172.21.15.22:0/1032245519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75640691d0 con 0x7f75641072c0 2024-04-20T21:45:03.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.284+0000 7f7561600640 1 -- 172.21.15.22:0/1032245519 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f754c01a070 con 0x7f75641072c0 2024-04-20T21:45:03.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.285+0000 7f7561600640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f75480a3da0 0x7f75480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.285+0000 7f7561600640 1 -- 172.21.15.22:0/1032245519 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f754c0d02a0 con 0x7f75641072c0 2024-04-20T21:45:03.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.285+0000 7f7563e00640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f75480a3da0 0x7f75480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.286+0000 7f7563e00640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f75480a3da0 0x7f75480a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f755402efd0 tx=0x7f755402cf90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:03.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.290+0000 7f7561600640 1 -- 172.21.15.22:0/1032245519 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f754c009de0 con 0x7f75641072c0 2024-04-20T21:45:03.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:03 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/71088298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:45:03.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:03 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2551766639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:45:03.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.502+0000 7f756b000640 1 -- 172.21.15.22:0/1032245519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7564102ba0 con 0x7f75641072c0 2024-04-20T21:45:03.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.503+0000 7f7561600640 1 -- 172.21.15.22:0/1032245519 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7f754c09c240 con 0x7f75641072c0 2024-04-20T21:45:03.502 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-04-20T21:45:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.506+0000 7f756b000640 1 -- 172.21.15.22:0/1032245519 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f75480a3da0 msgr2=0x7f75480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.506+0000 7f756b000640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f75480a3da0 0x7f75480a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f755402efd0 tx=0x7f755402cf90 comp rx=0 tx=0).stop 2024-04-20T21:45:03.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.506+0000 7f756b000640 1 -- 172.21.15.22:0/1032245519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75641072c0 msgr2=0x7f75641a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.506+0000 7f756b000640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75641072c0 0x7f75641a0cf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f754c00d770 tx=0x7f754c00dc30 comp rx=0 tx=0).stop 2024-04-20T21:45:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.506+0000 7f756b000640 1 -- 172.21.15.22:0/1032245519 shutdown_connections 2024-04-20T21:45:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.506+0000 7f756b000640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f75480a3da0 0x7f75480a6260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.506+0000 7f756b000640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f756419a960 0x7f756419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.506+0000 7f756b000640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75641072c0 0x7f75641a0cf0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.506+0000 7f756b000640 1 --2- 172.21.15.22:0/1032245519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75641021f0 0x7f75641a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.506+0000 7f756b000640 1 -- 172.21.15.22:0/1032245519 >> 172.21.15.22:0/1032245519 conn(0x7f75640fda70 msgr2=0x7f75640ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.506+0000 7f756b000640 1 -- 172.21.15.22:0/1032245519 shutdown_connections 2024-04-20T21:45:03.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.506+0000 7f756b000640 1 -- 172.21.15.22:0/1032245519 wait complete. 2024-04-20T21:45:03.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:45:03.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:03 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/71088298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:45:03.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:03 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2551766639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.744+0000 7fca43200640 1 -- 172.21.15.22:0/1447042527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca44109270 msgr2=0x7fca44110e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.744+0000 7fca43200640 1 --2- 172.21.15.22:0/1447042527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca44109270 0x7fca44110e40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009900 tx=0x7fca3c02bc40 comp rx=0 tx=0).stop 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.744+0000 7fca43200640 1 -- 172.21.15.22:0/1447042527 shutdown_connections 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.744+0000 7fca43200640 1 --2- 172.21.15.22:0/1447042527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca44109270 0x7fca44110e40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.744+0000 7fca43200640 1 --2- 172.21.15.22:0/1447042527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca44104b50 0x7fca44108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.744+0000 7fca43200640 1 --2- 172.21.15.22:0/1447042527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca441041a0 0x7fca44104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.745+0000 7fca43200640 1 -- 172.21.15.22:0/1447042527 >> 172.21.15.22:0/1447042527 conn(0x7fca440ffa00 msgr2=0x7fca44101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.745+0000 7fca43200640 1 -- 172.21.15.22:0/1447042527 shutdown_connections 2024-04-20T21:45:03.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.745+0000 7fca43200640 1 -- 172.21.15.22:0/1447042527 wait complete. 2024-04-20T21:45:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.746+0000 7fca43200640 1 Processor -- start 2024-04-20T21:45:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.746+0000 7fca43200640 1 -- start start 2024-04-20T21:45:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.746+0000 7fca43200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca441041a0 0x7fca441a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.746+0000 7fca43200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca44109270 0x7fca441a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.746+0000 7fca43200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca4419a5e0 0x7fca4419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.746+0000 7fca43200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fca44114360 con 0x7fca4419a5e0 2024-04-20T21:45:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.746+0000 7fca43200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca441141e0 con 0x7fca441041a0 2024-04-20T21:45:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.746+0000 7fca43200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca441144e0 con 0x7fca44109270 2024-04-20T21:45:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.746+0000 7fca42800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca4419a5e0 0x7fca4419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.746+0000 7fca42800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca4419a5e0 0x7fca4419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40744/0 (socket says 172.21.15.22:40744) 2024-04-20T21:45:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.747+0000 7fca42800640 1 -- 172.21.15.22:0/3241992148 learned_addr learned my addr 172.21.15.22:0/3241992148 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:45:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.747+0000 7fca41e00640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca441041a0 0x7fca441a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.747+0000 7fca41400640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca44109270 0x7fca441a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.747+0000 7fca42800640 1 -- 172.21.15.22:0/3241992148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca44109270 msgr2=0x7fca441a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.747+0000 7fca42800640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca44109270 0x7fca441a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.747+0000 7fca42800640 1 -- 172.21.15.22:0/3241992148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca441041a0 msgr2=0x7fca441a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.747+0000 7fca42800640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca441041a0 0x7fca441a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.747+0000 7fca42800640 1 -- 172.21.15.22:0/3241992148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4419b100 con 0x7fca4419a5e0 2024-04-20T21:45:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.747+0000 7fca42800640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca4419a5e0 0x7fca4419aa90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fca3c02efd0 tx=0x7fca3c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.748+0000 7fca3aa00640 1 -- 172.21.15.22:0/3241992148 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c041070 con 0x7fca4419a5e0 2024-04-20T21:45:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.748+0000 7fca43200640 1 -- 172.21.15.22:0/3241992148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4419b390 con 0x7fca4419a5e0 2024-04-20T21:45:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.748+0000 7fca3aa00640 1 -- 172.21.15.22:0/3241992148 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca3c02cca0 con 0x7fca4419a5e0 2024-04-20T21:45:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.748+0000 7fca3aa00640 1 -- 172.21.15.22:0/3241992148 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c003dc0 con 0x7fca4419a5e0 2024-04-20T21:45:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.748+0000 7fca43200640 1 -- 172.21.15.22:0/3241992148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca441a7580 con 0x7fca4419a5e0 2024-04-20T21:45:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.749+0000 7fca43200640 1 -- 172.21.15.22:0/3241992148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca440690f0 con 0x7fca4419a5e0 2024-04-20T21:45:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.751+0000 7fca3aa00640 1 -- 172.21.15.22:0/3241992148 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fca3c044070 con 0x7fca4419a5e0 2024-04-20T21:45:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.752+0000 7fca3aa00640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fca280a3d20 0x7fca280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.753+0000 7fca3aa00640 1 -- 172.21.15.22:0/3241992148 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7fca3c0f21c0 con 0x7fca4419a5e0 2024-04-20T21:45:03.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.753+0000 7fca41e00640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fca280a3d20 0x7fca280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:03.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.754+0000 7fca41e00640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fca280a3d20 0x7fca280a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009bd0 tx=0x7fca2c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.757+0000 7fca3aa00640 1 -- 172.21.15.22:0/3241992148 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fca3c0be270 con 0x7fca4419a5e0 2024-04-20T21:45:03.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:03 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/71088298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:45:03.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:03 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2551766639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:45:03.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.964+0000 7fca43200640 1 -- 172.21.15.22:0/3241992148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fca44104b50 con 0x7fca4419a5e0 2024-04-20T21:45:03.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.965+0000 7fca3aa00640 1 -- 172.21.15.22:0/3241992148 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7fca3c0c2570 con 0x7fca4419a5e0 2024-04-20T21:45:03.964 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-04-20T21:45:03.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.967+0000 7fca43200640 1 -- 172.21.15.22:0/3241992148 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fca280a3d20 msgr2=0x7fca280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.967+0000 7fca43200640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fca280a3d20 0x7fca280a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009bd0 tx=0x7fca2c005f90 comp rx=0 tx=0).stop 2024-04-20T21:45:03.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.968+0000 7fca43200640 1 -- 172.21.15.22:0/3241992148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca4419a5e0 msgr2=0x7fca4419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:03.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.968+0000 7fca43200640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca4419a5e0 0x7fca4419aa90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fca3c02efd0 tx=0x7fca3c0042e0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.968+0000 7fca43200640 1 -- 172.21.15.22:0/3241992148 shutdown_connections 2024-04-20T21:45:03.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.968+0000 7fca43200640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fca280a3d20 0x7fca280a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.968+0000 7fca43200640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca4419a5e0 0x7fca4419aa90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.968+0000 7fca43200640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca44109270 0x7fca441a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.968+0000 7fca43200640 1 --2- 172.21.15.22:0/3241992148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca441041a0 0x7fca441a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:03.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.968+0000 7fca43200640 1 -- 172.21.15.22:0/3241992148 >> 172.21.15.22:0/3241992148 conn(0x7fca440ffa00 msgr2=0x7fca44101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:03.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.968+0000 7fca43200640 1 -- 172.21.15.22:0/3241992148 shutdown_connections 2024-04-20T21:45:03.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:03.968+0000 7fca43200640 1 -- 172.21.15.22:0/3241992148 wait complete. 2024-04-20T21:45:03.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:45:04.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.202+0000 7fc2ada00640 1 -- 172.21.15.22:0/167797482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8100310 msgr2=0x7fc2a81006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:04.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.202+0000 7fc2ada00640 1 --2- 172.21.15.22:0/167797482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8100310 0x7fc2a81006f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc298009900 tx=0x7fc29802bc30 comp rx=0 tx=0).stop 2024-04-20T21:45:04.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.202+0000 7fc2ada00640 1 -- 172.21.15.22:0/167797482 shutdown_connections 2024-04-20T21:45:04.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.202+0000 7fc2ada00640 1 --2- 172.21.15.22:0/167797482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a810e850 0x7fc2a8110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.202+0000 7fc2ada00640 1 --2- 172.21.15.22:0/167797482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8100c30 0x7fc2a810e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.202+0000 7fc2ada00640 1 --2- 172.21.15.22:0/167797482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8100310 0x7fc2a81006f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.202+0000 7fc2ada00640 1 -- 172.21.15.22:0/167797482 >> 172.21.15.22:0/167797482 conn(0x7fc2a8075c60 msgr2=0x7fc2a8076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.202+0000 7fc2ada00640 1 -- 172.21.15.22:0/167797482 shutdown_connections 2024-04-20T21:45:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.202+0000 7fc2ada00640 1 -- 172.21.15.22:0/167797482 wait complete. 2024-04-20T21:45:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.203+0000 7fc2ada00640 1 Processor -- start 2024-04-20T21:45:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.203+0000 7fc2ada00640 1 -- start start 2024-04-20T21:45:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.203+0000 7fc2ada00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8100310 0x7fc2a81a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:04.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.203+0000 7fc2ada00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a8100c30 0x7fc2a81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:04.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2ada00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a810e850 0x7fc2a819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:04.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2ada00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8114440 con 0x7fc2a8100310 2024-04-20T21:45:04.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2ada00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a81142c0 con 0x7fc2a810e850 2024-04-20T21:45:04.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2ada00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a81145c0 con 0x7fc2a8100c30 2024-04-20T21:45:04.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2a7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8100310 0x7fc2a81a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:04.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2a7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8100310 0x7fc2a81a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40748/0 (socket says 172.21.15.22:40748) 2024-04-20T21:45:04.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2a7e00640 1 -- 172.21.15.22:0/1395171571 learned_addr learned my addr 172.21.15.22:0/1395171571 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:45:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc29fe00640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a8100c30 0x7fc2a81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2ad000640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a810e850 0x7fc2a819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2a7e00640 1 -- 172.21.15.22:0/1395171571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a8100c30 msgr2=0x7fc2a81a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2a7e00640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a8100c30 0x7fc2a81a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2a7e00640 1 -- 172.21.15.22:0/1395171571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a810e850 msgr2=0x7fc2a819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2a7e00640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a810e850 0x7fc2a819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.204+0000 7fc2a7e00640 1 -- 172.21.15.22:0/1395171571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a819b000 con 0x7fc2a8100310 2024-04-20T21:45:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.205+0000 7fc2a7e00640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8100310 0x7fc2a81a0680 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc29802c240 tx=0x7fc298004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.205+0000 7fc2a5600640 1 -- 172.21.15.22:0/1395171571 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298043020 con 0x7fc2a8100310 2024-04-20T21:45:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.205+0000 7fc2a5600640 1 -- 172.21.15.22:0/1395171571 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2980054e0 con 0x7fc2a8100310 2024-04-20T21:45:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.205+0000 7fc2ada00640 1 -- 172.21.15.22:0/1395171571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a819b290 con 0x7fc2a8100310 2024-04-20T21:45:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.205+0000 7fc2a5600640 1 -- 172.21.15.22:0/1395171571 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298003bb0 con 0x7fc2a8100310 2024-04-20T21:45:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.205+0000 7fc2ada00640 1 -- 172.21.15.22:0/1395171571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a81a7600 con 0x7fc2a8100310 2024-04-20T21:45:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.206+0000 7fc2ada00640 1 -- 172.21.15.22:0/1395171571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a819b600 con 0x7fc2a8100310 2024-04-20T21:45:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.209+0000 7fc2a5600640 1 -- 172.21.15.22:0/1395171571 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc298046020 con 0x7fc2a8100310 2024-04-20T21:45:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.210+0000 7fc2a5600640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc28c0a3d20 0x7fc28c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.210+0000 7fc2a5600640 1 -- 172.21.15.22:0/1395171571 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7fc2980f2710 con 0x7fc2a8100310 2024-04-20T21:45:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.210+0000 7fc29fe00640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc28c0a3d20 0x7fc28c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:04.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.211+0000 7fc29fe00640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc28c0a3d20 0x7fc28c0a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2900095f0 tx=0x7fc290005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.215+0000 7fc2a5600640 1 -- 172.21.15.22:0/1395171571 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc29804d020 con 0x7fc2a8100310 2024-04-20T21:45:04.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.439+0000 7fc2ada00640 1 -- 172.21.15.22:0/1395171571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc2a819bc70 con 0x7fc2a8100310 2024-04-20T21:45:04.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.440+0000 7fc2a5600640 1 -- 172.21.15.22:0/1395171571 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v103) ==== 81+0+945 (secure 0 0 0) 0x7fc2980bee70 con 0x7fc2a8100310 2024-04-20T21:45:04.439 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1815958949 2024-04-21T21:33:48.815984+0000 2024-04-20T21:45:04.439 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3359736029 2024-04-21T21:33:48.815984+0000 2024-04-20T21:45:04.440 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3485677447 2024-04-21T21:33:48.815984+0000 2024-04-20T21:45:04.440 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/250835174 2024-04-21T21:23:29.510166+0000 2024-04-20T21:45:04.440 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/2098569388 2024-04-21T21:23:29.510166+0000 2024-04-20T21:45:04.440 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/1911146506 2024-04-21T21:33:48.815984+0000 2024-04-20T21:45:04.440 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/2098569388 2024-04-21T21:23:29.510166+0000 2024-04-20T21:45:04.440 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1408863783 2024-04-21T21:33:48.815984+0000 2024-04-20T21:45:04.440 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2009335712 2024-04-21T21:23:29.510166+0000 2024-04-20T21:45:04.441 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/1911146506 2024-04-21T21:33:48.815984+0000 2024-04-20T21:45:04.441 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1022821242 2024-04-21T21:22:55.485151+0000 2024-04-20T21:45:04.441 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3042114972 2024-04-21T21:22:55.485151+0000 2024-04-20T21:45:04.441 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/3430200041 2024-04-21T21:22:55.485151+0000 2024-04-20T21:45:04.441 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1146611493 2024-04-21T21:22:55.485151+0000 2024-04-20T21:45:04.441 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/3430200041 2024-04-21T21:22:55.485151+0000 2024-04-20T21:45:04.441 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1810226571 2024-04-21T21:23:29.510166+0000 2024-04-20T21:45:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:listed 16 entries 2024-04-20T21:45:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.442+0000 7fc2ada00640 1 -- 172.21.15.22:0/1395171571 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc28c0a3d20 msgr2=0x7fc28c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.443+0000 7fc2ada00640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc28c0a3d20 0x7fc28c0a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2900095f0 tx=0x7fc290005f90 comp rx=0 tx=0).stop 2024-04-20T21:45:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.443+0000 7fc2ada00640 1 -- 172.21.15.22:0/1395171571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8100310 msgr2=0x7fc2a81a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:04.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.443+0000 7fc2ada00640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8100310 0x7fc2a81a0680 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc29802c240 tx=0x7fc298004360 comp rx=0 tx=0).stop 2024-04-20T21:45:04.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.443+0000 7fc2ada00640 1 -- 172.21.15.22:0/1395171571 shutdown_connections 2024-04-20T21:45:04.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.443+0000 7fc2ada00640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7fc28c0a3d20 0x7fc28c0a61e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:04.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.443+0000 7fc2ada00640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a810e850 0x7fc2a819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:04.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.443+0000 7fc2ada00640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a8100c30 0x7fc2a81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:04.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.443+0000 7fc2ada00640 1 --2- 172.21.15.22:0/1395171571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2a8100310 0x7fc2a81a0680 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:04.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.443+0000 7fc2ada00640 1 -- 172.21.15.22:0/1395171571 >> 172.21.15.22:0/1395171571 conn(0x7fc2a8075c60 msgr2=0x7fc2a810ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:04.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.443+0000 7fc2ada00640 1 -- 172.21.15.22:0/1395171571 shutdown_connections 2024-04-20T21:45:04.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:04.443+0000 7fc2ada00640 1 -- 172.21.15.22:0/1395171571 wait complete. 2024-04-20T21:45:04.461 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-20T21:45:04.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:45:04.461 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-20T21:45:04.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:04 smithi022 ceph-mon[16843]: pgmap v350: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:45:04.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:04 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1032245519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:45:04.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:04 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3241992148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:45:04.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:04 smithi033 ceph-mon[21626]: pgmap v350: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:45:04.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:04 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1032245519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:45:04.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:04 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3241992148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:45:04.578 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-04-20T21:45:04.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:45:04.579 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:45:04.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:45:04.651 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-04-20T21:45:04.688 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:45:04.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:45:04.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:45:04.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:04 smithi122 ceph-mon[22147]: pgmap v350: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T21:45:04.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:04 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1032245519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:45:04.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:04 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3241992148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:45:04.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:45:04.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-04-20T21:45:04.893 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-20T21:45:04.904 INFO:tasks.cephadm:Teardown begin 2024-04-20T21:45:04.905 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T21:45:04.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:05.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.078+0000 7f20d3200640 1 -- 172.21.15.22:0/3432833515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20d41041f0 msgr2=0x7f20d41045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:05.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.078+0000 7f20d3200640 1 --2- 172.21.15.22:0/3432833515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20d41041f0 0x7f20d41045d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009860 tx=0x7f20c402bb70 comp rx=0 tx=0).stop 2024-04-20T21:45:05.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.078+0000 7f20d3200640 1 -- 172.21.15.22:0/3432833515 shutdown_connections 2024-04-20T21:45:05.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.078+0000 7f20d3200640 1 --2- 172.21.15.22:0/3432833515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d41092c0 0x7f20d4110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.078+0000 7f20d3200640 1 --2- 172.21.15.22:0/3432833515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d4104ba0 0x7f20d4108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.078+0000 7f20d3200640 1 --2- 172.21.15.22:0/3432833515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20d41041f0 0x7f20d41045d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.078+0000 7f20d3200640 1 -- 172.21.15.22:0/3432833515 >> 172.21.15.22:0/3432833515 conn(0x7f20d40ffa30 msgr2=0x7f20d4101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:05.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.078+0000 7f20d3200640 1 -- 172.21.15.22:0/3432833515 shutdown_connections 2024-04-20T21:45:05.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.078+0000 7f20d3200640 1 -- 172.21.15.22:0/3432833515 wait complete. 2024-04-20T21:45:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.079+0000 7f20d3200640 1 Processor -- start 2024-04-20T21:45:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.079+0000 7f20d3200640 1 -- start start 2024-04-20T21:45:05.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.080+0000 7f20d3200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20d41041f0 0x7f20d41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.080+0000 7f20d3200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d41092c0 0x7f20d41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.080+0000 7f20d3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419a6d0 0x7f20d419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.080+0000 7f20d3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4114350 con 0x7f20d419a6d0 2024-04-20T21:45:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.080+0000 7f20d3200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41141d0 con 0x7f20d41092c0 2024-04-20T21:45:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.080+0000 7f20d3200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41144d0 con 0x7f20d41041f0 2024-04-20T21:45:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.080+0000 7f20d2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419a6d0 0x7f20d419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.080+0000 7f20d2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419a6d0 0x7f20d419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40756/0 (socket says 172.21.15.22:40756) 2024-04-20T21:45:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.080+0000 7f20d1400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d41092c0 0x7f20d41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.080+0000 7f20d2800640 1 -- 172.21.15.22:0/1786330346 learned_addr learned my addr 172.21.15.22:0/1786330346 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:45:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.080+0000 7f20d1e00640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20d41041f0 0x7f20d41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20d2800640 1 -- 172.21.15.22:0/1786330346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20d41041f0 msgr2=0x7f20d41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20d2800640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20d41041f0 0x7f20d41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20d2800640 1 -- 172.21.15.22:0/1786330346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d41092c0 msgr2=0x7f20d41a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20d2800640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d41092c0 0x7f20d41a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20d2800640 1 -- 172.21.15.22:0/1786330346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d419b2d0 con 0x7f20d419a6d0 2024-04-20T21:45:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20d1400640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d41092c0 0x7f20d41a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20d1e00640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20d41041f0 0x7f20d41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:45:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20d2800640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419a6d0 0x7f20d419ab50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00eb10 tx=0x7f20cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20caa00640 1 -- 172.21.15.22:0/1786330346 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc004b20 con 0x7f20d419a6d0 2024-04-20T21:45:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20d3200640 1 -- 172.21.15.22:0/1786330346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d41a73b0 con 0x7f20d419a6d0 2024-04-20T21:45:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20caa00640 1 -- 172.21.15.22:0/1786330346 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20cc004cc0 con 0x7f20d419a6d0 2024-04-20T21:45:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20caa00640 1 -- 172.21.15.22:0/1786330346 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc010760 con 0x7f20d419a6d0 2024-04-20T21:45:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.081+0000 7f20d3200640 1 -- 172.21.15.22:0/1786330346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d41a7760 con 0x7f20d419a6d0 2024-04-20T21:45:05.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.083+0000 7f20d3200640 1 -- 172.21.15.22:0/1786330346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d41056c0 con 0x7f20d419a6d0 2024-04-20T21:45:05.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.085+0000 7f20caa00640 1 -- 172.21.15.22:0/1786330346 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f20cc005020 con 0x7f20d419a6d0 2024-04-20T21:45:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.087+0000 7f20caa00640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f20b80a3d20 0x7f20b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.087+0000 7f20caa00640 1 -- 172.21.15.22:0/1786330346 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f20cc0d0e60 con 0x7f20d419a6d0 2024-04-20T21:45:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.087+0000 7f20d1e00640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f20b80a3d20 0x7f20b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.088+0000 7f20d1e00640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f20b80a3d20 0x7f20b80a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20c4000c00 tx=0x7f20c40058b0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:05.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.096+0000 7f20caa00640 1 -- 172.21.15.22:0/1786330346 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f20cc09d450 con 0x7f20d419a6d0 2024-04-20T21:45:05.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.336+0000 7f20d3200640 1 -- 172.21.15.22:0/1786330346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20d410c4f0 con 0x7f20d419a6d0 2024-04-20T21:45:05.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.338+0000 7f20caa00640 1 -- 172.21.15.22:0/1786330346 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f20cc09cd50 con 0x7f20d419a6d0 2024-04-20T21:45:05.337 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:45:05.337 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:45:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:45:05.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.340+0000 7f20d3200640 1 -- 172.21.15.22:0/1786330346 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f20b80a3d20 msgr2=0x7f20b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:05.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.340+0000 7f20d3200640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f20b80a3d20 0x7f20b80a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20c4000c00 tx=0x7f20c40058b0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.340+0000 7f20d3200640 1 -- 172.21.15.22:0/1786330346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419a6d0 msgr2=0x7f20d419ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:05.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.340+0000 7f20d3200640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419a6d0 0x7f20d419ab50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00eb10 tx=0x7f20cc00c2d0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.340+0000 7f20d3200640 1 -- 172.21.15.22:0/1786330346 shutdown_connections 2024-04-20T21:45:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.340+0000 7f20d3200640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f20b80a3d20 0x7f20b80a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.340+0000 7f20d3200640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20d419a6d0 0x7f20d419ab50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.340+0000 7f20d3200640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d41092c0 0x7f20d41a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.340+0000 7f20d3200640 1 --2- 172.21.15.22:0/1786330346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20d41041f0 0x7f20d41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.340+0000 7f20d3200640 1 -- 172.21.15.22:0/1786330346 >> 172.21.15.22:0/1786330346 conn(0x7f20d40ffa30 msgr2=0x7f20d4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:05.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.340+0000 7f20d3200640 1 -- 172.21.15.22:0/1786330346 shutdown_connections 2024-04-20T21:45:05.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.341+0000 7f20d3200640 1 -- 172.21.15.22:0/1786330346 wait complete. 2024-04-20T21:45:05.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:45:05.413 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:05 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1395171571' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T21:45:05.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.542+0000 7f05e2e00640 1 -- 172.21.15.22:0/1603278424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05dc1043b0 msgr2=0x7f05dc104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.542+0000 7f05e2e00640 1 --2- 172.21.15.22:0/1603278424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05dc1043b0 0x7f05dc104790 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009900 tx=0x7f05cc02bc30 comp rx=0 tx=0).stop 2024-04-20T21:45:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.542+0000 7f05e2e00640 1 -- 172.21.15.22:0/1603278424 shutdown_connections 2024-04-20T21:45:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.542+0000 7f05e2e00640 1 --2- 172.21.15.22:0/1603278424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05dc109480 0x7f05dc111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.542+0000 7f05e2e00640 1 --2- 172.21.15.22:0/1603278424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc104d60 0x7f05dc108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.542+0000 7f05e2e00640 1 --2- 172.21.15.22:0/1603278424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05dc1043b0 0x7f05dc104790 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.542+0000 7f05e2e00640 1 -- 172.21.15.22:0/1603278424 >> 172.21.15.22:0/1603278424 conn(0x7f05dc0ffc10 msgr2=0x7f05dc102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.542+0000 7f05e2e00640 1 -- 172.21.15.22:0/1603278424 shutdown_connections 2024-04-20T21:45:05.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.542+0000 7f05e2e00640 1 -- 172.21.15.22:0/1603278424 wait complete. 2024-04-20T21:45:05.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.543+0000 7f05e2e00640 1 Processor -- start 2024-04-20T21:45:05.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.543+0000 7f05e2e00640 1 -- start start 2024-04-20T21:45:05.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.544+0000 7f05e2e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05dc1043b0 0x7f05dc1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:05.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.544+0000 7f05e2e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc109480 0x7f05dc1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:05.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.544+0000 7f05e2e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05dc19a930 0x7f05dc19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:05.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.544+0000 7f05e2e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc114590 con 0x7f05dc109480 2024-04-20T21:45:05.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.544+0000 7f05e2e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc114410 con 0x7f05dc1043b0 2024-04-20T21:45:05.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.544+0000 7f05e2e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc114710 con 0x7f05dc19a930 2024-04-20T21:45:05.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.544+0000 7f05db400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc109480 0x7f05dc1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.544+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05dc1043b0 0x7f05dc1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.544+0000 7f05db400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc109480 0x7f05dc1a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40764/0 (socket says 172.21.15.22:40764) 2024-04-20T21:45:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.544+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05dc1043b0 0x7f05dc1a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50734/0 (socket says 172.21.15.22:50734) 2024-04-20T21:45:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.544+0000 7f05db400640 1 -- 172.21.15.22:0/3390472690 learned_addr learned my addr 172.21.15.22:0/3390472690 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:45:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.544+0000 7f05e0e00640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05dc19a930 0x7f05dc19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.545+0000 7f05dbe00640 1 -- 172.21.15.22:0/3390472690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05dc19a930 msgr2=0x7f05dc19ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.545+0000 7f05dbe00640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05dc19a930 0x7f05dc19ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.545+0000 7f05dbe00640 1 -- 172.21.15.22:0/3390472690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc109480 msgr2=0x7f05dc1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.545+0000 7f05dbe00640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc109480 0x7f05dc1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.545+0000 7f05dbe00640 1 -- 172.21.15.22:0/3390472690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc19b450 con 0x7f05dc1043b0 2024-04-20T21:45:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.545+0000 7f05db400640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc109480 0x7f05dc1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.545+0000 7f05dbe00640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05dc1043b0 0x7f05dc1a06e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f05cc02c170 tx=0x7f05cc004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.545+0000 7f05d8c00640 1 -- 172.21.15.22:0/3390472690 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc043020 con 0x7f05dc1043b0 2024-04-20T21:45:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.545+0000 7f05d8c00640 1 -- 172.21.15.22:0/3390472690 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05cc0054e0 con 0x7f05dc1043b0 2024-04-20T21:45:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.545+0000 7f05e2e00640 1 -- 172.21.15.22:0/3390472690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05dc19b6e0 con 0x7f05dc1043b0 2024-04-20T21:45:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.545+0000 7f05d8c00640 1 -- 172.21.15.22:0/3390472690 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc003b90 con 0x7f05dc1043b0 2024-04-20T21:45:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.546+0000 7f05e2e00640 1 -- 172.21.15.22:0/3390472690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05dc1a7890 con 0x7f05dc1043b0 2024-04-20T21:45:05.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.547+0000 7f05e2e00640 1 -- 172.21.15.22:0/3390472690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05dc0691d0 con 0x7f05dc1043b0 2024-04-20T21:45:05.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.550+0000 7f05d8c00640 1 -- 172.21.15.22:0/3390472690 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f05cc046020 con 0x7f05dc1043b0 2024-04-20T21:45:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.552+0000 7f05d8c00640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f05c00a3d20 0x7f05c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.552+0000 7f05d8c00640 1 -- 172.21.15.22:0/3390472690 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f05cc0f2de0 con 0x7f05dc1043b0 2024-04-20T21:45:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.553+0000 7f05db400640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f05c00a3d20 0x7f05c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.553+0000 7f05d8c00640 1 -- 172.21.15.22:0/3390472690 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f05cc04d020 con 0x7f05dc1043b0 2024-04-20T21:45:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.554+0000 7f05db400640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f05c00a3d20 0x7f05c00a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f05dc100130 tx=0x7f05c8005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:05.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:05 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1395171571' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T21:45:05.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:05 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1395171571' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T21:45:05.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.793+0000 7f05e2e00640 1 -- 172.21.15.22:0/3390472690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05dc19bdc0 con 0x7f05dc1043b0 2024-04-20T21:45:05.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.795+0000 7f05d8c00640 1 -- 172.21.15.22:0/3390472690 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11822 (secure 0 0 0) 0x7f05cc0bf4e0 con 0x7f05dc1043b0 2024-04-20T21:45:05.794 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:45:05.794 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2822771951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2822771951},{"type":"v1","addr":"172.21.15.22:6835","nonce":2822771951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/2696291995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":2696291995},{"type":"v1","addr":"172.21.15.22:6837","nonce":2696291995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/4122889060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":4122889060},{"type":"v1","addr":"172.21.15.22:6839","nonce":4122889060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2604771291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2604771291},{"type":"v1","addr":"172.21.15.22:6841","nonce":2604771291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/721989242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":721989242},{"type":"v1","addr":"172.21.15.33:6833","nonce":721989242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1883458757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1883458757},{"type":"v1","addr":"172.21.15.122:6833","nonce":1883458757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1440214109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1440214109},{"type":"v1","addr":"172.21.15.33:6835","nonce":1440214109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1181941460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1181941460},{"type":"v1","addr":"172.21.15.33:6837","nonce":1181941460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3637338060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3637338060},{"type":"v1","addr":"172.21.15.122:6835","nonce":3637338060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3716866957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3716866957},{"type":"v1","addr":"172.21.15.122:6837","nonce":3716866957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:31:27.176318+0000","modified":"2024-04-20T21:31:31.525301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/2642727200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2642727200},{"type":"v1","addr":"172.21.15.122:6839","nonce":2642727200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6839/300232041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":300232041},{"type":"v1","addr":"172.21.15.33:6839","nonce":300232041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-20T21:45:05.794 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 20 2024-04-20T21:45:05.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.798+0000 7f05e2e00640 1 -- 172.21.15.22:0/3390472690 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f05c00a3d20 msgr2=0x7f05c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:05.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.798+0000 7f05e2e00640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f05c00a3d20 0x7f05c00a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f05dc100130 tx=0x7f05c8005f90 comp rx=0 tx=0).stop 2024-04-20T21:45:05.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.798+0000 7f05e2e00640 1 -- 172.21.15.22:0/3390472690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05dc1043b0 msgr2=0x7f05dc1a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:05.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.798+0000 7f05e2e00640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05dc1043b0 0x7f05dc1a06e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f05cc02c170 tx=0x7f05cc004360 comp rx=0 tx=0).stop 2024-04-20T21:45:05.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.798+0000 7f05e2e00640 1 -- 172.21.15.22:0/3390472690 shutdown_connections 2024-04-20T21:45:05.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.798+0000 7f05e2e00640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f05c00a3d20 0x7f05c00a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.798+0000 7f05e2e00640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05dc19a930 0x7f05dc19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.798+0000 7f05e2e00640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05dc109480 0x7f05dc1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.798+0000 7f05e2e00640 1 --2- 172.21.15.22:0/3390472690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05dc1043b0 0x7f05dc1a06e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:05.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.798+0000 7f05e2e00640 1 -- 172.21.15.22:0/3390472690 >> 172.21.15.22:0/3390472690 conn(0x7f05dc0ffc10 msgr2=0x7f05dc100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:05.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.798+0000 7f05e2e00640 1 -- 172.21.15.22:0/3390472690 shutdown_connections 2024-04-20T21:45:05.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:05.798+0000 7f05e2e00640 1 -- 172.21.15.22:0/3390472690 wait complete. 2024-04-20T21:45:05.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:45:06.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.009+0000 7f3c3b200640 1 -- 172.21.15.22:0/1045606048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c069d10 msgr2=0x7f3c3c1069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.009+0000 7f3c3b200640 1 --2- 172.21.15.22:0/1045606048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c069d10 0x7f3c3c1069d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009860 tx=0x7f3c2402bb70 comp rx=0 tx=0).stop 2024-04-20T21:45:06.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.009+0000 7f3c3b200640 1 -- 172.21.15.22:0/1045606048 shutdown_connections 2024-04-20T21:45:06.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.009+0000 7f3c3b200640 1 --2- 172.21.15.22:0/1045606048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c107100 0x7f3c3c10eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.009+0000 7f3c3b200640 1 --2- 172.21.15.22:0/1045606048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c069d10 0x7f3c3c1069d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.009+0000 7f3c3b200640 1 --2- 172.21.15.22:0/1045606048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c069360 0x7f3c3c069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.010+0000 7f3c3b200640 1 -- 172.21.15.22:0/1045606048 >> 172.21.15.22:0/1045606048 conn(0x7f3c3c0767a0 msgr2=0x7f3c3c076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:06.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.010+0000 7f3c3b200640 1 -- 172.21.15.22:0/1045606048 shutdown_connections 2024-04-20T21:45:06.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.010+0000 7f3c3b200640 1 -- 172.21.15.22:0/1045606048 wait complete. 2024-04-20T21:45:06.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c3b200640 1 Processor -- start 2024-04-20T21:45:06.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c3b200640 1 -- start start 2024-04-20T21:45:06.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c3b200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c069360 0x7f3c3c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c3b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c107100 0x7f3c3c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c3b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c19a730 0x7f3c3c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c3b200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c112120 con 0x7f3c3c069360 2024-04-20T21:45:06.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c3b200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c111fa0 con 0x7f3c3c19a730 2024-04-20T21:45:06.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c3b200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c1122a0 con 0x7f3c3c107100 2024-04-20T21:45:06.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c39e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c069360 0x7f3c3c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c39e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c069360 0x7f3c3c1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40786/0 (socket says 172.21.15.22:40786) 2024-04-20T21:45:06.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c39e00640 1 -- 172.21.15.22:0/594648392 learned_addr learned my addr 172.21.15.22:0/594648392 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:45:06.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c33e00640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c107100 0x7f3c3c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.011+0000 7f3c3a800640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c19a730 0x7f3c3c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.012+0000 7f3c39e00640 1 -- 172.21.15.22:0/594648392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c107100 msgr2=0x7f3c3c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.012+0000 7f3c39e00640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c107100 0x7f3c3c1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.012+0000 7f3c39e00640 1 -- 172.21.15.22:0/594648392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c19a730 msgr2=0x7f3c3c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.012+0000 7f3c39e00640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c19a730 0x7f3c3c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.012+0000 7f3c39e00640 1 -- 172.21.15.22:0/594648392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3c19b330 con 0x7f3c3c069360 2024-04-20T21:45:06.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.012+0000 7f3c39e00640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c069360 0x7f3c3c1a0670 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00e9c0 tx=0x7f3c2c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:06.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.012+0000 7f3c32a00640 1 -- 172.21.15.22:0/594648392 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00cd80 con 0x7f3c3c069360 2024-04-20T21:45:06.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.012+0000 7f3c32a00640 1 -- 172.21.15.22:0/594648392 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2c0035f0 con 0x7f3c3c069360 2024-04-20T21:45:06.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.012+0000 7f3c3b200640 1 -- 172.21.15.22:0/594648392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3c1a7410 con 0x7f3c3c069360 2024-04-20T21:45:06.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.013+0000 7f3c3b200640 1 -- 172.21.15.22:0/594648392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c3c1a77c0 con 0x7f3c3c069360 2024-04-20T21:45:06.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.013+0000 7f3c32a00640 1 -- 172.21.15.22:0/594648392 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c010670 con 0x7f3c3c069360 2024-04-20T21:45:06.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.014+0000 7f3c3b200640 1 -- 172.21.15.22:0/594648392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c3c1034b0 con 0x7f3c3c069360 2024-04-20T21:45:06.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.016+0000 7f3c32a00640 1 -- 172.21.15.22:0/594648392 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3c2c0037a0 con 0x7f3c3c069360 2024-04-20T21:45:06.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.017+0000 7f3c32a00640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3c200a3d20 0x7f3c200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.017+0000 7f3c32a00640 1 -- 172.21.15.22:0/594648392 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8600+0+0 (secure 0 0 0) 0x7f3c2c0d0a40 con 0x7f3c3c069360 2024-04-20T21:45:06.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.017+0000 7f3c33e00640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3c200a3d20 0x7f3c200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.018+0000 7f3c33e00640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3c200a3d20 0x7f3c200a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3c24037040 tx=0x7f3c240056d0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:06.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.021+0000 7f3c32a00640 1 -- 172.21.15.22:0/594648392 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3c2c09d090 con 0x7f3c3c069360 2024-04-20T21:45:06.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.225+0000 7f3c3b200640 1 -- 172.21.15.22:0/594648392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c3c069d10 con 0x7f3c3c069360 2024-04-20T21:45:06.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.228+0000 7f3c32a00640 1 -- 172.21.15.22:0/594648392 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23239 (secure 0 0 0) 0x7f3c2c09c990 con 0x7f3c3c069360 2024-04-20T21:45:06.227 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-04-20T21:45:06.227 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":103,"fsid":"92c49c5c-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:22:14.364120+0000","modified":"2024-04-20T21:44:14.296211+0000","last_up_change":"2024-04-20T21:30:29.573291+0000","last_in_change":"2024-04-20T21:30:09.633897+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:26:15.688818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:31:25.506074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:31:26.126261+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67f06b5e-0fdc-4d2a-9355-fc7aa718d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6803","nonce":2391966324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6805","nonce":2391966324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6809","nonce":2391966324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2391966324},{"type":"v1","addr":"172.21.15.22:6807","nonce":2391966324}]},"public_addr":"172.21.15.22:6803/2391966324","cluster_addr":"172.21.15.22:6805/2391966324","heartbeat_back_addr":"172.21.15.22:6809/2391966324","heartbeat_front_addr":"172.21.15.22:6807/2391966324","state":["exists","up"]},{"osd":1,"uuid":"728fe9c7-163a-4b08-b7fe-5cd2e76aaa99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6801","nonce":2822429441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6803","nonce":2822429441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6807","nonce":2822429441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2822429441},{"type":"v1","addr":"172.21.15.33:6805","nonce":2822429441}]},"public_addr":"172.21.15.33:6801/2822429441","cluster_addr":"172.21.15.33:6803/2822429441","heartbeat_back_addr":"172.21.15.33:6807/2822429441","heartbeat_front_addr":"172.21.15.33:6805/2822429441","state":["exists","up"]},{"osd":2,"uuid":"cdddd7e5-5a5e-4d68-9849-420d9f891c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6801","nonce":4001148534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6803","nonce":4001148534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6807","nonce":4001148534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4001148534},{"type":"v1","addr":"172.21.15.122:6805","nonce":4001148534}]},"public_addr":"172.21.15.122:6801/4001148534","cluster_addr":"172.21.15.122:6803/4001148534","heartbeat_back_addr":"172.21.15.122:6807/4001148534","heartbeat_front_addr":"172.21.15.122:6805/4001148534","state":["exists","up"]},{"osd":3,"uuid":"9989d732-245e-417b-9a43-83c77374159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6811","nonce":1762015045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6813","nonce":1762015045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6817","nonce":1762015045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1762015045},{"type":"v1","addr":"172.21.15.22:6815","nonce":1762015045}]},"public_addr":"172.21.15.22:6811/1762015045","cluster_addr":"172.21.15.22:6813/1762015045","heartbeat_back_addr":"172.21.15.22:6817/1762015045","heartbeat_front_addr":"172.21.15.22:6815/1762015045","state":["exists","up"]},{"osd":4,"uuid":"12ff6d17-2779-4462-bb8b-1b50e117c78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6809","nonce":1935934998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6811","nonce":1935934998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6815","nonce":1935934998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1935934998},{"type":"v1","addr":"172.21.15.33:6813","nonce":1935934998}]},"public_addr":"172.21.15.33:6809/1935934998","cluster_addr":"172.21.15.33:6811/1935934998","heartbeat_back_addr":"172.21.15.33:6815/1935934998","heartbeat_front_addr":"172.21.15.33:6813/1935934998","state":["exists","up"]},{"osd":5,"uuid":"70a6256a-db56-4eab-bd2c-3f7a499cb166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6809","nonce":1738459470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6811","nonce":1738459470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6815","nonce":1738459470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1738459470},{"type":"v1","addr":"172.21.15.122:6813","nonce":1738459470}]},"public_addr":"172.21.15.122:6809/1738459470","cluster_addr":"172.21.15.122:6811/1738459470","heartbeat_back_addr":"172.21.15.122:6815/1738459470","heartbeat_front_addr":"172.21.15.122:6813/1738459470","state":["exists","up"]},{"osd":6,"uuid":"283c2314-3dbb-4f48-964d-127ae967d38f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6819","nonce":1631721286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6821","nonce":1631721286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6825","nonce":1631721286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1631721286},{"type":"v1","addr":"172.21.15.22:6823","nonce":1631721286}]},"public_addr":"172.21.15.22:6819/1631721286","cluster_addr":"172.21.15.22:6821/1631721286","heartbeat_back_addr":"172.21.15.22:6825/1631721286","heartbeat_front_addr":"172.21.15.22:6823/1631721286","state":["exists","up"]},{"osd":7,"uuid":"986f4852-193c-413d-8ffb-2faba217f4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6817","nonce":539722982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6819","nonce":539722982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6823","nonce":539722982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":539722982},{"type":"v1","addr":"172.21.15.33:6821","nonce":539722982}]},"public_addr":"172.21.15.33:6817/539722982","cluster_addr":"172.21.15.33:6819/539722982","heartbeat_back_addr":"172.21.15.33:6823/539722982","heartbeat_front_addr":"172.21.15.33:6821/539722982","state":["exists","up"]},{"osd":8,"uuid":"4a6f0da8-369c-4576-9105-13a02e09b03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6817","nonce":2876209556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6819","nonce":2876209556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6823","nonce":2876209556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2876209556},{"type":"v1","addr":"172.21.15.122:6821","nonce":2876209556}]},"public_addr":"172.21.15.122:6817/2876209556","cluster_addr":"172.21.15.122:6819/2876209556","heartbeat_back_addr":"172.21.15.122:6823/2876209556","heartbeat_front_addr":"172.21.15.122:6821/2876209556","state":["exists","up"]},{"osd":9,"uuid":"c35e9ebb-ebc5-4547-9490-09c2944b3e9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6827","nonce":1246646711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6829","nonce":1246646711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6833","nonce":1246646711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":1246646711},{"type":"v1","addr":"172.21.15.22:6831","nonce":1246646711}]},"public_addr":"172.21.15.22:6827/1246646711","cluster_addr":"172.21.15.22:6829/1246646711","heartbeat_back_addr":"172.21.15.22:6833/1246646711","heartbeat_front_addr":"172.21.15.22:6831/1246646711","state":["exists","up"]},{"osd":10,"uuid":"bc55492f-9b3b-40b3-ba6c-3d4c5b189c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6825","nonce":1408592244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6827","nonce":1408592244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6831","nonce":1408592244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1408592244},{"type":"v1","addr":"172.21.15.33:6829","nonce":1408592244}]},"public_addr":"172.21.15.33:6825/1408592244","cluster_addr":"172.21.15.33:6827/1408592244","heartbeat_back_addr":"172.21.15.33:6831/1408592244","heartbeat_front_addr":"172.21.15.33:6829/1408592244","state":["exists","up"]},{"osd":11,"uuid":"e35bd98e-f771-40fb-b428-3c182f1c59c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6825","nonce":3622105204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6827","nonce":3622105204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6831","nonce":3622105204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3622105204},{"type":"v1","addr":"172.21.15.122:6829","nonce":3622105204}]},"public_addr":"172.21.15.122:6825/3622105204","cluster_addr":"172.21.15.122:6827/3622105204","heartbeat_back_addr":"172.21.15.122:6831/3622105204","heartbeat_front_addr":"172.21.15.122:6829/3622105204","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:22.480889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:48.561261+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:12.985049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:44.881532+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:11.154384+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:36.256171+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:10.181605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:36.739242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:01.981987+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:35.180721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:02.307109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:30:27.563948+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/1815958949":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/3359736029":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/3485677447":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/250835174":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6801/1911146506":"2024-04-21T21:33:48.815984+0000","172.21.15.22:6800/2098569388":"2024-04-21T21:23:29.510166+0000","172.21.15.22:0/1408863783":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/2009335712":"2024-04-21T21:23:29.510166+0000","172.21.15.22:6800/1911146506":"2024-04-21T21:33:48.815984+0000","172.21.15.22:0/1022821242":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/3042114972":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6800/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1146611493":"2024-04-21T21:22:55.485151+0000","172.21.15.22:6801/3430200041":"2024-04-21T21:22:55.485151+0000","172.21.15.22:0/1810226571":"2024-04-21T21:23:29.510166+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:45:06.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.231+0000 7f3c3b200640 1 -- 172.21.15.22:0/594648392 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3c200a3d20 msgr2=0x7f3c200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.231+0000 7f3c3b200640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3c200a3d20 0x7f3c200a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3c24037040 tx=0x7f3c240056d0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.231+0000 7f3c3b200640 1 -- 172.21.15.22:0/594648392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c069360 msgr2=0x7f3c3c1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.231+0000 7f3c3b200640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c069360 0x7f3c3c1a0670 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00e9c0 tx=0x7f3c2c00edc0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.231+0000 7f3c3b200640 1 -- 172.21.15.22:0/594648392 shutdown_connections 2024-04-20T21:45:06.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.231+0000 7f3c3b200640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f3c200a3d20 0x7f3c200a61e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.231+0000 7f3c3b200640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c19a730 0x7f3c3c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.231+0000 7f3c3b200640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c107100 0x7f3c3c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.231+0000 7f3c3b200640 1 --2- 172.21.15.22:0/594648392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c3c069360 0x7f3c3c1a0670 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.231+0000 7f3c3b200640 1 -- 172.21.15.22:0/594648392 >> 172.21.15.22:0/594648392 conn(0x7f3c3c0767a0 msgr2=0x7f3c3c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:06.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.231+0000 7f3c3b200640 1 -- 172.21.15.22:0/594648392 shutdown_connections 2024-04-20T21:45:06.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.231+0000 7f3c3b200640 1 -- 172.21.15.22:0/594648392 wait complete. 2024-04-20T21:45:06.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:45:06.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:06 smithi022 ceph-mon[16843]: pgmap v351: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-04-20T21:45:06.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:06 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/1786330346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:45:06.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:06 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/3390472690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:45:06.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:06 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/594648392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:45:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.460+0000 7f4366c00640 1 -- 172.21.15.22:0/1113430172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4360100e20 msgr2=0x7f4360101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.460+0000 7f4366c00640 1 --2- 172.21.15.22:0/1113430172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4360100e20 0x7f4360101200 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4350009900 tx=0x7f435002bd10 comp rx=0 tx=0).stop 2024-04-20T21:45:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.460+0000 7f4366c00640 1 -- 172.21.15.22:0/1113430172 shutdown_connections 2024-04-20T21:45:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.460+0000 7f4366c00640 1 --2- 172.21.15.22:0/1113430172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4360106220 0x7f43601066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.460+0000 7f4366c00640 1 --2- 172.21.15.22:0/1113430172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4360101740 0x7f4360105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.460+0000 7f4366c00640 1 --2- 172.21.15.22:0/1113430172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4360100e20 0x7f4360101200 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.460+0000 7f4366c00640 1 -- 172.21.15.22:0/1113430172 >> 172.21.15.22:0/1113430172 conn(0x7f43600fc870 msgr2=0x7f43600fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.461+0000 7f4366c00640 1 -- 172.21.15.22:0/1113430172 shutdown_connections 2024-04-20T21:45:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.461+0000 7f4366c00640 1 -- 172.21.15.22:0/1113430172 wait complete. 2024-04-20T21:45:06.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.462+0000 7f4366c00640 1 Processor -- start 2024-04-20T21:45:06.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.462+0000 7f4366c00640 1 -- start start 2024-04-20T21:45:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.462+0000 7f4366c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4360101740 0x7f436019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.462+0000 7f4366c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4360106220 0x7f436019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.462+0000 7f4366c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43601a3850 0x7f43601a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.462+0000 7f4366c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4360118890 con 0x7f4360106220 2024-04-20T21:45:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.462+0000 7f4366c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4360118710 con 0x7f43601a3850 2024-04-20T21:45:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.462+0000 7f4366c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4360118a10 con 0x7f4360101740 2024-04-20T21:45:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.462+0000 7f4364e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4360106220 0x7f436019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.463+0000 7f4364e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4360106220 0x7f436019f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40810/0 (socket says 172.21.15.22:40810) 2024-04-20T21:45:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.463+0000 7f4364e00640 1 -- 172.21.15.22:0/2839186702 learned_addr learned my addr 172.21.15.22:0/2839186702 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:45:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.463+0000 7f4365800640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4360101740 0x7f436019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.463+0000 7f4366200640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43601a3850 0x7f43601a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.463+0000 7f4364e00640 1 -- 172.21.15.22:0/2839186702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4360101740 msgr2=0x7f436019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.463+0000 7f4364e00640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4360101740 0x7f436019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.463+0000 7f4364e00640 1 -- 172.21.15.22:0/2839186702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43601a3850 msgr2=0x7f43601a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.463+0000 7f4364e00640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43601a3850 0x7f43601a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.463+0000 7f4364e00640 1 -- 172.21.15.22:0/2839186702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43601a4340 con 0x7f4360106220 2024-04-20T21:45:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.463+0000 7f4366200640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43601a3850 0x7f43601a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:45:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.463+0000 7f4364e00640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4360106220 0x7f436019f4d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f435400e9f0 tx=0x7f435400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.464+0000 7f435e000640 1 -- 172.21.15.22:0/2839186702 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435400ccc0 con 0x7f4360106220 2024-04-20T21:45:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.464+0000 7f435e000640 1 -- 172.21.15.22:0/2839186702 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4354010910 con 0x7f4360106220 2024-04-20T21:45:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.464+0000 7f435e000640 1 -- 172.21.15.22:0/2839186702 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354018920 con 0x7f4360106220 2024-04-20T21:45:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.464+0000 7f4366c00640 1 -- 172.21.15.22:0/2839186702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43601a4630 con 0x7f4360106220 2024-04-20T21:45:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.464+0000 7f4366c00640 1 -- 172.21.15.22:0/2839186702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43601abdd0 con 0x7f4360106220 2024-04-20T21:45:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.465+0000 7f4366c00640 1 -- 172.21.15.22:0/2839186702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436006ae40 con 0x7f4360106220 2024-04-20T21:45:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.467+0000 7f435e000640 1 -- 172.21.15.22:0/2839186702 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4354010430 con 0x7f4360106220 2024-04-20T21:45:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.468+0000 7f435e000640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43480a3da0 0x7f43480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.468+0000 7f435e000640 1 -- 172.21.15.22:0/2839186702 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8644+0+0 (secure 0 0 0) 0x7f43540d0b20 con 0x7f4360106220 2024-04-20T21:45:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.468+0000 7f4365800640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43480a3da0 0x7f43480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.469+0000 7f4365800640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43480a3da0 0x7f43480a6260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4350036040 tx=0x7f4350005a20 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:06.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.472+0000 7f435e000640 1 -- 172.21.15.22:0/2839186702 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f435409d0e0 con 0x7f4360106220 2024-04-20T21:45:06.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:06 smithi033 ceph-mon[21626]: pgmap v351: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-04-20T21:45:06.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:06 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/1786330346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:45:06.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:06 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/3390472690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:45:06.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:06 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/594648392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:45:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.717+0000 7f4366c00640 1 -- 172.21.15.22:0/2839186702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f43601a02e0 con 0x7f4360106220 2024-04-20T21:45:06.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.722+0000 7f435e000640 1 -- 172.21.15.22:0/2839186702 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7f43601a02e0 con 0x7f4360106220 2024-04-20T21:45:06.721 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:45:06.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.725+0000 7f4366c00640 1 -- 172.21.15.22:0/2839186702 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43480a3da0 msgr2=0x7f43480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.725+0000 7f4366c00640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43480a3da0 0x7f43480a6260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4350036040 tx=0x7f4350005a20 comp rx=0 tx=0).stop 2024-04-20T21:45:06.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.725+0000 7f4366c00640 1 -- 172.21.15.22:0/2839186702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4360106220 msgr2=0x7f436019f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.725+0000 7f4366c00640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4360106220 0x7f436019f4d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f435400e9f0 tx=0x7f435400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.726+0000 7f4366c00640 1 -- 172.21.15.22:0/2839186702 shutdown_connections 2024-04-20T21:45:06.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.726+0000 7f4366c00640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43480a3da0 0x7f43480a6260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.726+0000 7f4366c00640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43601a3850 0x7f43601a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.726+0000 7f4366c00640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4360106220 0x7f436019f4d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.726+0000 7f4366c00640 1 --2- 172.21.15.22:0/2839186702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4360101740 0x7f436019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.726+0000 7f4366c00640 1 -- 172.21.15.22:0/2839186702 >> 172.21.15.22:0/2839186702 conn(0x7f43600fc870 msgr2=0x7f43600fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:06.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.726+0000 7f4366c00640 1 -- 172.21.15.22:0/2839186702 shutdown_connections 2024-04-20T21:45:06.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.726+0000 7f4366c00640 1 -- 172.21.15.22:0/2839186702 wait complete. 2024-04-20T21:45:06.744 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T21:45:06.764 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:06 smithi122 ceph-mon[22147]: pgmap v351: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-04-20T21:45:06.764 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:06 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/1786330346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:45:06.764 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:06 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/3390472690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:45:06.764 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:06 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/594648392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:45:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.942+0000 7f43aac00640 1 -- 172.21.15.22:0/3304639470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43a4109340 msgr2=0x7f43a4110f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.942+0000 7f43aac00640 1 --2- 172.21.15.22:0/3304639470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43a4109340 0x7f43a4110f10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f439c009ca0 tx=0x7f439c02c0f0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.942+0000 7f43aac00640 1 -- 172.21.15.22:0/3304639470 shutdown_connections 2024-04-20T21:45:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.942+0000 7f43aac00640 1 --2- 172.21.15.22:0/3304639470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43a4109340 0x7f43a4110f10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.942+0000 7f43aac00640 1 --2- 172.21.15.22:0/3304639470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a4104c20 0x7f43a4108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.942+0000 7f43aac00640 1 --2- 172.21.15.22:0/3304639470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43a4104270 0x7f43a4104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.942+0000 7f43aac00640 1 -- 172.21.15.22:0/3304639470 >> 172.21.15.22:0/3304639470 conn(0x7f43a40ffad0 msgr2=0x7f43a4101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.942+0000 7f43aac00640 1 -- 172.21.15.22:0/3304639470 shutdown_connections 2024-04-20T21:45:06.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.942+0000 7f43aac00640 1 -- 172.21.15.22:0/3304639470 wait complete. 2024-04-20T21:45:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.943+0000 7f43aac00640 1 Processor -- start 2024-04-20T21:45:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.943+0000 7f43aac00640 1 -- start start 2024-04-20T21:45:06.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.943+0000 7f43aac00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43a4104270 0x7f43a41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43aac00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43a4109340 0x7f43a41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43aac00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a419a720 0x7f43a419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43aac00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f43a41143e0 con 0x7f43a4109340 2024-04-20T21:45:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43aac00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43a4114260 con 0x7f43a419a720 2024-04-20T21:45:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43aac00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43a4114560 con 0x7f43a4104270 2024-04-20T21:45:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43a8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43a4109340 0x7f43a41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43a8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43a4109340 0x7f43a41a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40822/0 (socket says 172.21.15.22:40822) 2024-04-20T21:45:06.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43a9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43a4104270 0x7f43a41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43a8e00640 1 -- 172.21.15.22:0/287105709 learned_addr learned my addr 172.21.15.22:0/287105709 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:45:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43aa200640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a419a720 0x7f43a419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43aa200640 1 -- 172.21.15.22:0/287105709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43a4104270 msgr2=0x7f43a41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43aa200640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43a4104270 0x7f43a41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43aa200640 1 -- 172.21.15.22:0/287105709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43a4109340 msgr2=0x7f43a41a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43aa200640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43a4109340 0x7f43a41a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43aa200640 1 -- 172.21.15.22:0/287105709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a419b320 con 0x7f43a419a720 2024-04-20T21:45:06.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43a8e00640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43a4109340 0x7f43a41a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.944+0000 7f43a9800640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43a4104270 0x7f43a41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.945+0000 7f43aa200640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a419a720 0x7f43a419aba0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f439c009bf0 tx=0x7f439c032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.945+0000 7f43a2000640 1 -- 172.21.15.22:0/287105709 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c041020 con 0x7f43a419a720 2024-04-20T21:45:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.945+0000 7f43aac00640 1 -- 172.21.15.22:0/287105709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a419b5b0 con 0x7f43a419a720 2024-04-20T21:45:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.945+0000 7f43aac00640 1 -- 172.21.15.22:0/287105709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a41a76f0 con 0x7f43a419a720 2024-04-20T21:45:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.945+0000 7f43a2000640 1 -- 172.21.15.22:0/287105709 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f439c034cd0 con 0x7f43a419a720 2024-04-20T21:45:06.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.945+0000 7f43a2000640 1 -- 172.21.15.22:0/287105709 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c039040 con 0x7f43a419a720 2024-04-20T21:45:06.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.946+0000 7f43aac00640 1 -- 172.21.15.22:0/287105709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a4105740 con 0x7f43a419a720 2024-04-20T21:45:06.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.948+0000 7f43a2000640 1 -- 172.21.15.22:0/287105709 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f439c044020 con 0x7f43a419a720 2024-04-20T21:45:06.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.950+0000 7f43a2000640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43880a3d20 0x7f43880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.950+0000 7f43a2000640 1 -- 172.21.15.22:0/287105709 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8686+0+0 (secure 0 0 0) 0x7f439c0f2570 con 0x7f43a419a720 2024-04-20T21:45:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.950+0000 7f43a9800640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43880a3d20 0x7f43880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:06.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.951+0000 7f43a9800640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43880a3d20 0x7f43880a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4394009fc0 tx=0x7f4394005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:06.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:06.952+0000 7f43a2000640 1 -- 172.21.15.22:0/287105709 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f439c04b030 con 0x7f43a419a720 2024-04-20T21:45:07.025 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:45:06 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-i[52837]: ignoring --setuser ceph since I am not root 2024-04-20T21:45:07.025 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:45:06 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-i[52837]: ignoring --setgroup ceph since I am not root 2024-04-20T21:45:07.025 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:45:06 smithi122 ceph-mds[52841]: -- 172.21.15.122:0/1173717160 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600b54b1a40 con 0x5600b5484800 2024-04-20T21:45:07.025 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:45:06 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-i[52837]: starting mds.i at 2024-04-20T21:45:07.053 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:45:06 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b[51942]: ignoring --setuser ceph since I am not root 2024-04-20T21:45:07.053 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:45:06 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b[51942]: ignoring --setgroup ceph since I am not root 2024-04-20T21:45:07.054 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:45:06 smithi033 ceph-mds[51946]: -- 172.21.15.33:0/1750431795 <== mon.2 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c9538fa40 con 0x556c95362800 2024-04-20T21:45:07.054 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:45:06 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b[51942]: starting mds.b at 2024-04-20T21:45:07.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:06 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a[16839]: 2024-04-20T21:45:06.718+0000 7f2f99a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:45:07.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.187+0000 7f43aac00640 1 -- 172.21.15.22:0/287105709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f43a410c5d0 con 0x7f43a419a720 2024-04-20T21:45:07.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:07 smithi033 ceph-mon[21626]: osdmap e104: 12 total, 12 up, 12 in 2024-04-20T21:45:07.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:07 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2839186702' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T21:45:07.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:07 smithi033 ceph-mon[21626]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T21:45:07.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:07 smithi033 ceph-mon[21626]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:45:07.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:07 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2839186702' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T21:45:07.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:07 smithi033 ceph-mon[21626]: osdmap e105: 12 total, 12 up, 12 in 2024-04-20T21:45:07.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:07 smithi033 ceph-mon[21626]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-20T21:45:07.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:07 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/287105709' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:45:07.555 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:07 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:45:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:07 smithi022 ceph-mon[16843]: osdmap e104: 12 total, 12 up, 12 in 2024-04-20T21:45:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:07 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2839186702' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T21:45:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:07 smithi022 ceph-mon[16843]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T21:45:07.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:07 smithi022 ceph-mon[16843]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:45:07.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:07 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2839186702' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T21:45:07.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:07 smithi022 ceph-mon[16843]: osdmap e105: 12 total, 12 up, 12 in 2024-04-20T21:45:07.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:07 smithi022 ceph-mon[16843]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-20T21:45:07.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:07 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/287105709' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:45:07.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:07 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:45:07.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.736+0000 7f43a2000640 1 -- 172.21.15.22:0/287105709 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f439c0bec20 con 0x7f43a419a720 2024-04-20T21:45:07.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.741+0000 7f43aac00640 1 -- 172.21.15.22:0/287105709 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43880a3d20 msgr2=0x7f43880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:07.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.741+0000 7f43aac00640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43880a3d20 0x7f43880a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4394009fc0 tx=0x7f4394005f90 comp rx=0 tx=0).stop 2024-04-20T21:45:07.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.741+0000 7f43aac00640 1 -- 172.21.15.22:0/287105709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a419a720 msgr2=0x7f43a419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:07.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.741+0000 7f43aac00640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a419a720 0x7f43a419aba0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f439c009bf0 tx=0x7f439c032ea0 comp rx=0 tx=0).stop 2024-04-20T21:45:07.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.741+0000 7f43aac00640 1 -- 172.21.15.22:0/287105709 shutdown_connections 2024-04-20T21:45:07.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.741+0000 7f43aac00640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f43880a3d20 0x7f43880a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:07.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.741+0000 7f43aac00640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a419a720 0x7f43a419aba0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:07.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.741+0000 7f43aac00640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43a4109340 0x7f43a41a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:07.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.741+0000 7f43aac00640 1 --2- 172.21.15.22:0/287105709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43a4104270 0x7f43a41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:07.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.741+0000 7f43aac00640 1 -- 172.21.15.22:0/287105709 >> 172.21.15.22:0/287105709 conn(0x7f43a40ffad0 msgr2=0x7f43a4101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:07.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.742+0000 7f43aac00640 1 -- 172.21.15.22:0/287105709 shutdown_connections 2024-04-20T21:45:07.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.742+0000 7f43aac00640 1 -- 172.21.15.22:0/287105709 wait complete. 2024-04-20T21:45:07.762 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T21:45:07.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:07 smithi122 ceph-mon[22147]: osdmap e104: 12 total, 12 up, 12 in 2024-04-20T21:45:07.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:07 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2839186702' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T21:45:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:07 smithi122 ceph-mon[22147]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T21:45:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:07 smithi122 ceph-mon[22147]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:45:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:07 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2839186702' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T21:45:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:07 smithi122 ceph-mon[22147]: osdmap e105: 12 total, 12 up, 12 in 2024-04-20T21:45:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:07 smithi122 ceph-mon[22147]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-20T21:45:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:07 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/287105709' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:45:07.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:07 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:45:07.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.979+0000 7f8237e00640 1 -- 172.21.15.22:0/2579220554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8238074300 msgr2=0x7f82380746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:07.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.979+0000 7f8237e00640 1 --2- 172.21.15.22:0/2579220554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8238074300 0x7f82380746e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8228009860 tx=0x7f822802bc00 comp rx=0 tx=0).stop 2024-04-20T21:45:07.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.979+0000 7f8237e00640 1 -- 172.21.15.22:0/2579220554 shutdown_connections 2024-04-20T21:45:07.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.979+0000 7f8237e00640 1 --2- 172.21.15.22:0/2579220554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8238112d10 0x7f8238115100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:07.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.979+0000 7f8237e00640 1 --2- 172.21.15.22:0/2579220554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238074c20 0x7f82381127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:07.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.979+0000 7f8237e00640 1 --2- 172.21.15.22:0/2579220554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8238074300 0x7f82380746e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:07.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.979+0000 7f8237e00640 1 -- 172.21.15.22:0/2579220554 >> 172.21.15.22:0/2579220554 conn(0x7f82380fd830 msgr2=0x7f82380ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:07.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.979+0000 7f8237e00640 1 -- 172.21.15.22:0/2579220554 shutdown_connections 2024-04-20T21:45:07.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.979+0000 7f8237e00640 1 -- 172.21.15.22:0/2579220554 wait complete. 2024-04-20T21:45:07.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.980+0000 7f8237e00640 1 Processor -- start 2024-04-20T21:45:07.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.980+0000 7f8237e00640 1 -- start start 2024-04-20T21:45:07.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.980+0000 7f8237e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238074300 0x7f82381a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:07.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.980+0000 7f8237e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8238074c20 0x7f82381a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:07.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.980+0000 7f8237e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8238112d10 0x7f823819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:07.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8237e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8238118730 con 0x7f8238074c20 2024-04-20T21:45:07.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8237e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82381185b0 con 0x7f8238074300 2024-04-20T21:45:07.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8237e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f82381188b0 con 0x7f8238112d10 2024-04-20T21:45:07.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8236000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8238074c20 0x7f82381a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:07.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8236000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8238074c20 0x7f82381a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40852/0 (socket says 172.21.15.22:40852) 2024-04-20T21:45:07.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8236000640 1 -- 172.21.15.22:0/919090461 learned_addr learned my addr 172.21.15.22:0/919090461 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:45:07.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8236a00640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238074300 0x7f82381a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:07.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8237400640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8238112d10 0x7f823819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:07.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8236000640 1 -- 172.21.15.22:0/919090461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8238112d10 msgr2=0x7f823819ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:07.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8236000640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8238112d10 0x7f823819ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:07.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8236000640 1 -- 172.21.15.22:0/919090461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238074300 msgr2=0x7f82381a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:07.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8236000640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238074300 0x7f82381a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:07.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8236000640 1 -- 172.21.15.22:0/919090461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823819f250 con 0x7f8238074c20 2024-04-20T21:45:07.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.981+0000 7f8236a00640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238074300 0x7f82381a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:45:07.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.982+0000 7f8236000640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8238074c20 0x7f82381a4e60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f822400e9c0 tx=0x7f822400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:07.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.982+0000 7f823ca00640 1 -- 172.21.15.22:0/919090461 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822400cde0 con 0x7f8238074c20 2024-04-20T21:45:07.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.982+0000 7f823ca00640 1 -- 172.21.15.22:0/919090461 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82240035f0 con 0x7f8238074c20 2024-04-20T21:45:07.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.982+0000 7f8237e00640 1 -- 172.21.15.22:0/919090461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823819f540 con 0x7f8238074c20 2024-04-20T21:45:07.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.982+0000 7f823ca00640 1 -- 172.21.15.22:0/919090461 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224010690 con 0x7f8238074c20 2024-04-20T21:45:07.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.982+0000 7f8237e00640 1 -- 172.21.15.22:0/919090461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82381ab7e0 con 0x7f8238074c20 2024-04-20T21:45:07.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.983+0000 7f8237e00640 1 -- 172.21.15.22:0/919090461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82380690f0 con 0x7f8238074c20 2024-04-20T21:45:07.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.985+0000 7f823ca00640 1 -- 172.21.15.22:0/919090461 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f82240037a0 con 0x7f8238074c20 2024-04-20T21:45:07.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.988+0000 7f823ca00640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f821c0a3d60 0x7f821c0a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:07.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.988+0000 7f823ca00640 1 -- 172.21.15.22:0/919090461 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8686+0+0 (secure 0 0 0) 0x7f82240d0d60 con 0x7f8238074c20 2024-04-20T21:45:07.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.988+0000 7f8236a00640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f821c0a3d60 0x7f821c0a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:07.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.989+0000 7f8236a00640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f821c0a3d60 0x7f821c0a6220 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f822802c210 tx=0x7f822802c120 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:07.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:07.990+0000 7f823ca00640 1 -- 172.21.15.22:0/919090461 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f822409d300 con 0x7f8238074c20 2024-04-20T21:45:08.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.194+0000 7f8237e00640 1 -- 172.21.15.22:0/919090461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f823819ffc0 con 0x7f8238074c20 2024-04-20T21:45:08.553 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:08 smithi033 ceph-mon[21626]: pgmap v354: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 67 KiB/s wr, 29 op/s 2024-04-20T21:45:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:08 smithi033 ceph-mon[21626]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T21:45:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:08 smithi033 ceph-mon[21626]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T21:45:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:08 smithi033 ceph-mon[21626]: Cluster is now healthy 2024-04-20T21:45:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:08 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-20T21:45:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:08 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.122:6838/3394792410,v1:172.21.15.122:6839/3394792410] up:boot 2024-04-20T21:45:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:08 smithi033 ceph-mon[21626]: mds.? [v2:172.21.15.33:6838/219629192,v1:172.21.15.33:6839/219629192] up:boot 2024-04-20T21:45:08.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:08 smithi033 ceph-mon[21626]: fsmap 12 up:standby 2024-04-20T21:45:08.555 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:08 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:45:08.555 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:08 smithi033 ceph-mon[21626]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:45:08.555 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:08 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/919090461' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:45:08.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:08 smithi022 ceph-mon[16843]: pgmap v354: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 67 KiB/s wr, 29 op/s 2024-04-20T21:45:08.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:08 smithi022 ceph-mon[16843]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T21:45:08.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:08 smithi022 ceph-mon[16843]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T21:45:08.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:08 smithi022 ceph-mon[16843]: Cluster is now healthy 2024-04-20T21:45:08.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:08 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-20T21:45:08.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:08 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.122:6838/3394792410,v1:172.21.15.122:6839/3394792410] up:boot 2024-04-20T21:45:08.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:08 smithi022 ceph-mon[16843]: mds.? [v2:172.21.15.33:6838/219629192,v1:172.21.15.33:6839/219629192] up:boot 2024-04-20T21:45:08.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:08 smithi022 ceph-mon[16843]: fsmap 12 up:standby 2024-04-20T21:45:08.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:08 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:45:08.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:08 smithi022 ceph-mon[16843]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:45:08.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:08 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/919090461' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:45:08.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.736+0000 7f823ca00640 1 -- 172.21.15.22:0/919090461 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v106) ==== 181+0+0 (secure 0 0 0) 0x7f822409cc00 con 0x7f8238074c20 2024-04-20T21:45:08.736 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-04-20T21:45:08.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.744+0000 7f8237e00640 1 -- 172.21.15.22:0/919090461 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f821c0a3d60 msgr2=0x7f821c0a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:08.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.744+0000 7f8237e00640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f821c0a3d60 0x7f821c0a6220 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f822802c210 tx=0x7f822802c120 comp rx=0 tx=0).stop 2024-04-20T21:45:08.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.744+0000 7f8237e00640 1 -- 172.21.15.22:0/919090461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8238074c20 msgr2=0x7f82381a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:08.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.744+0000 7f8237e00640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8238074c20 0x7f82381a4e60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f822400e9c0 tx=0x7f822400ee80 comp rx=0 tx=0).stop 2024-04-20T21:45:08.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.744+0000 7f8237e00640 1 -- 172.21.15.22:0/919090461 shutdown_connections 2024-04-20T21:45:08.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.744+0000 7f8237e00640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f821c0a3d60 0x7f821c0a6220 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.744+0000 7f8237e00640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8238112d10 0x7f823819ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.744+0000 7f8237e00640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8238074c20 0x7f82381a4e60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.744+0000 7f8237e00640 1 --2- 172.21.15.22:0/919090461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238074300 0x7f82381a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.744+0000 7f8237e00640 1 -- 172.21.15.22:0/919090461 >> 172.21.15.22:0/919090461 conn(0x7f82380fd830 msgr2=0x7f82380ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:08.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.745+0000 7f8237e00640 1 -- 172.21.15.22:0/919090461 shutdown_connections 2024-04-20T21:45:08.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.745+0000 7f8237e00640 1 -- 172.21.15.22:0/919090461 wait complete. 2024-04-20T21:45:08.773 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T21:45:08.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:08 smithi122 ceph-mon[22147]: pgmap v354: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 67 KiB/s wr, 29 op/s 2024-04-20T21:45:08.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:08 smithi122 ceph-mon[22147]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T21:45:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:08 smithi122 ceph-mon[22147]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T21:45:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:08 smithi122 ceph-mon[22147]: Cluster is now healthy 2024-04-20T21:45:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:08 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-20T21:45:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:08 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.122:6838/3394792410,v1:172.21.15.122:6839/3394792410] up:boot 2024-04-20T21:45:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:08 smithi122 ceph-mon[22147]: mds.? [v2:172.21.15.33:6838/219629192,v1:172.21.15.33:6839/219629192] up:boot 2024-04-20T21:45:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:08 smithi122 ceph-mon[22147]: fsmap 12 up:standby 2024-04-20T21:45:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:08 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:45:08.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:08 smithi122 ceph-mon[22147]: from='mgr.15024 172.21.15.22:0/1907499287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:45:08.776 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:08 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/919090461' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:45:08.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.962+0000 7f8f8ce00640 1 -- 172.21.15.22:0/765289407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f88074cb0 msgr2=0x7f8f8810e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:08.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.962+0000 7f8f8ce00640 1 --2- 172.21.15.22:0/765289407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f88074cb0 0x7f8f8810e100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009900 tx=0x7f8f7002bcc0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.962+0000 7f8f8ce00640 1 -- 172.21.15.22:0/765289407 shutdown_connections 2024-04-20T21:45:08.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.962+0000 7f8f8ce00640 1 --2- 172.21.15.22:0/765289407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8810e640 0x7f8f88110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.962+0000 7f8f8ce00640 1 --2- 172.21.15.22:0/765289407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f88074cb0 0x7f8f8810e100 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.962+0000 7f8f8ce00640 1 --2- 172.21.15.22:0/765289407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f88074390 0x7f8f88074770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.962+0000 7f8f8ce00640 1 -- 172.21.15.22:0/765289407 >> 172.21.15.22:0/765289407 conn(0x7f8f880fd930 msgr2=0x7f8f880ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:08.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.962+0000 7f8f8ce00640 1 -- 172.21.15.22:0/765289407 shutdown_connections 2024-04-20T21:45:08.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.962+0000 7f8f8ce00640 1 -- 172.21.15.22:0/765289407 wait complete. 2024-04-20T21:45:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.963+0000 7f8f8ce00640 1 Processor -- start 2024-04-20T21:45:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.964+0000 7f8f8ce00640 1 -- start start 2024-04-20T21:45:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.964+0000 7f8f8ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f88074390 0x7f8f881a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.964+0000 7f8f8ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f88074cb0 0x7f8f881a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.964+0000 7f8f8ce00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8810e640 0x7f8f8819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:08.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.964+0000 7f8f8ce00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881143d0 con 0x7f8f88074390 2024-04-20T21:45:08.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.964+0000 7f8f8ce00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88114250 con 0x7f8f8810e640 2024-04-20T21:45:08.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.964+0000 7f8f8ce00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88114550 con 0x7f8f88074cb0 2024-04-20T21:45:08.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.964+0000 7f8f87400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f88074390 0x7f8f881a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:08.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.964+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8810e640 0x7f8f8819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:08.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.964+0000 7f8f7ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f88074cb0 0x7f8f881a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:08.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.965+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8810e640 0x7f8f8819a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.22:50836/0 (socket says 172.21.15.22:50836) 2024-04-20T21:45:08.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.965+0000 7f8f87e00640 1 -- 172.21.15.22:0/2990645217 learned_addr learned my addr 172.21.15.22:0/2990645217 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-04-20T21:45:08.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.965+0000 7f8f87400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f88074390 0x7f8f881a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40876/0 (socket says 172.21.15.22:40876) 2024-04-20T21:45:08.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.965+0000 7f8f7ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f88074cb0 0x7f8f881a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.22:51550/0 (socket says 172.21.15.22:51550) 2024-04-20T21:45:08.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.965+0000 7f8f87e00640 1 -- 172.21.15.22:0/2990645217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f88074cb0 msgr2=0x7f8f881a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:08.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.965+0000 7f8f87e00640 1 --2- 172.21.15.22:0/2990645217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f88074cb0 0x7f8f881a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.965+0000 7f8f87e00640 1 -- 172.21.15.22:0/2990645217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f88074390 msgr2=0x7f8f881a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:08.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.965+0000 7f8f87e00640 1 --2- 172.21.15.22:0/2990645217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f88074390 0x7f8f881a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:08.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.965+0000 7f8f87e00640 1 -- 172.21.15.22:0/2990645217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8819aff0 con 0x7f8f8810e640 2024-04-20T21:45:08.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.965+0000 7f8f87e00640 1 --2- 172.21.15.22:0/2990645217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8810e640 0x7f8f8819a820 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000eb10 tx=0x7f8f8000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:08.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.965+0000 7f8f84c00640 1 -- 172.21.15.22:0/2990645217 <== mon.2 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80004b20 con 0x7f8f8810e640 2024-04-20T21:45:08.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.966+0000 7f8f8ce00640 1 -- 172.21.15.22:0/2990645217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8819b2e0 con 0x7f8f8810e640 2024-04-20T21:45:08.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.966+0000 7f8f84c00640 1 -- 172.21.15.22:0/2990645217 <== mon.2 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f80004cc0 con 0x7f8f8810e640 2024-04-20T21:45:08.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.966+0000 7f8f84c00640 1 -- 172.21.15.22:0/2990645217 <== mon.2 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80010760 con 0x7f8f8810e640 2024-04-20T21:45:08.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.966+0000 7f8f8ce00640 1 -- 172.21.15.22:0/2990645217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f881a74b0 con 0x7f8f8810e640 2024-04-20T21:45:08.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.967+0000 7f8f8ce00640 1 -- 172.21.15.22:0/2990645217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f88075a40 con 0x7f8f8810e640 2024-04-20T21:45:08.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.969+0000 7f8f84c00640 1 -- 172.21.15.22:0/2990645217 <== mon.2 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8f80005020 con 0x7f8f8810e640 2024-04-20T21:45:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.970+0000 7f8f84c00640 1 --2- 172.21.15.22:0/2990645217 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f8f6c0a3d20 0x7f8f6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:45:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.971+0000 7f8f84c00640 1 -- 172.21.15.22:0/2990645217 <== mon.2 v2:172.21.15.33:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8132+0+0 (secure 0 0 0) 0x7f8f800d0d00 con 0x7f8f8810e640 2024-04-20T21:45:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.971+0000 7f8f87400640 1 --2- 172.21.15.22:0/2990645217 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f8f6c0a3d20 0x7f8f6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:45:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.971+0000 7f8f87400640 1 --2- 172.21.15.22:0/2990645217 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f8f6c0a3d20 0x7f8f6c0a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009bd0 tx=0x7f8f78005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:45:08.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:08.975+0000 7f8f84c00640 1 -- 172.21.15.22:0/2990645217 <== mon.2 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8f8009d4d0 con 0x7f8f8810e640 2024-04-20T21:45:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.205+0000 7f8f8ce00640 1 -- 172.21.15.22:0/2990645217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8f881097c0 con 0x7f8f8810e640 2024-04-20T21:45:09.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.755+0000 7f8f84c00640 1 -- 172.21.15.22:0/2990645217 <== mon.2 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v107) ==== 169+0+0 (secure 0 0 0) 0x7f8f8009cdd0 con 0x7f8f8810e640 2024-04-20T21:45:09.757 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-04-20T21:45:09.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.761+0000 7f8f8ce00640 1 -- 172.21.15.22:0/2990645217 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f8f6c0a3d20 msgr2=0x7f8f6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:09.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.761+0000 7f8f8ce00640 1 --2- 172.21.15.22:0/2990645217 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f8f6c0a3d20 0x7f8f6c0a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009bd0 tx=0x7f8f78005f90 comp rx=0 tx=0).stop 2024-04-20T21:45:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.761+0000 7f8f8ce00640 1 -- 172.21.15.22:0/2990645217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8810e640 msgr2=0x7f8f8819a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:45:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.761+0000 7f8f8ce00640 1 --2- 172.21.15.22:0/2990645217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8810e640 0x7f8f8819a820 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000eb10 tx=0x7f8f8000c2d0 comp rx=0 tx=0).stop 2024-04-20T21:45:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.761+0000 7f8f8ce00640 1 -- 172.21.15.22:0/2990645217 shutdown_connections 2024-04-20T21:45:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.761+0000 7f8f8ce00640 1 --2- 172.21.15.22:0/2990645217 >> [v2:172.21.15.22:6800/975472489,v1:172.21.15.22:6801/975472489] conn(0x7f8f6c0a3d20 0x7f8f6c0a61e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.761+0000 7f8f8ce00640 1 --2- 172.21.15.22:0/2990645217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8810e640 0x7f8f8819a820 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.761+0000 7f8f8ce00640 1 --2- 172.21.15.22:0/2990645217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f88074cb0 0x7f8f881a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.761+0000 7f8f8ce00640 1 --2- 172.21.15.22:0/2990645217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f88074390 0x7f8f881a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:45:09.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.761+0000 7f8f8ce00640 1 -- 172.21.15.22:0/2990645217 >> 172.21.15.22:0/2990645217 conn(0x7f8f880fd930 msgr2=0x7f8f880fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:45:09.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.762+0000 7f8f8ce00640 1 -- 172.21.15.22:0/2990645217 shutdown_connections 2024-04-20T21:45:09.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-04-20T21:45:09.762+0000 7f8f8ce00640 1 -- 172.21.15.22:0/2990645217 wait complete. 2024-04-20T21:45:09.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:45:09.862 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:45:09.894 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:45:09.922 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-20T21:45:09.922 DEBUG:teuthology.orchestra.run.smithi022:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T21:45:09.975 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T21:45:09.992 DEBUG:teuthology.orchestra.run.smithi122:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T21:45:10.008 INFO:tasks.cephadm:Stopping all daemons... 2024-04-20T21:45:10.008 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-20T21:45:10.008 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.a 2024-04-20T21:45:10.024 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:09 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/919090461' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T21:45:10.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:09 smithi122 ceph-mon[22147]: osdmap e106: 12 total, 12 up, 12 in 2024-04-20T21:45:10.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:09 smithi122 ceph-mon[22147]: pgmap v356: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:45:10.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:09 smithi122 ceph-mon[22147]: from='client.? 172.21.15.22:0/2990645217' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:45:10.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:09 smithi122 ceph-mon[22147]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:45:10.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:09 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/919090461' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T21:45:10.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:09 smithi033 ceph-mon[21626]: osdmap e106: 12 total, 12 up, 12 in 2024-04-20T21:45:10.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:09 smithi033 ceph-mon[21626]: pgmap v356: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:45:10.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:09 smithi033 ceph-mon[21626]: from='client.? 172.21.15.22:0/2990645217' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:45:10.054 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:09 smithi033 ceph-mon[21626]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:45:10.083 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:09 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/919090461' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T21:45:10.083 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:09 smithi022 ceph-mon[16843]: osdmap e106: 12 total, 12 up, 12 in 2024-04-20T21:45:10.083 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:09 smithi022 ceph-mon[16843]: pgmap v356: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:45:10.083 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:09 smithi022 ceph-mon[16843]: from='client.? 172.21.15.22:0/2990645217' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:45:10.083 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:09 smithi022 ceph-mon[16843]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:45:10.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:10 smithi022 systemd[1]: Stopping Ceph mon.a for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:10.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:10 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a[16839]: 2024-04-20T21:45:10.597+0000 7f2fa0800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:10.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:10 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a[16839]: 2024-04-20T21:45:10.597+0000 7f2fa0800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-20T21:45:10.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:10 smithi022 podman[78547]: 2024-04-20 21:45:10.711564202 +0000 UTC m=+0.257730141 container died 4f99bfab2067b8b8eb61f95fc827b57a5dc4e2949e812d4b08305a36cd942d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-20T21:45:11.343 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:11 smithi022 podman[78547]: 2024-04-20 21:45:11.128451144 +0000 UTC m=+0.674617063 container cleanup 4f99bfab2067b8b8eb61f95fc827b57a5dc4e2949e812d4b08305a36cd942d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=) 2024-04-20T21:45:11.343 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:11 smithi022 bash[78547]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a 2024-04-20T21:45:11.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Apr 20 21:45:11 smithi022 podman[78561]: 2024-04-20 21:45:11.34367836 +0000 UTC m=+0.602724929 container remove 4f99bfab2067b8b8eb61f95fc827b57a5dc4e2949e812d4b08305a36cd942d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS) 2024-04-20T21:45:11.821 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.a.service' 2024-04-20T21:45:11.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:11.963 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-20T21:45:11.963 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-20T21:45:11.963 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.b 2024-04-20T21:45:12.248 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:12 smithi033 systemd[1]: Stopping Ceph mon.b for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:12.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:12 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-b[21622]: 2024-04-20T21:45:12.250+0000 7f9a263f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:12.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:12 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-b[21622]: 2024-04-20T21:45:12.250+0000 7f9a263f2700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-20T21:45:12.554 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:12 smithi033 podman[57302]: 2024-04-20 21:45:12.363192174 +0000 UTC m=+0.201938567 container died 2ea070e847cc8cf1e26fdec595c3d8ebac83f31882853d8eb0460bec2694ee81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-20T21:45:13.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:12 smithi033 podman[57302]: 2024-04-20 21:45:12.825430813 +0000 UTC m=+0.664177200 container cleanup 2ea070e847cc8cf1e26fdec595c3d8ebac83f31882853d8eb0460bec2694ee81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-20T21:45:13.053 INFO:journalctl@ceph.mon.b.smithi033.stdout:Apr 20 21:45:12 smithi033 bash[57302]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-b 2024-04-20T21:45:13.268 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.b.service' 2024-04-20T21:45:13.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:13.312 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-20T21:45:13.312 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-20T21:45:13.312 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.c 2024-04-20T21:45:13.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:13 smithi122 systemd[1]: Stopping Ceph mon.c for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:14.024 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:13 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-c[22141]: 2024-04-20T21:45:13.723+0000 7f52f1e6c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:14.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:13 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-c[22141]: 2024-04-20T21:45:13.723+0000 7f52f1e6c700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-20T21:45:14.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:13 smithi122 podman[58505]: 2024-04-20 21:45:13.816144868 +0000 UTC m=+0.247415449 container died b8ff97a8380d4ea0fa269872f67a8ec4bd94b43090a86820b8e75c8d952d9e62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-04-20T21:45:14.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:14 smithi122 podman[58505]: 2024-04-20 21:45:14.216511811 +0000 UTC m=+0.647782398 container cleanup b8ff97a8380d4ea0fa269872f67a8ec4bd94b43090a86820b8e75c8d952d9e62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:45:14.311 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:14 smithi122 bash[58505]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-c 2024-04-20T21:45:14.646 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 20 21:45:14 smithi122 podman[58519]: 2024-04-20 21:45:14.485090902 +0000 UTC m=+0.666959682 container remove b8ff97a8380d4ea0fa269872f67a8ec4bd94b43090a86820b8e75c8d952d9e62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-20T21:45:14.664 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mon.c.service' 2024-04-20T21:45:14.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:14.711 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-20T21:45:14.711 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-20T21:45:14.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.x 2024-04-20T21:45:15.156 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:45:14 smithi022 systemd[1]: Stopping Ceph mgr.x for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:15.657 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:45:15 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:45:15.400+0000 7f242ca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:15.657 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:45:15 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x[17071]: 2024-04-20T21:45:15.400+0000 7f242ca00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-20T21:45:15.657 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:45:15 smithi022 podman[78667]: 2024-04-20 21:45:15.506357136 +0000 UTC m=+0.344428667 container died 3f9d41b7577e9669ba391993b737e7ff6eacccbd34d3538e76d306dea63e8437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-20T21:45:16.312 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:45:16 smithi022 podman[78667]: 2024-04-20 21:45:16.185334329 +0000 UTC m=+1.023405873 container cleanup 3f9d41b7577e9669ba391993b737e7ff6eacccbd34d3538e76d306dea63e8437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:45:16.312 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:45:16 smithi022 bash[78667]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x 2024-04-20T21:45:16.656 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Apr 20 21:45:16 smithi022 podman[78681]: 2024-04-20 21:45:16.436838413 +0000 UTC m=+0.906787436 container remove 3f9d41b7577e9669ba391993b737e7ff6eacccbd34d3538e76d306dea63e8437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-20T21:45:16.949 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.x.service' 2024-04-20T21:45:17.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:17.056 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-20T21:45:17.057 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-20T21:45:17.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.y 2024-04-20T21:45:17.407 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:45:17 smithi033 systemd[1]: Stopping Ceph mgr.y for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:17.803 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:45:17 smithi033 podman[57425]: 2024-04-20 21:45:17.408728568 +0000 UTC m=+0.147895322 container died f64fc78265e2e7446da8cbf1b18c0bdedd390e3f6d3a267c4e8fb274afd849c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-20T21:45:18.145 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:45:17 smithi033 podman[57425]: 2024-04-20 21:45:17.89635723 +0000 UTC m=+0.635523985 container cleanup f64fc78265e2e7446da8cbf1b18c0bdedd390e3f6d3a267c4e8fb274afd849c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-20T21:45:18.146 INFO:journalctl@ceph.mgr.y.smithi033.stdout:Apr 20 21:45:17 smithi033 bash[57425]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-y 2024-04-20T21:45:18.244 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.y.service' 2024-04-20T21:45:18.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:18.279 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-20T21:45:18.279 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-20T21:45:18.279 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.z 2024-04-20T21:45:18.623 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:45:18 smithi122 systemd[1]: Stopping Ceph mgr.z for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:18.903 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:45:18 smithi122 podman[58639]: 2024-04-20 21:45:18.625682621 +0000 UTC m=+0.155443662 container died 9a11fad32beeb9637ed9476ba1bed300af6d8c776073f1ee9beaa967a512cff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:45:19.254 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:45:18 smithi122 podman[58639]: 2024-04-20 21:45:18.979735399 +0000 UTC m=+0.509496438 container cleanup 9a11fad32beeb9637ed9476ba1bed300af6d8c776073f1ee9beaa967a512cff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-20T21:45:19.254 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 20 21:45:18 smithi122 bash[58639]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mgr-z 2024-04-20T21:45:19.392 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mgr.z.service' 2024-04-20T21:45:19.425 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:19.425 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-20T21:45:19.426 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-20T21:45:19.426 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.0 2024-04-20T21:45:19.906 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:19 smithi022 systemd[1]: Stopping Ceph osd.0 for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:20.407 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:20 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-0[29066]: 2024-04-20T21:45:20.038+0000 7fefc8a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:20.407 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:20 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-0[29066]: 2024-04-20T21:45:20.038+0000 7fefc8a00700 -1 osd.0 107 *** Got signal Terminated *** 2024-04-20T21:45:20.407 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:20 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-0[29066]: 2024-04-20T21:45:20.038+0000 7fefc8a00700 -1 osd.0 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:45:25.704 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:25 smithi022 podman[78789]: 2024-04-20 21:45:25.449208529 +0000 UTC m=+5.574462551 container died c38a89d36d8e848d4bf162d8dc4bf32a1c352d0e03b1a151d90179e997af7c91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-20T21:45:26.131 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:25 smithi022 podman[78789]: 2024-04-20 21:45:25.897533821 +0000 UTC m=+6.022787850 container cleanup c38a89d36d8e848d4bf162d8dc4bf32a1c352d0e03b1a151d90179e997af7c91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-20T21:45:26.131 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:25 smithi022 bash[78789]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-0 2024-04-20T21:45:26.406 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:26 smithi022 podman[78807]: 2024-04-20 21:45:26.132165052 +0000 UTC m=+0.665393702 container remove c38a89d36d8e848d4bf162d8dc4bf32a1c352d0e03b1a151d90179e997af7c91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-20T21:45:27.303 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:27 smithi022 podman[78898]: 2024-04-20 21:45:26.925997253 +0000 UTC m=+0.034991129 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:45:27.303 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:27 smithi022 podman[78898]: 2024-04-20 21:45:27.032962609 +0000 UTC m=+0.141956562 container create c05ee38a22762f15842848543d8130af18e185d6895d2d4808bcbf1037d40b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-20T21:45:27.657 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:27 smithi022 podman[78898]: 2024-04-20 21:45:27.304067594 +0000 UTC m=+0.413061514 container init c05ee38a22762f15842848543d8130af18e185d6895d2d4808bcbf1037d40b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-20T21:45:27.657 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:27 smithi022 podman[78898]: 2024-04-20 21:45:27.315977341 +0000 UTC m=+0.424971227 container start c05ee38a22762f15842848543d8130af18e185d6895d2d4808bcbf1037d40b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True) 2024-04-20T21:45:27.657 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:27 smithi022 podman[78898]: 2024-04-20 21:45:27.41872055 +0000 UTC m=+0.527714503 container attach c05ee38a22762f15842848543d8130af18e185d6895d2d4808bcbf1037d40b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-20T21:45:28.119 INFO:journalctl@ceph.osd.0.smithi022.stdout:Apr 20 21:45:27 smithi022 podman[78898]: 2024-04-20 21:45:27.789598777 +0000 UTC m=+0.898592744 container died c05ee38a22762f15842848543d8130af18e185d6895d2d4808bcbf1037d40b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-20T21:45:28.475 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.0.service' 2024-04-20T21:45:28.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:28.584 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-20T21:45:28.584 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-20T21:45:28.585 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.1 2024-04-20T21:45:28.873 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:28 smithi033 systemd[1]: Stopping Ceph osd.1 for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:29.303 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:28 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1[26204]: 2024-04-20T21:45:28.873+0000 7fe9e1d7d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:29.303 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:28 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1[26204]: 2024-04-20T21:45:28.873+0000 7fe9e1d7d700 -1 osd.1 107 *** Got signal Terminated *** 2024-04-20T21:45:29.303 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:28 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1[26204]: 2024-04-20T21:45:28.873+0000 7fe9e1d7d700 -1 osd.1 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:45:34.641 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:34 smithi033 podman[57552]: 2024-04-20 21:45:34.338099184 +0000 UTC m=+5.547966069 container died 459d5fa72b8e10d39a8d7a9cf440654d2803203488e355bc10cd439ec183b4cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:45:34.931 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:34 smithi033 podman[57552]: 2024-04-20 21:45:34.741540537 +0000 UTC m=+5.951407409 container cleanup 459d5fa72b8e10d39a8d7a9cf440654d2803203488e355bc10cd439ec183b4cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-20T21:45:34.931 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:34 smithi033 bash[57552]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1 2024-04-20T21:45:35.303 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:34 smithi033 podman[57583]: 2024-04-20 21:45:34.932897576 +0000 UTC m=+0.593467721 container remove 459d5fa72b8e10d39a8d7a9cf440654d2803203488e355bc10cd439ec183b4cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-20T21:45:35.617 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:35 smithi033 podman[57672]: 2024-04-20 21:45:35.272475475 +0000 UTC m=+0.018588832 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:45:35.618 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:35 smithi033 podman[57672]: 2024-04-20 21:45:35.400438447 +0000 UTC m=+0.146551806 container create cfa737581800414df7c5075ab007269c9d0b7882983118c87558ce2136092087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:45:35.618 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:35 smithi033 podman[57672]: 2024-04-20 21:45:35.619525451 +0000 UTC m=+0.365638819 container init cfa737581800414df7c5075ab007269c9d0b7882983118c87558ce2136092087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-20T21:45:35.934 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:35 smithi033 podman[57672]: 2024-04-20 21:45:35.6229146 +0000 UTC m=+0.369027962 container start cfa737581800414df7c5075ab007269c9d0b7882983118c87558ce2136092087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-20T21:45:35.934 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:35 smithi033 podman[57672]: 2024-04-20 21:45:35.70265975 +0000 UTC m=+0.448773107 container attach cfa737581800414df7c5075ab007269c9d0b7882983118c87558ce2136092087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-20T21:45:36.229 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 20 21:45:35 smithi033 podman[57672]: 2024-04-20 21:45:35.935820898 +0000 UTC m=+0.681934257 container died cfa737581800414df7c5075ab007269c9d0b7882983118c87558ce2136092087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-20T21:45:36.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.1.service' 2024-04-20T21:45:36.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:36.561 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-20T21:45:36.561 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-20T21:45:36.561 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.2 2024-04-20T21:45:36.932 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:36 smithi122 systemd[1]: Stopping Ceph osd.2 for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:37.274 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:36 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2[27302]: 2024-04-20T21:45:36.933+0000 7f5cb51f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:37.275 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:36 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2[27302]: 2024-04-20T21:45:36.933+0000 7f5cb51f8700 -1 osd.2 107 *** Got signal Terminated *** 2024-04-20T21:45:37.275 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:36 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2[27302]: 2024-04-20T21:45:36.933+0000 7f5cb51f8700 -1 osd.2 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:45:42.616 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:42 smithi122 podman[58766]: 2024-04-20 21:45:42.334852529 +0000 UTC m=+5.584258679 container died 6a89ee3089fed54d2d11a5cbd48193f9a91b22bba60a079c3b207ed82eef82e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-20T21:45:42.991 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:42 smithi122 podman[58766]: 2024-04-20 21:45:42.717093488 +0000 UTC m=+5.966499630 container cleanup 6a89ee3089fed54d2d11a5cbd48193f9a91b22bba60a079c3b207ed82eef82e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-20T21:45:42.991 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:42 smithi122 bash[58766]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2 2024-04-20T21:45:43.274 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:42 smithi122 podman[58784]: 2024-04-20 21:45:42.993260302 +0000 UTC m=+0.658592518 container remove 6a89ee3089fed54d2d11a5cbd48193f9a91b22bba60a079c3b207ed82eef82e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-20T21:45:43.620 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:43 smithi122 podman[58880]: 2024-04-20 21:45:43.233614018 +0000 UTC m=+0.017911757 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:45:43.620 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:43 smithi122 podman[58880]: 2024-04-20 21:45:43.369162106 +0000 UTC m=+0.153459847 container create f7be02763abb692828069c066dde34f524fab15de4f6d279634660b160107bb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-20T21:45:43.620 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:43 smithi122 podman[58880]: 2024-04-20 21:45:43.622077926 +0000 UTC m=+0.406375667 container init f7be02763abb692828069c066dde34f524fab15de4f6d279634660b160107bb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-20T21:45:43.921 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:43 smithi122 podman[58880]: 2024-04-20 21:45:43.625377309 +0000 UTC m=+0.409675054 container start f7be02763abb692828069c066dde34f524fab15de4f6d279634660b160107bb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:45:43.921 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:43 smithi122 podman[58880]: 2024-04-20 21:45:43.70517918 +0000 UTC m=+0.489476935 container attach f7be02763abb692828069c066dde34f524fab15de4f6d279634660b160107bb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-20T21:45:44.214 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 20 21:45:43 smithi122 podman[58880]: 2024-04-20 21:45:43.923349836 +0000 UTC m=+0.707647579 container died f7be02763abb692828069c066dde34f524fab15de4f6d279634660b160107bb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:45:44.500 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.2.service' 2024-04-20T21:45:44.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:44.540 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-20T21:45:44.540 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-20T21:45:44.540 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.3 2024-04-20T21:45:44.906 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:44 smithi022 systemd[1]: Stopping Ceph osd.3 for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:45.278 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:45 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3[35704]: 2024-04-20T21:45:45.121+0000 7f17edc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:45.279 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:45 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3[35704]: 2024-04-20T21:45:45.121+0000 7f17edc00700 -1 osd.3 107 *** Got signal Terminated *** 2024-04-20T21:45:45.279 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:45 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3[35704]: 2024-04-20T21:45:45.121+0000 7f17edc00700 -1 osd.3 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:45:46.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:46 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:46.090+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:46.906 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:46 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3[35704]: 2024-04-20T21:45:46.727+0000 7f17e9400700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.402715+0000 front 2024-04-20T21:45:24.403056+0000 (oldest deadline 2024-04-20T21:45:46.102844+0000) 2024-04-20T21:45:47.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:47 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:47.078+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:48.064 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:47 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3[35704]: 2024-04-20T21:45:47.761+0000 7f17e9400700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.402715+0000 front 2024-04-20T21:45:24.403056+0000 (oldest deadline 2024-04-20T21:45:46.102844+0000) 2024-04-20T21:45:48.303 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:47 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:47.866+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:45:48.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:48 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:48.065+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:49.065 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:48 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3[35704]: 2024-04-20T21:45:48.731+0000 7f17e9400700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.402715+0000 front 2024-04-20T21:45:24.403056+0000 (oldest deadline 2024-04-20T21:45:46.102844+0000) 2024-04-20T21:45:49.303 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:48 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:48.874+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:45:49.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:49.065+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:49.525 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:49 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:49.063+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:45:50.030 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:49 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3[35704]: 2024-04-20T21:45:49.779+0000 7f17e9400700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.402715+0000 front 2024-04-20T21:45:24.403056+0000 (oldest deadline 2024-04-20T21:45:46.102844+0000) 2024-04-20T21:45:50.274 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:50.017+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:45:50.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:49 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:49.825+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:45:50.303 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:49 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:49.842+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:45:50.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:50 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:50.030+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:50.819 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:50 smithi022 podman[79004]: 2024-04-20 21:45:50.548843548 +0000 UTC m=+5.567022127 container died 92af2d4bf4162e81f6d0ad693c1939e32e892f80c955597074b43bd50cdbd37b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-20T21:45:51.075 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:50 smithi022 podman[79004]: 2024-04-20 21:45:50.931565833 +0000 UTC m=+5.949744439 container cleanup 92af2d4bf4162e81f6d0ad693c1939e32e892f80c955597074b43bd50cdbd37b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-20T21:45:51.075 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:50 smithi022 bash[79004]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3 2024-04-20T21:45:51.274 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:50.991+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:45:51.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:50.843+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:45:51.303 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:50 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:50.844+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:45:51.406 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:51 smithi022 podman[79033]: 2024-04-20 21:45:51.124009415 +0000 UTC m=+0.560706694 container remove 92af2d4bf4162e81f6d0ad693c1939e32e892f80c955597074b43bd50cdbd37b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True) 2024-04-20T21:45:51.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:51 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:51.075+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:51.803 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:51 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:45:51.466+0000 7f9897f39700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:21.149505+0000 front 2024-04-20T21:45:21.149597+0000 (oldest deadline 2024-04-20T21:45:50.549920+0000) 2024-04-20T21:45:51.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:51 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:51.480+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:45:52.243 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:52 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:52.093+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:52.244 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:51 smithi022 podman[79118]: 2024-04-20 21:45:51.867527163 +0000 UTC m=+0.037696532 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:45:52.244 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:51 smithi022 podman[79118]: 2024-04-20 21:45:51.992059009 +0000 UTC m=+0.162228418 container create 591d7e7fc7a664447bc48e33d082e7eda28dcd468f3faea89aac93020e966518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-20T21:45:52.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:51 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:51.967+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:45:52.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:51 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:51.868+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:45:52.303 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:51 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:51.851+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:45:52.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:52 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:52.462+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:45:52.656 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:52 smithi022 podman[79118]: 2024-04-20 21:45:52.245335135 +0000 UTC m=+0.415504564 container init 591d7e7fc7a664447bc48e33d082e7eda28dcd468f3faea89aac93020e966518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-20T21:45:52.656 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:52 smithi022 podman[79118]: 2024-04-20 21:45:52.257274677 +0000 UTC m=+0.427444089 container start 591d7e7fc7a664447bc48e33d082e7eda28dcd468f3faea89aac93020e966518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:45:52.657 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:52 smithi022 podman[79118]: 2024-04-20 21:45:52.33613067 +0000 UTC m=+0.506300091 container attach 591d7e7fc7a664447bc48e33d082e7eda28dcd468f3faea89aac93020e966518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:45:52.803 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:52 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:45:52.470+0000 7f9897f39700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:21.149505+0000 front 2024-04-20T21:45:21.149597+0000 (oldest deadline 2024-04-20T21:45:50.549920+0000) 2024-04-20T21:45:53.076 INFO:journalctl@ceph.osd.3.smithi022.stdout:Apr 20 21:45:52 smithi022 podman[79118]: 2024-04-20 21:45:52.730584171 +0000 UTC m=+0.900753613 container died 591d7e7fc7a664447bc48e33d082e7eda28dcd468f3faea89aac93020e966518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-20T21:45:53.274 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:52.961+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:45:53.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:52.855+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:45:53.303 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:52 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:52.855+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:45:53.396 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:53 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:53.084+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:53.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.3.service' 2024-04-20T21:45:53.556 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:53.557 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-20T21:45:53.557 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-20T21:45:53.557 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.4 2024-04-20T21:45:53.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:53 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:53.487+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:45:53.803 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:53 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:45:53.467+0000 7f9897f39700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:21.149505+0000 front 2024-04-20T21:45:21.149597+0000 (oldest deadline 2024-04-20T21:45:50.549920+0000) 2024-04-20T21:45:53.803 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:53 smithi033 systemd[1]: Stopping Ceph osd.4 for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:54.274 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:53.914+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:45:54.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:53.816+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:45:54.303 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:53 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:53.818+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:45:54.303 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:53 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:45:53.867+0000 7f989f94d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:54.303 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:53 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:45:53.867+0000 7f989f94d700 -1 osd.4 107 *** Got signal Terminated *** 2024-04-20T21:45:54.304 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:53 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:45:53.867+0000 7f989f94d700 -1 osd.4 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:45:54.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:54 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:54.070+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:54.803 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:54 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:45:54.433+0000 7f9897f39700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:21.149505+0000 front 2024-04-20T21:45:21.149597+0000 (oldest deadline 2024-04-20T21:45:50.549920+0000) 2024-04-20T21:45:54.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:54 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:54.455+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:45:55.179 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:54 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:54.932+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:45:55.179 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:54 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:54.809+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:45:55.303 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:54 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:54.837+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:45:55.303 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:54 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:54.837+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:45:55.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:55 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:55.103+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:55.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:55 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:55.103+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:45:55.799 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:55 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:45:55.395+0000 7f9897f39700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:21.149505+0000 front 2024-04-20T21:45:21.149597+0000 (oldest deadline 2024-04-20T21:45:50.549920+0000) 2024-04-20T21:45:55.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:55 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:55.413+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:45:55.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:55 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:55.413+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:45:56.024 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:55.832+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:45:56.025 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:55.910+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:45:56.025 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:55.910+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:32.279999+0000 front 2024-04-20T21:45:32.279964+0000 (oldest deadline 2024-04-20T21:45:55.780037+0000) 2024-04-20T21:45:56.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:55 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:55.800+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:45:56.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:55 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:55.800+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:45:56.375 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:56 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:56.102+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:56.375 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:56 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:56.102+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:45:56.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:56 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:56.375+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:45:56.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:56 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:56.375+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:45:56.795 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:56 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:45:56.348+0000 7f9897f39700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:21.149505+0000 front 2024-04-20T21:45:21.149597+0000 (oldest deadline 2024-04-20T21:45:50.549920+0000) 2024-04-20T21:45:57.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:56 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:56.796+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:45:57.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:56 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:56.796+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:45:57.274 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:56.857+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:45:57.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:56.929+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:45:57.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:56.929+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:32.279999+0000 front 2024-04-20T21:45:32.279964+0000 (oldest deadline 2024-04-20T21:45:55.780037+0000) 2024-04-20T21:45:57.384 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:57 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:57.096+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:57.385 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:57 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:57.096+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:45:57.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:57 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:57.383+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:45:57.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:57 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:57.383+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:45:57.784 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:57 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:45:57.391+0000 7f9897f39700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:21.149505+0000 front 2024-04-20T21:45:21.149597+0000 (oldest deadline 2024-04-20T21:45:50.549920+0000) 2024-04-20T21:45:58.053 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:45:58 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:45:58.025+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:45:58.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:57 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:57.785+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:45:58.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:57 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:57.785+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:45:58.274 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:57 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:57.888+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:45:58.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:57 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:57.888+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:32.279999+0000 front 2024-04-20T21:45:32.279964+0000 (oldest deadline 2024-04-20T21:45:55.780037+0000) 2024-04-20T21:45:58.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:57 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:57.866+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:45:58.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:57 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:57.866+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:45:58.382 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:58 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:58.127+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:58.382 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:58 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:58.127+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:45:58.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:58 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:58.383+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:45:58.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:58 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:58.383+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:45:58.742 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:58 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4[33127]: 2024-04-20T21:45:58.412+0000 7f9897f39700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:21.149505+0000 front 2024-04-20T21:45:21.149597+0000 (oldest deadline 2024-04-20T21:45:50.549920+0000) 2024-04-20T21:45:59.024 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:58 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:58.743+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:45:59.024 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:58 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:58.743+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:45:59.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:45:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:45:58.929+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:45:59.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:58.930+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:45:59.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:58.930+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:32.279999+0000 front 2024-04-20T21:45:32.279964+0000 (oldest deadline 2024-04-20T21:45:55.780037+0000) 2024-04-20T21:45:59.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:58.875+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:45:59.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:58.875+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:45:59.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:45:59 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:45:59.025+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:45:59.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:59 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:59.152+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:45:59.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:45:59 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:45:59.152+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:45:59.708 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:59 smithi033 podman[57777]: 2024-04-20 21:45:59.416009391 +0000 UTC m=+5.658748699 container died 2091bf9e0c088f647abec6c8396b0a436800ee2f9746f5a1868bab493f1a8e6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-20T21:45:59.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:59 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:59.425+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:45:59.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:45:59 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:45:59.425+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:00.007 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:59 smithi033 podman[57777]: 2024-04-20 21:45:59.901638608 +0000 UTC m=+6.144377903 container cleanup 2091bf9e0c088f647abec6c8396b0a436800ee2f9746f5a1868bab493f1a8e6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-20T21:46:00.007 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:45:59 smithi033 bash[57777]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4 2024-04-20T21:46:00.007 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:59 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:59.709+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:00.007 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:45:59 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:45:59.709+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:00.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:45:59 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:45:59.883+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:00.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:59 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:59.947+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:46:00.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:45:59 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:45:59.948+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:32.279999+0000 front 2024-04-20T21:45:32.279964+0000 (oldest deadline 2024-04-20T21:45:55.780037+0000) 2024-04-20T21:46:00.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:59 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:59.870+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:00.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:45:59 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:45:59.870+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:00.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:00 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:00.007+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:00.303 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:46:00 smithi033 podman[57808]: 2024-04-20 21:46:00.170566362 +0000 UTC m=+0.745312553 container remove 2091bf9e0c088f647abec6c8396b0a436800ee2f9746f5a1868bab493f1a8e6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:46:00.462 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:00 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:00.166+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:00.462 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:00 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:00.166+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:00.803 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:00 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:00.700+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:00.803 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:00 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:00.700+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:00.804 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:46:00 smithi033 podman[57899]: 2024-04-20 21:46:00.431683725 +0000 UTC m=+0.018300430 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:00.804 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:46:00 smithi033 podman[57899]: 2024-04-20 21:46:00.542501879 +0000 UTC m=+0.129118571 container create b0b598ffe5f672376d19f71817e2a1b14f53492b569908844ec87b540717d934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-20T21:46:00.804 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:46:00 smithi033 podman[57899]: 2024-04-20 21:46:00.753554696 +0000 UTC m=+0.340171391 container init b0b598ffe5f672376d19f71817e2a1b14f53492b569908844ec87b540717d934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-20T21:46:00.804 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:46:00 smithi033 podman[57899]: 2024-04-20 21:46:00.756958575 +0000 UTC m=+0.343575269 container start b0b598ffe5f672376d19f71817e2a1b14f53492b569908844ec87b540717d934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-20T21:46:00.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:00 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:00.462+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:00.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:00 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:00.462+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:01.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:00 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:00.878+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:01.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:00 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:00.977+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:46:01.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:00 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:00.977+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:32.279999+0000 front 2024-04-20T21:45:32.279964+0000 (oldest deadline 2024-04-20T21:45:55.780037+0000) 2024-04-20T21:46:01.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:00 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:00.916+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:01.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:00 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:00.916+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:01.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:01 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:01.015+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:01.304 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:46:00 smithi033 podman[57899]: 2024-04-20 21:46:00.86171714 +0000 UTC m=+0.448333848 container attach b0b598ffe5f672376d19f71817e2a1b14f53492b569908844ec87b540717d934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-20T21:46:01.304 INFO:journalctl@ceph.osd.4.smithi033.stdout:Apr 20 21:46:01 smithi033 podman[57899]: 2024-04-20 21:46:01.066742924 +0000 UTC m=+0.653359614 container died b0b598ffe5f672376d19f71817e2a1b14f53492b569908844ec87b540717d934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-20T21:46:01.407 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:01.128+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:01.407 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:01.128+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:01.780 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:01 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:01.686+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:01.780 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:01 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:01.686+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:01.817 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.4.service' 2024-04-20T21:46:01.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:01.864 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-20T21:46:01.864 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-20T21:46:01.865 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.5 2024-04-20T21:46:01.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:01.427+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:01.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:01.427+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:02.198 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:01 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:01.891+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:02.198 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:01 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:01.956+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:02.199 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:01 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:01.956+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:02.199 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:01 smithi122 systemd[1]: Stopping Ceph osd.5 for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:02.199 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:01 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:01.967+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:46:02.199 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:01 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:01.967+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:32.279999+0000 front 2024-04-20T21:45:32.279964+0000 (oldest deadline 2024-04-20T21:45:55.780037+0000) 2024-04-20T21:46:02.398 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:02 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:02.090+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:02.398 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:02 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:02.090+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:02.524 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:02 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:02.199+0000 7f8de0a11700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:02.525 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:02 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:02.199+0000 7f8de0a11700 -1 osd.5 107 *** Got signal Terminated *** 2024-04-20T21:46:02.525 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:02 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:02.199+0000 7f8de0a11700 -1 osd.5 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:02.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:02 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:02.058+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:02.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:02 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:02.399+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:02.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:02 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:02.399+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:03.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:02 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:02.642+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:03.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:02 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:02.642+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:03.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:02 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:02.877+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:03.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:02 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:02.964+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:46:03.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:02 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:02.964+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:32.279999+0000 front 2024-04-20T21:45:32.279964+0000 (oldest deadline 2024-04-20T21:45:55.780037+0000) 2024-04-20T21:46:03.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:02 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:02.994+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:03.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:02 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:02.994+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:03.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:02 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:02.994+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:03.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:03 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:03.133+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:03.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:03 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:03.133+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:03.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:03 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:03.066+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:03.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:03 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:03.438+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:03.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:03 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:03.438+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:04.051 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:03 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:03.624+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:04.051 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:03 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:03.624+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:04.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:03 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:03.887+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:04.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:03 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:03.887+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:04.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:03 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:03.968+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:46:04.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:03 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:03.968+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:32.279999+0000 front 2024-04-20T21:45:32.279964+0000 (oldest deadline 2024-04-20T21:45:55.780037+0000) 2024-04-20T21:46:04.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:03 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:03.968+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.281048+0000 front 2024-04-20T21:45:39.281078+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:04.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:03 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:03.997+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:04.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:03 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:03.997+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:04.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:03 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:03.997+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:04.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:04 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:04.052+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:04.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:04 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:04.052+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:04.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:04 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:04.140+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:04.407 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:04 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:04.140+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:04.407 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:04 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:04.140+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.149557+0000 front 2024-04-20T21:45:41.149678+0000 (oldest deadline 2024-04-20T21:46:03.449820+0000) 2024-04-20T21:46:04.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:04 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:04.415+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:04.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:04 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:04.415+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:05.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:04 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:04.599+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:05.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:04 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:04.599+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:05.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:04 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:04.904+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:05.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:04 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:04.904+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:05.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:05 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:05.000+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:05.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:05 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:05.000+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:05.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:05 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:05.000+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:05.276 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:04 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:04.944+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:46:05.276 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:04 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:04.944+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:32.279999+0000 front 2024-04-20T21:45:32.279964+0000 (oldest deadline 2024-04-20T21:45:55.780037+0000) 2024-04-20T21:46:05.276 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:04 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:04.944+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.281048+0000 front 2024-04-20T21:45:39.281078+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:05.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:05 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:05.123+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:05.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:05 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:05.123+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:05.407 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:05 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:05.123+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.149557+0000 front 2024-04-20T21:45:41.149678+0000 (oldest deadline 2024-04-20T21:46:03.449820+0000) 2024-04-20T21:46:05.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:05 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:05.086+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:05.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:05 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:05.087+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:05.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:05 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:05.407+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:05.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:05 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:05.407+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:05.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:05 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:05.407+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:06.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:05 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:05.589+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:06.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:05 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:05.589+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:06.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:05 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:05.589+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:06.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:05 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:05.915+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:06.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:05 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:05.915+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:06.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:06 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:06.027+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:06.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:06 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:06.027+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:06.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:06 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:06.027+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:06.276 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:05 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:05.950+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:46:06.276 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:05 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:05.950+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:32.279999+0000 front 2024-04-20T21:45:32.279964+0000 (oldest deadline 2024-04-20T21:45:55.780037+0000) 2024-04-20T21:46:06.276 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:05 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:05.950+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.281048+0000 front 2024-04-20T21:45:39.281078+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:06.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:06 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:06.127+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:06.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:06 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:06.127+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:06.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:06 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:06.127+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.149557+0000 front 2024-04-20T21:45:41.149678+0000 (oldest deadline 2024-04-20T21:46:03.449820+0000) 2024-04-20T21:46:06.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:06 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:06.088+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:06.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:06 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:06.088+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:06.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:06 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:06.418+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:06.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:06 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:06.418+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:06.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:06 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:06.418+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:07.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:06 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:06.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:07.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:06 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:06.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:07.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:06 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:06.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:07.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:06 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:06.937+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:07.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:06 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:06.937+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:07.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:06 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:06.911+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.578698+0000 front 2024-04-20T21:45:23.578847+0000 (oldest deadline 2024-04-20T21:45:48.878543+0000) 2024-04-20T21:46:07.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:06 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:06.911+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:32.279999+0000 front 2024-04-20T21:45:32.279964+0000 (oldest deadline 2024-04-20T21:45:55.780037+0000) 2024-04-20T21:46:07.276 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:06 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5[34256]: 2024-04-20T21:46:06.911+0000 7f8dd8ffd700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.281048+0000 front 2024-04-20T21:45:39.281078+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:07.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:07 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:07.024+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:07.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:07 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:07.024+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:07.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:07 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:07.024+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:07.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:07 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:07.154+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:07.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:07 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:07.154+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:07.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:07 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:07.154+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.149557+0000 front 2024-04-20T21:45:41.149678+0000 (oldest deadline 2024-04-20T21:46:03.449820+0000) 2024-04-20T21:46:07.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:07 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:07.104+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:07.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:07 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:07.104+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:07.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:07 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:07.426+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:07.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:07 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:07.426+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:07.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:07 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:07.426+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:07.959 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:07 smithi122 podman[58998]: 2024-04-20 21:46:07.684795372 +0000 UTC m=+5.629926592 container died bee52ee40c05ff571ea3aff007f40e98cd5bd428538abd31662b51af0948c9cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5) 2024-04-20T21:46:07.959 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:07 smithi122 podman[58998]: 2024-04-20 21:46:07.961205088 +0000 UTC m=+5.906336316 container cleanup bee52ee40c05ff571ea3aff007f40e98cd5bd428538abd31662b51af0948c9cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-20T21:46:07.959 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:07 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:07.914+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:07.960 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:07 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:07.914+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:08.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:07 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:07.599+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:08.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:07 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:07.599+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:08.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:07 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:07.599+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:08.274 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:07 smithi122 bash[58998]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5 2024-04-20T21:46:08.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:08 smithi122 podman[59015]: 2024-04-20 21:46:08.195782445 +0000 UTC m=+0.510493121 container remove bee52ee40c05ff571ea3aff007f40e98cd5bd428538abd31662b51af0948c9cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-20T21:46:08.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:08 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:08.025+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:08.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:08 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:08.025+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:08.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:08 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:08.025+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:08.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:08 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:08.151+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:08.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:08 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:08.151+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:08.656 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:08 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:08.185+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:08.656 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:08 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:08.185+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:08.656 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:08 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:08.185+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.149557+0000 front 2024-04-20T21:45:41.149678+0000 (oldest deadline 2024-04-20T21:46:03.449820+0000) 2024-04-20T21:46:08.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:08 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:08.397+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:08.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:08 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:08.397+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:08.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:08 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:08.397+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:08.774 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:08 smithi122 podman[59104]: 2024-04-20 21:46:08.426492789 +0000 UTC m=+0.017112872 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:08.775 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:08 smithi122 podman[59104]: 2024-04-20 21:46:08.538264388 +0000 UTC m=+0.128884506 container create 869c64258838dff00c1c984421d81ac075d55fd882c32fdc9f074516ae896b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:46:08.775 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:08 smithi122 podman[59104]: 2024-04-20 21:46:08.76534912 +0000 UTC m=+0.355969194 container init 869c64258838dff00c1c984421d81ac075d55fd882c32fdc9f074516ae896b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-20T21:46:08.775 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:08 smithi122 podman[59104]: 2024-04-20 21:46:08.768436433 +0000 UTC m=+0.359056506 container start 869c64258838dff00c1c984421d81ac075d55fd882c32fdc9f074516ae896b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:46:09.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:08 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:08.622+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:09.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:08 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:08.622+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:09.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:08 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:08.622+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:09.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:08 smithi122 podman[59104]: 2024-04-20 21:46:08.873902494 +0000 UTC m=+0.464522599 container attach 869c64258838dff00c1c984421d81ac075d55fd882c32fdc9f074516ae896b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-20T21:46:09.275 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:09 smithi122 podman[59104]: 2024-04-20 21:46:09.062602863 +0000 UTC m=+0.653222930 container died 869c64258838dff00c1c984421d81ac075d55fd882c32fdc9f074516ae896b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:46:09.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:08 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:08.890+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:09.276 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:08 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:08.890+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:09.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:09 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:09.001+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:09.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:09 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:09.001+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:09.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:09 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:09.001+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:09.391 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:09 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:09.126+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:09.391 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:09 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:09.126+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:09.607 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.5.service' 2024-04-20T21:46:09.629 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:09 smithi122 podman[59130]: 2024-04-20 21:46:09.5857867 +0000 UTC m=+0.518300135 container remove 869c64258838dff00c1c984421d81ac075d55fd882c32fdc9f074516ae896b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-20T21:46:09.630 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:09 smithi122 systemd[1]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-20T21:46:09.630 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:09 smithi122 systemd[1]: Stopped Ceph osd.5 for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:46:09.630 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 20 21:46:09 smithi122 systemd[1]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.5.service: Consumed 33.350s CPU time. 2024-04-20T21:46:09.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:09.645 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-20T21:46:09.645 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-20T21:46:09.646 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.6 2024-04-20T21:46:09.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:09 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:09.367+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:09.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:09 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:09.367+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:09.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:09 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:09.367+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:09.656 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:09 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:09.190+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:09.657 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:09 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:09.190+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:09.657 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:09 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:09.190+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.149557+0000 front 2024-04-20T21:45:41.149678+0000 (oldest deadline 2024-04-20T21:46:03.449820+0000) 2024-04-20T21:46:10.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:09 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:09.670+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:10.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:09 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:09.670+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:10.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:09 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:09.670+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:10.156 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:09 smithi022 systemd[1]: Stopping Ceph osd.6 for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:10.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:09 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:09.914+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:10.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:09 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:09.914+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:10.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:10 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:10.037+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:10.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:10 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:10.037+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:10.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:10 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:10.037+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:10.554 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:10 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:10.126+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:10.554 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:10 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:10.126+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:10.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:10 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:10.415+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:10.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:10 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:10.415+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:10.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:10 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:10.415+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:10.657 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:10 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:10.169+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:10.657 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:10 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:10.169+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:10.657 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:10 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:10.169+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.149557+0000 front 2024-04-20T21:45:41.149678+0000 (oldest deadline 2024-04-20T21:46:03.449820+0000) 2024-04-20T21:46:10.658 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:10 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:10.264+0000 7fb566000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:10.658 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:10 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:10.264+0000 7fb566000700 -1 osd.6 107 *** Got signal Terminated *** 2024-04-20T21:46:10.658 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:10 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:10.264+0000 7fb566000700 -1 osd.6 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:11.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:10 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:10.689+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:11.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:10 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:10.689+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:11.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:10 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:10.689+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:11.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:10 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:10.931+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:11.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:10 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:10.931+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:11.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:11 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:11.056+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:11.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:11 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:11.056+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:11.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:11 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:11.056+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:11.406 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:11 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:11.380+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:11.406 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:11 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:11.380+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:11.407 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:11 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:11.380+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:11.407 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:11 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:11.149+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:11.407 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:11 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:11.149+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:11.407 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:11 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:11.149+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.149557+0000 front 2024-04-20T21:45:41.149678+0000 (oldest deadline 2024-04-20T21:46:03.449820+0000) 2024-04-20T21:46:11.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:11 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:11.121+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:11.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:11 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:11.121+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:12.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:11 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:11.696+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:12.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:11 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:11.696+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:12.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:11 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:11.696+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:12.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:11 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:11.928+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:12.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:11 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:11.928+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:12.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:11 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:11.928+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:12.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:12 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:12.052+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:12.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:12 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:12.052+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:12.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:12 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:12.052+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:12.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:12 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:12.100+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:12.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:12 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:12.100+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:12.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:12 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:12.343+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:12.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:12 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:12.343+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:12.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:12 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:12.343+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:12.657 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:12 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:12.194+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:12.657 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:12 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:12.194+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:12.657 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:12 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:12.194+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.149557+0000 front 2024-04-20T21:45:41.149678+0000 (oldest deadline 2024-04-20T21:46:03.449820+0000) 2024-04-20T21:46:13.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:12 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:12.660+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:13.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:12 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:12.660+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:13.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:12 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:12.660+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:13.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:12 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:12.908+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:13.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:12 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:12.908+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:13.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:12 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:12.908+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:13.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:13 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:13.096+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:13.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:13 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:13.096+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:13.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:13 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:13.096+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:13.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:13 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:13.080+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:13.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:13 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:13.080+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:13.656 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:13 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:13.177+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:13.656 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:13 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:13.177+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:13.657 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:13 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:13.177+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.149557+0000 front 2024-04-20T21:45:41.149678+0000 (oldest deadline 2024-04-20T21:46:03.449820+0000) 2024-04-20T21:46:13.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:13 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:13.345+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:13.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:13 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:13.345+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:13.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:13 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:13.345+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:14.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:13 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:13.694+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:14.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:13 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:13.694+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:14.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:13 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:13.694+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:14.176 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:13 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:13.894+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:14.176 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:13 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:13.894+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:14.176 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:13 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:13.894+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:14.177 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:14 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:14.119+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:14.177 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:14 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:14.119+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:14.177 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:14 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:14.119+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:14.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:14 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:14.076+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:14.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:14 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:14.076+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:14.558 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:14 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:14.222+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:14.558 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:14 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:14.222+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:14.559 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:14 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:14.222+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.149557+0000 front 2024-04-20T21:45:41.149678+0000 (oldest deadline 2024-04-20T21:46:03.449820+0000) 2024-04-20T21:46:14.559 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:14 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:14.332+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:14.559 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:14 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:14.332+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:14.559 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:14 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:14.332+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:14.559 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:14 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:14.332+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:15.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:14 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:14.682+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:15.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:14 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:14.682+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:15.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:14 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:14.682+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:15.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:14 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:14.682+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:15.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:14 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:14.862+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:15.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:14 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:14.862+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:15.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:14 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:14.862+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:15.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:15 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:15.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:15.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:15 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:15.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:15.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:15 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:15.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:15.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:15 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:15.060+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:15.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:15 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:15.060+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:15.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:15 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:15.060+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.827216+0000 front 2024-04-20T21:45:47.827372+0000 (oldest deadline 2024-04-20T21:46:14.827618+0000) 2024-04-20T21:46:15.589 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:15 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:15.228+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:20.044215+0000 front 2024-04-20T21:45:20.044312+0000 (oldest deadline 2024-04-20T21:45:45.944396+0000) 2024-04-20T21:46:15.589 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:15 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:15.228+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:28.846725+0000 front 2024-04-20T21:45:28.846861+0000 (oldest deadline 2024-04-20T21:45:54.746590+0000) 2024-04-20T21:46:15.589 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:15 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6[42652]: 2024-04-20T21:46:15.228+0000 7fb55c800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.149557+0000 front 2024-04-20T21:45:41.149678+0000 (oldest deadline 2024-04-20T21:46:03.449820+0000) 2024-04-20T21:46:15.590 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:15 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:15.360+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:15.590 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:15 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:15.360+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:15.590 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:15 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:15.360+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:15.590 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:15 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:15.360+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:15.906 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:15 smithi022 podman[79224]: 2024-04-20 21:46:15.756918279 +0000 UTC m=+5.634232721 container died 3a1633f88c5c0cca36d74be762af899ca87aad659b8ca80177483098b95580fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-20T21:46:16.044 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:15 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:15.663+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:16.044 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:15 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:15.663+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:16.044 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:15 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:15.663+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:16.044 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:15 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:15.663+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:16.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:15 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:15.892+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:16.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:15 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:15.892+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:16.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:15 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:15.892+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:16.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:16 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:16.119+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:16.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:16 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:16.119+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:16.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:16 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:16.119+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:16.294 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:16 smithi022 podman[79224]: 2024-04-20 21:46:16.165854682 +0000 UTC m=+6.043169174 container cleanup 3a1633f88c5c0cca36d74be762af899ca87aad659b8ca80177483098b95580fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-20T21:46:16.294 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:16 smithi022 bash[79224]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6 2024-04-20T21:46:16.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:16 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:16.045+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:16.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:16 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:16.045+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:16.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:16 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:16.045+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.827216+0000 front 2024-04-20T21:45:47.827372+0000 (oldest deadline 2024-04-20T21:46:14.827618+0000) 2024-04-20T21:46:16.656 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:16 smithi022 podman[79245]: 2024-04-20 21:46:16.350082476 +0000 UTC m=+0.582943782 container remove 3a1633f88c5c0cca36d74be762af899ca87aad659b8ca80177483098b95580fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True) 2024-04-20T21:46:16.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:16 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:16.356+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:16.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:16 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:16.356+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:16.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:16 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:16.356+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:16.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:16 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:16.356+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:16.996 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:16 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:16.672+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:16.996 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:16 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:16.672+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:16.997 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:16 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:16.672+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:16.997 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:16 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:16.672+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:17.274 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:17 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:17.097+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:17.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:17 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:17.097+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:17.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:17 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:17.097+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:17.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:16 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:16.866+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:17.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:16 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:16.866+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:17.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:16 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:16.866+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:17.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:16 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:16.998+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:17.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:16 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:16.998+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:17.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:16 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:16.998+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.827216+0000 front 2024-04-20T21:45:47.827372+0000 (oldest deadline 2024-04-20T21:46:14.827618+0000) 2024-04-20T21:46:17.541 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:17 smithi022 podman[79339]: 2024-04-20 21:46:17.090999925 +0000 UTC m=+0.036482747 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:17.541 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:17 smithi022 podman[79339]: 2024-04-20 21:46:17.242493868 +0000 UTC m=+0.187976708 container create 97bf57c849c752e1de2b90acc995b675d2c89504fc86b45bb0b60b729b71a9aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-20T21:46:17.542 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:17 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:17.352+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:17.542 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:17 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:17.352+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:17.542 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:17 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:17.352+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:17.542 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:17 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:17.352+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:17.906 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:17 smithi022 podman[79339]: 2024-04-20 21:46:17.646784507 +0000 UTC m=+0.592267373 container init 97bf57c849c752e1de2b90acc995b675d2c89504fc86b45bb0b60b729b71a9aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-04-20T21:46:17.907 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:17 smithi022 podman[79339]: 2024-04-20 21:46:17.659788189 +0000 UTC m=+0.605271044 container start 97bf57c849c752e1de2b90acc995b675d2c89504fc86b45bb0b60b729b71a9aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-20T21:46:17.907 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:17 smithi022 podman[79339]: 2024-04-20 21:46:17.736615103 +0000 UTC m=+0.682097946 container attach 97bf57c849c752e1de2b90acc995b675d2c89504fc86b45bb0b60b729b71a9aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-20T21:46:17.980 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:17 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:17.648+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:17.980 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:17 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:17.648+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:17.981 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:17 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:17.648+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:17.981 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:17 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:17.648+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:18.274 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:18 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:18.131+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:18.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:18 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:18.131+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:18.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:18 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:18.131+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:18.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:17 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:17.893+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:18.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:17 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:17.893+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:18.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:17 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:17.893+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:18.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:17 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:17.982+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:18.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:17 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:17.982+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:18.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:17 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:17.982+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.827216+0000 front 2024-04-20T21:45:47.827372+0000 (oldest deadline 2024-04-20T21:46:14.827618+0000) 2024-04-20T21:46:18.406 INFO:journalctl@ceph.osd.6.smithi022.stdout:Apr 20 21:46:18 smithi022 podman[79339]: 2024-04-20 21:46:18.120883323 +0000 UTC m=+1.066366188 container died 97bf57c849c752e1de2b90acc995b675d2c89504fc86b45bb0b60b729b71a9aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-20T21:46:18.406 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:18 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:18.351+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:18.407 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:18 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:18.351+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:18.407 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:18 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:18.351+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:18.407 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:18 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:18.351+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:18.952 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:18 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:18.610+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:18.953 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:18 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:18.610+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:18.953 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:18 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:18.610+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:18.953 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:18 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:18.610+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:19.058 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.6.service' 2024-04-20T21:46:19.131 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:18 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:18.859+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:19.132 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:18 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:18.859+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:19.132 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:18 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:18.859+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:19.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:19.221 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-20T21:46:19.221 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-20T21:46:19.221 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.7 2024-04-20T21:46:19.248 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:18 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:18.954+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:19.248 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:18 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:18.954+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:19.249 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:18 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:18.954+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.827216+0000 front 2024-04-20T21:45:47.827372+0000 (oldest deadline 2024-04-20T21:46:14.827618+0000) 2024-04-20T21:46:19.524 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:19 smithi033 systemd[1]: Stopping Ceph osd.7 for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:19.524 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:19 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:19.133+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:19.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:19 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:19.133+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:19.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:19 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:19.133+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:19.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:19 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:19.339+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:19.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:19 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:19.339+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:19.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:19 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:19.339+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:19.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:19 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:19.339+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:19.803 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:19 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:19.644+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:19.803 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:19 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:19.644+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:19.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:19 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:19.644+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:19.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:19 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:19.644+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:19.804 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:19 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:19.525+0000 7fc752474700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:19.804 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:19 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:19.525+0000 7fc752474700 -1 osd.7 107 *** Got signal Terminated *** 2024-04-20T21:46:19.804 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:19 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:19.525+0000 7fc752474700 -1 osd.7 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:20.147 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:19 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:19.886+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:20.147 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:19 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:19.886+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:20.147 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:19 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:19.886+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:20.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:19 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:19.998+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:20.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:19 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:19.998+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:20.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:19 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:19.998+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.827216+0000 front 2024-04-20T21:45:47.827372+0000 (oldest deadline 2024-04-20T21:46:14.827618+0000) 2024-04-20T21:46:20.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:20 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:20.148+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:20.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:20 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:20.148+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:20.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:20 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:20.148+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:20.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:20 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:20.386+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:20.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:20 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:20.386+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:20.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:20 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:20.386+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:20.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:20 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:20.386+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:21.010 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:20 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:20.674+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:21.010 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:20 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:20.674+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:21.010 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:20 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:20.674+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:21.010 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:20 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:20.674+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:21.136 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:20 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:20.873+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:21.137 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:20 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:20.873+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:21.137 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:20 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:20.873+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:21.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:21 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:21.011+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:21.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:21 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:21.011+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:21.304 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:21 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:21.011+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.827216+0000 front 2024-04-20T21:45:47.827372+0000 (oldest deadline 2024-04-20T21:46:14.827618+0000) 2024-04-20T21:46:21.304 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:21 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:21.011+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.828048+0000 front 2024-04-20T21:45:54.828000+0000 (oldest deadline 2024-04-20T21:46:20.728001+0000) 2024-04-20T21:46:21.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:21 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:21.138+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:21.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:21 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:21.138+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:21.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:21 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:21.138+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:21.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:21 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:21.419+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:21.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:21 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:21.420+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:21.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:21 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:21.420+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:21.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:21 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:21.420+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:22.046 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:21 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:21.698+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:22.047 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:21 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:21.698+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:22.047 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:21 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:21.698+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:22.047 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:21 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:21.698+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:22.179 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:21 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:21.908+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:22.179 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:21 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:21.908+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:22.180 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:21 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:21.908+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:22.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:22 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:22.048+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:22.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:22 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:22.048+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:22.304 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:22 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:22.048+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.827216+0000 front 2024-04-20T21:45:47.827372+0000 (oldest deadline 2024-04-20T21:46:14.827618+0000) 2024-04-20T21:46:22.304 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:22 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:22.048+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.828048+0000 front 2024-04-20T21:45:54.828000+0000 (oldest deadline 2024-04-20T21:46:20.728001+0000) 2024-04-20T21:46:22.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:22 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:22.180+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:22.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:22 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:22.180+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:22.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:22 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:22.180+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:22.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:22 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:22.460+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:22.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:22 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:22.460+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:22.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:22 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:22.460+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:22.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:22 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:22.460+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:23.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:22 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:22.709+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:23.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:22 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:22.709+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:23.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:22 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:22.709+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:23.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:22 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:22.710+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:23.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:22 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:22.710+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:23.168 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:22 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:22.912+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:23.168 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:22 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:22.912+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:23.168 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:22 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:22.912+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:23.168 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:22 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:22.912+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:58.484196+0000 front 2024-04-20T21:45:58.484135+0000 (oldest deadline 2024-04-20T21:46:22.484760+0000) 2024-04-20T21:46:23.524 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:23 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:23.169+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:23.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:23 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:23.169+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:23.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:23 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:23.169+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:23.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:23 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:23.080+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:23.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:23 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:23.080+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:23.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:23 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:23.080+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.827216+0000 front 2024-04-20T21:45:47.827372+0000 (oldest deadline 2024-04-20T21:46:14.827618+0000) 2024-04-20T21:46:23.553 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:23 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:23.080+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.828048+0000 front 2024-04-20T21:45:54.828000+0000 (oldest deadline 2024-04-20T21:46:20.728001+0000) 2024-04-20T21:46:23.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:23 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:23.427+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:23.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:23 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:23.427+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:23.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:23 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:23.427+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:23.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:23 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:23.427+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:24.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:23 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:23.680+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:24.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:23 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:23.680+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:24.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:23 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:23.680+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:24.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:23 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:23.680+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:24.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:23 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:23.680+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:24.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:23 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:23.901+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:24.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:23 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:23.901+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:24.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:23 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:23.901+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:24.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:23 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:23.901+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:58.484196+0000 front 2024-04-20T21:45:58.484135+0000 (oldest deadline 2024-04-20T21:46:22.484760+0000) 2024-04-20T21:46:24.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:24 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:24.121+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:24.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:24 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:24.121+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:24.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:24 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:24.121+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:24.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:24 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:24.121+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:24.554 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:24 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:24.121+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.323635+0000 front 2024-04-20T21:45:33.323671+0000 (oldest deadline 2024-04-20T21:45:57.423918+0000) 2024-04-20T21:46:24.554 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:24 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:24.121+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:41.525081+0000 front 2024-04-20T21:45:41.525113+0000 (oldest deadline 2024-04-20T21:46:03.225305+0000) 2024-04-20T21:46:24.554 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:24 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:24.121+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.827216+0000 front 2024-04-20T21:45:47.827372+0000 (oldest deadline 2024-04-20T21:46:14.827618+0000) 2024-04-20T21:46:24.554 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:24 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7[40134]: 2024-04-20T21:46:24.121+0000 7fc74ea68700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.828048+0000 front 2024-04-20T21:45:54.828000+0000 (oldest deadline 2024-04-20T21:46:20.728001+0000) 2024-04-20T21:46:24.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:24 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:24.388+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:24.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:24 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:24.388+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:24.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:24 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:24.388+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:24.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:24 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:24.388+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:25.017 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:24 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:24.646+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:25.017 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:24 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:24.646+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:25.017 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:24 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:24.646+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:25.017 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:24 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:24.646+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:25.017 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:24 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:24.646+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:25.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:24 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:24.876+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:25.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:24 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:24.876+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:25.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:24 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:24.876+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:25.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:24 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:24.876+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:58.484196+0000 front 2024-04-20T21:45:58.484135+0000 (oldest deadline 2024-04-20T21:46:22.484760+0000) 2024-04-20T21:46:25.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:25 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:25.101+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:25.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:25 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:25.101+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:25.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:25 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:25.101+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:25.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:25 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:25.101+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:25.303 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:25 smithi033 podman[58005]: 2024-04-20 21:46:25.01868109 +0000 UTC m=+5.597858918 container died 8f5b29468d5dee668acec7311787812a9be1a1a116bf76ef734f3a5eabbcd6fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-04-20T21:46:25.607 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:25 smithi033 podman[58005]: 2024-04-20 21:46:25.425797909 +0000 UTC m=+6.004975731 container cleanup 8f5b29468d5dee668acec7311787812a9be1a1a116bf76ef734f3a5eabbcd6fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-20T21:46:25.607 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:25 smithi033 bash[58005]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7 2024-04-20T21:46:25.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:25 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:25.387+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:25.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:25 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:25.387+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:25.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:25 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:25.387+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:25.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:25 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:25.387+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:25.968 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:25 smithi033 podman[58022]: 2024-04-20 21:46:25.635277203 +0000 UTC m=+0.619063463 container remove 8f5b29468d5dee668acec7311787812a9be1a1a116bf76ef734f3a5eabbcd6fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-20T21:46:25.968 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:25 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:25.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:25.968 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:25 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:25.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:25.968 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:25 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:25.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:25.969 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:25 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:25.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:25.969 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:25 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:25.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:26.228 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:25 smithi033 podman[58113]: 2024-04-20 21:46:25.872977111 +0000 UTC m=+0.018603375 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:26.229 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:25 smithi033 podman[58113]: 2024-04-20 21:46:25.986187643 +0000 UTC m=+0.131813905 container create 898b393393f5d5c7b165911d11d8d3fecd152fa21d265dc1cc3c73088cd9b999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-20T21:46:26.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:25 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:25.916+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:26.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:25 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:25.916+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:26.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:25 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:25.916+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:26.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:25 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:25.916+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:58.484196+0000 front 2024-04-20T21:45:58.484135+0000 (oldest deadline 2024-04-20T21:46:22.484760+0000) 2024-04-20T21:46:26.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:26 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:26.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:26.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:26 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:26.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:26.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:26 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:26.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:26.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:26 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:26.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:26.531 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:26 smithi033 podman[58113]: 2024-04-20 21:46:26.230689611 +0000 UTC m=+0.376315867 container init 898b393393f5d5c7b165911d11d8d3fecd152fa21d265dc1cc3c73088cd9b999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:46:26.531 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:26 smithi033 podman[58113]: 2024-04-20 21:46:26.233988069 +0000 UTC m=+0.379614328 container start 898b393393f5d5c7b165911d11d8d3fecd152fa21d265dc1cc3c73088cd9b999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-20T21:46:26.532 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:26 smithi033 podman[58113]: 2024-04-20 21:46:26.313820883 +0000 UTC m=+0.459447146 container attach 898b393393f5d5c7b165911d11d8d3fecd152fa21d265dc1cc3c73088cd9b999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True) 2024-04-20T21:46:26.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:26 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:26.364+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:26.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:26 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:26.364+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:26.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:26 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:26.364+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:26.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:26 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:26.364+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:26.803 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:26 smithi033 podman[58113]: 2024-04-20 21:46:26.533214444 +0000 UTC m=+0.678840690 container died 898b393393f5d5c7b165911d11d8d3fecd152fa21d265dc1cc3c73088cd9b999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True) 2024-04-20T21:46:26.803 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:26 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:26.646+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:26.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:26 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:26.646+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:26.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:26 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:26.646+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:26.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:26 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:26.646+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:26.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:26 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:26.646+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:27.136 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:26 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:26.870+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:27.136 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:26 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:26.870+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:27.136 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:26 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:26.870+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:27.137 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:26 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:26.870+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:58.484196+0000 front 2024-04-20T21:45:58.484135+0000 (oldest deadline 2024-04-20T21:46:22.484760+0000) 2024-04-20T21:46:27.229 INFO:journalctl@ceph.osd.7.smithi033.stdout:Apr 20 21:46:27 smithi033 podman[58140]: 2024-04-20 21:46:27.209266444 +0000 UTC m=+0.669648861 container remove 898b393393f5d5c7b165911d11d8d3fecd152fa21d265dc1cc3c73088cd9b999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True) 2024-04-20T21:46:27.235 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.7.service' 2024-04-20T21:46:27.316 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:27.316 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-20T21:46:27.317 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-20T21:46:27.317 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.8 2024-04-20T21:46:27.524 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:27 smithi122 systemd[1]: Stopping Ceph osd.8 for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:27.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:27 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:27.138+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:27.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:27 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:27.138+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:27.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:27 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:27.138+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:27.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:27 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:27.138+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:27.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:27 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:27.385+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:27.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:27 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:27.385+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:27.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:27 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:27.385+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:27.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:27 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:27.385+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:27.803 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:27 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:27.659+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:27.803 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:27 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:27.659+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:27.803 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:27 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:27.659+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:27.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:27 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:27.659+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:27.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:27 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:27.659+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:27.913 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:27 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:27.629+0000 7f53d5d44700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:27.913 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:27 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:27.629+0000 7f53d5d44700 -1 osd.8 107 *** Got signal Terminated *** 2024-04-20T21:46:27.913 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:27 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:27.629+0000 7f53d5d44700 -1 osd.8 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:27.913 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:27 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:27.914+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:27.913 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:27 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:27.914+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:27.913 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:27 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:27.914+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:27.913 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:27 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:27.914+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:58.484196+0000 front 2024-04-20T21:45:58.484135+0000 (oldest deadline 2024-04-20T21:46:22.484760+0000) 2024-04-20T21:46:28.274 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:28 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:28.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:28.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:28 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:28.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:28.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:28 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:28.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:28.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:28 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:28.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:28.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:28 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:28.349+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:28.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:28 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:28.349+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:28.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:28 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:28.349+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:28.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:28 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:28.349+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:28.803 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:28 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:28.617+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:28.803 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:28 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:28.617+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:28.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:28 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:28.617+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:28.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:28 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:28.617+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:28.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:28 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:28.617+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:29.274 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:29 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:29.120+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:29.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:29 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:29.120+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:29.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:29 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:29.120+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:29.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:29 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:29.120+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:29.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:28 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:28.914+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:29.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:28 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:28.914+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:29.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:28 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:28.914+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:29.276 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:28 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:28.914+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:58.484196+0000 front 2024-04-20T21:45:58.484135+0000 (oldest deadline 2024-04-20T21:46:22.484760+0000) 2024-04-20T21:46:29.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:29 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:29.321+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:29.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:29 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:29.321+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:29.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:29 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:29.321+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:29.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:29 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:29.321+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:30.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:29 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:29.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:30.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:29 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:29.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:30.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:29 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:29.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:30.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:29 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:29.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:30.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:29 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:29.609+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:30.274 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:30 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:30.149+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:30.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:30 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:30.149+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:30.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:30 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:30.149+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:30.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:30 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:30.149+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:30.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:29 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:29.901+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:30.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:29 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:29.901+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:30.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:29 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:29.901+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:30.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:29 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:29.901+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:58.484196+0000 front 2024-04-20T21:45:58.484135+0000 (oldest deadline 2024-04-20T21:46:22.484760+0000) 2024-04-20T21:46:30.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:30 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:30.337+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:30.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:30 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:30.337+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:30.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:30 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:30.337+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:30.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:30 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:30.337+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:30.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:30 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:30.337+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:04.229215+0000 front 2024-04-20T21:46:04.229322+0000 (oldest deadline 2024-04-20T21:46:29.529473+0000) 2024-04-20T21:46:31.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:30 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:30.573+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:31.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:30 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:30.573+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:31.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:30 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:30.573+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:31.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:30 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:30.573+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:31.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:30 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:30.573+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:31.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:30 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:30.573+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:31.274 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:31 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:31.128+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:31.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:31 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:31.128+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:31.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:31 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:31.128+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:31.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:31 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:31.128+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:31.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:30 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:30.898+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:31.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:30 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:30.898+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:31.275 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:30 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:30.898+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:31.276 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:30 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:30.898+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:58.484196+0000 front 2024-04-20T21:45:58.484135+0000 (oldest deadline 2024-04-20T21:46:22.484760+0000) 2024-04-20T21:46:31.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:31 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:31.351+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:31.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:31 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:31.351+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:31.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:31 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:31.351+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:31.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:31 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:31.351+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:31.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:31 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:31.351+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:04.229215+0000 front 2024-04-20T21:46:04.229322+0000 (oldest deadline 2024-04-20T21:46:29.529473+0000) 2024-04-20T21:46:32.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:31 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:31.623+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:32.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:31 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:31.623+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:32.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:31 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:31.623+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:32.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:31 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:31.623+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:32.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:31 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:31.623+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:32.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:31 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:31.623+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:32.137 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:31 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:31.851+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.278393+0000 front 2024-04-20T21:45:31.278479+0000 (oldest deadline 2024-04-20T21:45:58.279137+0000) 2024-04-20T21:46:32.137 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:31 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:31.851+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.481095+0000 front 2024-04-20T21:45:40.481123+0000 (oldest deadline 2024-04-20T21:46:03.381210+0000) 2024-04-20T21:46:32.137 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:31 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:31.851+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:47.982617+0000 front 2024-04-20T21:45:47.982746+0000 (oldest deadline 2024-04-20T21:46:11.482641+0000) 2024-04-20T21:46:32.137 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:31 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8[40804]: 2024-04-20T21:46:31.851+0000 7f53ce330700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:58.484196+0000 front 2024-04-20T21:45:58.484135+0000 (oldest deadline 2024-04-20T21:46:22.484760+0000) 2024-04-20T21:46:32.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:32 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:32.138+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:32.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:32 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:32.138+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:32.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:32 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:32.138+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:32.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:32 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:32.138+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:32.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:32.323+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:32.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:32.323+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:32.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:32.323+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:32.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:32.323+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:32.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:32 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:32.323+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:04.229215+0000 front 2024-04-20T21:46:04.229322+0000 (oldest deadline 2024-04-20T21:46:29.529473+0000) 2024-04-20T21:46:33.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:32 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:32.647+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:33.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:32 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:32.647+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:33.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:32 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:32.647+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:33.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:32 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:32.647+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:33.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:32 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:32.647+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:33.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:32 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:32.647+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:33.455 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:33 smithi122 podman[59222]: 2024-04-20 21:46:33.16699358 +0000 UTC m=+5.661255082 container died c1ef21c85fb731f5dc9fa8934909c98ea530adec74fdd8d6b92de46a4592728c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-20T21:46:33.455 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:33 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:33.172+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:33.455 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:33 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:33.172+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:33.456 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:33 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:33.172+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:33.456 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:33 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:33.172+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:33.456 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:33 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:33.172+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:33.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:33 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:33.317+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:33.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:33 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:33.317+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:33.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:33 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:33.317+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:33.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:33 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:33.317+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:33.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:33 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:33.317+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:04.229215+0000 front 2024-04-20T21:46:04.229322+0000 (oldest deadline 2024-04-20T21:46:29.529473+0000) 2024-04-20T21:46:33.774 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:33 smithi122 podman[59222]: 2024-04-20 21:46:33.538194281 +0000 UTC m=+6.032455788 container cleanup c1ef21c85fb731f5dc9fa8934909c98ea530adec74fdd8d6b92de46a4592728c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-20T21:46:33.775 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:33 smithi122 bash[59222]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8 2024-04-20T21:46:34.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:33 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:33.633+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:34.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:33 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:33.633+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:34.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:33 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:33.633+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:34.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:33 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:33.633+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:34.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:33 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:33.633+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:34.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:33 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:33.633+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:34.153 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:33 smithi122 podman[59241]: 2024-04-20 21:46:33.82309838 +0000 UTC m=+0.651249738 container remove c1ef21c85fb731f5dc9fa8934909c98ea530adec74fdd8d6b92de46a4592728c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-20T21:46:34.441 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:34 smithi122 podman[59334]: 2024-04-20 21:46:34.058602034 +0000 UTC m=+0.018086742 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:34.441 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:34 smithi122 podman[59334]: 2024-04-20 21:46:34.182137611 +0000 UTC m=+0.141622321 container create 80e446fb85af15d92a73c42a561fbae789b54c482e36c02d904fc4d054e74db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-20T21:46:34.441 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:34 smithi122 podman[59334]: 2024-04-20 21:46:34.442702385 +0000 UTC m=+0.402187092 container init 80e446fb85af15d92a73c42a561fbae789b54c482e36c02d904fc4d054e74db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:46:34.441 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:34 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:34.216+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:34.441 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:34 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:34.216+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:34.442 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:34 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:34.216+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:34.442 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:34 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:34.216+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:34.442 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:34 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:34.216+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:34.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:34 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:34.340+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:34.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:34 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:34.340+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:34.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:34 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:34.340+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:34.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:34 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:34.340+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:34.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:34 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:34.340+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:04.229215+0000 front 2024-04-20T21:46:04.229322+0000 (oldest deadline 2024-04-20T21:46:29.529473+0000) 2024-04-20T21:46:34.737 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:34 smithi122 podman[59334]: 2024-04-20 21:46:34.445800495 +0000 UTC m=+0.405285203 container start 80e446fb85af15d92a73c42a561fbae789b54c482e36c02d904fc4d054e74db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:46:34.737 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:34 smithi122 podman[59334]: 2024-04-20 21:46:34.517675646 +0000 UTC m=+0.477160370 container attach 80e446fb85af15d92a73c42a561fbae789b54c482e36c02d904fc4d054e74db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:46:35.024 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 20 21:46:34 smithi122 podman[59334]: 2024-04-20 21:46:34.739576741 +0000 UTC m=+0.699061449 container died 80e446fb85af15d92a73c42a561fbae789b54c482e36c02d904fc4d054e74db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-20T21:46:35.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:34 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:34.603+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:35.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:34 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:34.603+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:35.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:34 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:34.603+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:35.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:34 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:34.603+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:35.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:34 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:34.603+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:35.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:34 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:34.603+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:35.352 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:35 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:35.209+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:35.352 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:35 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:35.209+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:35.353 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:35 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:35.209+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:35.353 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:35 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:35.210+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:35.353 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:35 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:35.210+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:35.374 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.8.service' 2024-04-20T21:46:35.414 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:35.414 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-20T21:46:35.414 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-20T21:46:35.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.9 2024-04-20T21:46:35.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:35 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:35.317+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:35.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:35 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:35.317+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:35.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:35 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:35.317+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:35.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:35 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:35.317+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:35.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:35 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:35.317+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:04.229215+0000 front 2024-04-20T21:46:04.229322+0000 (oldest deadline 2024-04-20T21:46:29.529473+0000) 2024-04-20T21:46:35.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:35 smithi022 systemd[1]: Stopping Ceph osd.9 for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:36.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:35 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:35.639+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:36.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:35 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:35.639+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:36.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:35 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:35.639+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:36.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:35 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:35.639+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:36.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:35 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:35.639+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:36.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:35 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:35.639+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:36.321 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:36 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:36.007+0000 7f0c80a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:36.321 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:36 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:36.007+0000 7f0c80a00700 -1 osd.9 107 *** Got signal Terminated *** 2024-04-20T21:46:36.321 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:36 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:36.007+0000 7f0c80a00700 -1 osd.9 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:36.321 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:36 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:36.321+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:36.321 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:36 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:36.321+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:36.321 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:36 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:36.321+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:36.322 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:36 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:36.321+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:36.322 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:36 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:36.321+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:04.229215+0000 front 2024-04-20T21:46:04.229322+0000 (oldest deadline 2024-04-20T21:46:29.529473+0000) 2024-04-20T21:46:36.524 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:36 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:36.169+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:36.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:36 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:36.169+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:36.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:36 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:36.169+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:36.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:36 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:36.169+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:36.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:36 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:36.169+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:37.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:36 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:36.634+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:37.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:36 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:36.634+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:37.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:36 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:36.634+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:37.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:36 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:36.634+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:37.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:36 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:36.634+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:37.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:36 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:36.634+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:37.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:36 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:36.634+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:37.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:37 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:37.219+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:37.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:37 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:37.219+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:37.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:37 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:37.219+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:37.526 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:37 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:37.219+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:37.526 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:37 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:37.219+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:37.526 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:37 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:37.219+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:37.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:37 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:37.367+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:37.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:37 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:37.367+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:37.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:37 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:37.367+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:37.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:37 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:37.367+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:37.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:37 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:37.367+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:04.229215+0000 front 2024-04-20T21:46:04.229322+0000 (oldest deadline 2024-04-20T21:46:29.529473+0000) 2024-04-20T21:46:37.963 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:37 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:37.601+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:37.963 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:37 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:37.601+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:37.963 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:37 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:37.601+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:37.963 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:37 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:37.601+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:37.964 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:37 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:37.601+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:37.964 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:37 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:37.601+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:37.964 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:37 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:37.601+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:38.524 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:38 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:38.183+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:38.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:38 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:38.183+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:38.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:38 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:38.183+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:38.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:38 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:38.183+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:38.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:38 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:38.183+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:38.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:38 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:38.183+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:38.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:38 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:38.400+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:38.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:38 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:38.400+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:38.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:38 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:38.400+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:38.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:38 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:38.400+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:38.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:38 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:38.400+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:04.229215+0000 front 2024-04-20T21:46:04.229322+0000 (oldest deadline 2024-04-20T21:46:29.529473+0000) 2024-04-20T21:46:38.657 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:38 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:38.400+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:14.832060+0000 front 2024-04-20T21:46:14.831812+0000 (oldest deadline 2024-04-20T21:46:38.233510+0000) 2024-04-20T21:46:39.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:38 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:38.622+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:39.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:38 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:38.623+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:39.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:38 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:38.623+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:39.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:38 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:38.623+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:39.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:38 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:38.623+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:39.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:38 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:38.623+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:39.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:38 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:38.623+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:39.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:39 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:39.139+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:39.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:39 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:39.139+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:39.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:39 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:39.139+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:39.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:39 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:39.139+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:39.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:39 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:39.139+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:39.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:39 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:39.139+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:39.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:39.429+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:39.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:39.429+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:39.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:39.429+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:39.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:39.429+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:39.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:39.429+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:04.229215+0000 front 2024-04-20T21:46:04.229322+0000 (oldest deadline 2024-04-20T21:46:29.529473+0000) 2024-04-20T21:46:39.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:39 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:39.429+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:14.832060+0000 front 2024-04-20T21:46:14.831812+0000 (oldest deadline 2024-04-20T21:46:38.233510+0000) 2024-04-20T21:46:40.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:39 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:39.613+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:40.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:39 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:39.613+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:40.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:39 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:39.613+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:40.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:39 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:39.613+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:40.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:39 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:39.613+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:40.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:39 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:39.614+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:40.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:39 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:39.614+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:40.524 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:40 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:40.101+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:40.524 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:40 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:40.101+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:40.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:40 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:40.101+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:40.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:40 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:40.101+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:40.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:40 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:40.101+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:40.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:40 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:40.101+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:40.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:40 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:40.469+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:25.018514+0000 front 2024-04-20T21:45:25.018419+0000 (oldest deadline 2024-04-20T21:45:50.918500+0000) 2024-04-20T21:46:40.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:40 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:40.469+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.220944+0000 front 2024-04-20T21:45:33.220879+0000 (oldest deadline 2024-04-20T21:45:54.921158+0000) 2024-04-20T21:46:40.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:40 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:40.469+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:40.823130+0000 front 2024-04-20T21:45:40.823462+0000 (oldest deadline 2024-04-20T21:46:04.923230+0000) 2024-04-20T21:46:40.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:40 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:40.469+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:49.624876+0000 front 2024-04-20T21:45:49.625221+0000 (oldest deadline 2024-04-20T21:46:13.725071+0000) 2024-04-20T21:46:40.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:40 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:40.469+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:04.229215+0000 front 2024-04-20T21:46:04.229322+0000 (oldest deadline 2024-04-20T21:46:29.529473+0000) 2024-04-20T21:46:40.907 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:40 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9[49208]: 2024-04-20T21:46:40.469+0000 7f0c77200700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:14.832060+0000 front 2024-04-20T21:46:14.831812+0000 (oldest deadline 2024-04-20T21:46:38.233510+0000) 2024-04-20T21:46:41.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:40 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:40.580+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:41.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:40 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:40.580+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:41.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:40 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:40.580+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:41.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:40 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:40.580+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:41.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:40 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:40.580+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:41.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:40 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:40.580+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:41.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:40 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:40.580+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:41.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:41.140+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:41.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:41.140+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:41.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:41.140+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:41.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:41.140+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:41.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:41.140+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:41.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:41.140+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:41.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:41 smithi022 podman[79446]: 2024-04-20 21:46:41.487783167 +0000 UTC m=+5.634117861 container died a29e5ee3302c777554ded2f963debece1cddfa117328d5feeb2398569816c802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:46:42.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:41 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:41.595+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:42.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:41 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:41.595+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:42.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:41 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:41.595+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:42.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:41 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:41.595+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:42.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:41 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:41.595+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:42.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:41 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:41.595+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:42.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:41 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:41.595+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:42.249 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:42 smithi022 podman[79446]: 2024-04-20 21:46:42.007738617 +0000 UTC m=+6.154073338 container cleanup a29e5ee3302c777554ded2f963debece1cddfa117328d5feeb2398569816c802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True) 2024-04-20T21:46:42.250 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:42 smithi022 bash[79446]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9 2024-04-20T21:46:42.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:42 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:42.139+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:42.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:42 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:42.139+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:42.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:42 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:42.139+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:42.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:42 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:42.139+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:42.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:42 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:42.139+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:42.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:42 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:42.139+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:42.656 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:42 smithi022 podman[79463]: 2024-04-20 21:46:42.250324386 +0000 UTC m=+0.757887325 container remove a29e5ee3302c777554ded2f963debece1cddfa117328d5feeb2398569816c802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:46:43.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:42 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:42.559+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:43.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:42 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:42.559+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:43.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:42 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:42.559+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:43.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:42 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:42.559+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:43.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:42 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:42.559+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:43.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:42 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:42.559+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:43.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:42 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:42.559+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:43.354 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:43 smithi022 podman[79552]: 2024-04-20 21:46:42.984557534 +0000 UTC m=+0.034846629 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:43.354 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:43 smithi022 podman[79552]: 2024-04-20 21:46:43.10132091 +0000 UTC m=+0.151610027 container create 2ffa45e6c58558faef480e43372c4f3d6221f6567b2e23535f62823256e8c394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:46:43.354 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:43 smithi022 podman[79552]: 2024-04-20 21:46:43.354801779 +0000 UTC m=+0.405090918 container init 2ffa45e6c58558faef480e43372c4f3d6221f6567b2e23535f62823256e8c394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-20T21:46:43.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:43 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:43.109+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:43.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:43 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:43.109+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:43.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:43 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:43.109+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:43.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:43 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:43.109+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:43.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:43 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:43.109+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:43.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:43 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:43.109+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:43.606 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:43 smithi022 podman[79552]: 2024-04-20 21:46:43.365856531 +0000 UTC m=+0.416145670 container start 2ffa45e6c58558faef480e43372c4f3d6221f6567b2e23535f62823256e8c394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-20T21:46:43.606 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:43 smithi022 podman[79552]: 2024-04-20 21:46:43.453542996 +0000 UTC m=+0.503832175 container attach 2ffa45e6c58558faef480e43372c4f3d6221f6567b2e23535f62823256e8c394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-20T21:46:43.906 INFO:journalctl@ceph.osd.9.smithi022.stdout:Apr 20 21:46:43 smithi022 podman[79552]: 2024-04-20 21:46:43.844799643 +0000 UTC m=+0.895088739 container died 2ffa45e6c58558faef480e43372c4f3d6221f6567b2e23535f62823256e8c394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-20T21:46:44.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:43 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:43.564+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:44.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:43 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:43.564+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:44.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:43 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:43.564+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:44.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:43 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:43.564+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:44.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:43 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:43.564+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:44.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:43 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:43.564+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:44.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:43 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:43.564+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:44.426 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:44.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:44.426 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:44.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:44.426 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:44.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:44.427 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:44.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:44.427 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:44.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:44.427 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:44 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:44.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:44.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.9.service' 2024-04-20T21:46:44.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:44.744 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-20T21:46:44.744 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-20T21:46:44.744 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.10 2024-04-20T21:46:45.044 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:44 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:44.565+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:45.044 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:44 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:44.565+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:45.044 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:44 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:44.565+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:45.044 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:44 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:44.565+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:45.045 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:44 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:44.565+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:45.045 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:44 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:44.565+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:45.045 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:44 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:44.565+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:45.045 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:44 smithi033 systemd[1]: Stopping Ceph osd.10 for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:45.045 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:45.044+0000 7f25d6929700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:45.045 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:45.044+0000 7f25d6929700 -1 osd.10 107 *** Got signal Terminated *** 2024-04-20T21:46:45.045 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:45.044+0000 7f25d6929700 -1 osd.10 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:45.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:45 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:45.105+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:45.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:45 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:45.105+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:45.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:45 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:45.105+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:45.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:45 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:45.105+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:45.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:45 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:45.105+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:45.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:45 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:45.105+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:45.803 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:45.534+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:45.803 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:45.534+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:45.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:45.534+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:45.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:45.534+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:45.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:45.534+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:45.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:45.534+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:45.804 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:45 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:45.534+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:46.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:46 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:46.070+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:46.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:46 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:46.070+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:46.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:46 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:46.070+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:46.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:46 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:46.070+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:46.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:46 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:46.070+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:46.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:46 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:46.070+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:46.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:46 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:46.070+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:47.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:46 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:46.557+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:47.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:46 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:46.557+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:47.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:46 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:46.557+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:47.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:46 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:46.557+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:47.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:46 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:46.557+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:47.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:46 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:46.557+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:47.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:46 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:46.557+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:47.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:47 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:47.083+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:47.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:47 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:47.083+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:47.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:47 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:47.083+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:47.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:47 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:47.083+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:47.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:47 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:47.083+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:47.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:47 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:47.083+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:47.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:47 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:47.083+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:48.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:47 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:47.599+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:48.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:47 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:47.599+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:48.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:47 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:47.599+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:48.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:47 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:47.599+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:48.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:47 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:47.599+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:48.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:47 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:47.599+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:48.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:47 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:47.599+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:48.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:48 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:48.122+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:48.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:48 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:48.122+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:48.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:48 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:48.122+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:48.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:48 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:48.122+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:48.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:48 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:48.122+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:48.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:48 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:48.122+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:48.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:48 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:48.122+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:49.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:48 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:48.596+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:49.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:48 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:48.596+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:49.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:48 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:48.596+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:49.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:48 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:48.596+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:49.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:48 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:48.596+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:49.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:48 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:48.596+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:49.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:48 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:48.596+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:49.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:49 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:49.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:49.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:49 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:49.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:49.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:49 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:49.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:49.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:49 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:49.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:49.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:49 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:49.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:49.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:49 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:49.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:49.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:49 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:49.099+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:50.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:49 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:49.589+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:23.381121+0000 front 2024-04-20T21:45:23.381185+0000 (oldest deadline 2024-04-20T21:45:47.480885+0000) 2024-04-20T21:46:50.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:49 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:49.589+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:31.582124+0000 front 2024-04-20T21:45:31.582057+0000 (oldest deadline 2024-04-20T21:45:54.482174+0000) 2024-04-20T21:46:50.053 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:49 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:49.589+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:39.083778+0000 front 2024-04-20T21:45:39.083650+0000 (oldest deadline 2024-04-20T21:46:04.983620+0000) 2024-04-20T21:46:50.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:49 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:49.589+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-04-20T21:45:44.984150+0000 front 2024-04-20T21:45:44.984202+0000 (oldest deadline 2024-04-20T21:46:14.384698+0000) 2024-04-20T21:46:50.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:49 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:49.589+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:54.385214+0000 front 2024-04-20T21:45:54.385181+0000 (oldest deadline 2024-04-20T21:46:22.585615+0000) 2024-04-20T21:46:50.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:49 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:49.589+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.987260+0000 front 2024-04-20T21:46:05.987326+0000 (oldest deadline 2024-04-20T21:46:30.087262+0000) 2024-04-20T21:46:50.054 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:49 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10[46729]: 2024-04-20T21:46:49.589+0000 7f25d2f1d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:10.087805+0000 front 2024-04-20T21:46:10.087985+0000 (oldest deadline 2024-04-20T21:46:35.987728+0000) 2024-04-20T21:46:50.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:50.108+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:50.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:50.108+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:50.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:50.108+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:50.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:50.108+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:50.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:50.108+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:50.526 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:50.108+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:50.526 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:50 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:50.108+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:50.835 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:50 smithi033 podman[58233]: 2024-04-20 21:46:50.57667688 +0000 UTC m=+5.632884225 container died 27bb3d6239cb986e8dc49a9b08acb54a1831acde77e242a032c0c64d45ed95d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-20T21:46:51.089 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:51 smithi033 podman[58233]: 2024-04-20 21:46:51.070136941 +0000 UTC m=+6.126344293 container cleanup 27bb3d6239cb986e8dc49a9b08acb54a1831acde77e242a032c0c64d45ed95d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-20T21:46:51.089 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:51 smithi033 bash[58233]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10 2024-04-20T21:46:51.524 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:51 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:51.084+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:51.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:51 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:51.084+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:51.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:51 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:51.084+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:51.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:51 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:51.084+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:51.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:51 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:51.084+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:51.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:51 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:51.084+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:51.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:51 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:51.084+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:51.553 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:51 smithi033 podman[58249]: 2024-04-20 21:46:51.271508467 +0000 UTC m=+0.688231921 container remove 27bb3d6239cb986e8dc49a9b08acb54a1831acde77e242a032c0c64d45ed95d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-20T21:46:51.940 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:51 smithi033 podman[58337]: 2024-04-20 21:46:51.585318472 +0000 UTC m=+0.018360576 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:51.940 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:51 smithi033 podman[58337]: 2024-04-20 21:46:51.705957478 +0000 UTC m=+0.138999604 container create dd93cb7011a8a4139d67eba845c20dcd3be78d4c18bb4edb41fbb0a517c0532a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-20T21:46:51.940 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:51 smithi033 podman[58337]: 2024-04-20 21:46:51.942452539 +0000 UTC m=+0.375494623 container init dd93cb7011a8a4139d67eba845c20dcd3be78d4c18bb4edb41fbb0a517c0532a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-20T21:46:52.250 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:51 smithi033 podman[58337]: 2024-04-20 21:46:51.945814429 +0000 UTC m=+0.378856515 container start dd93cb7011a8a4139d67eba845c20dcd3be78d4c18bb4edb41fbb0a517c0532a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-20T21:46:52.250 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:52 smithi033 podman[58337]: 2024-04-20 21:46:52.034634705 +0000 UTC m=+0.467676805 container attach dd93cb7011a8a4139d67eba845c20dcd3be78d4c18bb4edb41fbb0a517c0532a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:46:52.524 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:52.096+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:52.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:52.096+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:52.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:52.096+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:52.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:52.096+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:52.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:52.096+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:52.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:52.096+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:52.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:52 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:52.096+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:52.543 INFO:journalctl@ceph.osd.10.smithi033.stdout:Apr 20 21:46:52 smithi033 podman[58337]: 2024-04-20 21:46:52.252620841 +0000 UTC m=+0.685662926 container died dd93cb7011a8a4139d67eba845c20dcd3be78d4c18bb4edb41fbb0a517c0532a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-04-20T21:46:52.833 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.10.service' 2024-04-20T21:46:52.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:52.874 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-20T21:46:52.874 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-20T21:46:52.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.11 2024-04-20T21:46:53.180 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:52 smithi122 systemd[1]: Stopping Ceph osd.11 for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:53.181 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:53.048+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:53.181 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:53.048+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:53.181 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:53.048+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:53.181 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:53.048+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:53.181 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:53.048+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:53.181 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:53.048+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:53.181 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:53.048+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:53.181 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:53.181+0000 7f19f1709700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:53.182 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:53.181+0000 7f19f1709700 -1 osd.11 107 *** Got signal Terminated *** 2024-04-20T21:46:53.182 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:53 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:53.181+0000 7f19f1709700 -1 osd.11 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:54.478 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:54 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:54.076+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:54.478 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:54 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:54.076+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:54.478 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:54 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:54.077+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:54.479 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:54 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:54.077+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:54.479 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:54 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:54.077+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:54.479 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:54 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:54.077+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:54.479 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:54 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:54.077+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:55.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:55.097+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:55.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:55.097+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:55.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:55.097+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:55.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:55.097+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:55.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:55.097+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:55.526 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:55.097+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:55.526 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:55 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:55.097+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:56.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:56.125+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:56.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:56.125+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:56.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:56.125+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:56.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:56.125+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:56.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:56.125+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:56.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:56.125+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:56.526 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:56.125+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:56.526 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:56 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:56.125+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6820 osd.8 since back 2024-04-20T21:46:30.248059+0000 front 2024-04-20T21:46:30.248081+0000 (oldest deadline 2024-04-20T21:46:55.548189+0000) 2024-04-20T21:46:57.524 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:57 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:57.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:57.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:57 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:57.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:57.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:57 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:57.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:57.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:57 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:57.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:57.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:57 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:57.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:57.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:57 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:57.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:57.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:57 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:57.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:57.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:57 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:57.114+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6820 osd.8 since back 2024-04-20T21:46:30.248059+0000 front 2024-04-20T21:46:30.248081+0000 (oldest deadline 2024-04-20T21:46:55.548189+0000) 2024-04-20T21:46:58.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:58.100+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-04-20T21:45:24.637099+0000 front 2024-04-20T21:45:24.637113+0000 (oldest deadline 2024-04-20T21:45:49.336931+0000) 2024-04-20T21:46:58.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:58.100+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6804 osd.1 since back 2024-04-20T21:45:33.839247+0000 front 2024-04-20T21:45:33.839058+0000 (oldest deadline 2024-04-20T21:45:57.339081+0000) 2024-04-20T21:46:58.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:58.100+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-20T21:45:37.339502+0000 front 2024-04-20T21:45:37.339407+0000 (oldest deadline 2024-04-20T21:46:02.540292+0000) 2024-04-20T21:46:58.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:58.100+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6812 osd.4 since back 2024-04-20T21:45:56.042402+0000 front 2024-04-20T21:45:56.042333+0000 (oldest deadline 2024-04-20T21:46:23.643077+0000) 2024-04-20T21:46:58.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:58.100+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-20T21:46:05.344065+0000 front 2024-04-20T21:46:05.343963+0000 (oldest deadline 2024-04-20T21:46:32.944539+0000) 2024-04-20T21:46:58.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:58.100+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-04-20T21:46:12.945530+0000 front 2024-04-20T21:46:12.945390+0000 (oldest deadline 2024-04-20T21:46:36.445034+0000) 2024-04-20T21:46:58.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:58.100+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.33:6820 osd.7 since back 2024-04-20T21:46:23.446473+0000 front 2024-04-20T21:46:23.446508+0000 (oldest deadline 2024-04-20T21:46:45.746409+0000) 2024-04-20T21:46:58.526 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:58 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11[47408]: 2024-04-20T21:46:58.100+0000 7f19edcfd700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.122:6820 osd.8 since back 2024-04-20T21:46:30.248059+0000 front 2024-04-20T21:46:30.248081+0000 (oldest deadline 2024-04-20T21:46:55.548189+0000) 2024-04-20T21:46:59.004 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:58 smithi122 podman[59454]: 2024-04-20 21:46:58.708940672 +0000 UTC m=+5.641620692 container died fc4581786c9fc0577e4fb3fab455f580a3aeecdb92b54de6e48347481efbeb58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-20T21:46:59.004 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:59 smithi122 podman[59454]: 2024-04-20 21:46:59.006450647 +0000 UTC m=+5.939130690 container cleanup fc4581786c9fc0577e4fb3fab455f580a3aeecdb92b54de6e48347481efbeb58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:46:59.274 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:59 smithi122 bash[59454]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11 2024-04-20T21:46:59.275 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:59 smithi122 podman[59471]: 2024-04-20 21:46:59.249174765 +0000 UTC m=+0.532339355 container remove fc4581786c9fc0577e4fb3fab455f580a3aeecdb92b54de6e48347481efbeb58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True) 2024-04-20T21:46:59.833 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:59 smithi122 podman[59564]: 2024-04-20 21:46:59.469573114 +0000 UTC m=+0.017691645 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:59.833 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:59 smithi122 podman[59564]: 2024-04-20 21:46:59.575034678 +0000 UTC m=+0.123153201 container create eee4c8a71727cf58ab0b8a7fc13ea0f3539423f381c009749e5e0f65b4239b7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-20T21:46:59.834 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:59 smithi122 podman[59564]: 2024-04-20 21:46:59.76952704 +0000 UTC m=+0.317645574 container init eee4c8a71727cf58ab0b8a7fc13ea0f3539423f381c009749e5e0f65b4239b7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-20T21:46:59.834 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:59 smithi122 podman[59564]: 2024-04-20 21:46:59.772740259 +0000 UTC m=+0.320858783 container start eee4c8a71727cf58ab0b8a7fc13ea0f3539423f381c009749e5e0f65b4239b7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True) 2024-04-20T21:46:59.834 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:46:59 smithi122 podman[59564]: 2024-04-20 21:46:59.835437409 +0000 UTC m=+0.383555940 container attach eee4c8a71727cf58ab0b8a7fc13ea0f3539423f381c009749e5e0f65b4239b7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:47:00.274 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:47:00 smithi122 podman[59564]: 2024-04-20 21:47:00.060184629 +0000 UTC m=+0.608303149 container died eee4c8a71727cf58ab0b8a7fc13ea0f3539423f381c009749e5e0f65b4239b7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-20T21:47:00.627 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 20 21:47:00 smithi122 podman[59590]: 2024-04-20 21:47:00.614041865 +0000 UTC m=+0.543940829 container remove eee4c8a71727cf58ab0b8a7fc13ea0f3539423f381c009749e5e0f65b4239b7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-20T21:47:00.632 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@osd.11.service' 2024-04-20T21:47:00.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:00.668 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-20T21:47:00.668 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-20T21:47:00.669 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.a 2024-04-20T21:47:01.156 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:47:00 smithi022 systemd[1]: Stopping Ceph mds.a for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:01.656 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:47:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-a[56141]: 2024-04-20T21:47:01.231+0000 7fb1a2800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:01.656 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:47:01 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-a[56141]: 2024-04-20T21:47:01.231+0000 7fb1a2800700 -1 mds.a *** got signal Terminated *** 2024-04-20T21:47:02.832 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:47:02 smithi022 podman[79671]: 2024-04-20 21:47:02.497624952 +0000 UTC m=+1.395349128 container died 9c4294888507995bd1473a6fbaa9e55ea8ee00a749262e6e84fb1de21ae1ad3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:47:03.156 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:47:02 smithi022 podman[79671]: 2024-04-20 21:47:02.83304274 +0000 UTC m=+1.730766950 container cleanup 9c4294888507995bd1473a6fbaa9e55ea8ee00a749262e6e84fb1de21ae1ad3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-20T21:47:03.157 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:47:02 smithi022 bash[79671]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-a 2024-04-20T21:47:03.157 INFO:journalctl@ceph.mds.a.smithi022.stdout:Apr 20 21:47:03 smithi022 podman[79684]: 2024-04-20 21:47:03.017373488 +0000 UTC m=+0.503585616 container remove 9c4294888507995bd1473a6fbaa9e55ea8ee00a749262e6e84fb1de21ae1ad3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:47:03.573 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.a.service' 2024-04-20T21:47:03.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:03.690 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-20T21:47:03.691 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-20T21:47:03.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.d 2024-04-20T21:47:04.156 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:47:03 smithi022 systemd[1]: Stopping Ceph mds.d for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:04.656 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:47:04 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-d[55170]: 2024-04-20T21:47:04.376+0000 7f783be00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:04.656 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:47:04 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-d[55170]: 2024-04-20T21:47:04.376+0000 7f783be00700 -1 mds.d *** got signal Terminated *** 2024-04-20T21:47:08.189 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:47:07 smithi022 podman[79789]: 2024-04-20 21:47:07.913978499 +0000 UTC m=+3.689972771 container died 5952b92e908cb3154d274fd78226902eabf158424fc130b8bf41f92cef439eb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:47:08.478 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:47:08 smithi022 podman[79789]: 2024-04-20 21:47:08.294350052 +0000 UTC m=+4.070344338 container cleanup 5952b92e908cb3154d274fd78226902eabf158424fc130b8bf41f92cef439eb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-d, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-20T21:47:08.479 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:47:08 smithi022 bash[79789]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-d 2024-04-20T21:47:08.903 INFO:journalctl@ceph.mds.d.smithi022.stdout:Apr 20 21:47:08 smithi022 podman[79803]: 2024-04-20 21:47:08.478626501 +0000 UTC m=+0.545995001 container remove 5952b92e908cb3154d274fd78226902eabf158424fc130b8bf41f92cef439eb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-d, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:47:08.926 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.d.service' 2024-04-20T21:47:09.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:09.033 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-20T21:47:09.033 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-20T21:47:09.033 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.g 2024-04-20T21:47:09.627 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:47:09 smithi022 systemd[1]: Stopping Ceph mds.g for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:09.906 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:47:09 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-g[55794]: 2024-04-20T21:47:09.628+0000 7f112b200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:09.906 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:47:09 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-g[55794]: 2024-04-20T21:47:09.628+0000 7f112b200700 -1 mds.g *** got signal Terminated *** 2024-04-20T21:47:13.906 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:47:13 smithi022 podman[79910]: 2024-04-20 21:47:13.649016098 +0000 UTC m=+4.145896357 container died 84dd5dfa8f035ae5915f92dc7288fed0b8334d5e0242f98f8a669367f16704c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:47:14.214 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:47:14 smithi022 podman[79910]: 2024-04-20 21:47:14.03123848 +0000 UTC m=+4.528118739 container cleanup 84dd5dfa8f035ae5915f92dc7288fed0b8334d5e0242f98f8a669367f16704c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-20T21:47:14.214 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:47:14 smithi022 bash[79910]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-g 2024-04-20T21:47:14.656 INFO:journalctl@ceph.mds.g.smithi022.stdout:Apr 20 21:47:14 smithi022 podman[79924]: 2024-04-20 21:47:14.215453622 +0000 UTC m=+0.554157681 container remove 84dd5dfa8f035ae5915f92dc7288fed0b8334d5e0242f98f8a669367f16704c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-20T21:47:14.702 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.g.service' 2024-04-20T21:47:14.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:14.816 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-20T21:47:14.816 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-20T21:47:14.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.j 2024-04-20T21:47:15.406 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:47:14 smithi022 systemd[1]: Stopping Ceph mds.j for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:15.906 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:47:15 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-j[54802]: 2024-04-20T21:47:15.424+0000 7f6c42800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:15.907 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:47:15 smithi022 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-j[54802]: 2024-04-20T21:47:15.424+0000 7f6c42800700 -1 mds.j *** got signal Terminated *** 2024-04-20T21:47:19.862 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:47:19 smithi022 podman[80030]: 2024-04-20 21:47:19.44480102 +0000 UTC m=+4.144221250 container died e1fde6631cfcba5dad119537dd3a1ed18ab49118a1c5db33810c414383c10fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:47:20.142 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:47:19 smithi022 podman[80030]: 2024-04-20 21:47:19.975821164 +0000 UTC m=+4.675241408 container cleanup e1fde6631cfcba5dad119537dd3a1ed18ab49118a1c5db33810c414383c10fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-20T21:47:20.142 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:47:19 smithi022 bash[80030]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-j 2024-04-20T21:47:20.406 INFO:journalctl@ceph.mds.j.smithi022.stdout:Apr 20 21:47:20 smithi022 podman[80044]: 2024-04-20 21:47:20.143325807 +0000 UTC m=+0.676923455 container remove e1fde6631cfcba5dad119537dd3a1ed18ab49118a1c5db33810c414383c10fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:47:20.611 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.j.service' 2024-04-20T21:47:20.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:20.721 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-20T21:47:20.721 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-20T21:47:20.721 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.b 2024-04-20T21:47:21.053 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:47:20 smithi033 systemd[1]: Stopping Ceph mds.b for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:21.325 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:47:21 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b[51942]: 2024-04-20T21:47:21.077+0000 7f1ee3dcb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:21.326 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:47:21 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b[51942]: 2024-04-20T21:47:21.077+0000 7f1ee3dcb700 -1 mds.b *** got signal Terminated *** 2024-04-20T21:47:23.053 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:47:22 smithi033 podman[58443]: 2024-04-20 21:47:22.789497176 +0000 UTC m=+1.865472291 container died ac595150a4206b95bd3afe73304d384f80202452b2a59474322011e3fb6f013e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True) 2024-04-20T21:47:23.521 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:47:23 smithi033 podman[58443]: 2024-04-20 21:47:23.271852788 +0000 UTC m=+2.347827904 container cleanup ac595150a4206b95bd3afe73304d384f80202452b2a59474322011e3fb6f013e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-20T21:47:23.521 INFO:journalctl@ceph.mds.b.smithi033.stdout:Apr 20 21:47:23 smithi033 bash[58443]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-b 2024-04-20T21:47:23.658 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.b.service' 2024-04-20T21:47:23.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:23.694 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-20T21:47:23.694 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-20T21:47:23.694 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.e 2024-04-20T21:47:24.053 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:47:23 smithi033 systemd[1]: Stopping Ceph mds.e for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:24.553 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:47:24 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-e[51327]: 2024-04-20T21:47:24.070+0000 7f5026b91700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:24.553 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:47:24 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-e[51327]: 2024-04-20T21:47:24.070+0000 7f5026b91700 -1 mds.e *** got signal Terminated *** 2024-04-20T21:47:27.036 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:47:26 smithi033 podman[58578]: 2024-04-20 21:47:26.780395325 +0000 UTC m=+2.866910550 container died bb6003d282922aa87bba475b98c50cde9aab9726f425872f38175705f2e76667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-20T21:47:27.303 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:47:27 smithi033 podman[58578]: 2024-04-20 21:47:27.137073423 +0000 UTC m=+3.223588650 container cleanup bb6003d282922aa87bba475b98c50cde9aab9726f425872f38175705f2e76667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-20T21:47:27.303 INFO:journalctl@ceph.mds.e.smithi033.stdout:Apr 20 21:47:27 smithi033 bash[58578]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-e 2024-04-20T21:47:27.552 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.e.service' 2024-04-20T21:47:27.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:27.586 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-20T21:47:27.586 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-20T21:47:27.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.h 2024-04-20T21:47:27.803 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:47:27 smithi033 systemd[1]: Stopping Ceph mds.h for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:28.303 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:47:27 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-h[51636]: 2024-04-20T21:47:27.907+0000 7f1461db9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:28.303 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:47:27 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-h[51636]: 2024-04-20T21:47:27.907+0000 7f1461db9700 -1 mds.h *** got signal Terminated *** 2024-04-20T21:47:31.514 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:47:31 smithi033 podman[58700]: 2024-04-20 21:47:31.242059977 +0000 UTC m=+3.436372005 container died fd7697cbf577a9405a3557b8cb29689be25e5516f7a089b958f15ba285b6a652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-20T21:47:31.776 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:47:31 smithi033 podman[58700]: 2024-04-20 21:47:31.615898297 +0000 UTC m=+3.810210326 container cleanup fd7697cbf577a9405a3557b8cb29689be25e5516f7a089b958f15ba285b6a652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-20T21:47:31.776 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:47:31 smithi033 bash[58700]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-h 2024-04-20T21:47:32.029 INFO:journalctl@ceph.mds.h.smithi033.stdout:Apr 20 21:47:31 smithi033 podman[58714]: 2024-04-20 21:47:31.875619667 +0000 UTC m=+0.625010692 container remove fd7697cbf577a9405a3557b8cb29689be25e5516f7a089b958f15ba285b6a652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-20T21:47:32.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.h.service' 2024-04-20T21:47:32.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:32.085 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-20T21:47:32.085 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-20T21:47:32.086 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.k 2024-04-20T21:47:32.303 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:47:32 smithi033 systemd[1]: Stopping Ceph mds.k for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:32.803 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:47:32 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-k[51006]: 2024-04-20T21:47:32.419+0000 7f6658901700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:32.803 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:47:32 smithi033 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-k[51006]: 2024-04-20T21:47:32.419+0000 7f6658901700 -1 mds.k *** got signal Terminated *** 2024-04-20T21:47:36.053 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:47:35 smithi033 podman[58821]: 2024-04-20 21:47:35.778581482 +0000 UTC m=+3.472865971 container died 7f796241d1be80257b800de7d7dfdbe5811f16c5a66d93f271fd27f243c50251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-20T21:47:36.415 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:47:36 smithi033 podman[58821]: 2024-04-20 21:47:36.077793004 +0000 UTC m=+3.772077499 container cleanup 7f796241d1be80257b800de7d7dfdbe5811f16c5a66d93f271fd27f243c50251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-20T21:47:36.416 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:47:36 smithi033 bash[58821]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-k 2024-04-20T21:47:36.416 INFO:journalctl@ceph.mds.k.smithi033.stdout:Apr 20 21:47:36 smithi033 podman[58835]: 2024-04-20 21:47:36.303789893 +0000 UTC m=+0.522015992 container remove 7f796241d1be80257b800de7d7dfdbe5811f16c5a66d93f271fd27f243c50251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:47:36.432 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.k.service' 2024-04-20T21:47:36.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:36.464 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-20T21:47:36.464 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-20T21:47:36.464 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.c 2024-04-20T21:47:36.774 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:47:36 smithi122 systemd[1]: Stopping Ceph mds.c for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:37.037 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:47:36 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-c[52530]: 2024-04-20T21:47:36.871+0000 7f90c2270700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:37.037 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:47:36 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-c[52530]: 2024-04-20T21:47:36.871+0000 7f90c2270700 -1 mds.c *** got signal Terminated *** 2024-04-20T21:47:40.812 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:47:40 smithi122 podman[59668]: 2024-04-20 21:47:40.542008128 +0000 UTC m=+3.886972354 container died d6f9c5f0e01c9f6149716e80b514258067e1f6f87295451ec8ee4d82407a903b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-20T21:47:41.188 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:47:40 smithi122 podman[59668]: 2024-04-20 21:47:40.913470559 +0000 UTC m=+4.258434784 container cleanup d6f9c5f0e01c9f6149716e80b514258067e1f6f87295451ec8ee4d82407a903b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-20T21:47:41.188 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 20 21:47:40 smithi122 bash[59668]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-c 2024-04-20T21:47:41.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.c.service' 2024-04-20T21:47:41.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:41.384 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-20T21:47:41.384 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-20T21:47:41.384 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.f 2024-04-20T21:47:41.716 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:47:41 smithi122 systemd[1]: Stopping Ceph mds.f for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:42.024 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:47:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-f[52216]: 2024-04-20T21:47:41.717+0000 7efe4813e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:42.025 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:47:41 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-f[52216]: 2024-04-20T21:47:41.717+0000 7efe4813e700 -1 mds.f *** got signal Terminated *** 2024-04-20T21:47:45.840 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:47:45 smithi122 podman[59800]: 2024-04-20 21:47:45.540685423 +0000 UTC m=+3.938906877 container died 669031ef3cd064a93190a7a9511b281cf47d0ac7b059c00254c09e917113bd91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-20T21:47:45.840 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:47:45 smithi122 podman[59800]: 2024-04-20 21:47:45.842619897 +0000 UTC m=+4.240841341 container cleanup 669031ef3cd064a93190a7a9511b281cf47d0ac7b059c00254c09e917113bd91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:47:46.099 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 20 21:47:45 smithi122 bash[59800]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-f 2024-04-20T21:47:46.260 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.f.service' 2024-04-20T21:47:46.291 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:46.292 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-20T21:47:46.292 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-20T21:47:46.292 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.i 2024-04-20T21:47:46.619 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:47:46 smithi122 systemd[1]: Stopping Ceph mds.i for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:47.024 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:47:46 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-i[52837]: 2024-04-20T21:47:46.621+0000 7f863d8e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:47.025 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:47:46 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-i[52837]: 2024-04-20T21:47:46.621+0000 7f863d8e0700 -1 mds.i *** got signal Terminated *** 2024-04-20T21:47:50.927 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:47:50 smithi122 podman[59921]: 2024-04-20 21:47:50.627176366 +0000 UTC m=+4.120160190 container died 18510cab3c10992bd574b17f5582c0cf1db691aa2c023ddaf921a72d8df8b94b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:47:50.927 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:47:50 smithi122 podman[59921]: 2024-04-20 21:47:50.929165257 +0000 UTC m=+4.422149087 container cleanup 18510cab3c10992bd574b17f5582c0cf1db691aa2c023ddaf921a72d8df8b94b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-20T21:47:51.203 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 20 21:47:50 smithi122 bash[59921]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-i 2024-04-20T21:47:51.347 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.i.service' 2024-04-20T21:47:51.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:51.378 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-20T21:47:51.378 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-20T21:47:51.378 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.l 2024-04-20T21:47:51.774 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:47:51 smithi122 systemd[1]: Stopping Ceph mds.l for 92c49c5c-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:51.775 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:47:51 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-l[51907]: 2024-04-20T21:47:51.698+0000 7f4ba28b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:51.775 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:47:51 smithi122 ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-l[51907]: 2024-04-20T21:47:51.698+0000 7f4ba28b7700 -1 mds.l *** got signal Terminated *** 2024-04-20T21:47:53.099 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:47:52 smithi122 podman[60043]: 2024-04-20 21:47:52.846849036 +0000 UTC m=+1.248636262 container died f62252f8aaf22fb2d3ae17ac5adb225afa849f3c0649d44be21cdbe3b0ebfce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:47:53.365 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:47:53 smithi122 podman[60043]: 2024-04-20 21:47:53.18300199 +0000 UTC m=+1.584789225 container cleanup f62252f8aaf22fb2d3ae17ac5adb225afa849f3c0649d44be21cdbe3b0ebfce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-20T21:47:53.365 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:47:53 smithi122 bash[60043]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-l 2024-04-20T21:47:53.615 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.l.service' 2024-04-20T21:47:53.635 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:47:53 smithi122 podman[60057]: 2024-04-20 21:47:53.367130265 +0000 UTC m=+0.521297563 container remove f62252f8aaf22fb2d3ae17ac5adb225afa849f3c0649d44be21cdbe3b0ebfce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968-mds-l, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-20T21:47:53.635 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:47:53 smithi122 systemd[1]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.l.service: Deactivated successfully. 2024-04-20T21:47:53.635 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:47:53 smithi122 systemd[1]: Stopped Ceph mds.l for 92c49c5c-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:47:53.635 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 20 21:47:53 smithi122 systemd[1]: ceph-92c49c5c-ff5b-11ee-bc93-c7b262605968@mds.l.service: Consumed 1.080s CPU time. 2024-04-20T21:47:53.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:53.645 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-20T21:47:53.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-20T21:47:53.977 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:47:59.001 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-20T21:47:59.200 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:48:01.703 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-20T21:48:01.889 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:48:04.226 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:48:04.324 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:48:04.355 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:48:04.380 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-20T21:48:04.383 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405/remote/smithi022/crash 2024-04-20T21:48:04.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/crash -- . 2024-04-20T21:48:04.490 INFO:teuthology.orchestra.run.smithi022.stderr:tar: /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T21:48:04.490 INFO:teuthology.orchestra.run.smithi022.stderr:tar: Error is not recoverable: exiting now 2024-04-20T21:48:04.495 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405/remote/smithi033/crash 2024-04-20T21:48:04.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/crash -- . 2024-04-20T21:48:04.541 INFO:teuthology.orchestra.run.smithi033.stderr:tar: /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T21:48:04.542 INFO:teuthology.orchestra.run.smithi033.stderr:tar: Error is not recoverable: exiting now 2024-04-20T21:48:04.544 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405/remote/smithi122/crash 2024-04-20T21:48:04.544 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/crash -- . 2024-04-20T21:48:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:tar: /var/lib/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T21:48:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:tar: Error is not recoverable: exiting now 2024-04-20T21:48:04.579 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-20T21:48:04.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-20T21:48:04.692 INFO:tasks.cephadm:Compressing logs... 2024-04-20T21:48:04.692 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T21:48:04.696 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T21:48:04.699 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T21:48:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.log 2024-04-20T21:48:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:: No such file or directory 2024-04-20T21:48:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.2.log 2024-04-20T21:48:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.f.log 2024-04-20T21:48:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-20T21:48:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.5.log 2024-04-20T21:48:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.c.log 2024-04-20T21:48:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mon.c.log 2024-04-20T21:48:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mgr.z.log 2024-04-20T21:48:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.i.log 2024-04-20T21:48:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.log 2024-04-20T21:48:04.732 INFO:teuthology.orchestra.run.smithi033.stderr:: No such file or directory 2024-04-20T21:48:04.732 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-20T21:48:04.732 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.7.log 2024-04-20T21:48:04.732 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.b.log 2024-04-20T21:48:04.732 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.4.log 2024-04-20T21:48:04.732 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.1.log 2024-04-20T21:48:04.732 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mgr.y.log 2024-04-20T21:48:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.h.log 2024-04-20T21:48:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.10.log 2024-04-20T21:48:04.733 INFO:teuthology.orchestra.run.smithi033.stderr: 87.4% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-20T21:48:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.h.log: 90.3%gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-20T21:48:04.733 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-20T21:48:04.733 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mgr.z.log: 87.4% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-20T21:48:04.734 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-20T21:48:04.734 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.i.log: 90.3% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-20T21:48:04.734 INFO:teuthology.orchestra.run.smithi122.stderr: 94.5% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-20T21:48:04.734 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.11.log 2024-04-20T21:48:04.734 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.8.log 2024-04-20T21:48:04.734 INFO:teuthology.orchestra.run.smithi122.stderr: 83.5% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-20T21:48:04.734 INFO:teuthology.orchestra.run.smithi122.stderr: 94.5% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-20T21:48:04.734 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.11.log: 91.9% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-20T21:48:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-20T21:48:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.l.log 2024-04-20T21:48:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T21:48:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.10.log: 94.5% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-20T21:48:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.e.log 2024-04-20T21:48:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-20T21:48:04.740 INFO:teuthology.orchestra.run.smithi033.stderr: 83.6% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-20T21:48:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mon.b.log 2024-04-20T21:48:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-20T21:48:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.k.log 2024-04-20T21:48:04.741 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-20T21:48:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mon.b.log: 94.5% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-20T21:48:04.743 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T21:48:04.748 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T21:48:04.748 INFO:teuthology.orchestra.run.smithi122.stderr: 91.0% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-20T21:48:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.k.log: 91.0% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-20T21:48:04.752 INFO:teuthology.orchestra.run.smithi022.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.log 2024-04-20T21:48:04.752 INFO:teuthology.orchestra.run.smithi022.stderr:: No such file or directory 2024-04-20T21:48:04.753 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-20T21:48:04.753 INFO:teuthology.orchestra.run.smithi033.stderr: 94.4% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-20T21:48:04.754 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose --/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.log: /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.g.log 2024-04-20T21:48:04.755 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.j.log 2024-04-20T21:48:04.756 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T21:48:04.756 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.3.log 2024-04-20T21:48:04.757 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-20T21:48:04.757 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.0.log 2024-04-20T21:48:04.758 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.3.log: 90.1% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-20T21:48:04.759 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mgr.x.log 2024-04-20T21:48:04.760 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mon.a.log 2024-04-20T21:48:04.761 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mgr.x.log: 94.4% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-20T21:48:04.761 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.6.log 2024-04-20T21:48:04.763 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-20T21:48:04.764 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.6.log: 94.5%gzip -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.j.log.gz -5 2024-04-20T21:48:04.764 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.a.log 2024-04-20T21:48:04.766 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-20T21:48:04.766 INFO:teuthology.orchestra.run.smithi022.stderr: 83.9% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-20T21:48:04.767 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.d.log 2024-04-20T21:48:04.770 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.9.log 2024-04-20T21:48:04.772 INFO:teuthology.orchestra.run.smithi022.stderr: 94.4% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-20T21:48:04.776 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T21:48:04.776 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.9.log: 94.3% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-20T21:48:04.786 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/cephadm.log: 90.9% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-20T21:48:04.789 INFO:teuthology.orchestra.run.smithi022.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T21:48:05.036 INFO:teuthology.orchestra.run.smithi122.stderr: 91.8% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-20T21:48:05.096 INFO:teuthology.orchestra.run.smithi033.stderr: 91.8% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-20T21:48:05.174 INFO:teuthology.orchestra.run.smithi022.stderr: 90.1% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-20T21:48:05.881 INFO:teuthology.orchestra.run.smithi022.stderr: 90.8% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-20T21:48:06.316 INFO:teuthology.orchestra.run.smithi033.stderr: 93.9% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-20T21:48:07.038 INFO:teuthology.orchestra.run.smithi022.stderr: 93.9% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-20T21:48:07.048 INFO:teuthology.orchestra.run.smithi022.stderr: 93.9% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-20T21:48:07.064 INFO:teuthology.orchestra.run.smithi122.stderr: 94.2% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-20T21:48:07.283 INFO:teuthology.orchestra.run.smithi122.stderr: 93.8% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-20T21:48:07.324 INFO:teuthology.orchestra.run.smithi022.stderr: 93.6% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-20T21:48:07.336 INFO:teuthology.orchestra.run.smithi022.stderr: 93.9% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-20T21:48:07.340 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-04-20T21:48:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:real 0m2.612s 2024-04-20T21:48:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:user 0m10.447s 2024-04-20T21:48:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m0.919s 2024-04-20T21:48:07.406 INFO:teuthology.orchestra.run.smithi033.stderr: 94.3% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-20T21:48:07.428 INFO:teuthology.orchestra.run.smithi122.stderr: 94.3% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-20T21:48:07.567 INFO:teuthology.orchestra.run.smithi033.stderr: 94.0% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-20T21:48:07.618 INFO:teuthology.orchestra.run.smithi122.stderr: 94.6% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-20T21:48:07.796 INFO:teuthology.orchestra.run.smithi033.stderr: 94.3% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-20T21:48:09.074 INFO:teuthology.orchestra.run.smithi122.stderr: 95.6% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-20T21:48:09.075 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-20T21:48:09.076 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m4.363s 2024-04-20T21:48:09.076 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m14.647s 2024-04-20T21:48:09.076 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.546s 2024-04-20T21:49:17.783 INFO:teuthology.orchestra.run.smithi033.stderr: 93.6% -- replaced with /var/log/ceph/92c49c5c-ff5b-11ee-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-20T21:49:17.784 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-20T21:49:17.785 INFO:teuthology.orchestra.run.smithi033.stderr:real 1m13.070s 2024-04-20T21:49:17.785 INFO:teuthology.orchestra.run.smithi033.stderr:user 1m21.167s 2024-04-20T21:49:17.785 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m2.344s 2024-04-20T21:49:17.785 INFO:tasks.cephadm:Archiving logs... 2024-04-20T21:49:17.786 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405/remote/smithi022/log 2024-04-20T21:49:17.787 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T21:49:19.237 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405/remote/smithi033/log 2024-04-20T21:49:19.238 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T21:49:30.639 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405/remote/smithi122/log 2024-04-20T21:49:30.640 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T21:49:32.456 INFO:tasks.cephadm:Removing cluster... 2024-04-20T21:49:32.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 --force 2024-04-20T21:49:32.786 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:49:33.382 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 --force 2024-04-20T21:49:33.582 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:49:33.940 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 92c49c5c-ff5b-11ee-bc93-c7b262605968 --force 2024-04-20T21:49:34.130 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 92c49c5c-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:49:34.420 INFO:tasks.cephadm:Removing cephadm ... 2024-04-20T21:49:34.421 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T21:49:34.483 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T21:49:34.501 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T21:49:34.516 INFO:tasks.cephadm:Teardown complete 2024-04-20T21:49:34.517 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-20T21:49:34.528 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-20T21:49:34.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T21:49:34.535 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T21:49:34.544 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T21:49:34.691 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-20T21:49:34.701 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-20T21:49:34.702 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:49:34.710 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:49:34.712 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:49:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:49:34.726 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:49:34.729 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:49:34.729 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-04-20T21:49:34.730 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 37 -860us[ -860us] +/- 82ms 2024-04-20T21:49:34.730 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv02.front.sepia.ceph.com 3 8 377 234 -551us[ -506us] +/- 60ms 2024-04-20T21:49:34.730 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +1493us[+1493us] +/- 70ms 2024-04-20T21:49:34.730 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:49:34.731 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:49:34.732 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-04-20T21:49:34.732 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 -900us[ -900us] +/- 82ms 2024-04-20T21:49:34.732 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -588us[ -561us] +/- 63ms 2024-04-20T21:49:34.732 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +1470us[+1470us] +/- 71ms 2024-04-20T21:49:34.732 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:49:34.758 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:49:34.768 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:49:34.769 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-04-20T21:49:34.769 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 -858us[ -828us] +/- 82ms 2024-04-20T21:49:34.769 INFO:teuthology.orchestra.run.smithi022.stdout:^* hv02.front.sepia.ceph.com 3 7 377 37 -538us[ -508us] +/- 63ms 2024-04-20T21:49:34.769 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 235 +1491us[+1540us] +/- 68ms 2024-04-20T21:49:34.769 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:49:34.770 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-20T21:49:34.784 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-20T21:49:34.785 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-20T21:49:34.795 DEBUG:teuthology.orchestra.run.smithi022:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T21:49:34.973 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T21:49:35.030 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T21:49:35.081 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:49:35.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:49:35.177 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:49:35.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:49:35.207 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:49:35.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:49:35.233 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-20T21:49:35.244 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-20T21:49:35.254 INFO:teuthology.task.internal:Duration was 2260.011634 seconds 2024-04-20T21:49:35.255 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-20T21:49:35.265 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-20T21:49:35.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T21:49:35.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T21:49:35.276 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T21:49:35.312 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:49:35.314 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:49:35.407 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:49:35.817 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-20T21:49:35.817 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2024-04-20T21:49:35.817 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T21:49:35.894 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-04-20T21:49:35.894 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T21:49:35.925 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-04-20T21:49:35.926 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T21:49:35.953 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-20T21:49:35.953 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T21:49:35.961 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T21:49:35.968 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T21:49:36.046 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-20T21:49:36.047 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T21:49:36.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T21:49:36.055 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T21:49:36.518 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-20T21:49:36.529 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-20T21:49:36.529 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T21:49:36.617 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T21:49:36.644 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T21:49:36.667 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-20T21:49:36.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:49:36.686 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:49:36.689 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:49:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-04-20T21:49:36.730 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-04-20T21:49:36.783 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2024-04-20T21:49:36.827 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:49:36.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:49:36.880 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:49:36.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:49:36.895 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:49:36.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:49:36.909 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-20T21:49:36.920 INFO:teuthology.task.internal:Transferring archived files... 2024-04-20T21:49:36.922 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405/remote/smithi022 2024-04-20T21:49:36.922 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T21:49:37.026 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405/remote/smithi033 2024-04-20T21:49:37.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T21:49:37.062 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405/remote/smithi122 2024-04-20T21:49:37.062 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T21:49:37.094 INFO:teuthology.task.internal:Removing archive directory... 2024-04-20T21:49:37.094 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T21:49:37.101 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T21:49:37.104 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T21:49:37.154 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-20T21:49:37.166 INFO:teuthology.task.internal:Not uploading archives. 2024-04-20T21:49:37.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-20T21:49:37.177 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-20T21:49:37.177 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T21:49:37.183 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T21:49:37.185 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T21:49:37.199 INFO:teuthology.orchestra.run.smithi033.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:49 /home/ubuntu/cephtest 2024-04-20T21:49:37.200 INFO:teuthology.orchestra.run.smithi122.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:49 /home/ubuntu/cephtest 2024-04-20T21:49:37.234 INFO:teuthology.orchestra.run.smithi022.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:49 /home/ubuntu/cephtest 2024-04-20T21:49:37.237 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-20T21:49:37.249 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-20T21:49:37.274 INFO:teuthology.nuke:Checking targets against current locks 2024-04-20T21:49:37.297 DEBUG:teuthology.nuke:shortname: smithi022 2024-04-20T21:49:37.298 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T21:49:37.319 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:43.907839', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAXRcctz8qWnJkfkKzMfepnmRHdg99/wZeDzBv2PXjD1R27OlTS8c9NJ5ZyJ/wD1rLapEjN6LAnt4wyA1i8gbPo='} 2024-04-20T21:49:37.324 DEBUG:teuthology.nuke:shortname: smithi033 2024-04-20T21:49:37.324 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T21:49:37.346 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:43.911319', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBII995n8+j7s9P+oKxzAF7j1Qrg2cvouQaBQxRHEw4S/nTOqFycPqHN9eaS5EdEjDbTwZRdvRl0tOA59InC24zU='} 2024-04-20T21:49:37.351 DEBUG:teuthology.nuke:shortname: smithi122 2024-04-20T21:49:37.352 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T21:49:37.381 INFO:teuthology.orchestra.console.smithi022:Power off 2024-04-20T21:49:37.381 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T21:49:37.392 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:43.909793', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHh2kZNUTGlXQpaIf/Re/UpcoG6gxbpABInNp+oGhB2UtHQIcVHuDq6Y/TXERCv+GxePAkH0S56PO/9VIe4+3eE='} 2024-04-20T21:49:37.406 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off 2024-04-20T21:49:37.407 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:49:37.434 INFO:teuthology.orchestra.console.smithi033:Power off 2024-04-20T21:49:37.434 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T21:49:37.447 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-04-20T21:49:37.460 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2024-04-20T21:49:37.461 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:49:37.476 INFO:teuthology.orchestra.console.smithi122:Power off 2024-04-20T21:49:37.477 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T21:49:37.487 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-20T21:49:37.499 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off 2024-04-20T21:49:37.499 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:49:37.523 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-20T21:49:41.448 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:49:41.489 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:49:41.524 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:49:41.558 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-04-20T21:49:41.599 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-20T21:49:41.634 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-20T21:49:45.562 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:49:45.601 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:49:45.635 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:49:45.673 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2024-04-20T21:49:45.711 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-20T21:49:45.745 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2024-04-20T21:49:45.774 INFO:teuthology.orchestra.console.smithi022:Power off completed 2024-04-20T21:49:45.846 INFO:teuthology.orchestra.console.smithi122:Power off completed 2024-04-20T21:49:49.716 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:49:49.826 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-04-20T21:49:49.927 INFO:teuthology.orchestra.console.smithi033:Power off completed 2024-04-20T21:49:50.034 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} duration: 2260.011634349823 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi022 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=7e9ec60b3de24c11955d7d0e6bf852e1 status: fail success: false 2024-04-20T21:49:50.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T21:49:50.120 INFO:teuthology.run:FAIL